A method for rapid detection and segmentation of skin lesion area

Abstract

The application relates to a skin lesion area rapid detection and segmentation method, which comprises the following steps: given an input X element R C×H×W , which becomes Y element R C×HW after remodeling operation, and 4 times expanded attention map Att element R 4C×HW is obtained by calculating the relevance of the query vector and the memory unit; given an input feature map X and a randomly initialized learnable tensor P; the size of the tensor P is adjusted by using bilinear interpolation to match the size of the X, and a depth separable convolution is used on the P; the feature map is uniformly cut into four parts X1, X2, X3 and X4 along the channel dimension. The application improves and introduces Hadamard product attention module to extract multi-angle pathological features of different shaft groupings of the feature map, fuses multi-scale context information, aggregates cross-dimension information, improves the representation ability of the model, designs a new boundary loss function, and puts the boundary information into the model learning process, encourages the model to pay attention to the boundary details, is excellent in various segmentation performance indexes, and can accurately extract a lesion area.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing, computer-aided diagnosis, and artificial intelligence, specifically to a method for rapid detection and segmentation of skin lesion areas. Background Technology

[0002] Melanoma, usually referring to malignant melanoma, is a highly malignant tumor originating from melanocytes. It is caused by the abnormal growth of melanocytes in the skin and has a high mortality rate. If detected early, melanoma can be cured with simple outpatient surgery; however, if diagnosed at an advanced stage, the patient survival rate drops from 99% to 23%. The accuracy rate of clinical diagnosis of early melanoma relying on the naked eye is only slightly above 60%, while computer-aided diagnostic systems can automatically analyze dermoscopic images, accurately segmenting the lesion area and improving the accuracy of doctors' diagnosis of melanoma, thereby reducing the severe impact of the disease on patients.

[0003] For example, Chinese Patent 202310862791.8 proposes a classification and detection method, device, equipment, and storage medium for pigmented skin diseases. This method employs a classification model based on DenseNet and Xception network architectures, incorporating an attention mechanism, to output feature vectors for both the lesion region image and the original sample image. A feature fusion module then fuses the feature vectors from the lesion region image and the original sample image to obtain the classification result for the original sample image. This improves the classification model's ability to identify skin lesions.

[0004] In recent years, research combining convolutional neural networks (CNNs) and Transformers has gained increasing attention. CNNs are used to process local features and establish local correlations, further improving the performance of medical image segmentation. However, due to the quadratic complexity of Transformers, they require enormous computational resources, making them difficult to apply in real-world medical environments. Furthermore, boundary information is more important than other regions; emphasizing boundaries can reduce the influence of other regions and achieve more accurate results. Therefore, a fast detection and segmentation method for skin lesions is proposed. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for rapid detection and segmentation of skin lesion areas. This method has advantages such as aggregating cross-dimensional information, improving the model's representation ability, incorporating boundary information into the model learning process, encouraging the model to focus on boundary details, and accurately extracting lesion areas.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for rapid detection and segmentation of skin lesion areas, characterized by comprising the following steps:

[0007] Step 1: Improve the external attention module

[0008] Given an input After being remodeled and put into operation, it becomes A 4-fold expanded attention map is obtained by calculating the correlation between query vectors and memory units. ;

[0009] Step 2: Construct the Hadamard product attention module

[0010] S1. Given an input feature map X and a randomly initialized learnable tensor P;

[0011] S2. Use bilinear interpolation to adjust the size of tensor P to match the size of X, and apply depthwise separable convolution to P.

[0012] S3. Divide the feature map evenly into four parts along the channel dimension: X1, X2, X3, and X4.

[0013] S4. For the first three groups, perform Hadamard product operations on height-width, channel-height, channel-width and learnable tensors. For the fourth group, use depthwise separable convolution operations.

[0014] S5. Finally, the information from different angles is concatenated along the channel dimension and depthwise separable convolution is used to integrate the information.

[0015] Step 3: Design a multi-scale integrated bridge

[0016] S1. First, bilinear interpolation is used to adjust the high-level feature Y. h The size is used to match the low-level feature X1;

[0017] S2. Then, the two feature maps are divided into four parts along the channel dimension. A set of low-level features and a set of high-level features are concatenated to obtain four sets of fused features. The results are concatenated along the channel dimension and ordinary convolution with a kernel size of 1×1 is used to achieve feature interaction at different scales to obtain feature map Z.

[0018] S4. Local feature interactions between channels are captured by a 2D convolution with a kernel size of 1×1. Then, after passing through the sigmoid activation function, residual connections are used to adjust the order of the three branch dimensions and perform feature fusion in a unified manner to complete the extraction of multi-dimensional information and obtain the aggregated output Z0.

[0019] Step 4: Design the boundary loss function

[0020] S1. Use the Laplace operator to extract the boundary between the prediction results and the true labels;

[0021] S2. Calculate the error between the true boundary and the predicted boundary based on the mean square error;

[0022] S3, The constructed overall loss function;

[0023] S4. Use binary cross-entropy loss as the dominant loss function of the network and boundary loss as the auxiliary loss function of the network. Use α=0.8 and β=0.3 as the weight coefficients of the hybrid loss function.

[0024] Furthermore, step one also includes the following steps:

[0025] S1. First, map the input to a high-dimensional space;

[0026] S2, then the attention map is multiplied by the shared weights in another memory unit to produce a refinement.

[0027] Feature map Z;

[0028] S3. Reshape the obtained overall feature information to restore it to the original input dimension;

[0029] S4. After passing through a 1×1 two-dimensional convolution, the result is added to the original information to obtain the final feature map output.

[0030] Preferably, the multi-scale fusion bridge module has two inputs: low-level feature X. l and high-level features Y h .

[0031] Furthermore, in step three, different groups apply dilated convolutions with different dilation rates to extract information at different scales. The feature map Z is modeled with attention in the three dimensions of height, width, and number of channels. After adjusting the dimensional order of Z, it is fed into the squeeze transformer. Global average pooling and standard pooling are used in parallel for dual information interaction to enhance the information content and discriminative power of the feature descriptor.

[0032] Preferably, after step four is completed, an experiment is conducted. The experiment uses a hardware platform with an Intel(R) Core(TM) i5-12490F CPU, 32GB of memory, and an NVIDIA 3060 graphics card, and runs on the Ubuntu 20.04 operating system.

[0033] Compared with the prior art, the technical solution of this application has the following beneficial effects:

[0034] 1. This invention improves and introduces the Hadamard product attention module to extract pathological features from different axes of the feature map.

[0035] 2. This invention designs a multi-scale aggregation bridge, which integrates multi-scale contextual information and aggregates cross-dimensional information to improve the model's representation ability.

[0036] 3. This invention designs a new boundary loss function that incorporates boundary information into the model learning process, encourages the model to pay attention to boundary details, and performs excellently in various segmentation performance indicators, and can accurately extract lesion areas. Attached Figure Description

[0037] Figure 1 This is a flowchart of the rapid detection and segmentation method for skin lesions according to the present invention;

[0038] Figure 2 This is a flowchart of the improved external attention module of the present invention;

[0039] Figure 3 This is a flowchart of the Hadamard product attention module of the present invention;

[0040] Figure 4 This is a flowchart of the multi-scale aggregation bridge of the present invention;

[0041] Figure 5 Here is a flowchart of the boundary loss function.

[0042] Figure 6 The images show the segmentation results of dermoscopy images using different models of this invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Example 1

[0045] Please see Figure 1 and Figure 2 This embodiment provides a method for rapid detection and segmentation of skin lesion areas, including an improved external attention module, the steps of which are as follows:

[0046] Given an input After being remodeled and put into operation, it becomes A 4-fold expanded attention map is obtained by calculating the correlation between query vectors and memory units. .

[0047] It also includes the following steps:

[0048] S1. First, map the input to a high-dimensional space;

[0049] S2, then the attention map is multiplied by the shared weights in another memory unit to produce a refinement.

[0050] Feature map Z;

[0051] S3. Reshape the obtained overall feature information to restore it to the original input dimension;

[0052] S4. After passing through a 1×1 two-dimensional convolution, the result is added to the original information to obtain the final feature map output.

[0053] Example 2

[0054] Please see Figure 1 and Figure 3 A method for rapid detection and segmentation of skin lesion areas includes constructing a Hadamard product attention module, the steps of which are as follows:

[0055] S1. Given an input feature map X and a randomly initialized learnable tensor P;

[0056] S2. Use bilinear interpolation to adjust the size of tensor P to match the size of X, and apply depthwise separable convolution to P.

[0057] S3. Divide the feature map evenly into four parts along the channel dimension: X1, X2, X3, and X4.

[0058] S4. For the first three groups, perform Hadamard product operations on height-width, channel-height, channel-width and learnable tensors. For the fourth group, use depthwise separable convolution operations.

[0059] S5. Finally, the information from different angles is concatenated along the channel dimension and depthwise separable convolution is used to integrate the information.

[0060] Specifically, the Hadamard product attention module is represented by the following formula:

[0061]

[0062] Where DW represents a depthwise separable convolution with a kernel size of 3, BI represents bilinear interpolation, LN represents layer normalization, and Cat() represents a cascade operation on the channel dimension.

[0063] Example 3

[0064] Please see Figure 4 A rapid detection and segmentation method for skin lesions includes designing a multi-scale fusion bridge, the steps of which are as follows:

[0065] S1. First, bilinear interpolation is used to adjust the high-level feature Y. h The size is used to match the low-level feature X1;

[0066] S2. Then, the two feature maps are divided into four parts along the channel dimension. A set of low-level features and a set of high-level features are concatenated to obtain four sets of fused features. The results are concatenated along the channel dimension and ordinary convolution with a kernel size of 1×1 is used to achieve feature interaction at different scales to obtain feature map Z.

[0067] S4. Local feature interactions between channels are captured by a 2D convolution with a kernel size of 1×1. Then, after passing through the sigmoid activation function, residual connections are used to adjust the order of the three branch dimensions and perform feature fusion in a unified manner to complete the extraction of multi-dimensional information and obtain the aggregated output Z0.

[0068] Specifically, the multi-scale fusion bridge module has two inputs: low-level features X l and high-level features Y h Different groups are subjected to dilated convolutions with different dilation rates to extract information at different scales. Attention models are then applied to the feature map Z along the three dimensions of height, width, and number of channels. After adjusting the dimensional order of Z, the data is fed into a compression transformer, where global average pooling and standard pooling are used in parallel for dual information interaction to enhance the information content and discriminative power of the feature descriptors.

[0069] Furthermore, the calculation formula for the multi-scale fusion bridge is as follows:

[0070]

[0071] in, This indicates that the input feature map is divided into 4 groups along the channel dimension. DC() represents dilated convolution, and Avg() and Std() represent average pooling and standard pooling, respectively. It is the sigmoid activation function. The functions represent element-wise multiplication, Add() represents addition, and Cat() represents cascading operations along the channel dimension.

[0072] Example 4

[0073] Please see Figure 5 A method for rapid detection and segmentation of skin lesion areas, including the design of a boundary loss function, comprises the following steps:

[0074] S1. Use the Laplace operator to extract the boundary between the prediction results and the true labels;

[0075] S2. Calculate the error between the true boundary and the predicted boundary based on the mean square error;

[0076] S3, The constructed overall loss function;

[0077] S4. Use binary cross-entropy loss as the dominant loss function of the network and boundary loss as the auxiliary loss function of the network. Use α=0.8 and β=0.3 as the weight coefficients of the hybrid loss function.

[0078] Specifically, the formula for calculating the error between the true boundary and the predicted boundary based on the mean square error is as follows:

[0079]

[0080] in Let be the mean square error function. To predict the predicted boundary value of the i-th pixel in the boundary map, Let be the true boundary value of the i-th pixel in the true boundary map.

[0081] The overall loss function is:

[0082]

[0083] Where α and β are the weights of the loss function.

[0084] Experimental Example

[0085] Please see Figure 6 A rapid detection and segmentation method for skin lesions was proposed. The method utilized a hardware platform with an Intel(R) Core(TM) i5-12490F CPU, 32GB of RAM, and an NVIDIA 3060 graphics card, running on Ubuntu 20.04. The experimental results are shown in the table below.

[0086]

[0087] In summary, this invention improves and introduces a Hadamard product attention module to extract pathological features from different axes of the feature map. A multi-scale aggregation bridge is designed to fuse multi-scale contextual information and aggregate cross-dimensional information, thereby improving the model's representational ability. A novel boundary loss function is designed to incorporate boundary information into the model learning process, encouraging the model to focus on boundary details. This results in excellent performance across various segmentation metrics and accurate extraction of lesion regions.

[0088] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for rapid detection and segmentation of skin lesion areas, characterized in that, Includes the following steps: Step 1: Improve the external attention module Given an input After being remodeled and put into operation, it becomes A 4-fold expanded attention map is obtained by calculating the correlation between query vectors and memory units. ; Step 2: Construct the Hadamard product attention module S1. Given an input feature map X and a randomly initialized learnable tensor P; S2. Use bilinear interpolation to adjust the size of tensor P to match the size of X, and apply depthwise separable convolution to P. S3. Divide the feature map evenly into four parts along the channel dimension: X1, X2, X3, and X4. S4. For the first three groups, perform Hadamard product operations on height-width, channel-height, channel-width and learnable tensors. For the fourth group, use depthwise separable convolution operations. S5. Finally, the information from different angles is concatenated along the channel dimension and depthwise separable convolution is used to integrate the information. Step 3: Design a multi-scale integrated bridge S1. First, bilinear interpolation is used to adjust the high-level feature Y. h The size is used to match the low-level feature X1; S2. Then, the two feature maps are divided into four parts along the channel dimension. A set of low-level features and a set of high-level features are concatenated to obtain four sets of fused features. The results are concatenated along the channel dimension and ordinary convolution with a kernel size of 1×1 is used to achieve feature interaction at different scales to obtain feature map Z. S4. Local feature interactions between channels are captured by a 2D convolution with a kernel size of 1×1. Then, after passing through the sigmoid activation function, residual connections are used to adjust the order of the three branch dimensions and perform feature fusion in a unified manner to complete the extraction of multi-dimensional information and obtain the aggregated output Z0. Step 4: Design the boundary loss function S1. Use the Laplace operator to extract the boundary between the prediction results and the true labels; S2. Calculate the error between the true boundary and the predicted boundary based on the mean square error; S3, The constructed overall loss function; S4. Use binary cross-entropy loss as the dominant loss function of the network and boundary loss as the auxiliary loss function of the network. Use α=0.8 and β=0.3 as the weight coefficients of the hybrid loss function.

2. The method for rapid detection and segmentation of skin lesion areas according to claim 1, characterized in that, Step one also includes the following steps: S1. First, map the input to a high-dimensional space; S2, then the attention map is multiplied by the shared weights in another memory unit to produce a refinement. Feature map Z; S3. Reshape the obtained overall feature information to restore it to the original input dimension; S4. After passing through a 1×1 two-dimensional convolution, the result is added to the original information to obtain the final feature map output.

3. The method for rapid detection and segmentation of skin lesion areas according to claim 1, characterized in that, The Hadamard product attention module in step two is represented by the following formula: Where DW represents a depthwise separable convolution with a kernel size of 3, BI represents bilinear interpolation, LN represents layer normalization, and Cat() represents a cascade operation on the channel dimension.

4. The method for rapid detection and segmentation of skin lesion areas according to claim 1, characterized in that, The multi-scale fusion bridge module has two inputs: low-level feature X. l and high-level features Y h .

5. The method for rapid detection and segmentation of skin lesion areas according to claim 1, characterized in that, In step three, different groups apply dilated convolutions with different dilation rates to extract information at different scales. The feature map Z is modeled with attention in the three dimensions of height, width, and number of channels. After adjusting the dimensional order of Z, it is fed into the squeeze transformer. Global average pooling and standard pooling are used in parallel for dual information interaction to enhance the information content and discriminative power of the feature descriptor.

6. The method for rapid detection and segmentation of skin lesion areas according to claim 1, characterized in that, The calculation formula for the multi-scale fusion bridge is as follows: in, This indicates that the input feature map is divided into 4 groups along the channel dimension. DC() represents dilated convolution, and Avg() and Std() represent average pooling and standard pooling, respectively. It is the sigmoid activation function. The functions represent element-wise multiplication, Add() represents addition, and Cat() represents cascading operations along the channel dimension.

7. The method for rapid detection and segmentation of skin lesion areas according to claim 1, characterized in that, The formula for calculating the error between the true boundary and the predicted boundary based on the mean square error is as follows: in Let be the mean square error function. To predict the predicted boundary value of the i-th pixel in the boundary map, Let be the true boundary value of the i-th pixel in the true boundary map.

8. The method for rapid detection and segmentation of skin lesion areas according to claim 1, characterized in that, The overall loss function in step four is: Where α and β are the weights of the loss function.

9. The method for rapid detection and segmentation of skin lesion areas according to claim 1, characterized in that, After step four is completed, an experiment is conducted. The experiment uses a hardware platform with an Intel(R) Core(TM) i5-12490F CPU, 32GB of memory, and an NVIDIA 3060 graphics card, and runs on the Ubuntu 20.04 operating system.

Images