Skin lesion image segmentation method based on feature interaction fusion

Abstract

The invention discloses a skin lesion image segmentation method based on feature interaction fusion, and relates to the technical field of medical image segmentation. Comprising the following steps: firstly, carrying out size normalization and noise suppression on a dermatoscope image, and expanding a data set through random rotation and overturning operation; inputting the training data into the network model for training to obtain a trained model weight; and finally, inputting a to-be-segmented skin lesion image into the trained network model to realize automatic segmentation of a skin lesion area. According to the invention, a focusing feature interaction fusion module is designed, and deep complementary fusion of CNN and Transform features is realized; a multi-scale context attention module is constructed, multi-scale accurate feature extraction is realized by extracting spatial information from local scale to global scale, and meanwhile, the calculation efficiency is kept. The invention aims to solve the problems of complex shape, variable size, low contrast and the like of the lesion area of the skin lesion image, and improve the segmentation precision and robustness of the skin lesion image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image segmentation, and more particularly to a skin lesion image segmentation method based on feature interaction fusion. BACKGROUND

[0002] With the continuous rise of skin cancer and other skin lesions, accurate identification and segmentation of skin lesion areas are of great significance for early detection and treatment of diseases. However, skin lesion images usually have the following challenges: first, the boundary of the lesion area is fuzzy and the shape is complex, which is easy to be confused with the surrounding skin tissue; second, the size of the lesion varies greatly, and traditional segmentation methods are difficult to consider multi-scale features; third, the image contrast is low and the noise is more, which brings great difficulty to automatic segmentation.

[0003] At present, the automatic segmentation method based on convolutional neural network (CNN) performs excellently in extracting local features, but due to its inherent local receptive field limitation, it is difficult to capture the global context information in skin lesion images, resulting in deficiencies in boundary segmentation and semantic understanding. At the same time, the Transformer structure has made remarkable progress in natural language processing and computer vision in recent years, and with its powerful global modeling capability, it provides a new idea for medical image segmentation. However, the Transformer has certain limitations in processing fine-grained features, and it is difficult to accurately depict the local details of the lesion. Therefore, the fusion of the advantages of CNN and Transformer becomes an ideal solution.

[0004] However, the current mainstream fusion strategy mostly adopts a simple feature concatenation or addition method, which fails to fully exploit the complementary relationship and synergistic potential between the two. In addition, the existing attention mechanism, while improving performance, often introduces a large amount of computational overhead, and usually only focuses on a single scale or global context, lacking comprehensive modeling capability for multi-scale features. These technical bottlenecks seriously restrict the segmentation accuracy and clinical application value of the model in complex skin lesion scenarios.

[0005] Therefore, there is an urgent need for a new image segmentation method that can efficiently fuse local and global features, have multi-scale modeling capability, and maintain low computational cost, thereby improving the accuracy and robustness of skin lesion area segmentation. SUMMARY

[0006] Therefore, the present application provides a skin lesion image segmentation method based on feature interaction fusion to solve the problems of low segmentation accuracy and weak scale perception in the prior art, thereby improving the recognition and processing capability of complex lesion areas.

[0007] To achieve the above purpose, the present application adopts the following technical solutions:

[0008] The embodiment of the present application provides a skin lesion image segmentation method based on feature interaction fusion, comprising the following steps:

[0009] Image preprocessing and data enhancement: the dermoscope image is subjected to size normalization and noise suppression, and the data set is expanded through random rotation and flipping operation;

[0010] Model construction: a deep segmentation network combining CNN and Transformer is constructed, comprising: a feature extraction double encoder for extracting local features based on a convolutional neural network and global features based on a Transformer; a focused feature interaction fusion module for realizing complementary interaction and fusion between convolutional neural network features and Transformer features; a multi-scale context attention module for introducing multi-scale spatial perception ability and enhancing feature representation; a decoder: through upsampling to restore the resolution, and channel splicing with the encoder features transmitted through the jump connection, after convolution, batch normalization and activation function processing, the segmentation result is reconstructed layer by layer, and finally the segmentation probability map of the skin lesion area is output;

[0011] Model training: the images of the data set are input into the deep segmentation network, the network weight is optimized by using the joint loss function of Dice loss and binary cross entropy loss, and the early stopping strategy is combined to prevent overfitting;

[0012] Segmentation prediction: the dermoscope image to be segmented is input into the trained deep segmentation network, and the segmentation mask map of the skin lesion area is output.

[0013] In one embodiment, in the step of model construction, the feature extraction double encoder comprises a SwinTransformer encoder and a U-Net encoder;

[0014] The Swin Transformer encoder comprises four layers, each layer is subjected to feature extraction by a Swin Transformer block, and the output feature dimensions are 、 、 、 respectively, for capturing long-distance dependency relationships of the lesion;

[0015] The U-Net encoder comprises five layers, each layer is composed of a convolutional layer and a pooling layer, and the output feature dimensions are 、 、 、 、 respectively, for extracting local detailed features of the lesion.

[0016] In an embodiment, in the step of model construction, the multi-scale context attention module receives the features extracted by the first layer convolution of the U-Net encoder, and performs multi-scale spatial information extraction and fusion.

[0017] The focus feature interaction fusion module receives the features extracted by the second to fifth layers of the U-Net encoder and the features extracted by the first to fourth layers of the Swin-Transformer encoder, and realizes layer-by-layer fusion.

[0018] In an embodiment, the focus feature interaction fusion module performs the following operations:

[0019] Feature focusing: enhance features through the SE module, first perform 1x1 convolution channel matching on the Swin Transformer encoder feature map; and perform channel attention enhancement on the feature map from the U-Net encoder;

[0020] Feature interaction: generate a reverse attention mask, cross-weight to supplement missing information;

[0021] Feature fusion: concatenate the interaction features and the directly added features, and output through a 1x1 convolution.

[0022] In an embodiment, in the step of feature focusing, the Swin Transformer encoder feature map is matched in channel number through a 1x1 convolution layer, and then input to the SE module to enhance its effective features, as follows: ;

[0023] For the feature map from the U-Net encoder , it is directly input to the SE module for feature enhancement, and the processing process is as follows: ;

[0024] wherein, represents a 1x1 convolution operation, represents an SE module, represents a Sigmoid activation function, represents a channel-wise element multiplication operation.

[0025] In an embodiment, in the step of feature interaction, the SE module enhanced Transformer features and CNN features are respectively input to the Sigmoid function to generate an attention mask; by taking the complement of the attention mask, the corresponding reverse attention mask is obtained; the reverse attention mask is used to cross-weight the heterogeneous features, and the calculation process is as follows: ; ;

[0026] wherein, and respectively represent the Transformer feature map and the CNN feature map after channel attention processing; the symbol represents a Sigmoid activation function, J represents an all-one matrix, and the output is subtracted to form complementary weights; represents an element-wise multiplication operation.

[0027] In an embodiment, in the feature fusion step, the Transformer feature map and the CNN feature map after cross-feature weighting processing are element-wise added and fused, and a 1x1 convolution is performed to generate preliminary fusion features as the first group of fusion features;

[0028] The Transformer feature map and the CNN feature map after focus enhancement are also element-wise added to obtain the second group of fusion features;

[0029] The first group of fusion features and the second group of fusion features are spliced, and a 1x1 convolution is performed to adjust the number of channels to generate a fusion output feature map ; the expression is as follows: ;

[0030] wherein, represents a feature splicing operation.

[0031] In an embodiment, the multi-scale context attention module performs the following operations:

[0032] Feature division: receiving an input feature map , and dividing it into four sub-feature maps , , and in the channel dimension, wherein each sub-feature map has a size of ;

[0033] Multi-scale feature extraction: through four parallel branches, different scales of image features are captured on the four sub-feature maps through different receptive field structures;

[0034] Feature fusion and output: the feature maps , , and In the channel dimension splicing, a fusion feature map is formed, and finally an output feature map is obtained .

[0035] In one embodiment, in the step of multi-scale feature extraction:

[0036] The first branch is used for fine feature extraction on the input feature by point-wise convolution .

[0037] The second branch is used for local feature extraction on the input feature by channel-wise convolution and point-wise convolution operations .

[0038] The third branch is used for medium-range context information extraction on the input feature by applying channel-wise convolution, channel-wise atrous convolution and point-wise convolution .

[0039] The fourth branch first down-samples the input feature to extract important features while preserving their location information, and then realizes large-scale modeling by channel-wise convolution, channel-wise atrous convolution and point-wise convolution, and finally up-samples the feature map back to the original size through the location information .

[0040] According to the technical solution described above, compared with the prior art, the present application has the following technical advantages:

[0041] Firstly, the present application effectively realizes the deep complementary fusion of CNN and Transformer features by introducing a focus feature interaction fusion module. This module preserves the local details of the image while strengthening the global semantic information, significantly improving the model's performance in handling fuzzy boundaries and complex structures. Secondly, the present application uses a multi-scale context attention module, which can accurately capture multi-scale information from small lesions to large structures while maintaining low computational cost, thereby improving the model's adaptability to targets of different sizes. In addition, the entire model structure design focuses on lightweight and efficiency, making it easy to deploy in clinical auxiliary diagnosis systems or embedded devices, with good practical application feasibility. Finally, through experiments on multiple mainstream skin disease image datasets, the method significantly outperforms existing technologies in segmentation accuracy, fully demonstrating its effectiveness and practical value. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0043] Figure 1 A feature interaction fusion-based skin lesion image segmentation method flowchart is provided for the present application.

[0044] Figure 2 A whole network architecture schematic diagram of a skin lesion image segmentation model is provided for the present application.

[0045] Figure 3 A structure schematic diagram of a focused feature interaction fusion module is provided for the present application.

[0046] Figure 4 A structure schematic diagram of a multi-scale context attention module is provided for the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0048] Referring to Figure 1 The embodiments of the present application disclose a feature interaction fusion-based skin lesion image segmentation method, which comprises the following steps S1-S4 for convenience of description:

[0049] S1: Image preprocessing and data enhancement: normalizing the size and suppressing the noise of the dermoscope image, and expanding the data set through random rotation and flipping operations;

[0050] This step is used to build a data set. First, the collected original skin lesion image is standardized, including size unification and noise suppression operations, to ensure the consistency of the input data. Then, the data set is expanded using data enhancement techniques to enhance the generalization ability of the model. The enhancement means includes random rotation and flipping to form diversified data samples for robustness improvement in the training phase.

[0051] S2: Model construction: constructing a deep segmentation network that fuses CNN and Transformer, as shown in Figure 2 includes:

[0052] Feature extraction dual encoder, used to extract local features based on convolutional neural network and global features based on Transformer respectively;

[0053] Focused feature interaction fusion module, used to realize the complementary interaction and fusion between the convolutional neural network features and the Transformer features;

[0054] A multi-scale context attention module is used to introduce multi-scale spatial perception ability and enhance feature representation.

[0055] The decoder recovers the resolution by upsampling, and performs channel splicing with the encoder features transmitted by the skip connection, and then performs convolution, batch normalization and activation function processing, and finally reconstructs the skin lesion image segmentation result layer by layer to output the segmentation probability map of the skin lesion region.

[0056] The application designs a deep segmentation network based on a CNN and a Transformer fusion structure, wherein the feature extraction double encoder comprises a Swin Transformer encoder and a U-Net encoder.

[0057] The Swin Transformer encoder comprises four layers, each of which extracts features by a Swin Transformer block. The input image first enters a linear embedding layer as an initial processing layer of the Swin Transformer encoder, divides the image into non-overlapping image blocks, and converts the pixel values of each image block into an embedding vector through linear projection. The output feature dimensions of the four-layer structure are respectively , , , , which are used to capture long-distance dependencies of the lesion.

[0058] The U-Net encoder comprises five layers, each of which is composed of a convolution layer and a pooling layer. According to the input image dimension , , , , , the feature dimensions output by the U-Net encoder are respectively

[0059] It is emphasized that the features extracted by the first layer convolution of the U-Net encoder are input into the multi-scale context attention module, and through the extraction and fusion of multi-scale spatial information, the scale perception from local to global is realized under the premise of keeping low computational complexity, and the recognition ability of the model to different size lesion regions is enhanced. Subsequently, the features extracted by the second to fifth layers of the U-Net encoder are input into the focus feature interaction fusion module together with the features extracted by the first to fourth layers of the Swin-Transformer encoder, so as to realize layer-by-layer fusion. The decoder gradually restores the detail resolution of the skin lesion image through upsampling and skip connection. Finally, the network outputs a probability map through the Sigmoid activation function, and a binary skin lesion region segmentation mask is finally generated through threshold processing.

[0060] S3: Model training: input the images of the data set (using dermoscopy images with real masks) into the deep segmentation network, and optimize the network weight by using the joint loss function of Dice loss and binary cross entropy loss, combined with the early stopping strategy to prevent overfitting.

[0061] After completing the model structure design, the preprocessed and enhanced images are input into the constructed network model for training. The model training adopts the Adam optimizer, the initial learning rate is set to 0.00001, the Dice loss and cross entropy loss are jointly optimized to ensure that the model has good discrimination ability at the pixel level. At the same time, the early stopping mechanism is introduced to prevent model overfitting.

[0062] S4: Segmentation prediction: input the dermoscopy image to be segmented into the trained deep segmentation network, and output the segmentation mask of the skin lesion region.

[0063] The trained network model is used for actual dermoscopy image segmentation. After inputting the image to be detected, the model can automatically output the segmentation result of the lesion region. The output is a mask image with the same size as the original image. This process does not require human intervention and can be widely used in skin disease early auxiliary diagnosis systems.

[0064] The focus feature interaction fusion module designed in the application realizes the deep complementary fusion of CNN and Transformer features; the multi-scale context attention module is constructed to realize multi-scale accurate feature extraction by extracting local to global scale spatial information while keeping the computational efficiency. The application can solve the problems of complex shape, variable size and low contrast of skin lesion image lesion regions, and improve the segmentation accuracy and robustness of skin lesion images.

[0065] Further, the focus feature interaction fusion module in step S2 above has the structure as shown in Figure 3As shown, to realize the deep complementary fusion of CNN and Transformer features, it can be described as the following 3 steps:

[0066] 1. Focus on key features. This step is used to receive feature maps from the Transformer encoder and the CNN encoder respectively. First, the feature map output by the Transformer encoder is subjected to 1x1 convolution to match the number of channels, and then both the feature map of the CNN encoder and the feature map of the Transformer encoder are input into the SE module to extract key channel information, thereby obtaining the Transformer feature map and the CNN feature map respectively after channel attention enhancement.

[0067] Specifically, to suppress redundant information and highlight key information, the SE module is introduced in the focus feature interaction fusion module of the present application to strengthen the discriminative feature channels in the Transformer and CNN branches, and to realize the extraction of key features. The process equation is as follows: (1) ;

[0068] For the feature map from the CNN encoder, it is directly input into the SE module for feature enhancement, and the processing process is as follows: (2) ;

[0069] In these two formulas, represents the feature map from the Transformer, represents the feature map output from the CNN branch (U-Net encoder). represents the 1x1 convolution operation, represents the SE module, represents the Sigmoid activation function, represents the element-by-channel multiplication operation.

[0070] 2. Recall the ignored details. This step is used to generate a reverse attention mask, which can be used to highlight important areas that the current encoder has failed to pay attention to. The reverse attention mask is generated using the feature map of the other encoder, and the feature map of the current encoder is cross- weighted based on the reverse attention mask to supplement the key information that the respective encoder has missed in the feature expression process.

[0071] Specifically, to further realize the complementary fusion between features, the focus feature interaction fusion module of the application proposes a reverse attention mechanism to make full use of the complementarity of Transformer and CNN in local and global feature modeling. The method is as follows: first, the SE module enhanced Transformer features and CNN features are respectively input into the Sigmoid function to generate attention masks. By taking the complement of the attention mask, the corresponding reverse attention mask is obtained. Then, the reverse attention is used to cross-weight the heterogeneous features, thereby realizing the complementary recovery of detailed information, and the calculation process is as follows: (3); (4);

[0072] wherein, and respectively represent the Transformer feature map and the CNN feature map after channel attention processing; the symbol represents the Sigmoid activation function, J represents the all-one matrix, and the output is subtracted to form a complementary weight; represents the element-wise multiplication operation.

[0073] 3. Fusion of complementary information.

[0074] Specifically, in the phase of fusing complementary information, first, the Transformer feature map and the CNN feature map after cross-feature weighting processing are element-wise added and fused, and a 1x1 convolution is used to generate preliminary fusion features. At the same time, the focus enhanced Transformer feature map and the CNN feature map are also element-wise added to obtain another group of fusion features. Then, the two types of features are spliced, and a 1x1 convolution is used to adjust the channel number to generate a fusion output feature map . The expression is as follows: (5);

[0075] wherein, represents the feature splicing operation.

[0076] Further, the multi-scale context attention module of the above step S2, the structure is shown in Figure 4 , which is used to process the input feature map to obtain multi-scale feature information, thereby improving the recognition ability of the model to multi-scale lesion targets. This module can balance the efficiency and representation ability of feature extraction, and is suitable for resource-constrained scenarios. The specific implementation method is as follows 1)~3) steps:

[0077] 1) Feature division:

[0078] First, the input feature map is denoted as , where H, W and C represent the height, width and channel number of the feature map, respectively. The feature map is evenly divided into four sub-feature maps in the channel dimension, i.e. , where each sub-feature map .

[0079] 2) Multi-scale feature extraction: Through four parallel branches, different scales of image features are captured on the four sub-feature maps respectively through different receptive field structures;

[0080] The first branch takes the sub-feature map as input, first uses a set of one-dimensional convolution operations corresponding to Figure 4 pointwise convolution (input channel number = C / 4, output channel number = C, convolution kernel size = 1, step = 1) to expand its channel dimension. Then, batch normalization (BN) and GeLU activation function are applied in turn to enhance the nonlinear feature expression ability. Finally, another set of pointwise convolution operations with parameters (input channel number = C, output channel number = C / 4, convolution kernel size = 1, step = 1) are applied to compress the channels, and the output features are normalized by Sigmoid activation, and then element-wise multiplied with the original input to obtain the final output of the branch .

[0081] The second branch processes the sub-feature map , which specifically includes a channel-wise convolution operation to avoid cross-channel calculation overhead, with parameters input channel number = C / 4, output channel number = C / 4, convolution kernel size = 3, step = 1, padding = 1, group number = C / 4. Next, after further processing by batch normalization (BN) and GeLU activation function, a pointwise convolution (input channel number = C / 4, output channel number = C / 4, convolution kernel size = 1, step = 1) is applied to integrate the information between channels. Finally, after processing by the Sigmoid function, element-wise multiplication is performed with the original input to form the branch output .

[0082] The third branch takes the sub-feature map as input to obtain medium-dimensional feature information. For The following operations are sequentially applied: the first step uses the per-channel convolution parameters (input channel number = C / 4, output channel number = C / 4, convolution kernel size = 5, step = 1, padding = 2, and group number = C / 4); the second step uses the per-channel atrous convolution parameters (input channel number = C / 4, output channel number = C / 4, convolution kernel size = 7, step = 1, padding = 9, group number = C / 4, and dilation coefficient = 3). After the above two steps, point-wise convolution (input channel number = C / 4, output channel number = C / 8, convolution kernel size = 1, and step = 1) is performed in the channel dimension to splice and integrate cross-channel information. Finally, Sigmoid activation is performed, and the output of the branch is obtained by multiplying the input .

[0083] The fourth branch aims to capture sparse global feature relationships. First, the important features are extracted from the by a downsampling operation and are reconstructed into while retaining their original position information for subsequent restoration. The is sequentially subjected to the following operations: the first step uses a group of per-channel convolutions (parameters: input channel number = C / 4, output channel number = C / 4, convolution kernel size = 5, step = 1, padding = 2, and group number = C / 4) to obtain intermediate features ; the second step applies per-channel atrous convolution (input channel number = C / 4, output channel number = C / 4, convolution kernel size = 7, step = 1, padding = 9, group number = C / 4, and dilation coefficient = 3) to obtain ; the third step performs point-wise convolution (input channel number = C / 4, output channel number = C / 8, convolution kernel size = 1, and step = 1) on and respectively, and then splices them in the channel dimension to form a fusion feature; the fourth step up-samples the feature map to the original size through the position information. The fifth step obtains the final output by performing Sigmoid activation and multiplying the original .

[0084] 3) Feature fusion and output:

[0085] The output features obtained by the above four branches are spliced in the channel dimension to form a fusion feature map, and the final output feature map is obtained.

[0086] The following network training configuration is adopted in this embodiment: the maximum number of iterations is set to 100, the batch size is set to 16, and the initial learning rate is set to 0.00001. In the design of the loss function, a hybrid form combining binary cross-entropy loss and Dice loss is adopted. In addition, the Adam optimizer is used for back propagation to update the model parameters. During the training process, an early stopping strategy is implemented, and the training is terminated when the performance of the validation set no longer improves, and the current optimal model weight is saved. After iterative optimization, the model gradually learns discriminative feature representations, and finally obtains a skin lesion image segmentation model with optimal performance on the validation set.

[0087] On the ISIC2018 dataset, this embodiment comprehensively compares the performance of the proposed network model with the current mainstream image segmentation models. The models involved in the comparison include: U-Net, U-Net++, Attention U-Net, TransUNet, I2U-Net, and CSWin-UNet. The evaluation indicators include accuracy, precision, recall, Dice coefficient (DSC), and Jaccard Index, to comprehensively measure the performance of each model in the skin lesion image segmentation task. Table 1 shows the comparison results of each model on the ISIC2018 dataset. ISIC2018 is a skin lesion analysis dataset released by the International Skin Imaging Collaboration (ISIC), mainly used to promote the application of machine learning in skin cancer detection.

[0088] Table 1. Skin lesion segmentation performance of different networks on ISIC2018.

[0089] Model Accuracy (%) Precision (%) Recall (%) Dice coefficient (%) Jaccard coefficient (%) U-Net 91.66 85.05 85.15 85.10 74.06 U-Net++ 91.92 84.66 86.96 85.77 75.10 Attention U-Net 92.26 86.48 85.72 86.09 75.59 TransUNet 92.67 87.80 85.89 86.78 76.65 I 2 U-Net 91.71 86.28 83.67 84.95 73.85 CSWin-UNet 92.75 88.91 84.64 86.73 76.56 The invention 93.22 84.73 92.43 88.41 79.22

[0090] As shown in Table 1, the performance of the method of the present application is compared with other methods in terms of performance indicators. It can be clearly found from the table that compared with other methods, the method of the present application generally achieves the best performance on the ISIC2018 skin lesion image dataset. Specifically, the accuracy, precision, recall, Dice coefficient, and Jaccard coefficient of the present application are 93.22%, 84.73%, 92.43%, 88.41%, and 79.22%, respectively.

[0091] To verify the performance improvement effect of the network module of the present application, five groups of ablation experiments are designed in this embodiment: first, a model with only the output feature dimension of the U-Net adjusted is taken as the baseline (subnetwork 1); on this basis, the focus feature interaction fusion module without the feature focusing part (subnetwork 2) and the complete focus feature interaction fusion module (subnetwork 3) are added respectively to evaluate the independent contribution of the interaction structure and the feature focusing mechanism to the performance; then, the multiscale context attention module is integrated on the subnetwork 1 (subnetwork 4) to test the independent effect of the attention mechanism; finally, the focus feature interaction fusion module and the scale-aware attention module are introduced at the same time (subnetwork 5) to form a complete model. By comparing the experimental results of each subnetwork, the contribution of the two modules and their combination to the performance improvement is systematically evaluated, thereby verifying the rationality and practicability of the design of the present application. Table 2 shows the comparison results of segmentation performance on the ISIC2018 dataset.

[0092] Table 2. Ablation study of key components on ISIC2018 dataset.

[0093] Model Accuracy Precision Recall Dice coefficient (%) Jaccard coefficient (%) Subnetwork 1 91.44 84.62 84.89 84.73 73.51 Subnetwork 2 92.52 83.74 91.00 87.19 77.30 Subnetwork 3 92.63 82.97 92.81 87.58 77.91 Subnetwork 4 92.58 87.68 85.49 86.57 76.32 Subnetwork 5 93.22 84.73 92.43 88.41 79.22

[0094] The experimental data in the table clearly show that enhancing a module alone can bring certain performance improvement, and the effective combination of each module also shows the optimal overall performance.

[0095] Finally, the skin lesion image to be processed can be input into the skin lesion image segmentation model, and the segmentation result of the corresponding skin lesion image is output.

[0096] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between each embodiment can be referred to each other. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0097] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A skin lesion image segmentation method based on feature interaction fusion, characterized in that, The method comprises the following steps: Image preprocessing and data enhancement: normalizing the size and suppressing the noise of the dermoscopy image, and expanding the data set through random rotation and flipping operations; Model construction: constructing a deep segmentation network combining CNN and Transformer, including: a double-encoder feature extraction module for extracting local features based on a convolutional neural network and global features based on a Transformer; a focused feature interaction fusion module for realizing complementary interaction and fusion between convolutional neural network features and Transformer features; a multi-scale context attention module for introducing multi-scale spatial perception ability and enhancing feature representation; a decoder: restoring the resolution through upsampling, and performing channel splicing with the encoder features transmitted through the skip connection, and then processing through convolution, batch normalization and activation function to reconstruct the segmentation result layer by layer, and finally outputting the segmentation probability map of the skin lesion area; Model training: inputting the images of the data set into the deep segmentation network, and optimizing the network weights using the joint loss function of Dice loss and binary cross-entropy loss, and combining the early stopping strategy to prevent overfitting; Segmentation prediction: inputting the dermoscopy image to be segmented into the trained deep segmentation network to output the segmentation mask of the skin lesion area.

2. The skin lesion image segmentation method based on feature interaction fusion according to claim 1, characterized in that, In the step of model construction, the double-encoder feature extraction module comprises a Swin Transformer encoder and a U-Net encoder; The Swin Transformer encoder comprises a four-layer structure, each layer is subjected to feature extraction by a Swin Transformer block, and the output feature dimensions are , , , respectively, for capturing long-distance dependencies of the lesion. The U-Net encoder comprises a five-layer structure, each layer being composed of a convolutional layer and a pooling layer, and the output feature dimensions are 64, 128, 256, 512 and 1024 respectively , , , , for extracting local detailed features of the lesion.

3. The skin lesion image segmentation method based on feature interaction fusion according to claim 2, characterized in that, In the step of model construction, the multi-scale context attention module receives the features extracted by the first layer convolution of the U-Net encoder for multi-scale spatial information extraction and fusion; The focused feature interaction fusion module receives the features extracted by the second to fifth layers of the U-Net encoder and the features extracted by the first to fourth layers of the Swin-Transformer encoder for layer-by-layer fusion.

4. The skin lesion image segmentation method based on feature interaction fusion according to claim 3, characterized in that, The focused feature interaction fusion module performs the following operations: Feature focusing: enhancing the features through an SE module, performing 1x1 convolution channel matching on the Swin Transformer encoder feature map first; and performing channel attention enhancement on the feature map from the U-Net encoder; Feature interaction: generating a reverse attention mask to cross-weight and supplement the missing information; Feature fusion: concatenating the interaction features and the direct addition features, and outputting through 1x1 convolution.

5. The skin lesion image segmentation method based on feature interaction fusion according to claim 4, characterized in that, In the feature focusing step, the feature map from the Swin Transformer encoder The channel number is matched by a 1x1 convolutional layer, and then input into the SE module to enhance the effective features, as follows: ； For the feature map from the U-Net encoder , the SE module is directly inputted for feature enhancement, and the processing process is as follows: ； wherein, denotes a 1 x 1 convolution operation, denotes an SE module, denotes a Sigmoid activation function, denotes an element-wise multiplication operation per channel.

6. The skin lesion image segmentation method based on feature interaction fusion according to claim 5, characterized in that, In the feature interaction step, the Transformer features and CNN features enhanced by the SE module are respectively input into the Sigmoid function to generate an attention mask; the corresponding reverse attention mask is obtained by taking the complement of the attention mask; the reverse attention mask is used to cross-weight the heterogeneous features, and the calculation process is as follows: ； ； wherein, and respectively represent the transformed feature map and the CNN feature map after passing through the channel attention processing; the symbol represents a Sigmoid activation function, J represents an all-one matrix, and the output is subtracted to form a complementary weight; represents an element-wise multiplication operation.

7. The skin lesion image segmentation method based on feature interaction fusion according to claim 6, characterized in that, In the feature fusion step, the cross-feature weighted processed Transformer feature map is element-wise added with the CNN feature map to generate a preliminary fusion feature through 1x1 convolution as the first group of fusion features. Focusing enhanced transformer feature map with cnn feature map Similarly, element-wise addition is performed to obtain a second set of fused features; The first group of fusion features and the second group of fusion features are spliced, and a fusion output feature map is generated by adjusting the number of channels through 1*1 convolution ; the expression is as follows: ； wherein represents a feature stitching operation.

8. The skin lesion image segmentation method based on feature interaction fusion according to claim 3, characterized in that, The multi-scale context attention module performs the following operations: Feature division: receive input feature map , and divide it into four sub-feature maps on the channel dimension , , and , where the size of each sub-feature map is ; Multi-scale feature extraction: through four parallel branches, different scale image features are captured on the four sub-feature maps through different receptive field structures; Feature fusion and output: the feature maps output by the four parallel branches are spliced in the channel dimension to form a fused feature map, and finally an output feature map is obtained , , and In the channel dimension, the feature maps output by the four parallel branches are spliced to form a fused feature map, and finally an output feature map is obtained .

9. The skin lesion image segmentation method based on feature interaction fusion according to claim 8, characterized in that, In the step of multi-scale feature extraction: The first branch uses point-wise convolution for processing the input features extracting fine features; The second branch is used for local feature extraction on the input features through a channel-wise convolution and a point-wise convolution operation ; The third branch extracts medium-range context information by applying a channel-wise convolution, a channel-wise atrous convolution, and a point-wise convolution on the input features ​ The fourth branch first processes the input features Down-sampling extracts important features while preserving their location information; then, large-scale modeling is achieved through channel-wise convolution, channel-wise dilated convolution, and point-wise convolution, and the features are up-sampled back to the original size through location information.

Images