Medical image segmentation method based on wavelet transform and attention mechanism

Abstract

The invention provides a medical image segmentation method based on wavelet transform and an attention mechanism, and belongs to the technical field of image segmentation of medical image processing. The technical problem that semantic differences between global features and features of coders and decoders at the same level cannot be captured due to limitation of a traditional CNN receptive field is solved. According to the technical scheme, the method comprises the following steps of 1, preprocessing an input polyp medical image; step 2, constructing an encoder decoder network based on wavelet transform and an attention mechanism; 3, inputting the preprocessed polyp image data set into an encoder-decoder network of a wavelet transform and attention mechanism for training, and reserving an optimal model; and step 4, after training is completed, inputting the polyp medical image verification set into the obtained optimal model. And detecting the focus segmentation effect of the model in the polyp medical image verification set. According to the method, the polyp lesion can be segmented more accurately, and the accuracy and robustness are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a medical image segmentation method based on an encoder-decoder network of wavelet transform and attention mechanism. Background Art

[0002] Medical image segmentation is one of the important tasks in the field of medical image processing. Its purpose is to extract specific areas from medical images, such as organs, tissues, lesion areas, etc. This technology plays a vital role in early diagnosis, treatment planning and disease monitoring of diseases. For example, early monitoring of colorectal cancer depends on the accurate segmentation of colorectal polyps. Accurate medical image segmentation can provide clinicians with complete disease information, allowing doctors to accurately detect and locate the disease in the early stages, so as to intervene in treatment as early as possible. Therefore, medical image segmentation is crucial for disease diagnosis and subsequent treatment plan decision-making.

[0003] Early medical image segmentation mainly relied on traditional methods such as edge detection, template matching, statistical shape models, and active contours. These methods are relatively simple and suitable for some simple or low-noise images. However, when the noise and contrast in the image are low and the boundaries are blurred, traditional methods often cannot accurately distinguish different tissues or lesion areas. With the development of deep learning technology, deep learning technology has shown great potential in the field of medical image segmentation, especially the application of convolutional neural networks in medical image segmentation has made breakthroughs. Compared with traditional segmentation methods, deep learning can automatically learn complex feature representations from large amounts of data, thereby improving the accuracy and robustness of segmentation.

[0004] Although convolutional neural networks represented by U-Net have achieved remarkable success in the field of medical image segmentation, encoder-decoder networks such as U-Net still have some shortcomings that limit the segmentation performance of the model. The core design concept of U-Net is to rely on local information. For tasks that need to capture large-scale context (for example, segmentation of the entire organ), the design of U-Net may not be able to efficiently process global context information. Secondly, the jump connection of U-Net effectively helps the decoder to recover the spatial information of the image by directly connecting the low-level feature map in the encoder to the corresponding layer in the decoder. However, the simple connection of the low-level feature map of the decoder with the high-level feature map of the corresponding encoder does not consider the semantic gap between the high-level and low-level feature maps. The simple feature connection method limits the role of the jump connection. Wavelet transform is a powerful tool for signal and image processing. It can decompose signals into components in different frequency ranges and retain their spatial position information. Wavelet transform decomposes signals at multiple scales through scaling functions and wavelet functions, which can capture global features and local details at the same time. The low-frequency component is the global feature, which can appropriately supplement the global feature capture capability of the U-Net network. The attention mechanism achieves selective transmission and enhancement of information through a dynamic weight allocation strategy, making the fusion between features more efficient. Similarly, the attention mechanism can also alleviate the semantic gap between features.

[0005] At present, medical images have problems such as data scarcity, low data quality, and complex structure. The application methods of deep learning in the field of medical image segmentation still need to be continuously improved. Specific improvement directions include lightweight models for more efficient application in actual clinical practice. By combining multiple tasks, the model can learn more useful information at the same time, thereby improving segmentation accuracy. Design a model structure that can be adaptively adjusted according to the input data, or use a dynamic deep learning framework (such as automatically adjusting the number of network layers, convolution kernel size, etc.) to improve performance in segmentation tasks. Combined with the optimization of algorithms and model frameworks and data enhancement, medical image segmentation technology can provide a more reliable basis for clinical diagnosis.

[0006] How to solve the above technical problems is the subject faced by the present invention. Summary of the invention

[0007] The technical problem to be solved by the present invention is how to increase the receptive field of convolutional neural network structures such as U-Net without greatly increasing parameters and calculation amount, alleviate the semantic gap between high-level and low-level features of U-Net jump connections, and achieve a more accurate polyp lesion medical image segmentation method. The method of the present invention provides an encoder-decoder segmentation method based on wavelet transform and attention mechanism.

[0008] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is specifically: a medical image segmentation method based on an encoder-decoder network of wavelet transform and attention mechanism, comprising the following steps:

[0009] Step 1: Preprocess the input polyp medical images, perform data enhancement, and divide the dataset into training set, test set, and validation set in a ratio of 8:1:1;

[0010] Step 2: construct an encoder-decoder network based on wavelet transform and attention mechanism, wherein the encoder-decoder network based on wavelet transform and attention mechanism uses modules based on wavelet transform and convolution to extract global and local features, and then alleviates the semantic gap between encoder-decoder features of the same level through modules based on attention mechanism, allowing the network to better perform feature fusion of encoders and decoders of the same level;

[0011] Step 3: Input the preprocessed polyp medical image dataset into the encoder-decoder network based on wavelet transform and attention mechanism for training to obtain the optimal model;

[0012] Step 4: After the training is completed, the polyp medical image verification set is input into the obtained optimal model to detect the segmentation results of the polyp lesions in the polyp medical images.

[0013] The specific process in step 1 is as follows:

[0014] Step 1.1: Divide the polyp medical image dataset into training set, test set and validation set in a ratio of 8:1:1.

[0015] Step 1.2: Use cv2 to load the images and masks in the training and test sets of polyp medical images, and resize the images and masks to 256×256.

[0016] Step 1.3: The mask is grayscaled before conversion to a PIL image, and then the pixel values ​​are normalized to [0,1] by dividing by 255. The grayscale and normalization formulas are as follows:

[0017]

[0018] Among them, R, G, B are the values ​​of the red, green, and blue channels of the image respectively, and Out is the grayscale and normalized image.

[0019] Step 1.4: Convert the image from BGR format to RGB format.

[0020] Step 1.5: Perform random scaling, random horizontal flipping, random vertical flipping, random cropping, and standardization on the training set data to perform data enhancement and preprocessing. This data enhancement process not only improves the robustness and generalization ability of the model by introducing diversity and uncertainty, but also increases the attention to target details, making the model more adaptable to slight changes in data distribution. Mean = (0.402, 0.402, 0.402), Std = (0.298, 0.298, 0.298), the standardization formula is as follows:

[0021]

[0022] Among them, Input is the input image, Mean is the mean, Std is the standard deviation, and Normalized is the image after normalization.

[0023] The specific process in step 2 is as follows:

[0024] Step 2.1: Use wavelet transform and convolution as a dual-branch structure to reconstruct the convolution layer of U-Net. The wavelet branch extracts global features, and the convolution branches independently complete their own feature extraction processes. The process is formulated as follows:

[0025]

[0026] Among them, F is the input image of each convolution block, DTWBranch is the wavelet transform branch, ConvBranch is the standard convolution branch, and Y i is the output feature map of each branch;

[0027] Step 2.2: Convolution branch;

[0028] Use 7×7 depth-separable convolution to extract spatial features, then normalize the convolutional features, use 1×1 convolution to increase the feature dimension, then use GELU activation function, and then use 1×1 point-by-point convolution to compress the channel dimension, then normalize the point-by-point convolution features, then make residual connection between the original input and the normalized features, and finally perform GELU on the residual connection features. The process is formulated as follows:

[0029] F 1，1 =DWConv 7×7 (F)

[0030] F 1，2 =BN(F 1，1 )

[0031] F 1，3 =Conv 1×1 (F 1，2)

[0032] F 1，4 =GELU(F 1，3 )

[0033] F 1，5 =PWConv 1×1 (F 1，4 )

[0034] F 1，6 =BN(F 1，4 )

[0035] F 1，7 =F+F 1，6

[0036] F 1，8 =GELU(F 1，7 )

[0037] Among them, DWConv 7×7 is a 7×7 depth separable convolution, BN is normalized, Conv 1×1 is a 1×1 standard convolution, GELU is the activation function, PWConv 1×1 is the point convolution, F 1，1 、F 1，2 、F 1，3 、F 1，4 、F 1，5 、F 1，6 、F 1，7 , respectively, the feature maps after the corresponding operations, F 1.8 is the output feature map of the convolution branch;

[0038] Step 2.3: wavelet branching;

[0039] The image is processed using a two-dimensional wavelet transform to obtain a low-frequency sub-image and three high-frequency sub-images. The low-frequency sub-image mainly represents the global information of the image, that is, the main shape, outline and background in the image, while removing the details and high-frequency information. The three high-frequency sub-images provide detailed information of the image, including edges, textures, and local changes, which are used to describe the local features of the image. Only the low-frequency sub-image, that is, the global features, is retained in the wavelet branch. The wavelet transform branch formula is as follows:

[0040]

[0041] Among them, h and g are filters of Haar wavelet decomposition, h is a low-pass filter, and g is a high-pass filter;

[0042]

[0043] Among them, f(x, y) is the input image, x and y represent the horizontal and vertical positions of the image respectively, and fL (x,y) and f H (x, y) represent the low-frequency and high-frequency components in the horizontal direction, respectively, and f LL (x, y) indicates that both the horizontal and vertical directions are low-frequency components, which is also the final output feature map of the wavelet branch. LH (x, y) represents the horizontal low frequency and vertical high frequency components, that is, the horizontal edge information, f HL (x, y) represents horizontal high frequency and vertical low frequency components, i.e. vertical edge information, f HH (x, y) indicates that both the horizontal and vertical directions are high-frequency components, that is, diagonal detail information, k is the index of the filter, indicating the position of the filter weight in the sliding window;

[0044] Step 2.4: Merge branches;

[0045] Before the features of the convolution branch and the wavelet branch are fused, the output f of the wavelet branch is first LL The output F of the convolution branch 1，8 Align on the channel, and then integrate the two by element-by-element addition to integrate the local features of the convolution branch with the global information of the wavelet branch. The process is formulated as follows:

[0046] f ll =ChannelAdjust(f LL , F 1，8 )

[0047] F fused =F 1，8 +F ll

[0048] Among them, ChannelAdjust is the channel alignment operation, F fused The feature image is the fusion output of the wavelet branch and the convolution branch;

[0049] Step 2.5: Use the module built based on the attention mechanism to fused the features F in step 2.4 fused The decoder feature D of the corresponding stage extracts the spatial and channel attention weight coefficients, and then the weight coefficients are successively applied to F fused The weighted two feature maps are obtained, and finally the two feature maps are combined with F fused Integration, in order to alleviate the semantic gap between the two, the process is formulated as follows:

[0050] D concat =Concat(AvgPool(D),MaxPool(D))

[0051] Att spatial =σ(Conv 1×1 (Dconcat ))

[0052]

[0053] z max =max i，j D(:,i,j)

[0054] Att channel =σ(MLP(z avg +z max ))

[0055] F spatial =F fused ⊙Att spatial

[0056] F channel =F fused ⊙Att channel

[0057] F out =F fused +F spatial +F channel

[0058] Where D is F fused The corresponding feature map of the decoder at the same level, AvgPool and MaxPool are average pooling and maximum pooling respectively, σ is the activation function sigmoid, Att spatial is the spatial attention weight of D, H and W are the height and width of D, Concat is concatenation along the channel dimension, z avg and z max They are the channel features obtained after D global average pooling and D global maximum pooling, MLP is a multi-layer perceptron, Att channel is the channel attention weight of D, ⊙ is the element-by-element multiplication, F spatial and F channel D and Att spatial and Att channel The weighted feature map after weighting, F out It is the feature map after weighted feature enhancement.

[0059] The specific process in step 3 is as follows:

[0060] Step 3.1: Use the data loader to load data in batches and use GPU to accelerate data loading;

[0061] Step 3.2: Choose an appropriate loss function: a mixture of cross entropy and Dice coefficient;

[0062] Step 3.3: Select the encoder-decoder network based on wavelet transform and attention mechanism designed by the method of the present invention as the training model, set the training epoch to 400, the batch size to 4, the initial learning rate to 0.01, and use the Adam optimizer;

[0063] Step 3.4: Use Dice, mIoU, Accuracy, Recall, and Precision as evaluation indicators of the model to evaluate the training and test results. The function expressions of each evaluation indicator are as follows:

[0064]

[0065] Among them, TP, TN, FP, and FN represent the true positive, true negative, false positive, and false negative classes respectively. Dice represents the degree of overlap between the predicted result and the true result. mIoU represents the overlap ratio between the predicted area and the true area. Accuracy represents the overall accuracy of the model prediction. Recall represents the recall rate, which measures how many pixels in the real target area are correctly predicted as positive. Precision represents the precision rate, which measures how many pixels predicted as positive are truly positive.

[0066] Step 3.5: Use the encoder-decoder network based on wavelet transform and attention mechanism to train on the training set and save the optimal model parameters;

[0067] Step 4: After the model training is completed, the optimal model in the training is used to verify the polyp medical image verification set to detect the segmentation effect of the polyp lesions in the polyp medical images.

[0068] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0069] 1. The present invention combines the characteristics and advantages of different medical image segmentation methods and designs a relatively lightweight fully automatic medical image segmentation network, which improves the performance of the model in segmenting lesions without increasing the number of parameters and calculations.

[0070] 2. The single-path convolution layer of the U-Net convolution layer is reconstructed using wavelet transform and convolution as two branches. The convolution path and wavelet transform path capture the local features and global features of the image respectively, and finally integrate the features. In the convolution path, a larger-sized depthwise separable convolution is used to replace the two small-sized convolution kernels of U-Net, and the depthwise separable convolution is used to improve the computational efficiency of the model while increasing the receptive field. The depthwise separable convolution itself improves efficiency by reducing parameters and computation, but it may still not be able to fully extract diverse features. Therefore, the introduction of the inverted bottleneck structure can control the changes in the feature map channels, allowing the network to learn complex features more flexibly, thereby further improving performance. The low-frequency part obtained by wavelet transform mainly represents the low-frequency information of the image, that is, the global structure of the image. The feature map obtained by the wavelet transform path is used as a supplement to the global features of the convolution path to improve the model's ability to capture global features.

[0071] 3. The method of the present invention proposes a SACA module based on channel attention and spatial attention. The module uses the channel attention mechanism and the spatial attention mechanism to extract the weight coefficients of the encoder features in the channel and space dimensions respectively, and then weights the corresponding encoder features with the two weight coefficients in the space and channel dimensions, and fuses the weighted encoder features with the original encoder features to alleviate the semantic gap problem between the high and low features of the encoder and decoder in the jump connection. The SACA module improves the semantic consistency of high and low-level features in the jump connection, helps to more effectively fuse these features, and improves the overall performance of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0073] Figure 1 The figure is a schematic diagram of the overall process of an embodiment of the method of the present invention.

[0074] Figure 2 Schematic diagram of the encoder-decoder network structure based on wavelet transform and attention mechanism in the method described in the embodiment of the method of the present invention.

[0075] Figure 3 Schematic diagram of the convolutional block structure of the encoder-decoder network in the embodiment of the method of the present invention.

[0076] Figure 4 Schematic diagram of the structure of the SACA module used in the method embodiment of the present invention.

[0077] Figure 5 Schematic diagram of the structure of the spatial attention module in the SACA module in the embodiment of the method of the present invention.

[0078] Figure 6 Schematic diagram of the structure of the channel attention module in the SACA module in the embodiment of the method of the present invention. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0080] Example 1

[0081] Figure 1 Schematic diagram of the overall process of the method of this embodiment. First, the polyp medical image dataset is divided into a training set, a test set, and a validation set in a ratio of 8:1:1; then, the images in the training set are preprocessed and data enhanced. Next, the training parameters of the encoder-decoder network based on wavelet transform and attention mechanism are set. Then, the model is trained on the preprocessed and data enhanced training set, and the optimal model parameters are saved. After the training is completed, the optimal model parameters are loaded, and the polyp lesion segmentation is performed on the validation set to test the segmentation performance of the model.

[0082] The specific implementation steps of the above method are as follows:

[0083] Step 1.1: Divide the polyp medical image dataset into training set, test set and validation set in a ratio of 8:1:1.

[0084] Step 1.2: Use cv2 to load the images and masks in the training and test sets of polyp medical images, and resize the images and masks to 256×256.

[0085] Step 1.3: The mask is grayscaled before conversion to a PIL image, and then the pixel values ​​are normalized to [0,1] by dividing by 255. The grayscale and normalization formulas are as follows:

[0086]

[0087] Among them, R, G, B are the values ​​of the red, green, and blue channels of the image respectively, and Out is the grayscale and normalized image.

[0088] Step 1.4: Convert the image from BGR format to RGB format.

[0089] Step 1.5: Perform random scaling, random horizontal flipping, random vertical flipping, random cropping, and standardization on the training set data to perform data enhancement and preprocessing. This data enhancement process not only improves the robustness and generalization ability of the model by introducing diversity and uncertainty, but also increases the attention to target details, making the model more adaptable to slight changes in data distribution. Mean = (0.402, 0.402, 0.402), Std = (0.298, 0.298, 0.298), the standardization formula is as follows:

[0090]

[0091] Among them, Input is the input image, Mean is the mean, Std is the standard deviation, and Normalized is the image after normalization.

[0092] Step 2.1: Use wavelet transform and convolution as a dual-branch structure to reconstruct the convolution layer of U-Net. The wavelet branch extracts global features, and the convolution branches independently complete their own feature extraction processes. The process is formulated as follows:

[0093]

[0094] Among them, F is the input image of each convolution block, DTWBranch is the wavelet transform branch, ConvBranch is the standard convolution branch, and Y i is the output feature map of each branch;

[0095] Step 2.2: Convolution branch;

[0096] Use 7×7 depth-separable convolution to extract spatial features, then use the convolution features to normalize, and then use 1×1 convolution to increase the feature dimension, and then use the GELU activation function, and then use 1×1 point-by-point convolution to compress the channel dimension, and then normalize the point-by-point convolution features, and then make residual connections between the original input and the normalized features, and finally perform GELU on the residual connection features. The process is formulated as follows:

[0097] F 1，1 =DWConv 7×7 (F)

[0098] F 1，2 =BN(F 1，1 )

[0099] F 1，3 =Conv 1×1 (F 1，2 )

[0100] F1，4 =GELU(F 1，3 )

[0101] F 1，5 =PWConv 1×1 (F 1，4 )

[0102] F 1，6 =BN(F 1，4 )

[0103] F 1，7 =F+F 1，6

[0104] F 1，8 =GELU(F 1,7 )

[0105] Among them, DWConv 7×7 is a 7×7 depth separable convolution, BN is normalized, Conv 1×1 is a 1×1 standard convolution, GELU is the activation function, PWConv 1×1 is the point convolution, F 1，1 、F 1，2 、F 1，3 、F 1，4 、F 1，5 、F 1，6 、F 1，7 , respectively, the feature maps after the corresponding operations, F 1.8 is the output feature map of the convolution branch;

[0106] Step 2.3: wavelet branching;

[0107] The image is processed using a two-dimensional wavelet transform to obtain a low-frequency sub-image and three high-frequency sub-images. The low-frequency sub-image mainly represents the global information of the image, that is, the main shape, outline and background in the image, while the details and high-frequency information are removed. The three high-frequency sub-images provide detailed information of the image, including edges, textures, local changes, etc., which are mainly used to describe the local features of the image. Only the low-frequency sub-image, that is, the global features, is retained in the wavelet branch. The wavelet transform branch formula is as follows:

[0108]

[0109] Among them, h and g are filters of Haar wavelet decomposition, h is a low-pass filter, and g is a high-pass filter;

[0110]

[0111] Where f(x, y) is the input image, x and y represent the horizontal and vertical positions of the image, respectively, and fL(x, y) and f H(x, y) represent the low-frequency and high-frequency components in the horizontal direction, respectively, and f LL (x, y) indicates that both the horizontal and vertical directions are low-frequency components, which is also the final output feature map of the wavelet branch. LH (x, y) represents the horizontal low frequency and vertical high frequency components, that is, the horizontal edge information, f HL (x, y) represents horizontal high frequency and vertical low frequency components, i.e. vertical edge information, f HH (x, y) indicates that both the horizontal and vertical directions are high-frequency components, that is, diagonal detail information, k is the index of the filter, indicating the position of the filter weight in the sliding window;

[0112] Step 2.4: Merge branches;

[0113] Before the features of the convolution branch and the wavelet branch are fused, the output f of the wavelet branch is first LL The output F of the convolution branch 1，8 Align on the channel, and then integrate the two by element-by-element addition to integrate the local features of the convolution branch with the global information of the wavelet branch. The process is formulated as follows:

[0114] F ll =ChannelAdjust(f LL , F 1，8 )

[0115] F fused =F 1，8 +F ll

[0116] Among them, ChannelAdjust is the channel alignment operation, F fused The feature image is the fusion output of the wavelet branch and the convolution branch;

[0117] Step 2.5: Use the module built based on the attention mechanism to fused the features F in step 2.4 fused The decoder feature D of the corresponding stage extracts the spatial and channel attention weight coefficients, and then the weight coefficients are successively applied to F fused The weighted two feature maps are obtained, and finally the two feature maps are combined with F fused Integration, in order to alleviate the semantic gap between the two, the process is formulated as follows:

[0118] D concat =Concat(AvgPool(D),MaxPool(D))

[0119] Att spatial =σ(Conv 1×1 (D concat ))

[0120]

[0121] z max =max i，j D(:,i,j)

[0122] Att channel =σ(MLP(z avg +z max ))

[0123] F spatial =F fused ⊙Att spatial

[0124] F channel =F fused ⊙Att channel

[0125] F out =F fused +F spatial +F channel

[0126] Where D is F fused The corresponding feature map of the decoder at the same level, AvgPool and MaxPool are average pooling and maximum pooling respectively, σ is the activation function sigmoid, Att spatial is the spatial attention weight of D, H and W are the height and width of D, Concat is concatenation along the channel dimension, z avg and z max They are the channel features obtained after D global average pooling and D global maximum pooling, MLP is a multi-layer perceptron, Att channel is the channel attention weight of D, ⊙ is the element-by-element multiplication, F spatial and F channel D and Att spatial and Att channel The weighted feature map after weighting, F out It is the feature map after weighted feature enhancement.

[0127] Step 3.1: Use the data loader to load data in batches and use GPU to accelerate data loading and improve training efficiency;

[0128] Step 3.2: Choose an appropriate loss function: a mixture of cross entropy and Dice coefficient.

[0129] Step 3.3: Select the encoder-decoder network based on wavelet transform and attention mechanism designed by the method of this embodiment as the training model, set the training epoch to 400, the batch size to 4, the initial learning rate to 0.01, and use the Adam optimizer;

[0130] Step 3.4: Use Dice, mIoU, Accuracy, Recall, and Precision as evaluation indicators of the model to evaluate the training and test results. The function expressions of each evaluation indicator are as follows:

[0131]

[0132] Among them, TP, TN, FP, and FN represent the true positive, true negative, false positive, and false negative classes respectively. Dice represents the degree of overlap between the predicted result and the true result. mIoU represents the overlap ratio between the predicted area and the true area. Accuracy represents the overall accuracy of the model prediction. Recall represents the recall rate, which measures how many pixels in the real target area are correctly predicted as positive. Precision represents the precision rate, which measures how many pixels predicted as positive are truly positive.

[0133] Step 3.5: Use the encoder-decoder network based on wavelet transform and attention mechanism to train on the training set and save the optimal model parameters;

[0134] Step 4: After the model training is completed, the optimal model in the training is used to verify the polyp medical image verification set to detect the segmentation effect of the polyp lesions in the polyp medical images.

[0135] The method of this embodiment is compared with the current mainstream polyp lesion segmentation models (U-Net, DCSAU-Net, ConvUNeXt, AMSUnet) on the CVC-ClinicDB dataset, and the results are shown in Table 1. The evaluation indicators in Example 1 are Dice similarity coefficient, mean intersection over union (mIoU), accuracy, recall, and precision.

[0136] Table 1 Performance comparison of segmentation models on the CVC-ClinicDB dataset

[0137] Model Dice QUR Accuracy Recall Precision U-Net 0.8562 0.8409 0.9821 0.8751 0.8803 DCSAU-Net 0.8757 0.8711 0.9862 0.9035 0.8984 ConvUNeXt 0.8808 0.8775 0.9851 0.9066 0.9331 AMSUnet 0.8957 0.8823 0.9860 0.9253 0.9258 The method of this embodiment 0.9198 0.9083 0.9895 0.9422 0.9325

[0138] According to Table 1, the experimental data are compared in detail.

[0139] The Dice of the method in this embodiment is 0.9198, which is 6.36% higher than that of U-Net, 4.41% higher than that of DCSAU-Net, 3.90% higher than that of ConvUNeXt, and 2.41% higher than that of AMSUnet.

[0140] The mIoU of the method in this embodiment is 0.9083, which is 6.74% higher than that of UNet, 3.72% higher than that of DCSAU-Net, 3.31% higher than that of ConvUNeXt, and 2.60% higher than that of AMSUnet.

[0141] The accuracy of the method in this embodiment is 0.9895, which is 0.74% higher than that of U-Net, 0.33% higher than that of DCSAU-Net, 0.44% higher than that of ConvUNeXt, and 0.35% higher than that of AMSUnet.

[0142] The Recall of the method in this embodiment is 0.9422, which is 6.71% higher than that of U-Net, 3.87% higher than that of DCSAU-Net, 3.56% higher than that of ConvUNeXt, and 1.69% higher than that of AMSUnet.

[0143] The Precision of the method in this embodiment is 0.9325, which is 5.22% higher than that of U-Net, 3.41% higher than that of DCSAU-Net, 0.06% lower than that of ConvUNeXt, and 0.67% higher than that of AMSUnet.

[0144] In general, the method of this embodiment achieves significant performance improvement in the polyp image segmentation task through effective improvement strategies, and has obvious improvements in Dice, Precision, Recall, and mIoU indicators. The Precision is only 0.06% lower than ConvUNeXt but still better than most models. According to the results of the comparative experiments, it is concluded that when the method of this embodiment performs polyp segmentation tasks on the CVC-ClinicDB dataset, the accuracy of segmentation and the similarity and overlap between the predicted polyp lesion area and the actual polyp lesion area are better than other methods in the comparative experiment. This proves the effectiveness and superiority of the method of this embodiment in performing polyp lesion segmentation on the CVC-ClinicDB dataset.

[0145] Example 2

[0146] This embodiment is compared with the traditional UNet, and the improved network HCTNet and MSDANet models. The experimental data are shown in Table 2, which further verifies the effectiveness of the method of this embodiment in polyp lesion segmentation. Experimental Example 3 uses the CVC-ColonDB polyp medical image dataset, and the evaluation indicators include Dice similarity coefficient, mean intersection over union (mIoU), accuracy, recall, and precision.

[0147] Table 2 Performance comparison of different segmentation models on CVC-ColonDB dataset (Example 2)

[0148] Model Dice QUR Accuracy Recall Precision U-Net 0.8105 0.8002 0.9653 0.8291 0.8772 HCTNet 0.8267 0.8036 0.9788 0.8573 0.8998 MSDANet 0.8212 0.8098 0.9796 0.8553 0.9031 The method of this embodiment 0.8653 0.8495 0.9839 0.8622 0.9245

[0149] In comparison with the three models of UNet, HCTNet, and MSDANet, the method of this embodiment shows significant advantages in all five evaluation indicators.

[0150] Specifically, the Dice of the method in this embodiment is 0.8653, which is 5.48% higher than that of U-Net, 4.41% higher than that of MSDANet, and 3.86% higher than that of HCTNet.

[0151] The mIoU of the method in this embodiment is 0.8495, which is 4.93% higher than that of U-Net, 4.59% higher than that of HCTNet, and 3.97% higher than that of MSDANet.

[0152] The accuracy of the method in this embodiment is 0.9839, which is 1.86% higher than that of UNet, 0.51% higher than that of HCTNet, and 0.43% higher than that of MSDANet.

[0153] The Recall of the method in this embodiment is 0.8622, which is 3.31% higher than that of U-Net, 0.51% higher than that of HCTNet, and 0.43% higher than that of MSDANet.

[0154] The Precision of the method in this embodiment is 0.9245, which is 4.73% higher than that of UNet, 2.46% higher than that of HCTNet, and 2.14% higher than that of MSDANet.

[0155] Through the comparison of evaluation indicators on the CVC-ColonDB dataset, it can be seen that the method of this embodiment is superior to UNet, HCTNet, and MSDANet in terms of Dice, mIoU, Accuracy, Recall, and Precision indicators. This shows that this method has higher comprehensive performance in segmentation tasks, can more accurately identify target areas, and balances accuracy and recall, demonstrating its adaptability and robustness in complex scenarios. In addition, this also verifies the effectiveness and superiority of the method of this embodiment in improving segmentation effects.

[0156] Example 3

[0157] This example further compares the segmentation performance of the method of this example with other popular segmentation network models in the polyp lesion segmentation task on the Kvasir-SEG dataset. The comparison models include three models: U-Net, AMSUnet, and DCSAU-Net. The purpose is to verify the superiority of the method of this example in the polyp lesion segmentation task. The evaluation indicators are: Dice similarity coefficient, mean intersection over union (mIoU), and accuracy.

[0158] Table 3 Performance comparison of different segmentation models on Kvasir-SEG dataset (Example 4)

[0159] Segmentation Model Dice QUR Accuracy U-Net 0.8296 0.8279 0.9145 AMSUnet 0.8511 0.8525 0.9252 DCSAU-Net 0.8658 0.8642 0.9316 The method of this embodiment 0.8977 0.8893 0.9572

[0160] In comparison with models such as UNet, AMSUnet, and DCSAU-Net, the method in this embodiment shows obvious advantages. Specifically,

[0161] The Dice of the method in this embodiment is 0.8977, which is 5.81% higher than U-Net, 3.66% higher than AMSUnet, and 3.19% higher than DCSAU-Net. This shows that the overlap between the polyp lesion area segmented by the method in this embodiment and the real polyp lesion area is the highest, that is, the segmentation performance is relatively good.

[0162] The mIoU of the method in this embodiment is 0.8802, which is 6.14% higher than UNet, 3.68% higher than AMSUnet, and 2.51% higher than DCSAU-Net, indicating that the segmentation accuracy of this method is relatively good.

[0163] The accuracy of the method in this embodiment is 0.9572, which is 5.27% higher than UNet, 3.20% higher than AMSUnet, and 2.56% higher than DCSAU-Net. This proves that the method in this embodiment has a significant advantage in overall prediction accuracy.

[0164] The comparison results of the evaluation indicators on the Kvasir-SEG dataset show that the method of this embodiment is superior to the UNet, AMSUnet, and DCSAU-Net models in three indicators, namely Dice, mIoU, and Accuracy. This shows that the method has significant advantages in segmentation performance, can more accurately extract the target area, and effectively reduce background interference, thereby improving the accuracy and robustness of segmentation. This also verifies the superiority of the method under complex backgrounds and diverse targets.

[0165] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A medical image segmentation method based on wavelet transform and attention mechanism, characterized in that: The following steps are involved: Step 1: Preprocess the input polyp medical images, perform data enhancement, and divide the dataset into training set, test set, and validation set in a ratio of 8:1:1; Step 2: construct an encoder-decoder network based on wavelet transform and attention mechanism, wherein the encoder-decoder network based on wavelet transform and attention mechanism uses modules based on wavelet transform and convolution to extract global and local features, and then alleviates the semantic gap between encoder-decoder features of the same level through modules based on attention mechanism, allowing the network to perform feature fusion of encoders and decoders of the same level; Step 3: Use the preprocessed and data-enhanced polyp medical image training set to train in the encoder-decoder network based on wavelet transform and attention mechanism, and retain the optimal model; Step 4: After the model training is completed, load the optimal model retained from the training to test the segmentation performance of the model on the polyp medical image validation set.

2. According to claim 1, a medical image segmentation method based on wavelet transform and attention mechanism is characterized in that: The step 1 comprises the following steps: Step 1.1: Divide the polyp medical image dataset into training set, test set and validation set in a ratio of 8:1:1; Step 1.2: Use cv2 to load the images and masks in the training and test sets of polyp medical images, and resize the images and masks to 256×256; Step 1.3: The mask is grayscaled before conversion to a PIL image, and then the pixel values ​​are normalized to [0,1] by dividing by 255. The grayscale and normalization formulas are as follows: Among them, R, G, B are the values ​​of the red, green, and blue channels of the image respectively, and Out is the grayscale and normalized image; Step 1.4: Convert the image from BGR format to RGB format; Step 1.5: Perform random scaling, random horizontal flipping, random vertical flipping, random cropping, and standardization operations on the training set data to perform data enhancement and preprocessing. Mean = (0.402, 0.402, 0.402), Std = (0.298, 0.298, 0.298), and the standardization formula is as follows: Among them, Input is the input image, Mean is the mean, Std is the standard deviation, and Normalized is the image after normalization.

3. The medical image segmentation method based on wavelet transform and attention mechanism according to claim 1, characterized in that: The step 2 comprises the following steps: Step 2.1: Use wavelet transform and convolution as a dual-branch structure to reconstruct the convolution layer of U-Net. The wavelet branch extracts global features, and the convolution branches independently complete their own feature extraction processes. The process is formulated as follows: Among them, F is the input image of each convolution block, DTWBranch is the wavelet transform branch, ConvBranch is the standard convolution branch, and Y i is the output feature map of each branch; Step 2.2: Convolution branch; Use 7×7 depth-separable convolution to extract spatial features, then normalize the convolutional features, use 1×1 convolution to increase the feature dimension, then use GELU activation function, and then use 1×1 point-by-point convolution to compress the channel dimension, then normalize the point-by-point convolution features, then make residual connection between the original input and the normalized features, and finally perform GELU on the residual connection features. The process is formulated as follows: F 1，1 =DWConv 7×7 (F) F 1，2 =BN(F 1，1 ) F 1，3 =Conv 1×1 (F 1，2 ) F 1，4 =GEL(F 1，3 ) F 1，5 =PWConv 1×1 (F 1，4 ) F 1，6 =BN(F 1，4 ) F 1，7 =F+F 1，6 F 1，8 =GEL(F 1，7 ) Among them, DWConv 7×7 is a 7×7 depth separable convolution, BN is normalized, Conv 1×1 is a 1×1 standard convolution, GELU is the activation function, PWConv 1×1 is the point convolution, F 1，1 、F 1，2 、F 1，3 、F 1，4 、F 1，5 、F 1，6 、F 1，7 , respectively, the feature maps after the corresponding operations, F 1.8 is the output feature map of the convolution branch; Step 2.3: wavelet branching; The image is processed using a two-dimensional wavelet transform to obtain a low-frequency sub-image and three high-frequency sub-images. The low-frequency sub-image mainly represents the global information of the image, that is, the main shape, outline and background in the image, while removing the details and high-frequency information. The three high-frequency sub-images provide detailed information of the image, including edges, textures, and local changes, which are used to describe the local features of the image. Only the low-frequency sub-image, that is, the global features, is retained in the wavelet branch. The wavelet transform branch formula is as follows: Among them, h and g are filters of Haar wavelet decomposition, h is a low-pass filter, and g is a high-pass filter; Among them, f(x, y) is the input image, x and y represent the horizontal and vertical positions of the image respectively, and f L (x, y) and f H (x, y) represent the low-frequency and high-frequency components in the horizontal direction, respectively, and f LL (x, y) indicates that both the horizontal and vertical directions are low-frequency components, which is also the final output feature map of the wavelet branch. LH (x, y) represents the horizontal low frequency and vertical high frequency components, that is, the horizontal edge information, f HL (x, y) represents horizontal high frequency and vertical low frequency components, i.e. vertical edge information, f HH (x, y) indicates that both horizontal and vertical directions are high-frequency components, i.e., diagonal detail information. k is the index of the filter, indicating the position of the filter weight in the sliding window. Step 2.4: Merge branches; Before the features of the convolution branch and the wavelet branch are fused, the output f of the wavelet branch is first LL The output F of the convolution branch 1，8 Align on the channel, and then integrate the two by element-by-element addition to integrate the local features of the convolution branch with the global information of the wavelet branch. The process is formulated as follows: F ll =ChannelAdjust(f LL ,F 1，8 ) F fused =F 1，8 +F ll Among them, ChannelAdjust is the channel alignment operation, F fused The feature image is the fusion output of the wavelet branch and the convolution branch; Step 2.5: Use the module built based on the attention mechanism to fused the features F in step 2.4 fused The decoder feature D of the corresponding stage extracts the spatial and channel attention weight coefficients, and then the weight coefficients are applied to F in turn fused The weighted two feature maps are obtained, and finally the two feature maps are combined with F fused Integration, in order to alleviate the semantic gap between the two, the process is formulated as follows: D concat =Concat(AvgPool(D),MaxPool(D)) To spatial =σ(Conv 1×1 (D concat )) z max =max i，j D(:,i,j) Att channel =σ(MLP(z avg +with max )) F spatial =F fused ⊙Att spatial F channel =F fused ⊙Att channel F out =F fused +F spatial +F channel Where D is F fused The corresponding feature map of the decoder at the same level, AvgPool and MaxPool are average pooling and maximum pooling respectively, σ is the activation function sigmoid, Att spatial is the spatial attention weight of D, H and W are the height and width of D, Concat is concatenation along the channel dimension, z avg and z max They are the channel features obtained after D global average pooling and D global maximum pooling, MLP is a multi-layer perceptron, Att channel is the channel attention weight of D, ⊙ is the element-by-element multiplication, F spatial and F channel D and Att spatial and Att channel The weighted feature map after weighting, F out It is the feature map after weighted feature enhancement.

4. The medical image segmentation method based on wavelet transform and attention mechanism according to claim 1, characterized in that: The step 3 comprises the following steps: Step 3.1: Use the data loader to load data in batches and use GPU to accelerate data loading; Step 3.2: Choose an appropriate loss function: Cross Entropy and Dic e Coefficient mixing; Step 3.3: Select the encoder-decoder network based on wavelet transform and attention mechanism designed by the method of the present invention as the training model, set the training epoch to 400, the batch size to 4, the initial learning rate to 0.01, and use the Adam optimizer; Step 3.4: Use Dice, mIoU, Accuracy, Recall, and Precision as evaluation indicators of the model to evaluate the training and test results. The function expressions of each evaluation indicator are as follows: Among them, TP, TN, FP, and FN represent the true positive, true negative, false positive, and false negative classes respectively. Dice represents the degree of overlap between the predicted result and the true result. mIoU represents the overlap ratio between the predicted area and the true area. Accuracy represents the overall accuracy of the model prediction. Recall represents the recall rate, which measures how many pixels in the real target area are correctly predicted as positive. Precision represents the precision rate, which measures how many pixels predicted as positive are truly positive. Step 3.5: Use the encoder-decoder network based on wavelet transform and attention mechanism to train on the training set and save the optimal model parameters; Step 4: After the model training is completed, the optimal model in the training is used to verify the polyp medical image verification set to detect the segmentation effect of the polyp lesions in the polyp medical images.

Images