Mammary gland ultrasonic tumor segmentation method based on VGG and MESCA

Abstract

The invention discloses a breast ultrasonic tumor segmentation method based on VGG and MESCA, and the method comprises the following steps: S1, obtaining a breast ultrasonic tumor image, and obtaining a data set; s2, preprocessing the acquired data set; s3, constructing an improved UNet model based on the VGG; s4, substituting the preprocessed data set into the improved UNet model to train the network; s5, inputting the breast ultrasonic tumor image to be segmented into the trained improved UNet network, and outputting a segmentation result; according to the method, the Unet model is improved, local and global information in the image can be better captured, and the global feature extraction capability is enhanced.

Description

Technical Field

[0001] The invention relates to the technical field of breast tumor segmentation, in particular to a breast ultrasound tumor segmentation method based on VGG and MESCA. Background Art

[0002] At present, breast tumor segmentation can be divided into threshold-based segmentation methods, statistical learning model-based segmentation methods, graph theory-based segmentation methods, active contour-based segmentation methods, and deep learning-based segmentation methods. Compared with deep learning-based segmentation methods, the first four segmentation methods have some problems, such as limited feature extraction capabilities, sensitivity to noise and image quality, and insufficient generalization capabilities. In recent years, deep learning-based breast tumor segmentation methods have become more robust and recognizable and can automatically capture the complex features of different tumor types.

[0003] Fully convolutional neural networks (FCNs) have brought revolutionary progress to the field of image semantic segmentation and opened a new chapter in understanding images at the semantic level. Since then, many network models based on semantic segmentation have been proposed and developed. U-Net is a classic image segmentation network. It is a symmetrical U-shaped architecture that includes an encoder-decoder part. It encodes context information by gradually reducing feature maps through continuous convolution and pooling, and gradually restores the original resolution of the image through upsampling and convolution operations to retain more detail information. The VGG network model proposed by the Visual Geometry Group of the University of Oxford improves the ability to extract image features by increasing the depth of the network. Its continuously stacked convolutional layers can achieve the effect of larger convolution kernels, which can effectively capture more local information and integrate it into global features. VGG16 contains 13 convolutional layers and 3 fully connected layers. Compared with other VGG models, such as VGG19, VGG16 is a better balance in depth. It improves performance by increasing the depth of the network. At the same time, it has fewer parameters and calculations than VGG19, and is faster in training and reasoning. However, the UNet based on VGG16 is not capable of extracting features from target areas with complex and irregular boundaries, and is also limited in capturing contextual information in a wider range, which can easily lead to insufficient understanding of the global structure and texture, and lead to problems such as unclear boundary segmentation and slow prediction speed. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above and / or existing problems in the existing breast ultrasound tumor segmentation images, the present invention is proposed.

[0006] Therefore, the purpose of the present invention is to provide a breast ultrasound tumor segmentation method based on VGG and MESCA. The present invention improves the Unet model, which can better capture the local and global information in the image and enhance the ability to extract global features.

[0007] To solve the above technical problems, the present invention provides the following technical solutions: a breast ultrasound tumor segmentation method based on VGG and MESCA, comprising the following steps: S1 obtains breast ultrasound tumor images and obtains a data set; S2 preprocesses the acquired data set; S3 builds an improved UNet model based on VGG; S4 brings the preprocessed data set into the improved UNet model to train the network; S5 inputs the breast ultrasound tumor image to be segmented into the trained improved UNet network and outputs the segmentation result.

[0008] As a preferred solution of the breast ultrasound tumor segmentation method based on VGG and MESCA in the present invention, step S3 specifically includes the following steps: S3.1 uses VGG16 as the feature extraction network of UNet, corresponding to the first, second, third, fourth and fifth layers of the UNet encoder part; S3.2 uses depthwise separable convolutional modules in the encoder instead of the original ordinary convolutional modules; S3.3 Add a Batch Normalization layer after each convolution of the encoder and output it through the nonlinear activation function ReLU; S3.4 For each layer of the encoder, add a SE attention module after all depth-wise separable convolutional layer operations; The S3.5 UNet decoder uses the encoder in the original UNet and adds a MESCA module before each upsampling.

[0009] As a preferred solution of the breast ultrasound tumor segmentation method based on VGG and MESCA in the present invention, the specific steps of obtaining the data set in step S1 are: S1.1 The public breast ultrasound tumor dataset BUSI was used as the original dataset; S1.2 Randomly select m1 images from a dataset containing n images, perform Laplace sharpening on the selected images as a breast ultrasound tumor dataset, and randomly select m2 images from the dataset as a training set. 21 pictures as the validation set, m 22 pictures as the test set; Among them, the number of m1 is less than n, and m2=m 21 +m 22 .

[0010] As a preferred solution of the breast ultrasound tumor segmentation method based on VGG and MESCA in the present invention, in which: in step 3.4, the SE attention module enhances the feature expression capability through a lightweight channel attention mechanism, and the specific implementation process is as follows: First, the global information of each channel is extracted through global average pooling, and then the channel weights are generated through two layers of full connection, and finally the input features are weighted.

[0011] As a preferred solution of the breast ultrasound tumor segmentation method based on VGG and MESCA in the present invention, wherein: in step S3.5, the MESCA module is a median-enhanced spatial channel attention module, and the implementation process is as follows: the input feature is feature adjusted through a 1×1 point-by-point convolution, and then enters the channel attention module; in the channel attention module, global features are extracted respectively through global average pooling, maximum pooling and median pooling, and each feature is processed by two point-by-point convolutions and an activation function to obtain an attention map, which is then added to fuse the global characteristics of each channel; the feature map weighted by the channel attention is further extracted through a 5×5 deep convolution for preliminary spatial features; then multiple deep convolution layers of heterogeneous sizes are used to extract multi-scale spatial information, and they are fused into a spatial attention map; finally, the input features are weighted.

[0012] As a preferred solution of the breast ultrasound tumor segmentation method based on VGG and MESCA in the present invention, in which: in step S4, the training parameters of the UNet model are specifically: using the Adam optimizer, the initial learning rate is set to 0.001, the number of training iterations is set to 200, the learning rate is dynamically adjusted in an exponential decay manner according to the number of iterations, and the minimum learning rate is set to 0.0001.

[0013] As a preferred solution of the breast ultrasound tumor segmentation method based on VGG and MESCA in the present invention, wherein: in step S5, the parameter corresponding to the minimum loss value in the training cycle is selected as the final weight to segment the breast ultrasound tumor image.

[0014] As a preferred solution of the breast ultrasound tumor segmentation method based on VGG and MESCA in the present invention, the binary cross entropy function is used as the model training loss function, and the specific formula is: , where y is a binary label 0 or 1, p(y) is the probability that the output belongs to the y label, and N represents the total number of training times.

[0015] Compared with the prior art, the present invention has the following technical effects: the present invention greatly reduces the redundant parameters of the encoder part by adopting deep separable convolution, reduces the model complexity of the feature extraction network, and improves the operation efficiency. By adding a batch normalization layer, the data distribution is improved and the network convergence speed is accelerated. By adding the SE module, the global information modeling of the features is performed, and the weight of each channel is dynamically adjusted, which effectively enhances the model's attention to key features. The addition of the MESCA module in the decoder part combines convolution kernels of various sizes to extract multi-scale features, so that the model can better capture local and global information in the image, and is particularly suitable for processing images with multi-scale information, thereby improving the expression ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them: Figure 1 It is a flowchart of the present invention.

[0017] Figure 2 This is the improved UNet network model diagram of the present invention.

[0018] Figure 3 This is the schematic diagram of VGG16 used in the present invention.

[0019] Figure 4 Schematic diagram of the depthwise separable convolution used in the present invention.

[0020] Figure 5 Schematic diagram of the SE attention module used in the present invention.

[0021] Figure 6 Schematic diagram of the MESCA attention module used in the present invention.

[0022] Figure 7 This is a comparison chart of the segmentation effects of the model used in the present invention and other models.

[0023] Figure 8 A comparison chart of the network segmentation effects using the comparative method. DETAILED DESCRIPTION

[0024] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0027] Example 1 like Figure 1 to Figure 6 As shown, it is the first embodiment of the present invention. This embodiment provides a breast ultrasound tumor segmentation method based on VGG and MESCA. The use of the present invention can better capture local and global information in the image.

[0028] A breast ultrasound tumor segmentation method based on VGG and MESCA comprises the following steps: S1 obtains breast ultrasound tumor images and data sets. The specific steps are: S1.1 The public breast ultrasound tumor dataset BUSI was used as the original dataset; S1.2 Randomly select m1 images from a dataset containing n images, perform Laplace sharpening on the selected images as a breast ultrasound tumor dataset, and randomly select m2 images from the dataset as a training set. 21 pictures as the validation set, m 22 pictures as the test set; Among them, the number of m1 is less than n, and m2=m 21 +m 22 ; S2 preprocesses the acquired data set, including horizontal flipping, vertical flipping, scaling, and Laplace sharpening operations (the above operations are conventional operations of image preprocessing and are existing technologies); S3 builds an improved UNet model based on VGG, specifically, S3.1 uses VGG16 as the feature extraction network of UNet, corresponding to the first, second, third, fourth and fifth layers of the UNet encoder part; S3.2 uses depthwise separable convolutional modules in the encoder instead of the original ordinary convolutional modules; S3.3 After each convolution of the encoder, a Batch Normalization layer is added to perform normalization operations to alleviate the gradient vanishing or gradient exploding problems during training, and the output is obtained through the nonlinear activation function ReLU (which is the existing technology); S3.4 For each layer of the encoder, an SE attention module is added after all depth-wise separable convolutional layer operations. The SE attention module enhances the feature expression capability through a lightweight channel attention mechanism. The specific implementation process is to first extract the global information of each channel through global average pooling, then generate channel weights through two layers of full connection, and finally weight the input features; The decoder of S3.5 UNet uses the encoder in the original UNet, and adds a MESCA module before each upsampling. The MESCA module is a median-enhanced spatial channel attention module. The implementation process is as follows: the input features are feature adjusted through a 1×1 point-by-point convolution, and then enter the channel attention module; in the channel attention module, the average, maximum and median of the channel are calculated in the spatial dimension through global average pooling, maximum pooling and median pooling respectively, and the global features are extracted. Each extracted feature is processed by two point-by-point convolutions and activation functions (which are existing technologies) to obtain an attention map, which is then added to fuse the global characteristics of each channel; the feature map weighted by the channel attention is further extracted through a 5×5 deep convolution to extract preliminary spatial features; then multiple deep convolution layers of heterogeneous sizes are used to extract multi-scale spatial information and fuse them into a spatial attention map; finally, the input features are weighted; S4 brings the preprocessed data set into the improved UNet model to train the network. The specific training parameters are: using the Adam optimizer, the initial learning rate is set to 0.001, the number of training iterations is set to 200, the learning rate is dynamically adjusted in an exponential decay manner according to the number of iterations, the minimum learning rate is set to 0.0001, and the binary cross entropy function is used as the model training loss function Loss The specific formula is , where y is a binary label 0 or 1, p(y) is the probability that the output belongs to label y, i is the i-th sample, starting from 1, and N represents the total number of training times; S5 inputs the breast ultrasound tumor image to be segmented into the trained improved UNet network, selects the parameter corresponding to the minimum loss value in the training cycle as the final weight, segments the breast ultrasound tumor image, and outputs the segmentation result.

[0029] The present invention greatly reduces the redundant parameters of the encoder part by adopting deep separable convolution, reduces the model complexity of the feature extraction network, and improves the operation efficiency. By adding a batch normalization layer, the data distribution is improved and the network convergence speed is accelerated; by adding the SE module, the global information modeling of the features is performed, and the weight of each channel is dynamically adjusted, which effectively enhances the model's attention to key features. The addition of the MESCA module in the decoder part combines a variety of convolution kernels of different sizes to extract multi-scale features, so that the model can better capture local and global information in the image, and is particularly suitable for processing images with multi-scale information, thereby improving the model's expression ability.

[0030] Example 2 Reference Figure 1 to Figure 7 , which is the second embodiment of the present application. The difference from Embodiment 1 is that this embodiment provides a breast ultrasound tumor segmentation method based on VGG and MESCA. This embodiment uses experiments to verify the technical effects of the present application.

[0031] The hardware environment configuration used for implementation is: Windows 11 64-bit operating system, CPU is Intel(R) Core(TM) i5-12450H 2.00 GHz, GPU is GeForce GTX 4060, and 16GB running memory; the software environment configuration is: Python 3.9, deep learning framework is Pytorch2.0.0, CUDA11.8.

[0032] The present invention discloses a breast ultrasound tumor segmentation method based on VGG and MESCA, which comprises the following steps: S1 obtains a breast ultrasound tumor dataset, which uses the public breast ultrasound tumor dataset BUSI as the original dataset. The present invention uses labelme software to annotate breast ultrasound tumor images, names the background part as "_background" with a value of 0, and names the tumor area as "_busi_" with a value of 1. A json format file is generated, which is converted into a labeled grayscale image in png format through code; 600 images were selected from the 780 images in the BUSI dataset. The selected images were scaled, Laplace sharpened, and used as the original breast ultrasound tumor dataset, and randomly divided into a training set and a test set in a ratio of 8:1. S2 preprocesses the acquired data set; S3 builds an improved UNet model based on VGG. The UNet network consists of two parts, the encoder and the decoder, each of which contains five layers. The first two layers of the encoder each contain two 3×3 depth-separable convolutions, RELU activation functions and SE attention modules, and the last three layers each contain three 3×3 depth-separable convolutions, RELU activation functions and SE attention modules. Starting from the first layer, each downsampling is performed through the maximum pooling before entering the next layer. After the convolution activation in the fifth layer, upsampling is performed using bilinear interpolation. Each layer of the decoder contains two 3×3 ordinary convolutions and ReLU activation functions. The output of the previous layer is processed by the MESCA attention module and concat operation is performed with the corresponding encoder layer, and then enters the next layer through convolution and activation. The decoder finally passes through a normal convolution layer with a channel number of 2 to achieve foreground and background segmentation; the specific improvement steps are as follows, S3.1 uses VGG16 as the feature extraction network of UNet, corresponding to the first, second, third, fourth and fifth layers of the UNet encoder part, as shown in Figure 3 As shown in the figure, the first 13 convolutional layers after removing the fully connected layer of VGG16 are the encoder part of UNet; S3.2 uses a depth-wise separable convolution module in the encoder to replace the original ordinary convolution module, such as Figure 4 As shown in the figure, the specific operation is: split the standard 3×3 convolution into 3×3 depth convolution equal to the number of input channels and 1×1 point-by-point convolution equal to the number of output channels, and perform channel-by-channel and point-by-point operations respectively. Compared with the parameter amount of standard convolution 3×3×3×2, the parameter amount of depth-separable convolution is only 3×3×1×3+1×1×3×2, which greatly reduces the number of parameters and improves the computational efficiency. S3.3 Add a Batch Normalization layer after each convolution of the encoder and output it through the nonlinear activation function ReLU; S3.4 For each layer of the encoder, an SE attention module is added after all depth-separable convolutional layer operations to enhance the network's learning representation ability by adaptively adjusting the weights between channels. Its implementation process is as follows: Figure 5 As shown in the figure: First, the global information of each channel is extracted through global average pooling, then the channel weights are generated through two layers of full connection, and finally the input features are weighted; this method can dynamically adjust the contribution of the channel, highlight important features and suppress unimportant features, thereby improving the model performance; The S3.5 UNet decoder uses the deconvolution layer in the original UNet and adds a MESCA module before each upsampling. Its implementation process is as follows: Figure 6(In the figure, Shared MLP is a shared multi-layer perception mechanism; Conv2d1×1 represents point-by-point convolution; Channel-wise Attention is channel-level attention, Refined Channel Features is the improved channel features, Multi-depthconv is a multi-scale deep convolution; Channel Attention is channel attention; SpatialAttention is spatial attention; Output Feature is output feature) as shown: The input feature (Input Feature) is first adjusted by a 1×1 point-by-point convolution, and then enters the channel attention module. In the channel attention, global features are extracted by global average pooling (AvgPool), maximum pooling (MaxPool) and median pooling (MedianPool). Each feature is processed by two point-by-point convolutions and an activation function (Sigmoid) to obtain an attention map, which is then added to fuse the global characteristics of each channel. The feature map weighted by the channel attention is further extracted through a 5×5 deep convolution to extract preliminary spatial features. Subsequently, multiple deep convolution layers of heterogeneous sizes are used to extract multi-scale spatial information and fuse them into a spatial attention map. Finally, by weighting the input features, the feature enhancement of fusion channel and spatial attention is achieved, thereby improving the network's expression ability and classification performance. The channel attention of this module extracts global statistical information through global pooling operations, and the spatial attention captures spatial features of different scales through multi-scale deep convolution, providing richer feature representation and enhancing the model's feature extraction ability.

[0033] Figure 2 The improved UNet network shown in the figure has a 3-channel 256×256 pixel image as input. After each convolution layer of the encoder, the number of channels output is 64, 128, 256, and 512 respectively. The resolution is halved after each downsampling. After each convolution layer of the decoder, the number of channels output is 512, 256, 128, and 64 respectively. The resolution is doubled after each upsampling and restored to the same size as the input after decoding.

[0034] S4 The preprocessed data set is brought into the improved UNet model to train the network. The training parameters of the model are as follows: using the Adam optimizer, the initial learning rate is set to 0.001, the number of training iterations is set to 200, the learning rate is dynamically adjusted in an exponential decay manner according to the number of iterations, the minimum learning rate is set to 0.0001, and the binary cross entropy function is used as the training loss function of the model of the present invention. The specific formula is, where y is a binary label 0 or 1, p(y) is the probability that the output belongs to the y label, and N represents the total number of training times; S5 inputs the breast ultrasound tumor image to be segmented into the trained improved UNet network, selects the parameter corresponding to the minimum loss value in the training cycle as the final weight, segments the breast ultrasound tumor image, and outputs the segmentation result.

[0035] The present invention uses a confusion matrix to statistically classify the results, and the confusion matrix of the classification results is shown in the table.

[0036]

[0037] Among them, TP (True Positive) means that the pixels belonging to the tumor in the image are correctly classified; FP (False Positive) means that the non-tumor in the image is misclassified as a tumor; FN (False Negative) means that the tumor area in the image is misclassified as the background; TN (True Negative) means that the non-tumor in the image is correctly classified.

[0038] Based on the confusion matrix, this experiment uses three indicators: intersection over union (IoU), mean intersection over union (MIoU) and Dice score to evaluate network performance. The higher the score, the better the network segmentation effect and the better the model. The specific formula is as follows:

[0039] The mainstream semantic segmentation algorithms UNet, MobilenetV3-UNet, and Vgg16-UNet are selected as the comparison algorithms of the method of the present invention. Figure 7 This is a comparison chart of network segmentation effects using different algorithms. Figure 7 (a) is the original image, Figure 7 (b) is the label map corresponding to the original image. Figure 7 (c)- Figure 7 (f) The segmentation results obtained by the mainstream segmentation algorithms UNet, MobilenetV3-UNet, Vgg16-UNet and the proposed method, where the white part represents the tumor area and the black part represents the background. Figure 7 (c) It can be observed that the edge information of the tumor extracted by the original UNet network lacks details. Figure 7 (d) and Figure 7 (e) It can be seen that although MobilenetV3-UNet and Vgg16-UNet can extract the approximate contour of breast ultrasound tumors, the edge information is relatively rough and even segmentation errors occur. The method proposed in the present invention can better extract breast tumors in various situations. The following table compares the segmentation evaluation indicators of different network models on the self-made data set. Compared with other algorithms, there are significant improvements in intersection-over-union ratio, average intersection-over-union ratio and Dice score.

[0040] Comparative Example The difference from Example 2 is that this comparative example provides a breast ultrasound tumor segmentation method based on VGG and MESCA. The difference from Example 2 is step S3. This comparative example provides a total of 4 different comparison algorithms.

[0041] The difference between the first comparison algorithm and Example 1 is that: S3 builds the VGG16-UNet1 model. The UNet network consists of two parts, the encoder and the decoder. Each part consists of five layers. The first two layers of the encoder each contain two 3×3 depth-separable convolutions and RELU activation functions. The third and fourth layers each contain three 3×3 depth-separable convolutions and RELU activation functions. The fifth layer contains three 3×3 depth-separable convolutions and SE attention modules. Starting from the first layer, each downsampling is performed through the maximum pooling before entering the next layer. After the convolution activation in the fifth layer, upsampling is performed using bilinear interpolation. Each layer of the decoder contains two 3×3 ordinary convolutions and ReLU activation functions. The output of the previous layer is processed by the MESCA attention module and concat operation is performed with the corresponding encoder layer, and then convolution and activation are performed to enter the next layer. The decoder finally passes through a normal convolution layer with a channel number of 2 to achieve foreground and background segmentation; the specific steps are as follows: S3.1 uses VGG16 as the feature extraction network of UNet, corresponding to the first, second, third, fourth and fifth layers of the UNet encoder part, as shown in Figure 3 As shown in the figure, the first 13 convolutional layers after removing the fully connected layer of VGG16 are the encoder part of UNet; S3.2 uses a depth-wise separable convolution module in the encoder to replace the original ordinary convolution module, such as Figure 4 As shown in the figure, the specific operation is: split the standard 3×3 convolution into 3×3 depth convolution equal to the number of input channels and 1×1 point-by-point convolution equal to the number of output channels, and perform channel-by-channel and point-by-point operations respectively. Compared with the parameter amount of standard convolution 3×3×3×2, the parameter amount of depth-separable convolution is only 3×3×1×3+1×1×3×2, which greatly reduces the number of parameters and improves the computational efficiency. S3.3 Add a normalization layer after each convolution of the encoder and output it through the nonlinear activation function ReLU; S3.4 For the fifth layer of the encoder, the SE attention module is added after all the depth-separable convolutional layer operations to enhance the network's learning representation ability by adaptively adjusting the weights between channels. Its implementation process is as follows: Figure 5As shown in the figure: First, the global information of each channel is extracted through global average pooling, then the channel weights are generated through two layers of full connection, and finally the input features are weighted; this method can dynamically adjust the contribution of the channel, highlight important features and suppress unimportant features, thereby improving the model performance; The S3.5 UNet decoder uses the deconvolution layer in the original UNet and adds a MESCA module before each upsampling. Its implementation process is as follows: Figure 6 As shown in the figure: The input features are first adjusted by a 1×1 point-by-point convolution, and then enter the channel attention module. In the channel attention, global features are extracted by global average pooling, maximum pooling and median pooling respectively. Each feature is processed by two point-by-point convolutions and activation functions to obtain an attention map, which is then added to fuse the global characteristics of each channel. The feature map weighted by the channel attention is further extracted through a 5×5 deep convolution. Subsequently, multiple deep convolution layers of heterogeneous sizes are used to extract multi-scale spatial information and fuse them into a spatial attention map. Finally, by weighting the input features, feature enhancement of the fusion channel and spatial attention is achieved, thereby improving the network's expression ability and classification performance. The channel attention of this module extracts global statistical information through global pooling operations, and the spatial attention captures spatial features of different scales through multi-scale deep convolutions, providing richer feature representations and enhancing the model's feature extraction capabilities.

[0042] S4 The preprocessed data set is brought into the improved UNet model to train the network. The training parameters of the model are as follows: using the Adam optimizer, the initial learning rate is set to 0.001, the number of training iterations is set to 200, the learning rate is dynamically adjusted in an exponential decay manner according to the number of iterations, the minimum learning rate is set to 0.0001, and the binary cross entropy function is used as the training loss function of the model of the present invention. The specific formula is: , where y is a binary label 0 or 1, p(y) is the probability that the output belongs to the y label, and N is the total number of training times; S5 inputs the breast ultrasound tumor image to be segmented into the trained improved UNet network, selects the parameter corresponding to the minimum loss value in the training cycle as the final weight, segments the breast ultrasound tumor image, and outputs the segmentation result.

[0043] The difference between the second comparison algorithm and Example 1 is that: S3 builds the VGG16-UNet2 model. The UNet network consists of two parts, the encoder and the decoder. Each part contains five layers. The first two layers of the encoder each contain two 3×3 depth-separable convolutions, RELU activation functions and MESCA modules. The last three layers each contain three 3×3 depth-separable convolutions, RELU activation functions and MESCA modules. Starting from the first layer, each downsampling is performed through the maximum pooling before entering the next layer. After the convolution activation in the fifth layer, upsampling is performed using bilinear interpolation. Each layer of the decoder contains two 3×3 ordinary convolutions and ReLU activation functions. The output of the previous layer is processed by the MESCA attention module and concat operation is performed with the corresponding encoder layer, and then convolution and activation are performed to enter the next layer. The decoder finally passes through a normal convolution layer with a channel number of 2 to achieve foreground and background segmentation; the specific steps are as follows: S3.1 uses VGG16 as the feature extraction network of UNet, corresponding to the first, second, third, fourth and fifth layers of the UNet encoder part, as shown in Figure 3 As shown in the figure, the first 13 convolutional layers after removing the fully connected layer of VGG16 are the encoder part of UNet; S3.2 uses a depth-wise separable convolution module in the encoder to replace the original ordinary convolution module, such as Figure 4 As shown in the figure, the specific operation is: split the standard 3×3 convolution into 3×3 depth convolution equal to the number of input channels and 1×1 point-by-point convolution equal to the number of output channels, and perform channel-by-channel and point-by-point operations respectively. Compared with the parameter amount of standard convolution 3×3×3×2, the parameter amount of depth-separable convolution is only 3×3×1×3+1×1×3×2, which greatly reduces the number of parameters and improves the computational efficiency. S3.3 Add a normalization layer after each convolution of the encoder and output it through the nonlinear activation function ReLU; S3.4 For each layer of the encoder, add the MESCA attention module after all depth-separable convolutional layer operations. Its implementation process is as follows Figure 6As shown: The input features are first adjusted through a 1×1 point-by-point convolution, and then enter the channel attention module. In the channel attention, global features are extracted through global average pooling, maximum pooling and median pooling respectively. Each feature is processed by two point-by-point convolutions and activation functions to obtain an attention map, which is then added to fuse the global characteristics of each channel. The feature map weighted by the channel attention is further subjected to a 5×5 deep convolution to extract preliminary spatial features. Subsequently, multiple deep convolutional layers of heterogeneous sizes are used to extract multi-scale spatial information and fuse them into a spatial attention map. Finally, by weighting the input features, feature enhancement of the fusion channel and spatial attention is achieved, thereby improving the network's expression ability and classification performance. The channel attention of this module extracts global statistical information through global pooling operations, and the spatial attention captures spatial features of different scales through multi-scale deep convolutions, providing richer feature representations and enhancing the model's feature extraction capabilities. The S3.5 UNet decoder uses the deconvolution layer in the original UNet and adds a MESCA module before each upsampling. Its implementation process is as follows: Figure 6 As shown in the figure: The input features are first adjusted by a 1×1 point-by-point convolution, and then enter the channel attention module. In the channel attention, global features are extracted by global average pooling, maximum pooling and median pooling respectively. Each feature is processed by two point-by-point convolutions and activation functions to obtain an attention map, which is then added to fuse the global characteristics of each channel. The feature map weighted by the channel attention is further extracted through a 5×5 deep convolution. Subsequently, multiple deep convolution layers of heterogeneous sizes are used to extract multi-scale spatial information and fuse them into a spatial attention map. Finally, by weighting the input features, feature enhancement of the fusion channel and spatial attention is achieved, thereby improving the network's expression ability and classification performance. The channel attention of this module extracts global statistical information through global pooling operations, and the spatial attention captures spatial features of different scales through multi-scale deep convolutions, providing richer feature representations and enhancing the model's feature extraction capabilities.

[0044] The difference between the third comparison algorithm and embodiment 1 is that: S3 is built on the VGG16-UNet3 model. The UNet network consists of two parts, the encoder and the decoder. Each part consists of five layers. The first two layers of the encoder each contain two 3×3 depth-separable convolutions, RELU activation functions and SE attention modules. The last three layers each contain three 3×3 depth-separable convolutions, RELU activation functions and SE attention modules. Starting from the first layer, each downsampling is performed through the maximum pooling before entering the next layer. After the convolution activation in the fifth layer, upsampling is performed using bilinear interpolation. Each layer of the decoder contains two 3×3 depth-separable convolutions and ReLU activation functions. The output of the previous layer is processed by the MESCA attention module and concat operation is performed with the corresponding encoder layer, and then convolution and activation are performed to enter the next layer. The decoder finally passes through a normal convolution layer with a channel number of 2 to achieve foreground and background segmentation; the improvement steps are as follows: S3.1 uses VGG16 as the feature extraction network of UNet, corresponding to the first, second, third, fourth and fifth layers of the UNet encoder part, as shown in Figure 3 As shown in the figure, the first 13 convolutional layers after removing the fully connected layer of VGG16 are the encoder part of UNet; S3.2 uses depth-wise separable convolutional modules in the encoder and decoder instead of the original ordinary convolutional modules, such as Figure 4 As shown in the figure, the specific operation is: split the standard 3×3 convolution into 3×3 depth convolution equal to the number of input channels and 1×1 point-by-point convolution equal to the number of output channels, and perform channel-by-channel and point-by-point operations respectively. Compared with the parameter amount of standard convolution 3×3×3×2, the parameter amount of depth-separable convolution is only 3×3×1×3+1×1×3×2, which greatly reduces the number of parameters and improves the computational efficiency. S3.3 Add a normalization layer after each convolution of the encoder and output it through the nonlinear activation function ReLU; S3.4 For each layer of the encoder, an SE attention module is added after all depth-separable convolutional layer operations to enhance the network's learning representation ability by adaptively adjusting the weights between channels. Its implementation process is as follows: Figure 5 As shown in the figure: First, the global information of each channel is extracted through global average pooling, then the channel weights are generated through two layers of full connection, and finally the input features are weighted; this method can dynamically adjust the contribution of the channel, highlight important features and suppress unimportant features, thereby improving the model performance; The S3.5 UNet decoder uses the deconvolution layer in the original UNet and adds a MESCA module before each upsampling. Its implementation process is as follows: Figure 6As shown in the figure: The input features are first adjusted by a 1×1 point-by-point convolution, and then enter the channel attention module. In the channel attention, global features are extracted by global average pooling, maximum pooling and median pooling respectively. Each feature is processed by two point-by-point convolutions and activation functions to obtain an attention map, which is then added to fuse the global characteristics of each channel. The feature map weighted by the channel attention is further extracted through a 5×5 deep convolution. Subsequently, multiple deep convolution layers of heterogeneous sizes are used to extract multi-scale spatial information and fuse them into a spatial attention map. Finally, by weighting the input features, feature enhancement of the fusion channel and spatial attention is achieved, thereby improving the network's expression ability and classification performance. The channel attention of this module extracts global statistical information through global pooling operations, and the spatial attention captures spatial features of different scales through multi-scale deep convolutions, providing richer feature representations and enhancing the model's feature extraction capabilities.

[0045] The difference between the fourth comparison algorithm and Example 1 is that: S3 is built on the VGG16-UNet4 model. The UNet network consists of two parts, the encoder and the decoder, each of which contains five layers. The first two layers of the encoder each contain two 3×3 depth-separable convolutions, RELU activation functions and SE attention modules, and the last three layers each contain three 3×3 depth-separable convolutions, RELU activation functions and SE attention modules. Starting from the first layer, each downsampling is performed through the maximum pooling before entering the next layer. After the convolution activation in the fifth layer, upsampling is performed using bilinear interpolation. Each layer of the decoder contains two 3×3 ordinary convolutions and ReLU activation functions. The output of the previous layer is concat-operated with the encoder layer, and then processed by convolution, activation function and MESCA attention module before entering the next layer. The decoder finally passes through a normal convolution layer with a channel number of 2 to achieve foreground and background segmentation. The specific improvement steps are as follows: S3.1 uses VGG16 as the feature extraction network of UNet, corresponding to the first, second, third, fourth and fifth layers of the UNet encoder part, as shown in Figure 3 As shown in the figure, the first 13 convolutional layers after removing the fully connected layer of VGG16 are the encoder part of UNet; S3.2 uses a depth-wise separable convolution module in the encoder to replace the original ordinary convolution module, such as Figure 4 As shown in the figure, the specific operation is: split the standard 3×3 convolution into 3×3 depth convolution equal to the number of input channels and 1×1 point-by-point convolution equal to the number of output channels, and perform channel-by-channel and point-by-point operations respectively. Compared with the parameter amount of standard convolution 3×3×3×2, the parameter amount of depth-separable convolution is only 3×3×1×3+1×1×3×2, which greatly reduces the number of parameters and improves the computational efficiency. S3.3 Add a normalization layer after each convolution of the encoder and output it through the nonlinear activation function ReLU; S3.4 For each layer of the encoder, an SE attention module is added after all depth-separable convolutional layer operations to enhance the network's learning representation ability by adaptively adjusting the weights between channels. Its implementation process is as follows: Figure 5 As shown in the figure: First, the global information of each channel is extracted through global average pooling, then the channel weights are generated through two layers of full connection, and finally the input features are weighted; this method can dynamically adjust the contribution of the channel, highlight important features and suppress unimportant features, thereby improving the model performance; The S3.5 UNet decoder uses the deconvolution layer in the original UNet and adds a MESCA module after each upsampling. Its implementation process is as follows: Figure 6 As shown in the figure: The input features are first adjusted by a 1×1 point-by-point convolution, and then enter the channel attention module. In the channel attention, global features are extracted by global average pooling, maximum pooling and median pooling respectively. Each feature is processed by two point-by-point convolutions and activation functions to obtain an attention map, which is then added to fuse the global characteristics of each channel. The feature map weighted by the channel attention is further extracted through a 5×5 deep convolution. Subsequently, multiple deep convolution layers of heterogeneous sizes are used to extract multi-scale spatial information and fuse them into a spatial attention map. Finally, by weighting the input features, feature enhancement of the fusion channel and spatial attention is achieved, thereby improving the network's expression ability and classification performance. The channel attention of this module extracts global statistical information through global pooling operations, and the spatial attention captures spatial features of different scales through multi-scale deep convolutions, providing richer feature representations and enhancing the model's feature extraction capabilities.

[0046] Figure 8 This is a comparison chart of the network segmentation effect of the method of this comparative example. VGG16-UNet1, VGG16-UNet2, VGG16-UNet3 and VGG16-UNet4 are selected as the comparison algorithms of the method of the present invention. Figure 8 (a) is the original image, Figure 8 (b) is the label map corresponding to the original image. Figure 8 (c)- Figure 8 (g) The segmentation results obtained by using VGG16-UNet1, VGG16-UNet2, VGG16-UNet3, VGG16-UNet4 and the proposed method, where the white part represents the tumor area and the black part represents the background. Figure 8(c)8(d)8(f)It can be observed that VGG16-UNet1, VGG16-UNet2, and VGG16-UNet4 have weaker ability to extract edge information of tumors, and the extraction is relatively rough. Figure 8 (e) The VGG16-UNet3 network has large-scale errors in tumor region extraction and the segmentation is not accurate enough. The method proposed in the present invention can better extract breast tumors in various situations. The following table compares the segmentation evaluation indicators of different network models on the self-made data set. Compared with other algorithms, the intersection-over-union ratio, average intersection-over-union ratio and Dice score are improved.

[0047]

[0048] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.

[0049] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0050] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or part thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques described in the present invention, the present invention also includes the computer itself. The computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on a display.

[0051] As used in this application, the terms "component", "module", "system", etc. are intended to refer to a computer-related entity, which can be hardware, firmware, a combination of hardware and software, software, or software in operation. For example, a component can be, but is not limited to: a process running on a processor, a processor, an object, an executable file, a thread in execution, a program, and / or a computer. As an example, an application running on a computing device and the computing device can both be components. One or more components can exist in a process and / or thread in execution, and a component can be located in a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures thereon. These components can communicate in a local and / or remote process manner, such as according to a signal having one or more data packets (e.g., data from a component that interacts with another component in a local system, a distributed system, and / or interacts with other systems in a signal manner through a network such as the Internet).

[0052] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the technical solutions of the present invention.

Claims

1. A breast ultrasound tumor segmentation method based on VGG and MESCA, characterized by: It includes the following steps, S1 obtains breast ultrasound tumor images and obtains a data set; S2 preprocesses the acquired data set; S3 builds an improved UNet model based on VGG; S4 brings the preprocessed data set into the improved UNet model to train the network; S5 inputs the breast ultrasound tumor image to be segmented into the trained improved UNet network and outputs the segmentation result.

2. The breast ultrasound tumor segmentation method based on VGG and MESCA as claimed in claim 1, characterized in that: Step S3 specifically The following steps are included: S3.1 uses VGG16 as the feature extraction network of UNet, corresponding to the first, second, third, fourth and fifth layers of the UNet encoder part; S3.2 uses depthwise separable convolutional modules in the encoder instead of the original ordinary convolutional modules; S3.3 Add a Batch Normalization layer after each convolution of the encoder and output it through the nonlinear activation function ReLU; S3.4 For each layer of the encoder, add a SE attention module after all depth-wise separable convolutional layer operations; The S3.5 UNet decoder uses the encoder in the original UNet and adds a MESCA module before each upsampling.

3. The breast ultrasound tumor segmentation method based on VGG and MESCA as claimed in claim 1, characterized in that: The specific steps of obtaining the data set in step S1 are: S1.1 The public breast ultrasound tumor dataset BUSI was used as the original dataset; S1.2 Randomly select m1 images from a dataset containing n images, perform Laplace sharpening on the selected images as a breast ultrasound tumor dataset, and randomly select m2 images from the dataset as a training set. 21 pictures as the validation set, m 22 pictures as the test set; Among them, the number of m1 is less than n, and m2=m 21 +m 22 .

4. The breast ultrasound tumor segmentation method based on VGG and MESCA as claimed in claim 2, characterized in that: In step 3.4, the SE attention module enhances the feature expression capability through a lightweight channel attention mechanism. The specific implementation process is as follows: First, the global information of each channel is extracted through global average pooling, and then the channel weights are generated through two layers of full connection, and finally the input features are weighted.

5. The breast ultrasound tumor segmentation method based on VGG and MESCA as claimed in claim 2, characterized in that: In step S3.5, the MESCA module is a median-enhanced spatial channel attention module. The implementation process is as follows: the input features are feature adjusted through a 1×1 point-by-point convolution, and then enter the channel attention module; in the channel attention module, global features are extracted through global average pooling, maximum pooling and median pooling respectively, and each feature is processed by two point-by-point convolutions and activation functions to obtain an attention map, which is then added to fuse the global characteristics of each channel; the feature map weighted by the channel attention is further extracted through a 5×5 deep convolution to extract preliminary spatial features; Multiple deep convolutional layers of heterogeneous sizes are then used to extract multi-scale spatial information and fused into a spatial attention map; finally, the input features are weighted.

6. The breast ultrasound tumor segmentation method based on VGG and MESCA as claimed in claim 2, characterized in that: In step S4, the training parameters of the UNet model are as follows: using the Adam optimizer, the initial learning rate is set to 0.001, the number of training iterations is set to 200, the learning rate is dynamically adjusted in an exponential decay manner according to the number of iterations, and the minimum learning rate is set to 0.0001.

7. The breast ultrasound tumor segmentation method based on VGG and MESCA as claimed in claim 2, characterized in that: In step S5, the parameter corresponding to the minimum loss value in the training cycle is selected as the final weight to segment the breast ultrasound tumor image.

8. The breast ultrasound tumor segmentation method based on VGG and MESCA as claimed in claim 6, characterized in that: The binary cross entropy function is used as the model training loss function. The specific formula is: , where y is a binary label 0 or 1, p(y) is the probability that the output belongs to the y label, and N represents the total number of training times.

Images