Method and system for segmenting breast masses

Abstract

The application discloses a breast lump segmentation method and system, comprising the following steps: step 1, obtaining the medical image of the breast lump of a patient, pre-processing the medical image, and labeling the pre-processed remote sensing image to make a data set; step 2, constructing an MM-3DUNet model for segmenting the breast lump, training the MM-3DUNet model by using the data set, and obtaining an optimized MM-3DUNet model; and step 3, segmenting the breast lump of the medical image by using the optimized MM-3DUNet model. The application reduces the calculation complexity of the model, and improves the calculation efficiency and segmentation accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image recognition, and particularly relates to a breast lump segmentation method and system. BACKGROUND

[0002] Breast cancer has become the most common malignant tumor type in women worldwide and is one of the leading causes of death from cancer in women. In recent years, with the development of magnetic resonance examination technology and means, multi-modal magnetic resonance imaging (MRI) scanning method has become the most sensitive imaging technology for breast cancer. This method can provide detailed anatomical images and accurately depict soft tissue structures, and has excellent potential in the diagnosis, preoperative planning and prognosis of breast cancer. Among them, dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) sequence plays an important role in the discovery and diagnosis of breast cancer lesions. DCE-MRI sequence can help discover significantly enhanced lesions, understand the size, signal and morphological characteristics of the lesions, and not only can qualitative diagnosis be made according to the morphological characteristics, enhancement mode, etc. of the lesions, but also can be used for comparison of the lesions before and after chemotherapy to evaluate the treatment effect.

[0003] Traditional breast cancer pathology identification based on MRI images usually relies on doctors' naked eye observation and subjective experience for judgment. Due to the heterogeneity and atypicality of breast tumor shape characteristics itself, as well as the differences in professional knowledge, operation process and working state among doctors, misdiagnosis or missed diagnosis risk is easy to occur. Computer-aided diagnosis (CAD) technology is increasingly becoming an important research field to help doctors interpret breast cancer pathology images, which can effectively improve the accuracy and consistency of diagnosis, reduce the workload of doctors and improve the efficiency of diagnosis. One of the most common symptoms of breast cancer is the presence of a lump, so exploring how to accurately and efficiently detect the size, shape, edge and other information of the lump in dense breast tissue has become an important issue in the process of computer-aided diagnosis technology in breast cancer diagnosis. In recent years, with the rapid development of deep learning and computer vision technology, the method of using convolutional neural network (CNN) for semantic segmentation of breast cancer lump has achieved remarkable results. This method is based on the semantic features contained in the image, which combines the pixels belonging to the same target class, generates a mask that highlights different regions of interest, and finally segments the image into several regions with unique visual properties and specific types. Because deep learning algorithm has a more complex feature vector space calculation framework and deeper operation level, it can learn different levels of semantic features from shallow to deep in the image of complex scene without relying on manual feature extraction, and realize higher precision end-to-end image semantic segmentation. Among them, UNet19 network adopts symmetrical encoding-decoding structure and jump connection structure, which can realize fast and accurate semantic segmentation under the condition of less training sample amount. With its excellent structure design and wide applicability, it has become the first choice of medical image segmentation model bottom framework.

[0004] 2D convolutional neural network has achieved some success in MRI image segmentation, but it ignores the spatial continuity between slices, resulting in the loss of important three-dimensional context information and relatively low segmentation accuracy. Milletari et al. inspired by the 2D structure of UNet, improved the network upsampling and downsampling layers and the loss function, and proposed a 3D convolutional neural network VNet to improve the segmentation accuracy of three-dimensional images. After that, more and more researchers began to try to improve the 3DUNet and proposed 3DUNet network variants with different structural characteristics. Zhou et al. fused 2D convolutional structure in the encoding stage of 3DUNet network, respectively from 2D and 3D MRI images to obtain the features of the continuous slices and fuse them, and proposed a dimension fusion D-UNet, which enables the network to learn more rich feature representation. Although these improvement schemes have achieved remarkable accuracy improvement in medical image segmentation tasks by introducing complex network architecture, dimension fusion strategy and rich auxiliary modules, etc. However, the model parameters and computational complexity have increased dramatically. At the same time, the high training cost also limits its popularity and application flexibility, which leads to difficulties in actual deployment and application, such as in mobile medical devices, edge computing platforms and other resource-constrained environments. These models will face great challenges, and there is an urgent need for lightweight strategies to balance accuracy and efficiency to meet the diverse needs of medical image segmentation. SUMMARY

[0005] An object of the present application is to provide a breast lump segmentation method that reduces the computational complexity of the model and improves the computational efficiency and segmentation accuracy.

[0006] To solve the above technical problems, the present application adopts the following technical solutions:

[0007] A breast lump segmentation method, comprising the following steps:

[0008] Step 1: Obtain the medical image of the patient's breast lump, pre-process it, and label the pre-processed remote sensing image to make a data set;

[0009] Step 2: Construct a MM-3DUNet model for segmenting breast lumps, train the MM-3DUNet model with the data set, and obtain an optimized MM-3DUNet model;

[0010] Step 3: Use the optimized MM-3DUNet model to segment the breast lump of the medical image.

[0011] Further, in step 1, the pre-processing of the image includes image cropping, image contrast adjustment and Gaussian blur processing on the obtained medical image.

[0012] Further, the MM-3DUNet model constructed in step 2 includes an encoder, a decoder, a 1x1x1 convolution kernel, a sigmoid function and an auxiliary classification task bypass.

[0013] The encoder is used to extract multi-level semantic features of the input image and compress the feature map size; the decoder is used to recover the feature map size and locate the key information, and finally output a target image with the same size as the original image; the 1x1x1 convolution kernel is used to compress the channel number of the final feature map to the label number, and the sigmoid function is used to convert the prediction result into a probability value, with a value range of 0-1; the auxiliary classification task bypass is used to provide beneficial supervision signals for the main task of breast cancer tumor segmentation.

[0014] Further, the encoder includes a plurality of progressive calculation modules; each calculation module includes a plurality of repeated feature extraction units, and each feature extraction unit includes a channel expansion convolution unit, an inverse residual structure unit and an activation function.

[0015] The decoder includes a plurality of progressive calculation modules of repeated feature recovery units, and the structure of the feature recovery unit is basically the same as that of the feature extraction unit, wherein a transposed convolution is used at the end of the feature recovery unit to expand the feature map size, and the image channel number is halved and the size is doubled after feature recovery of each calculation module.

[0016] Further, the channel expansion convolution unit includes a channel expansion module, a depth convolution module and a channel compression module.

[0017] In the channel expansion module, for an input feature with a size of CinxDxHxW, a 1x1x1 convolution kernel is used to expand the channels of the input feature to txCin, t is the channel expansion factor, and the output feature map size is (txCin)xDxHxW.

[0018] In the depth convolution module, a 3D depth separable convolution is used to independently extract features for each channel, and the output feature map dimension is still (txCin)xDxHxW.

[0019] In the channel compression module, another convolution kernel is used to compress the channel number of the feature map from txCin to Cout.

[0020] Wherein, after each convolution calculation of the channel expansion module and the depth convolution module, the combination of LeakyReLU activation function and Instance Norm normalization calculation is used for nonlinear change and channel normalization processing of the feature map; in the channel compression module, after the feature map is convolved in the low-dimensional space, a linear activation function is used for linear change processing of the feature map.

[0021] Further, the 3D depth separable convolution includes 3D depth convolution and 3D pointwise convolution;

[0022] Wherein, the 3D depth convolution is independently convolved on each input channel, and the calculation process is that assuming that the input feature map has C in channels, the image size is D0xH0xW0, D0, H0 and W0 respectively represent the depth, height and width of the input feature, C in convolution kernels with the size of K D x K H x K W are used for separate convolution calculation with the input on each channel, and the output feature map channel number is still C in .

[0023] The calculation process of 3D pointwise convolution is: using C out 1x1x1 convolution kernel with C in channel number, combining the feature map output by 3D depth convolution in channel dimension, finally obtaining the output feature map with C out channel number and D1xH1xW1 size.

[0024] Further, the reverse residual structure unit first expands the channel of the input feature map, extracts the feature, and then compresses it, and if the channel number of the input feature map and the output feature map is the same, the input feature map is directly added to the output feature map to form a residual connection.

[0025] Further, the auxiliary classification task bypass includes a 3x3 convolution kernel, a max pooling layer and a linear activation layer, the feature map is input into the auxiliary classification task bypass after the feature recovery output of the decoder, first mapped to two-dimensional space, then the type feature of the input image is extracted using a 3x3 convolution kernel, and then the data structure is adjusted through the max pooling layer, and finally the same number of results as the type is output through the linear activation layer.

[0026] Further, in the step 2 training process, Adam is used as the parameter optimizer, and gradient descent method is used for training, when training, Dice Loss is used as the loss function of the segmentation task, and cross entropy CE Loss is used as the loss function of the auxiliary classification task bypass.

[0027] wherein, the Dice Loss takes all pixels of one class as a whole to calculate the loss function value, directly takes the segmentation effect evaluation index as the loss function to supervise the network, and the calculation formulas of the Dice Loss and the CE Loss are as follows:

[0028]

[0029] In the formula, p(c i ) is the label value of the i th pixel in the class C, which is 0 or 1; g(c i ) is the prediction probability of the i th pixel belonging to the class C; I is the intersection of the prediction value and the label value; H and W are the height and width of the feature map respectively; U is the union of the prediction value and the label value; and ε is a smoothing coefficient.

[0030] Another object of the present application is to provide a system for implementing the breast lump segmentation method described above, comprising:

[0031] A data acquisition and processing module is configured to acquire medical images of a patient's breast lump, pre-process the medical images, and label the pre-processed remote sensing images to generate a data set.

[0032] A model construction and training module is configured to construct an MM-3DUNet model, train the MM-3DUNet model using the data set, and obtain an optimized MM-3DUNet model.

[0033] A breast lump segmentation module is configured to segment the medical images using the optimized MM-3DUNet model.

[0034] Compared with the prior art, the MM-3DUNet model constructed by the present application creatively adds a channel expansion convolution unit and a reverse residual structure, realizes a substantial increase in network computing efficiency and accuracy by adding an adjustable feature dimension high-dimensional mapping and completely retaining effective information in the input features, and further improves the segmentation accuracy of breast cancer lumps by adding an auxiliary classification task bypass in the decoding stage of the model to provide additional supervision signals for the segmentation main task while substantially reducing the computational complexity. Experimental results show that, compared with the classic 3DUNet network, the Parameters and FLOPs of the MM-3DUNet are reduced by 63.16% and 80.90% respectively, the model computational complexity is substantially reduced, the IoU of the MM-3DUNet is increased by 1.30% compared with the 3DUNet in terms of segmentation accuracy, the model is lightweight, the accuracy of breast cancer lump segmentation is improved, the computing efficiency is improved, the stability and applicability of the network are enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 For the breast cancer pathological tissue tag of the embodiment of the application: (a) breast tissue section; (b) breast cancer mass tag;

[0036] Figure 2 For the framework schematic diagram of the MM-3DUNet model of the embodiment of the application;

[0037] Figure 3 For the structural schematic diagram of the channel expansion convolution unit of the embodiment of the application;

[0038] Figure 4 For the structural schematic diagram of the reverse residual structure unit of the embodiment of the application;

[0039] Figure 5 For the structural schematic diagram of the auxiliary classification task bypass of the embodiment of the application;

[0040] Figure 6 For the loss value change curve comparison of the DUNet and the MM-3DUNet in verification of the embodiment of the application;

[0041] Figure 7 For the IoU change curve comparison of the 3DUNet and the MM-3DUNet in verification of the embodiment of the application;

[0042] Figure 8 For the breast cancer mass segmentation precision comparison of the network before and after using the channel expansion convolution unit on the test set of the embodiment of the application: (a) breast tissue section; (b) breast cancer mass tag; (c) 3DUNet network segmentation result; (d) MM-3DUNet network (expansion factor = 6) segmentation result; (e) MM-3DUNet network (expansion factor = 12) segmentation result, the number in the upper left corner of the segmentation effect diagram is the IoU value;

[0043] Figure 9 For the breast cancer mass segmentation precision comparison of the network before and after using the channel expansion convolution unit on the test set of the embodiment of the application: (a) breast tissue section; (b) breast cancer mass tag; (c) 3DUNet network segmentation result; (d) MM-3DUNet network segmentation result; (e) MM-3DUNet+ACT network segmentation result, the number in the upper left corner of the segmentation effect diagram is the IoU value. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work belong to the scope of protection of the present application.

[0045] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0046] The present application will be further described below with reference to specific embodiments, but not as a limitation of the present application.

[0047] The present application discloses a breast lump segmentation method, comprising the following steps:

[0048] Step 1, obtain the medical image of the patient's breast lump, pre-process it, and label the pre-processed remote sensing image to make a data set;

[0049] In this embodiment, the nuclear magnetic resonance image (RMI) of the breast part of the patient in the hospital is obtained. The nuclear magnetic resonance image of the breast is obtained by the magnetic resonance device of the hospital, and the model is Philips Achieva 3.0T. The image data is mainly collected in the breast to obtain detailed information of the breast structure. Some configuration parameters during collection include: magnetic field strength is 3.0T, TR is 2000ms, TE is 30ms, matrix size is 256*256, layer thickness is 1mm, and interval is 0.5mm. The above configuration parameters are set to ensure the clarity and resolution of the data, so as to carry out subsequent image segmentation and analysis work.

[0050] In this embodiment, 219 breast tissue samples are obtained, and each sample contains about 180 slices in the MRI image. The professional doctors label the obtained samples, and the labeling target is the mask of the breast tumor area. The professional doctors accurately locate the breast tumor area by checking the nuclear magnetic resonance image, and use professional labeling tools to draw and label the area according to the area, to generate the mask of the breast tumor. This process ensures the accuracy and reliability of the labeling, and provides a reliable data basis for subsequent medical image segmentation analysis.

[0051] The labeled images are preprocessed, which includes image cropping, image contrast adjustment and Gaussian blur processing on the obtained medical images. Image cropping is to crop the obtained original image into image blocks of the same size, thereby increasing the number of samples. The image contrast adjustment method is: set a threshold, compare the contrast parameter value of the image with the threshold, when the contrast parameter value of the image is less than the threshold, the contrast of the image is increased, and when the contrast parameter value of the image is greater than the threshold, the contrast of the image is reduced, thereby reducing the contrast difference of images obtained in different batches. Gaussian blur is an image smoothing method, which reduces noise and details in the image by applying a Gaussian filter, which helps to remove unnecessary fine differences and abnormal noise, thereby focusing on the main features of the image. The preprocessed images are sorted into a data set, and divided into training set and test set according to the ratio of 4:3:3.

[0052] Step 2, constructing MM-3DUNet model for segmenting breast lumps, training MM-3DUNet model with data set, and obtaining optimized MM-3DUNet model;

[0053] In view of the problems of excessive model parameter quantity, high computing resource demand and serious memory consumption of the existing 3DUNet network in actual application, its application in resource-limited environment and small devices is limited. The present embodiment constructs a lightweight semantic segmentation network MM-3DUNet model for extracting breast cancer lumps in MRI images. The model enhances the feature expression capability in the information flow process by constructing channel expansion convolution units and reverse residual blocks, and additionally increases an auxiliary classification task bypass at the end of the model to distinguish lesions and normal parts, which provides beneficial supervision signals for the main task of breast cancer lump segmentation, effectively reduces the model parameters and computing amount while ensuring the segmentation accuracy of the model. Specifically, as shown in Figure 2 The structure of the MM-3DUNet model is as follows:

[0054] The main body of the MM-3DUNet model includes a gradually shrinking downsampling path and a symmetric expanding upsampling path, i.e., the encoder and the decoder of the network. The encoder is used to extract the multi-level semantic features of the input image and compress the feature map size, including four progressive calculation modules. Each calculation module contains two repeated feature extraction units. Each feature extraction unit includes two 1x1x1 convolution kernels, a channel expansion convolution unit, an inverse residual block, and a subsequent activation function. At the end of the second feature extraction unit, there is a max pooling layer for compressing the feature map size. After feature extraction of each calculation module, the number of image channels is doubled and the size is halved. The decoder is used to restore the feature map size and locate the key information, and finally outputs a target image with the same size as the original image, which is also composed of progressive calculation modules containing two repeated feature recovery units. The structure of the feature recovery unit is basically the same as that of the feature extraction unit, except that a transposed convolution is used at the end of the feature recovery unit to expand the feature map size. After feature recovery of each calculation module, the number of image channels is halved and the size is doubled. The calculation modules at corresponding levels between the encoder and the decoder are connected using a skip connection structure to realize the mutual fusion of low-level semantic information and high-level semantic information. In the MM-3DUNet model, a 1x1x1 convolution kernel is used to compress the number of channels of the final feature map to the number of labels, and a sigmoid function is used to convert the network prediction result to a probability value with a range of 0-1. In addition, an additional classification task bypass is added at the end of the decoder, which includes a 3x3 convolution kernel, a max pooling layer, and a linear activation layer, for distinguishing normal parts from lesion parts.

[0055] Channel expansion convolution unit

[0056] Compared with low-dimensional space, in high-dimensional space calculation, more complex feature learning mode helps the model to mine the hidden information in the data that is difficult to capture through low-dimensional features, so as to more comprehensively describe the data features and allow the deep learning model to capture more subtle and complex relationships and patterns in the data. Therefore, the embodiment proposes a channel expansion convolution unit, which first expands the dimension of the input feature, maps the low-dimensional input feature to the high-dimensional space for feature extraction calculation, improves the depth of feature extraction, reduces the feature dimension, reduces the difficulty of subsequent layer calculation, and enhances the model calculation efficiency. Without increasing a large amount of calculation and parameters, the expression ability and nonlinearity of the network are enhanced. See Figure 3 The channel expansion convolution unit mainly includes a channel expansion module, a deep convolution module, and a channel compression module.

[0057] In the channel expansion module, for a feature map with a size of C inThe input feature of ×D×H×W is first expanded to t×C in channels using a 1×1×1 convolution kernel, where t is a channel expansion factor used to adjust the degree of channel expansion, and an output feature map with richer dimension information is obtained, with a size of (t×C in )×D×H×W. In the depth convolution module, a 3D depthwise separable convolution with a size of 3×3×3 is used to independently extract features for each channel, reduce the model calculation amount while maintaining stable internal spatial structure information, and the output feature map dimension is still (t×C in )×D×H×W. Finally, in the channel compression module, another 1×1×1 convolution kernel is used to compress the channel number of the feature map from t×C in to C out , which significantly reduces the model calculation amount and parameter amount. It is worth noting that after each convolution calculation in the channel expansion and depth convolution stage, a combination of a LeakyReLU activation function and an Instance Norm normalization calculation is used for nonlinear transformation and channel normalization of the feature map, enhancing the nonlinearity and stability of feature extraction. However, in the channel compression stage, when the feature map is convolved in a low-dimensional space, a linear activation function is used for linear transformation of the feature map to avoid nonlinear loss of information and preserve more original image information.

[0058] 3D depthwise separable convolution

[0059] The present embodiment uses 3D depthwise separable convolution to replace the simple 3D convolution in the classic 3DU-Net, so as to effectively reduce the parameter amount and calculation complexity in the model calculation process. 3D depthwise separable convolution mainly includes two main steps: 3D depthwise convolution and 3D pointwise convolution. Among them, 3D depthwise convolution is independently convolved on each input channel, rather than combining all input channels with each convolution kernel for calculation as in standard 3D convolution. Assuming that the input feature map has C in channels, and the image size is D0×H0×W0, D0, H0 and W0 represent the depth, height and width of the input feature respectively. C in convolution kernels with a size of K D ×K H ×K W are used to perform independent convolution calculation with the input on each channel, and the output feature map channel number is still C in . The calculation process of 3D pointwise convolution is similar to that of standard convolution calculation, using C out channel number C ina 1x1x1 convolution kernel, combining the feature maps output by the 3D deep convolution in the channel dimension, and finally obtaining an output feature map with a channel number of C out , a size of D1xH1xW1.

[0060] In the entire calculation process of the 3D deep separable convolution, the parameter amount P 3D-ds and the calculation amount F 3D-ds of the network are as follows:

[0061] P 3D-ds = C in × K D × K H × K W + C in × C out (1)

[0062] F 3D-ds = D × H × W × C in × (K D × K H × K W + C out ) (2)

[0063] If only the standard 3D convolution operation is used, to achieve the same feature extraction effect, the parameter amount P 3D-standard and the calculation amount F 3D-standard of the network are as follows:

[0064] P 3D-standard = C in × K D × K H × K W × C out (3)

[0065] F 3D-standard = D × H × W × C in × K D × K H × K W × C out (4)

[0066] The parameter amount and the calculation amount of the 3D deep separable convolution proposed in the present research are compared with those of the standard 3D convolution operation:

[0067]

[0068]

[0069] It can be found that after using the 3D deep separable convolution, the parameter amount and the calculation amount of the network are only The model parameter amount and the calculation complexity are greatly reduced.

[0070] Reverse residual structure unit

[0071] In this embodiment, a reverse residual structure is added on the basis of the channel expansion convolution unit. Unlike the calculation process of the traditional residual block, the input features pass through multiple convolution layers, and an identity mapping is added to connect the input features and the output features while the number of channels remains unchanged. This embodiment adopts the opposite strategy, which first expands the channels of the input features, extracts the features, and then compresses them. If the number of input and output channels is the same, the input feature map is directly added to the output feature map to form a residual connection. In this way, the transmission efficiency of information in the calculation unit is improved, while part of the information of the input features is preserved, effectively alleviating the gradient vanishing problem and enhancing the feature representation capability. See Figure 4 The specific implementation is as follows: assuming that there is an input feature map with a size of HxWxC (heightxwidthxchannel number), first use 1x1 convolution to expand the number of channels from C to tC (t is an expansion factor, for example, 6), and the output size becomes HxWxtC. Then apply a 3x3 depth separable convolution to keep the output size unchanged, that is, HxWxtC. Again, use 1x1 convolution to compress the number of channels from tC back to C, and obtain the output size HxWxC.

[0072] Auxiliary classification task bypass

[0073] In the breast cancer tumor segmentation task, the global feature difference between the lesion site and the normal site can be fully utilized to improve the accuracy and stability of breast cancer tumor segmentation. Therefore, an auxiliary classification task bypass is added at the decoding end of the MM-3DUNet model, based on the correlation between breast cancer tumor segmentation and lesion and normal site classification, the image features extracted in the encoding stage are shared, and additional supervision signals are learned from the perspective of global feature information. In this way, more global context information can be provided to the network decoder to promote the model to learn more generalized feature representations, avoid overfitting of the main task, and improve the performance of the semantic segmentation task. The structure of the auxiliary classification task bypass is shown in Figure 5 As shown in the figure, it includes a 3x3 convolution kernel, a max pooling layer, and a linear activation layer. After the feature recovery output, the feature map is first mapped to a two-dimensional space, then the type features of the input image are extracted using a 3x3 convolution kernel, and then the data structure is adjusted through the pooling layer. Finally, the linear activation layer outputs the same number of results as the type.

[0074] Training of the MM-3DUNet model

[0075] The training, validation and testing processes of the model were performed under the deep learning framework Pytorch v1.13. The CPU of the computer was 13th Gen Intel(R) Core(TM) i5-13600K, the memory was 64 GB, the GPU was NVIDIA GeForce RTX3090, and CUDA12.2 computing platform and cuDNN8.9.1 were used for hardware acceleration.

[0076] The training epoch was set to 130 and the batch size was set to 24 during training. Adam was used as the parameter optimizer, and the weight drop rate was set to 0.0001 and the weight coefficient was set to 0.95. Dice Coefficient Loss (Dice Loss) was used as the loss function for the segmentation task, and Cross-Entropy (CE Loss) loss was used as the loss function for the auxiliary classification task. Among them, Dice Loss takes all pixels of a class as a whole to calculate the loss function value, directly uses the segmentation effect evaluation index as the loss function to supervise the network, and ignores a large number of background pixels when calculating the intersection-over-union, thereby solving the problem of unbalanced positive and negative samples, so the convergence speed is faster. The calculation formulas of Dice Loss and CE Loss are as follows:

[0077]

[0078] In the formula, p(c i ) is the label value of the ith pixel in class C, which is 0 or 1; g(c i ) is the predicted probability of the ith pixel belonging to class C; I is the intersection of the predicted value and the label value; H and W are the height and width of the feature map, respectively; U is the union of the predicted value and the label value; ε is a smoothing coefficient to prevent the denominator from being 0, usually taking a very small positive number. At the same time, the Cosine Annealing Learning Rate was used as the rate decay strategy to update the learning rate of the network training process, and the calculation formula is as follows:

[0079]

[0080] In the formula, lr t is the learning rate of the ith training; lr0 is the initial learning rate, which is lr0 = 0.0001; T is the number of training times in a cosine period, which is the same as the epoch of the network.

[0081] Evaluation index

[0082] In the image semantic segmentation task, in order to comprehensively evaluate the segmentation performance of the deep learning network, not only the accuracy of the classification of a single target pixel is calculated, but also the coincidence degree of the overall segmentation result of the region and the segmentation label is compared. Therefore, the present embodiment uses the intersection over union (IoU) as the evaluation index of the MM-3DUNet model to evaluate the accuracy of the model. The IoU represents the ratio of the intersection and the union of the number of pixels in the predicted target region and the number of pixels in the actual labeled target region in the segmentation result, which reflects the overlapping degree of the segmentation result and the labeled result of the class, and the calculation formula is as follows:

[0083]

[0084] In the formula, TP is the number of pixels labeled as a target class and correctly predicted as a target class; FP is the number of pixels labeled as a non-target class but incorrectly predicted as a target class; TN is the number of pixels labeled as a non-target class and correctly predicted as a non-target class; and FN is the number of pixels labeled as a target class but incorrectly predicted as a non-target class.

[0085] In addition, in order to evaluate the change of the computational complexity of the improved segmentation network, the floating point operations (FLOPs) and the number of parameters (Parameters) are used as evaluation indexes. Among them, the FLOPs represents the number of floating point operations required for one forward propagation of the model, which directly reflects the workload of the model in the calculation process. The higher the FLOPs value of the model, the more floating point operations it performs in the calculation, resulting in higher computational cost and longer calculation time. Parameters represent the total number of all parameters including weights and biases contained in the model, which not only affects the computational complexity of the model, but also determines the amount of data and training time required for model training. Too many parameters may cause overfitting of the model, while too few parameters may limit the representation ability of the model.

[0086] Step 3, using the optimized MM-3DUNet model to segment medical images.

[0087] In order to illustrate the beneficial effects of the MM-3DUNet model of the present embodiment, its performance is compared with that of the existing model. The comparative example is based on the same breast cancer pathological image dataset and model parameter settings, and the classic 3DUNet and the MM-3DUNet of the present embodiment are trained respectively to verify the superiority of the MM-3DUNet model in performance and computational efficiency. The loss function and IoU change processes of the two on the validation set are shown in Figure 6 and Figure 7 From Figure 6As can be seen from the above table, the loss values of the 3DUNet and MM-3DUNet models gradually decrease with the increase of the round. Notably, the loss value of the MM-3DUNet model decreases significantly faster, which indicates that the MM-3DUNet model has higher learning efficiency, can capture key features in the data faster, and can fully mine useful information in the data. Figure 7 As can be seen from the comparison of the IoU curves of the 3DUNet and the MM-3DUNet, both of them gradually increase with the increase of the round, and the IoU of the MM-3DUNet increases significantly faster, which further verifies the efficiency of the MM-3DUNet model in the training process. At the same time, the IoU curve of the MM-3DUNet fluctuates less, which indicates that it can more stably improve the performance and is not easily disturbed by the fluctuation or noise of the training data, and has better superiority and reliability in the breast cancer tumor segmentation task.

[0088] Performance of 3D depth separable convolution

[0089] To verify the effectiveness of the 3D depth separable convolution proposed in the embodiment in reducing the network parameter quantity and the computational complexity, the embodiment uses the classical 3DUNet with simple 3D convolution as the baseline model (Baseline), and replaces the convolution unit with the 3D depth separable convolution, trains and tests the two networks on the same data set, and compares the accuracy and the computational complexity of the two networks on the test set, and the results are shown in Table 1. As can be seen from the table, after replacing the simple 3D convolution with the 3D depth separable convolution, the computational complexity of the breast cancer tumor segmentation network is significantly reduced. Among them, the Parameters is reduced by 1.57M compared with the Baseline, which is only 11.8% of the original, and the FLOPs is reduced by 88.05G compared with the Baseline, which is only 23.5% of the original. In addition, the IoU of the 3D depth separable convolution is almost unchanged compared with the Baseline, only reduced by 0.97%, and still maintains a high segmentation accuracy.

[0090] The above results show that the 3D depth separable convolution proposed in the embodiment can efficiently adapt to the complex structural characteristics of three-dimensional data, capture the spatial correlation and depth correlation in three-dimensional data. Under the premise of not significantly reducing the model performance, it can greatly reduce the number of floating point operations required for the network during the forward propagation and the parameter quantity generated, so that the model can process data faster during the training and inference process, realize higher computational efficiency and lower resource consumption, and provide feasibility for the deployment of the model on resource-constrained devices such as mobile devices, embedded devices, etc.

[0091] Table 1 Comparison of breast tumor segmentation performance of simple 3D convolution and 3D depthwise separable convolution

[0092]

[0093] Performance of channel expansion convolution unit

[0094] This embodiment further designs a channel expansion convolution unit on the basis of using 3D depthwise separable convolution to help improve the accuracy of breast tumor feature extraction. In order to explore the effectiveness of this structure, the comparative experiments shown in Table 2 are designed, by setting different channel expansion factors, adjusting the number of channels of the network expansion layer to change the diversity of feature learning, and comparing the performance of different networks on the test set. From the results in the table, it can be seen that when the channel expansion factor is small, the network computation complexity decreases significantly, but the segmentation accuracy decreases compared with Baseline. After adjusting the channel expansion factor in MM-3DUNet from 6 to 12, the IoU increases by 1.46%, which is 0.49 higher than the classic 3DUNet. In terms of model computation complexity, at this time, the Parameters of MM-3DUNet is reduced by 1.45M compared with Baseline, and the FLOPs is reduced by 73.31G.

[0095] Table 2 Comparison of breast tumor segmentation performance before and after using channel expansion convolution unit

[0096]

[0097] To further evaluate the influence of the channel expansion convolution unit on the segmentation performance, the visual results of the breast tumor segmentation accuracy of different models in the test set are compared, and the results are shown in Figure 8 From Figure 8 , it can be seen that after directly increasing the channel expansion convolution, the segmentation accuracy of the model decreases, compared with 3DUNet, the edge details of the segmentation result are missing more, and the identification accuracy of the fuzzy area is insufficient. After adjusting the channel expansion factor to a suitable value, the segmentation accuracy of the network for complex areas in the image is significantly improved compared with Baseline, and the outline restoration for the breast tumor area is more fine.

[0098] The experimental results show that the channel expansion convolution unit can effectively adjust the complexity and feature extraction ability of the model, help the model to remove redundant information while retaining key features, and obtain more compact and effective feature representation, thereby balancing the expression ability and computation complexity of the model. Using 3D depthwise separable convolution can reduce the network computation load and improve the segmentation accuracy.

[0099] Performance of auxiliary classification task bypass

[0100] On the basis of the existing MM-3DUNet, the auxiliary classification task bypass is further increased, and after training and verification on the same data set, the performances of different networks on the test set are compared, as shown in Table 3. As can be seen from the table, after adding the auxiliary classification task bypass, the IoU of MM-3DUNet increases by 1.30% compared with 3DUNet, and the Parameters and FLOPs are reduced by 63.16% and 80.90% respectively compared with 3DUNet. The results show that adding the auxiliary classification task bypass can provide additional supervision signals for the network about breast cancer lesions, promote the model to learn more general feature representation, and prevent overfitting by appropriately increasing the complexity of the network, thereby enhancing the segmentation performance of the main task.

[0101] Table 3 Comparison of breast cancer tumor segmentation performance before and after adding auxiliary classification task bypass

[0102]

[0103] Figure 9 The segmentation visual effects of the network before and after adding the auxiliary classification task bypass are given, and from the figure it can be seen that after adding the auxiliary classification task bypass in the network, the network is more accurate in segmenting the boundary of the breast cancer tumor, and the segmentation result is more consistent with the actual situation. With the help of the auxiliary classification task bypass, the relationship between the target region in space and its surrounding pixels is better understood, so the network is more accurate in segmenting the boundary of the pathological site. In addition, from the visual effect, it can also be observed that the network after adding the auxiliary classification task bypass performs better in handling complex cases, and can better cope with challenging problems such as variable tumor morphology and fuzzy boundary, thus providing a more reliable basis for early diagnosis and treatment of breast cancer.

[0104] The above are only the preferred embodiments of the present application, and do not limit the implementation and protection scope of the present application. For those skilled in the art, it should be realized that any equivalent replacement and obvious changes made by applying the content of the present application should be included in the protection scope of the present application.

Claims

1. A method for segmenting a breast mass, characterized in that: The following steps are involved: Step 1: Obtain medical images of the patient's breast mass, preprocess them, and annotate the preprocessed remote sensing images to create a dataset; Step 2: construct an MM-3DUNet model for breast mass segmentation, train the MM-3DUNet model using the dataset, and obtain an optimized MM-3DUNet model; Step 3: Use the optimized MM-3DUNet model to perform breast mass segmentation on medical images; The MM-3DUNet model constructed in step 2 includes an encoder, a decoder, a 1×1×1 convolution kernel, a sigmoid function, and an auxiliary classification task bypass. The encoder and decoder are connected via a skip connection structure. The encoder extracts multi-level semantic features from the input image and compresses the feature map size. The decoder restores the feature map size and locates key information, ultimately outputting a target image of the same size as the original image. A 1×1×1 convolution kernel compresses the number of channels in the final feature map to the number of labels. The sigmoid function converts the prediction results into probability values ​​ranging from 0 to 1. The auxiliary classification task bypass provides a useful supervisory signal for the main task of breast cancer mass segmentation. The encoder includes multiple progressive computing modules; each computing module includes multiple repeated feature extraction units, each feature extraction unit includes a channel dilation convolution unit, an inverse residual structure unit and an activation function; The decoder consists of a progressive computation module with multiple repeated feature recovery units. The structure of the feature recovery unit is basically the same as that of the feature extraction unit. In particular, transposed convolution is used at the end of the feature recovery unit to expand the size of the feature map. After each computation module has restored the features, the number of image channels is halved and the size is doubled. The channel expansion convolution unit includes a channel expansion module, a depth convolution module and a channel compression module; In the channel expansion module, for an input feature of size Cin×D×H×W, a 1×1×1 convolution kernel is used to expand the channel of the input feature to t×Cin, where t is the channel expansion factor, and the output feature map size is (t×Cin)×D×H×W; In the depth convolution module, 3D depth-separable convolution is used to extract independent features for each channel, and the output feature map dimension is still (t×Cin)×D×H×W; In the channel compression module, the number of channels of the feature map is compressed from t×Cin to Cout through another convolution kernel; Among them, after each convolution calculation in the channel expansion module and the depth convolution module, a combination of Leaky ReLU activation function and Instance Norm normalization calculation is used to perform nonlinear changes in the feature map and channel normalization processing; in the channel compression module, after the feature map is convolved in the low-dimensional space, a linear activation function is used to perform linear changes in the feature map; 3D depth-wise separable convolution includes 3D depth-wise convolution and 3D point-by-point convolution; Among them, 3D depth convolution is performed independently on each input channel. The calculation process is to assume that the input feature map has C in channels, the image size is D0×H0×W0, D0, H0 and W0 represent the depth, height and width of the input feature respectively, using C in The size is K D ×K H ×K W The convolution kernel is convolved with the input on each channel separately, and the number of channels of the output feature map is still C in ; The calculation process of 3D point-by-point convolution is: using C out The number of channels is C in The 1×1×1 convolution kernel combines the feature maps output by the 3D depth convolution in the channel dimension, and finally obtains a channel number C out The output feature map of size is D1×H1×W1; The auxiliary classification task bypass includes a 3×3 convolution kernel, a maximum pooling layer, and a linear activation layer. The feature map is recovered by the decoder and input into the auxiliary classification task bypass. It is first mapped to a two-dimensional space, and then a 3×3 convolution kernel is used to extract the type features of the input image. The data structure is then adjusted through the maximum pooling layer, and finally a linear activation layer is used to output the same number of results as the number of types. During the training process of step 2, Adam is used as the parameter optimizer, and the gradient descent method is used for training. During training, Dice Loss is used as the loss function for the segmentation task, and Cross Entropy CE Loss is used as the loss function for the auxiliary classification task bypass. Among them, Dice Loss calculates the loss function value for all pixels of a category as a whole, and directly uses the segmentation effect evaluation index as the loss function to supervise the network. The calculation formulas of Dice Loss and CE Loss are: ； ； In the formula, p(c i ) is the label value of the i-th pixel in category C, which can be 0 or 1; g(c i ) is the predicted probability that the i-th pixel belongs to category C; I is the intersection of the predicted value and the label value; H and W are the height and width of the feature map respectively; U is the union of the predicted value and the label value; ε is the smoothing coefficient.

2. The method for segmenting a breast lump according to claim 1, wherein: In step 1, image preprocessing includes image cropping, image contrast adjustment, and Gaussian blur processing on the obtained medical image.

3. The method for segmenting a breast lump according to claim 1, wherein: The reverse residual structure unit first expands the channels of the input feature map, extracts the features, and then compresses them. If the number of channels of the input feature map and the output feature map is the same, the input feature map is directly added to the output feature map to form a residual connection.

4. A system for implementing the breast mass segmentation method according to any one of claims 1 to 3, characterized in that: include: The data acquisition and processing module is used to obtain medical images of the patient's breast mass, pre-process them, and annotate the pre-processed remote sensing images to produce a data set; The model building and training module is used to build the MM-3DUNet model and train the MM-3DUNet model using the dataset to obtain the optimized MM-3DUNet model; The breast mass segmentation module is used to perform breast mass segmentation on medical images using the optimized MM-3DUNet model.

Images