Breast lump segmentation method and system

Abstract

The invention discloses a breast lump segmentation method and system, and the method comprises the following steps: 1, obtaining a medical image of a breast lump of a patient, carrying out the preprocessing of the medical image, and carrying out the marking of a preprocessed remote sensing image, and making a data set; step 2, constructing an MM-3DUNet model used for segmenting the breast lumps, and training the MM-3DUNet model by adopting the data set, so as to obtain an optimized MM-3DUNet model; and step 3, carrying out breast lump segmentation on the medical image by adopting the optimized MM-3DUNet model. According to the method, the calculation complexity of the model is reduced, and the calculation efficiency and the segmentation precision are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of medical image recognition, and in particular relates to a breast mass segmentation method and system. Background Art

[0002] Breast cancer has become the most common type of malignant tumor in women worldwide and one of the main causes of cancer death in women. In recent years, with the progress of magnetic resonance examination technology and methods, multimodal magnetic resonance imaging (MRI) scanning method has become the most sensitive imaging technology for breast cancer. This method can provide detailed anatomical images and depict soft tissue structures with high precision, showing excellent potential in the diagnosis, preoperative planning and prognosis of breast cancer. Among them, dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) sequence plays an important role in the discovery and diagnosis of breast cancer lesions. DCE-MRI sequence can help find significantly enhanced lesions, understand the size, signal and morphological characteristics of lesions, not only can qualitative diagnosis be made based on the morphological characteristics and enhancement mode of lesions, but also can be used to compare lesions before and after chemotherapy and evaluate the treatment effect.

[0003] Traditional breast cancer pathological tissue recognition based on MRI images usually relies on the doctor's naked eye observation and subjective experience for judgment. Due to the heterogeneity and atypicality of breast tumor shape characteristics, as well as the differences in professional knowledge, operation procedures, and working conditions between individual doctors, there is a risk of misdiagnosis or missed diagnosis. Computer-aided diagnosis (CAD) technology is increasingly becoming an important research field to help doctors interpret breast cancer pathological images. It can effectively improve the accuracy and consistency of diagnosis, while reducing the workload of doctors and improving diagnostic efficiency. One of the most common symptoms of breast cancer is the presence of a mass. Therefore, exploring how to accurately and efficiently detect the size, shape, and edge information of a mass in dense breast tissue has become an important topic of computer-aided diagnosis technology in the diagnosis of breast cancer. In recent years, with the rapid development of deep learning and computer vision technology, the semantic segmentation method of breast cancer masses using convolutional neural networks (CNNs) has achieved remarkable results. Based on the various semantic features contained in the image, this method merges pixels belonging to the same target category, generates masks that highlight different regions of interest, and finally segments the image into several regions with visually unique properties and specific types. Since the deep learning algorithm has a more complex feature vector space calculation framework and a deeper operation level, it can learn different levels of semantic features from complex scene images from shallow to deep without relying on manual feature extraction, and achieve more accurate end-to-end image semantic segmentation. Among them, the UNet19 network adopts a symmetrical encoding-decoding structure and a skip connection structure, which can achieve fast and accurate semantic segmentation effects with a small number of training samples. With its excellent structural design and wide applicability, it has become the main choice for the underlying framework of medical image segmentation models.

[0004] 2D convolutional neural networks have achieved some success in MRI image segmentation, but they ignore the spatial continuity between slices, resulting in the loss of important 3D context information and relatively low segmentation accuracy. Inspired by the 2D structure UNet, Milletari et al. proposed a 3D convolutional neural network VNet to improve the segmentation accuracy of 3D images by improving the upsampling and downsampling layers and loss functions on the network. After that, more and more researchers began to try to improve 3DUNet and proposed 3DUNet network variants with different structural characteristics. Zhou et al. fused the 2D convolution structure in the 3DUNet network encoding stage, obtained continuous slices from 2D and 3D MRI images, and fused their respective features. They proposed a dimension-fused D-UNet, which enables the network to learn richer feature representations. Although these improved schemes have achieved remarkable accuracy improvements in medical image segmentation tasks by introducing complex network architectures, dimension fusion strategies, and rich auxiliary modules, they are accompanied by a sharp increase in model parameters and a significant increase in computational complexity. At the same time, the high training cost also limits its popularity and application flexibility, which leads to difficulties in actual deployment and application. For example, in resource-constrained environments such as mobile medical devices and edge computing platforms, these models will face huge challenges. There is an urgent need for lightweight strategies to balance accuracy and efficiency to meet the diverse needs of medical image segmentation. Summary of the invention

[0005] An object of the present invention is to provide a method for segmenting breast masses in view of the deficiencies of the prior art, which method reduces the computational complexity of the model and improves the computational efficiency and segmentation accuracy.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A method for segmenting a breast lump comprises the following steps:

[0008] Step 1, obtaining medical images of breast masses of patients, preprocessing them, and annotating the preprocessed remote sensing images to produce a data set;

[0009] Step 2, constructing an MM-3DUNet model for segmenting breast masses, training the MM-3DUNet model using the data set, and obtaining an optimized MM-3DUNet model;

[0010] Step 3: Use the optimized MM-3DUNet model to segment breast masses in medical images.

[0011] Furthermore, in step 1, preprocessing the image includes performing image cropping, image contrast adjustment, and Gaussian blur processing on the obtained medical image.

[0012] Furthermore, the MM-3DUNet model constructed in step 2 includes an encoder, a decoder, a 1×1×1 convolution kernel, a sigmoid function, and an auxiliary classification task bypass, and the encoder and the decoder are connected by a skip connection structure;

[0013] Among them, the encoder is used to extract the multi-level semantic features of the input image and compress the feature map size; the decoder is used to restore the feature map size and locate key information, and finally output a target image of the same size as the original image; the 1×1×1 convolution kernel is used to compress the number of channels of the final feature map into the number of labels, and the sigmoid function is used to convert the prediction result into a probability value ranging from 0 to 1; the auxiliary classification task bypass is used to provide beneficial supervision signals for the main task of breast cancer mass segmentation.

[0014] Furthermore, the encoder includes a plurality of progressive computing modules; wherein each computing module includes a plurality of repeated feature extraction units, each feature extraction unit including a channel dilation convolution unit, a reverse residual structure unit and an activation function;

[0015] The decoder contains a progressive computation module of multiple repeated feature recovery units. The structure of the feature recovery unit is basically the same as that of the feature extraction unit. At the end of the feature recovery unit, a transposed convolution is used to expand the size of the feature map. After feature recovery in each computation module, the number of image channels is halved and the size is doubled.

[0016] Further, the channel dilation convolution unit includes a channel dilation module, a depth convolution module and a channel compression module;

[0017] In the channel expansion module, for the input feature of size Cin×D×H×W, a 1×1×1 convolution kernel is used to expand the channel of the input feature to t×Cin, where t is the channel expansion factor, and the output feature map size is (t×Cin)×D×H×W;

[0018] In the deep convolution module, 3D depth-separable convolution is used to extract independent features for each channel, and the output feature map dimension is still (t×Cin)×D×H×W;

[0019] In the channel compression module, the number of channels of the feature map is compressed from t×Cin to Cout through another convolution kernel;

[0020] Among them, after each convolution calculation in the channel expansion module and the deep convolution module, the combination of LeakyReLU activation function and Instance Norm normalization calculation is used to perform nonlinear changes and channel normalization processing on the feature map; in the channel compression module, after the feature map is convolved in the low-dimensional space, the linear activation function is used to perform linear changes on the feature map.

[0021] Furthermore, 3D depthwise separable convolution includes 3D depthwise convolution and 3D pointwise convolution;

[0022] Among them, the 3D deep convolution performs convolution operations independently on each input channel. The calculation process is as follows: assuming that the input feature map has C in channels, the image size is D0×H0×W0, D0, H0 and W0 represent the depth, height and width of the input feature respectively, and C in The size is K D ×K H ×K W The convolution kernel performs separate convolution calculations with the input on each channel, and the number of channels of the output feature map is still C in ;

[0023] The calculation process of 3D point-by-point convolution is: using C out The number of channels is C in The 1×1×1 convolution kernel combines the feature maps output by the 3D deep convolution in the channel dimension, and finally obtains a channel number C out The output feature map of size is D1×H1×W1.

[0024] Furthermore, the inverse residual structure unit first expands the channels of the input feature map, extracts the features and then compresses them. If the number of channels of the input feature map and the output feature map is the same, the input feature map is directly added to the output feature map to form a residual connection.

[0025] Furthermore, the auxiliary classification task bypass includes a 3×3 convolution kernel, a maximum pooling layer, and a linear activation layer. The feature map is input into the auxiliary classification task bypass after the feature recovery output of the decoder. It is first mapped to a two-dimensional space, and then a convolution kernel of size 3×3 is used to extract the type features of the input image. The data structure is then adjusted through a maximum pooling layer, and finally a linear activation layer is used to output a result with the same number of types.

[0026] Furthermore, in the training process of step 2, Adam is used as the parameter optimizer, and the gradient descent method is used for training. During training, Dice Loss is used as the loss function of the segmentation task, and Cross Entropy CE Loss is used as the loss function of the auxiliary classification task bypass.

[0027] Among them, Dice Loss takes all pixels of a category as a whole to calculate the loss function value, and directly uses the segmentation effect evaluation index as the loss function to supervise the network. The calculation formulas of Dice Loss and CE Loss are:

[0028]

[0029] In the formula, p(c i ) is the label value of the i-th pixel in category C, which can be 0 or 1; g(c i ) is the predicted probability that the i-th pixel belongs to category C; I is the intersection of the predicted value and the label value; H and W are the height and width of the feature map respectively; U is the union of the predicted value and the label value; ε is the smoothing coefficient.

[0030] Another object of the present invention is to provide a system for implementing the above-mentioned breast mass segmentation method, comprising:

[0031] The data acquisition and processing module is used to obtain medical images of breast masses of patients, pre-process them, and annotate the pre-processed remote sensing images to produce data sets;

[0032] The model building and training module is used to build the MM-3DUNet model and use the data set to train the MM-3DUNet model to obtain an optimized MM-3DUNet model;

[0033] The breast mass segmentation module is used to perform breast mass segmentation on medical images using the optimized MM-3DUNet model.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows: the MM-3DUNet model constructed by the present invention creatively adds a channel dilation convolution unit and an inverse residual structure, and achieves a significant improvement in network computing efficiency and accuracy by adding an adjustable feature dimension high-dimensional mapping and fully retaining the effective information in the input features; in addition, an auxiliary classification task bypass is added in the decoding stage of the model to provide an additional supervision signal for the main segmentation task, and the segmentation accuracy of breast cancer masses is further improved while greatly reducing the computational complexity. Experimental results show that compared with the classic 3DUNet network, the Parameters and FLOPs of MM-3DUNet are reduced by 63.16% and 80.90% respectively, and the model computational complexity is greatly reduced. In terms of segmentation accuracy, the IoU of MM-3DUNet is increased by 1.30% compared with 3DUNet, which improves the accuracy of breast cancer mass segmentation while achieving model lightweight, and also improves the computational efficiency, and enhances the stability and applicability of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Breast cancer pathological tissue label according to the embodiment of the present invention: (a) breast tissue section; (b) breast cancer mass label;

[0036] Figure 2 Schematic diagram of the framework of the MM-3DUNet model of an embodiment of the present invention;

[0037] Figure 3 Schematic diagram of the structure of a channel dilation convolution unit according to an embodiment of the present invention;

[0038] Figure 4 Schematic diagram of the structure of the reverse residual structure unit according to an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the structure of the auxiliary classification task bypass according to an embodiment of the present invention;

[0040] Figure 6 Comparison of loss value change curves between DUNet and MM-3DUNet in verification in the embodiment of the present invention;

[0041] Figure 7 Comparison of IoU change curves between 3DUNet and MM-3DUNet in verification in the embodiment of the present invention;

[0042] Figure 8 This is a comparison of the breast cancer mass segmentation accuracy of the network before and after using the channel dilation convolution unit on the test set in the embodiment of the present invention, where (a) breast tissue slice; (b) breast cancer mass label; (c) 3DUNet network segmentation result; (d) MM-3DUNet network (expansion factor = 6) segmentation result; (e) MM-3DUNet network (expansion factor = 12) segmentation result, and the number in the upper left corner of the segmentation effect diagram is the IoU value;

[0043] Fig. 9 Comparison of the breast cancer mass segmentation accuracy of the network before and after using the channel dilation convolution unit on the test set in the embodiment of the present invention: (a) breast tissue slice; (b) breast cancer mass label; (c) 3DUNet network segmentation result; (d) MM-3DUNet network segmentation result; (e) MM-3DUNet+ACT network segmentation result. The number in the upper left corner of the segmentation effect diagram is the IoU value. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present invention will be described clearly and completely below in combination with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0046] The present invention will be further described below in conjunction with specific embodiments, but they are not intended to be limiting of the present invention.

[0047] The embodiment of the present invention discloses a method for segmenting a breast lump, comprising the following steps:

[0048] Step 1, obtaining medical images of breast masses of patients, preprocessing them, and annotating the preprocessed remote sensing images to produce a data set;

[0049] In this embodiment, a nuclear magnetic resonance image (RMI) of the breast of a hospital patient is obtained. The nuclear magnetic resonance image of the breast is obtained by a magnetic resonance device in the hospital, and the model is Philips Achieva 3.0T. The acquisition of image data is mainly focused on the breast to obtain detailed information on the breast structure. Some configuration parameters during acquisition include: magnetic field strength is 3.0T, TR is 2000ms, TE is 30ms, matrix size is 256*256, layer thickness is 1mm, and interval is 0.5mm. The setting of the above configuration parameters is intended to ensure the clarity and resolution of the data so as to carry out subsequent image segmentation and analysis.

[0050] In this embodiment, 219 breast tissue samples were obtained, and the number of slices contained in each sample MRI image was about 180. Professional doctors annotated the obtained samples, and the annotated target was a mask of the breast tumor area. Professional doctors accurately located the breast tumor area by viewing the nuclear magnetic resonance image, and used professional annotation tools to describe and annotate the area to generate a mask of the breast tumor. This process ensures the accuracy and reliability of the annotation, and provides a reliable data basis for subsequent medical image segmentation analysis.

[0051] The annotated images are preprocessed, including image cropping, image contrast adjustment, and Gaussian blur processing of the obtained medical images. Image cropping is to crop the original image into image blocks of the same size to increase the number of samples. The image contrast adjustment method is: set a threshold, compare the contrast parameter value of the image with the threshold, when the contrast parameter value of the image is less than the threshold, the contrast of the image increases, and when the contrast parameter value of the image is greater than the threshold, the contrast of the image decreases, thereby reducing the contrast difference of images obtained in different batches. Gaussian blur is an image smoothing method that reduces noise and details in the image by applying a Gaussian filter, which helps to remove unnecessary subtle differences and abnormal noise, thereby focusing on the main features of the image. The preprocessed images are organized into a data set and divided into a training set and a test set in a ratio of 4:3:3.

[0052] Step 2, constructing an MM-3DUNet model for segmenting breast masses, training the MM-3DUNet model using the data set, and obtaining an optimized MM-3DUNet model;

[0053] In view of the fact that the existing 3DUNet network has problems such as too large model parameters, high computing resource requirements, and severe memory consumption in practical applications, its application in resource-constrained environments and small devices is limited. This embodiment constructs a lightweight semantic segmentation network MM-3DUNet model for extracting breast cancer masses in MRI images. The model enhances the feature expression ability in the process of information flow by constructing channel dilation convolution units and reverse residual blocks; and adds an auxiliary classification task bypass at the end of the model to distinguish between lesions and normal parts, providing a beneficial supervision signal for the main task of breast cancer mass segmentation, while ensuring the accuracy of model segmentation, effectively reducing model parameters and calculations. Specifically, Figure 2 As shown, the MM-3DUNet model structure is:

[0054] The main body of the MM-3DUNet model includes a gradually shrinking downsampling path and a symmetrically expanding upsampling path, namely the encoder and decoder of the network. Among them, the encoder is used to extract the multi-level semantic features of the input image and compress the feature map size, including 4 progressive calculation modules. Each calculation module contains 2 repeated feature extraction units. Each feature extraction unit includes 2 1×1×1 convolution kernels, a channel expansion convolution unit, a reverse residual block, and a subsequent activation function. There is a maximum pooling layer at the end of the second feature extraction unit to compress the feature map size. After feature extraction in each calculation module, the number of image channels doubles and the size is halved. The decoder is used to restore the feature map size and locate key information, and finally outputs a target image of the same size as the original image. It is also composed of a progressive calculation module containing 2 repeated feature recovery units. The structure of the feature recovery unit is basically the same as that of the feature extraction unit. The difference is that a transposed convolution is used at the end of the feature recovery unit to expand the feature map size. After feature recovery in each calculation module, the number of image channels is halved and the size is doubled. The computational modules of the corresponding layers of the encoder and decoder are connected using a skip connection structure to achieve the mutual fusion of low-level semantic information and high-level semantic information. At the end of the MM-3DUNet model, a 1×1×1 convolution kernel is used to compress the number of channels of the final feature map into the number of labels, and the sigmoid function is used to convert the network prediction results into probability values ​​ranging from 0 to 1. In addition, an additional classification task bypass is added at the end of the decoder. The classification task bypass includes a 3×3 convolution kernel, a maximum pooling layer, and a linear activation layer to distinguish the input image into normal parts and lesion parts.

[0055] Channel dilated convolutional unit

[0056] Compared with low-dimensional space, in the calculation of high-dimensional space, more complex feature learning mode helps the model to mine the information hidden in the data that is difficult to capture through low-dimensional features, thereby more comprehensively describing the data features and allowing deep learning models to capture more subtle and complex relationships and patterns in the data. Therefore, this embodiment proposes a channel dilation convolution unit, which first expands the dimension of the input feature, maps the low-dimensional input features to the high-dimensional space for feature extraction calculation, improves the depth of feature extraction, and then reduces the feature dimension, reduces the difficulty of subsequent layer calculations, and enhances the model calculation efficiency. Enhance the expression ability and nonlinearity of the network without increasing a large amount of calculations and parameters. See Figure 3 ,The channel dilation convolution unit mainly includes the channel dilation module, the depth convolution module and the channel compression module.

[0057] In the channel expansion module, for a size of C in×D×H×W input features, first use a 1×1×1 convolution kernel to expand the channel of the input feature to t×C in , t is the channel expansion factor, which is used to adjust the degree of channel expansion and output a feature map with richer dimensional information, with a size of (t×C in )×D×H×W. In the deep convolution module, a 3D depth-separable convolution of size 3×3×3 is used to extract independent features for each channel, reducing the amount of model calculation while maintaining stable internal spatial structure information. The output feature map dimension is still (t×C in )×D×H×W. Finally, in the channel compression module, another 1×1×1 convolution kernel is used to reduce the number of channels of the feature map from t×C in Compress to C out , significantly reducing the amount of model calculations and parameters. It is worth noting that after each convolution calculation in the channel expansion and depth convolution stages, a combination of a LeakyReLU activation function and an Instance Norm normalization calculation is used to perform nonlinear changes in the feature map and channel normalization, enhancing the nonlinearity and stability of feature extraction. In the channel compression stage, after the feature map is convolved in a low-dimensional space, a linear activation function is used to perform linear changes in the feature map, avoiding nonlinear loss of information and retaining more original information of the image.

[0058] 3D depth-wise separable convolution

[0059] This embodiment uses 3D depth separable convolution to replace the simple 3D convolution in the classic 3DU-Net to effectively reduce the number of parameters and computational complexity in the model calculation process. 3D depth separable convolution can be decomposed into two main steps: 3D depth convolution and 3D point-by-point convolution. Among them, 3D depth convolution performs convolution operations on each input channel independently, instead of combining all input channels with each convolution kernel as in standard 3D convolution. Assume that the input feature map has C in channels, and the image size is D0×H0×W0, where D0, H0, and W0 represent the depth, height, and width of the input feature, respectively. in The size is K D ×K H ×K W The convolution kernel performs separate convolution calculations with the input on each channel, and the number of channels of the output feature map is still C in The calculation process of 3D point-by-point convolution is similar to that of standard convolution, using C out The number of channels is C inThe 1×1×1 convolution kernel combines the feature maps output by the 3D deep convolution in the channel dimension, and finally obtains a channel number C out The output feature map of size is D1×H1×W1.

[0060] In the entire 3D depth separable convolution calculation process, the network parameter quantity P 3D-ds And the amount of calculation F 3D-ds As shown below:

[0061] P 3D-ds =C in ×K D ×K H ×K W +C in ×C out (1)

[0062] F 3D-ds =D×H×W×C in ×(K D ×K H ×K W +C out ) (2)

[0063] If only standard 3D convolution operations are used to achieve the same feature extraction effect, the network parameter P 3D-standard And the amount of calculation F 3D-standard Then:

[0064] P 3D-standard =C in ×K D ×K H ×K W ×C out (3)

[0065] F 3D-standard =D×H×W×C in ×K D ×K H ×K W ×C out (4)

[0066] Compare the number of parameters and computational complexity of the 3D depthwise separable convolution proposed in this study with the standard 3D convolution operation:

[0067]

[0068]

[0069] It can be found that after using 3D depth-separable convolution, the number of network parameters and the amount of computation are only the original The number of model parameters and computational complexity are greatly reduced.

[0070] Reverse residual building block

[0071] In this embodiment, an inverse residual structure is added on the basis of the channel expansion convolution unit. Different from the traditional residual block calculation process, the input features pass through multiple convolution layers, and an identity mapping is added to connect the input features with the output features while keeping the number of channels unchanged. This embodiment adopts the opposite strategy, which is to expand the channels of the input features first, extract the features and then compress them. If the number of input and output channels is the same, the input feature map is directly added to the output feature map to form a residual connection. This improves the efficiency of information transmission in the calculation unit while retaining part of the input feature information, effectively alleviating the gradient vanishing problem and enhancing the feature representation capability. See Figure 4 , the specific implementation method is as follows: Assuming there is an input feature map with a size of H×W×C (height x width x number of channels), first use 1x1 convolution to expand the number of channels from C to tC (t is the expansion factor, such as 6), and the output size becomes H×W×tC. Then apply 3x3 depthwise separable convolution to keep the output size unchanged, that is, H×W×tC. Use 1x1 convolution again to compress the number of channels from tC back to C, and get the output size H×W×C.

[0072] Auxiliary classification task bypass

[0073] In the breast cancer mass segmentation task, we can consider making full use of the global feature differences between the lesion site and the normal site, so as to improve the accuracy and stability of breast cancer mass segmentation. Therefore, an auxiliary classification task bypass is added to the decoding end of the MM-3DUNet model. Based on the correlation between the two tasks of breast cancer mass segmentation and lesion and normal site classification, the image features extracted in the encoding stage are shared, and additional supervision signals are added from the perspective of global feature information learning. This can provide more global context information for the network decoder, promote the model to learn more generalized feature representations, avoid overfitting of the main task, and improve the performance of semantic segmentation tasks. The structure of the auxiliary classification task bypass is as follows: Figure 5 As shown in the figure, it includes a 3×3 convolution kernel, a maximum pooling layer, and a linear activation layer. After the feature map is restored and output, it is first mapped to a two-dimensional space. Then, a convolution kernel of size 3×3 is used to extract the type features of the input image. The data structure is adjusted through the pooling layer, and finally, the linear activation layer outputs the same result as the number of types.

[0074] Training of MM-3DUNet model

[0075] The training, validation, and testing processes of the model were all carried out under the deep learning framework Pytorch v1.13. The computer's CPU is a 13th Gen Intel(R) Core(TM) i5-13600K, the memory is 64GB, the GPU is an NVIDIA GeForce RTX3090, and the CUDA12.2 computing platform and cuDNN8.9.1 are used for hardware acceleration.

[0076] During training, the training round (epoch) is set to 130, and the batch size (batch size) is set to 24. Adam is used as the parameter optimizer, and the weight reduction rate is set to 0.0001 and the weight coefficient is set to 0.95. Dice Coefficient Loss (Dice Loss) is used as the loss function of the segmentation task, and Cross-Entropy (CELoss) loss is used as the loss function of the auxiliary classification task. Among them, Dice Loss calculates the loss function value of all pixels of a category as a whole, directly uses the segmentation effect evaluation index as the loss function to supervise the network, and ignores a large number of background pixels when calculating the intersection-union ratio, thereby solving the problem of imbalance between positive and negative samples, so the convergence speed is faster. The calculation formulas of DiceLoss and CE Loss are:

[0077]

[0078] In the formula, p(c i ) is the label value of the i-th pixel in category C, which can be 0 or 1; g(c i ) is the predicted probability that the i-th pixel belongs to category C; I is the intersection of the predicted value and the label value; H and W are the height and width of the feature map respectively; U is the union of the predicted value and the label value; ε is the smoothing coefficient to prevent the denominator from being 0, and is usually taken as a very small positive number. At the same time, the cosine annealing learning rate is used as the rate decay strategy to update the learning rate of the network training process. The calculation formula is as follows:

[0079]

[0080] In the formula, lr t is the learning rate of the training; lr0 is the initial learning rate, lr0=0.0001; T is the number of training times of a cosine cycle, which is the same as the epoch of the network.

[0081] Evaluation indicators

[0082] In the image semantic segmentation task, in order to comprehensively evaluate the segmentation performance of the deep learning network, it is necessary not only to calculate the accuracy of the classification of a single target pixel, but also to compare the degree of overlap between the overall segmentation result of the region and the segmentation label. Therefore, this embodiment uses the intersection over union (IoU) as the evaluation index of the MM-3DUNet model to evaluate the accuracy of the model. IoU represents the ratio of the intersection and union of the number of pixels in the predicted target area and the number of pixels in the actual labeled target area in the segmentation result, reflecting the degree of overlap between the segmentation result and the labeled result of this category. The calculation formula is as follows:

[0083]

[0084] Where TP is the number of pixels labeled as target and correctly predicted as target; FP is the number of pixels labeled as non-target but incorrectly predicted as target; TN is the number of pixels labeled as non-target and correctly predicted as non-target; FN is the number of pixels labeled as target but incorrectly predicted as non-target.

[0085] In addition, in order to evaluate the changes in the computational complexity of the improved segmentation network, the number of floating point operations (FLOPs) and the number of parameters (Parameters) are used as evaluation indicators. Among them, FLOPs represents the number of floating point operations required for the model to perform a forward propagation, which directly reflects the workload of the model during the calculation process. The higher the FLOPs value of the model, the more floating point operations it performs when calculating, which brings higher computational costs and longer calculation time. Parameters represents the total number of all parameters contained in the model, including weights and biases. It not only affects the computational complexity of the model, but also determines the amount of data and training time required for model training. Too many parameters may cause the model to overfit, while too few parameters may limit the representation ability of the model.

[0086] Step 3: Use the optimized MM-3DUNet model to segment medical images.

[0087] In order to illustrate the beneficial effects of the MM-3DUNet model of this embodiment, it is compared with the performance of the existing model. Based on the same breast cancer pathology image dataset and model parameter settings, the classic 3DUNet and the MM-3DUNet of this embodiment are trained to verify the superiority of the MM-3DUNet model in performance and computational efficiency. The loss function and IoU change process of the two on the validation set are shown in Figure 2. Figure 6 and Figure 7 As shown. Figure 6It can be seen that the loss values ​​of the 3DUNet and MM-3DUNet models gradually decrease with the increase of rounds. It is worth noting that the loss value of the MM-3DUNet model decreases significantly faster, which shows that the MM-3DUNet model has higher learning efficiency, can capture the key features in the data faster, adjust the model parameters accordingly, and can fully mine the useful information in the data. Figure 7 From the IoU curve changes of 3DUNet and MM-3DUNet, we can find that both of them gradually increase with the increase of rounds, among which the IoU of MM-3DUNet increases significantly faster, which further verifies the efficiency of the MM-3DUNet model during the training process. At the same time, the IoU curve of MM-3DUNet has a smaller fluctuation range, indicating that it can improve performance more stably and is not easily disturbed by fluctuations or noise in training data, and has better superiority and reliability in the task of breast cancer mass segmentation.

[0088] Performance of 3D Depthwise Separable Convolution

[0089] In order to verify the effectiveness of the 3D deep separable convolution proposed in this embodiment in reducing the number of network parameters and computational complexity, this embodiment uses the classic 3DUNet with simple 3D convolution as the baseline model (Baseline), and replaces the convolution unit with 3D deep separable convolution, trains and tests on the same data set, and compares the accuracy and computational complexity of the two networks on the test set. The results are shown in Table 1. As can be seen from the table, after replacing the simple 3D convolution with 3D deep separable convolution, the computational complexity of the breast cancer mass segmentation network is significantly reduced. Among them, Parameters is reduced by 1.57M compared with the Baseline, which is only 11.8% of the original, and FLOPs is reduced by 88.05G compared with the Baseline, which is only 23.5% of the original. In addition, the IoU using 3D deep separable convolution is almost unchanged compared with the Baseline, only slightly reduced by 0.97%, and still maintains a high segmentation accuracy.

[0090] The above results show that the 3D depth-separable convolution proposed in this embodiment can efficiently adapt to the complex structural characteristics of three-dimensional data and capture the spatial correlation and depth correlation in three-dimensional data. Under the premise of not significantly reducing the performance of the model, it can greatly reduce the number of floating-point operations required for the network to perform forward propagation and the amount of parameters generated, so that the model can process data faster during training and inference, achieve higher computing efficiency and lower resource consumption, and provide feasibility for the deployment of the model on resource-constrained devices such as mobile devices and embedded devices.

[0091] Table 1 Comparison of breast cancer mass segmentation performance between simple 3D convolution and 3D depthwise separable convolution

[0092]

[0093] Performance of channel-dilated convolutional units

[0094] Based on the use of 3D depth-separable convolution, this embodiment further designs a channel expansion convolution unit to help improve the accuracy of breast cancer mass feature extraction. In order to explore the effectiveness of this structure, a comparative experiment as shown in Table 2 was designed. By setting different channel expansion factors, the number of channels in the network expansion layer was adjusted to change the diversity of feature learning and compare the performance of different networks on the test set. From the results in the table, it can be seen that when the channel expansion factor is small, the network calculation complexity decreases significantly, but the segmentation accuracy decreases compared with the Baseline. After adjusting the channel expansion factor in MM-3DUNet from 6 to 12, the IoU increased by 1.46%, which is 0.49 higher than the classic 3DUNet. In terms of model calculation complexity, at this time, the Parameters of MM-3DUNet are reduced by 1.45M compared with the Baseline, and the FLOPs are reduced by 73.31G.

[0095] Table 2 Comparison of breast cancer mass segmentation performance before and after using channel dilation convolutional unit

[0096]

[0097] To further evaluate the impact of channel dilation convolutional units on segmentation performance, visual results of breast cancer mass segmentation accuracy of different models in the test set were compared. Figure 8 As shown. Figure 8 It can be seen that after directly adding channel expansion convolution, the segmentation accuracy of the model has decreased. Compared with 3DUNet, the edge details of the segmentation results are more missing, and the recognition accuracy of fuzzy areas is insufficient. After adjusting the channel expansion factor to an appropriate value, the network has significantly improved the segmentation precision of complex areas in the image compared with the Baseline, and the contour restoration of the breast cancer mass area is more precise.

[0098] Experimental results show that the channel dilation convolution unit can effectively adjust the complexity and feature extraction ability of the model, helping the model to remove redundant information while retaining key features, and obtain a more compact and effective feature representation, thereby balancing the model's expressive power and computational complexity. When used in conjunction with 3D depthwise separable convolution, it can reduce the network's computational load and improve segmentation accuracy.

[0099] Performance of bypassing auxiliary classification tasks

[0100] On the basis of the existing MM-3DUNet, an auxiliary classification task bypass was further added, and after training and verification on the same dataset, the performance of different networks on the test set was compared, as shown in Table 3. As can be seen from the table, after adding the auxiliary classification task bypass, the IoU of MM-3DUNet increased by 1.30% compared with 3DUNet, and the parameters and FLOPs were reduced by 63.16% and 80.90% respectively compared with 3DUNet. The results show that adding the auxiliary classification task bypass can provide the network with additional supervision signals about breast cancer lesions, promote the model to learn more generalized feature representations, and prevent overfitting by appropriately increasing the complexity of the network, thereby enhancing the segmentation performance of the main task.

[0101] Table 3 Comparison of breast cancer mass segmentation performance before and after adding auxiliary classification task bypass

[0102]

[0103] Fig. 9 The visual effect comparison of the segmentation before and after the network adds the auxiliary classification task bypass is given. It can be seen from the figure that after adding the auxiliary classification task bypass to the network, the network has a more refined boundary segmentation of breast cancer masses, and the segmentation result is more consistent with the actual situation. With the help of the auxiliary classification task bypass, the relationship between the target area and its surrounding pixels in space is better grasped, so the network is more accurate in segmenting the boundary of the pathological part. In addition, it can be observed from the visual effect that the network after adding the auxiliary classification task bypass performs better in dealing with complex cases, and can better deal with challenging problems such as variable mass morphology and blurred boundaries, providing a more reliable basis for the early diagnosis and treatment of breast cancer.

[0104] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of the specification of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for segmenting a breast lump, characterized in that: The following steps are involved: Step 1, obtaining medical images of breast masses of patients, preprocessing them, and annotating the preprocessed remote sensing images to produce a data set; Step 2, constructing an MM-3DUNet model for segmenting breast masses, training the MM-3DUNet model using the data set, and obtaining an optimized MM-3DUNet model; Step 3: Use the optimized MM-3DUNet model to segment breast masses in medical images.

2. The method for segmenting a breast lump according to claim 1, characterized in that: In step 1, image preprocessing includes image cropping, image contrast adjustment, and Gaussian blur processing of the obtained medical image.

3. The method for segmenting a breast lump according to claim 1, characterized in that: The MM-3DUNet model constructed in step 2 includes an encoder, a decoder, a 1×1×1 convolution kernel, a sigmoid function, and an auxiliary classification task bypass. The encoder and decoder are connected by a skip connection structure; Among them, the encoder is used to extract the multi-level semantic features of the input image and compress the feature map size; the decoder is used to restore the feature map size and locate key information, and finally output the target image of the same size as the original image. The 1×1×1 convolution kernel is used to compress the number of channels of the final feature map into the number of labels, and the sigmoid function is used to convert the prediction result into a probability value ranging from 0 to 1; the auxiliary classification task bypass is used to provide beneficial supervision signals for the main task of breast cancer mass segmentation.

4. The method for segmenting a breast lump according to claim 3, characterized in that: The encoder includes a plurality of progressive computing modules; each computing module includes a plurality of repeated feature extraction units, each feature extraction unit includes a channel dilation convolution unit, a reverse residual structure unit and an activation function; The decoder contains a progressive computation module of multiple repeated feature recovery units. The structure of the feature recovery unit is basically the same as that of the feature extraction unit. At the end of the feature recovery unit, a transposed convolution is used to expand the size of the feature map. After feature recovery in each computation module, the number of image channels is halved and the size is doubled.

5. The method for segmenting a breast lump according to claim 4, characterized in that: The channel expansion convolution unit includes a channel expansion module, a depth convolution module and a channel compression module; In the channel expansion module, for the input feature of size Cin×D×H×W, a 1×1×1 convolution kernel is used to expand the channel of the input feature to t×Cin, where t is the channel expansion factor, and the output feature map size is (t×Cin)×D×H×W; In the deep convolution module, 3D depth-separable convolution is used to extract independent features for each channel, and the output feature map dimension is still (t×Cin)×D×H×W; In the channel compression module, the number of channels of the feature map is compressed from t×Cin to Cout through another convolution kernel; Among them, after each convolution calculation in the channel expansion module and the deep convolution module, a combination of Leaky ReLU activation function and Instance Norm normalization calculation is used to perform nonlinear changes and channel normalization processing on the feature map; in the channel compression module, after the feature map is convolved in the low-dimensional space, a linear activation function is used to perform linear changes on the feature map.

6. The method for segmenting a breast lump according to claim 5, characterized in that: 3D depth-wise separable convolution includes 3D depth-wise convolution and 3D point-wise convolution; Among them, the 3D deep convolution performs convolution operations independently on each input channel. The calculation process is as follows: assuming that the input feature map has C in channels, the image size is D0×H0×W0, D0, H0 and W0 represent the depth, height and width of the input feature respectively, and C in The size is K D ×K H ×K W The convolution kernel performs separate convolution calculations with the input on each channel, and the number of channels of the output feature map is still C in ; The calculation process of 3D point-by-point convolution is: using C out The number of channels is C in The 1×1×1 convolution kernel combines the feature maps output by the 3D deep convolution in the channel dimension, and finally obtains a channel number C out The output feature map of size is D1×H1×W1.

7. The method for segmenting a breast lump according to claim 4, characterized in that: The reverse residual structure unit first expands the channels of the input feature map, extracts the features, and then compresses them. If the number of channels of the input feature map and the output feature map is the same, the input feature map is directly added to the output feature map to form a residual connection.

8. The method for segmenting a breast lump according to claim 3, characterized in that: The auxiliary classification task bypass includes a 3×3 convolution kernel, a maximum pooling layer, and a linear activation layer. The feature map is input into the auxiliary classification task bypass after the feature recovery output of the decoder. It is first mapped to a two-dimensional space, and then a convolution kernel of size 3×3 is used to extract the type features of the input image. The data structure is then adjusted through a maximum pooling layer, and finally a linear activation layer is used to output a result with the same number of types.

9. The method for segmenting a breast lump according to claim 1, characterized in that: In step 2, during the training process, Adam is used as the parameter optimizer, and the gradient descent method is used for training. During training, Dice Loss is used as the loss function for the segmentation task, and Cross Entropy CE Loss is used as the loss function for the auxiliary classification task bypass. Among them, Dice Loss takes all pixels of a category as a whole to calculate the loss function value, and directly uses the segmentation effect evaluation index as the loss function to supervise the network. The calculation formulas of Dice Loss and CE Loss are: In the formula, p(c i ) is the label value of the i-th pixel in category C, which can be 0 or 1; g(c i ) is the predicted probability that the i-th pixel belongs to category C; I is the intersection of the predicted value and the label value; H and W are the height and width of the feature map respectively; U is the union of the predicted value and the label value; ε is the smoothing coefficient.

10. A system for implementing the breast mass segmentation method according to any one of claims 1 to 9, characterized in that: include: The data acquisition and processing module is used to obtain medical images of breast masses of patients, pre-process them, and annotate the pre-processed remote sensing images to produce data sets; The model building and training module is used to build the MM-3DUNet model and train the MM-3DUNet model using the data set to obtain the optimized MM-3DUNet model; The breast mass segmentation module is used to perform breast mass segmentation on medical images using the optimized MM-3DUNet model.

Images