Breast Mass Image Segmentation Method and System Based on Pruned U-Net++

Abstract

The present invention provides a breast mass image segmentation method and system based on pruned U-Net++, which relates to the technical field of ultrasonic image segmentation. The breast mass image segmentation model of pruned U-Net++ constructed in the present invention is based on the U-Net++ network, uses skip connections to fuse the feature expressions of each branch U-Net, and introduces residual connections to replace the dense connections in the U-Net++ method, and uses this model to segment the obtained original breast ultrasound images. The pruned U-Net++ breast ultrasound image segmentation model constructed in the present invention can avoid the problem of model overfitting caused by the small data set problem in medical image data when segmenting the masses in breast ultrasound images, and improves the generalization ability of the ultrasound image breast mass segmentation model; at the same time, the scale of the model parameters is much smaller than that of the prior art, reducing the complexity of the network model, shortening the calculation time of network training, reducing the memory occupation of network training, and improving the model training efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image segmentation, and particularly relates to a method and system for segmenting breast mass images based on pruned U-Net++. Background Art

[0002] Breast ultrasound examination is one of the most commonly used methods for detecting and classifying breast abnormalities. Doctors give diagnostic opinions and classify breast diseases based on the physical signs of patients and the characteristics of breast masses in ultrasound images. However, limited by the diagnostic experience and energy of doctors, there are problems such as long time consumption and low accuracy in distinguishing breast masses in ultrasound images. With the rapid development of computer technology, computer-aided diagnosis technology has gradually participated in the medical diagnosis process. How to efficiently and accurately identify and segment masses in complex breast ultrasound images is an urgent problem to be solved currently.

[0003] Currently, methods for ultrasonic image segmentation include traditional methods based on image processing (such as fixed threshold segmentation method, histogram bimodal method, etc.) and methods based on deep learning (such as U-Net, U-Net++, etc.).

[0004] Since prior knowledge is the core of traditional methods based on image processing, however, the analysis of the characteristics of breast ultrasound images through prior knowledge has limitations. Therefore, relying solely on traditional segmentation methods cannot accurately segment ultrasound images at the pixel level. The U-Net method directly connects deep and shallow features with large differences, which will cause some shallow features to be lost, thus increasing the learning difficulty of the network. In the U-Net++ method, the combination of long and short connections is used to replace the original single long connection, which will increase the complexity of the network model and reduce the network training efficiency. In addition, there is a problem of small data sets in medical image data, which will lead to overfitting, and is not conducive to the improvement of segmentation accuracy and the generalization of the segmentation model. At the same time, the U-Net++ method does not consider pruning the redundant information related to different levels, which will lead to long calculation time and low efficiency. It can be seen that the existing technology cannot achieve efficient and accurate segmentation of breast ultrasound images in the case of small data sets. Summary of the Invention

[0005] (1) Technical Problems to be Solved

[0006] Aiming at the deficiencies of the existing technology, the present invention provides a method and system for segmenting breast mass images based on pruned U-Net++, which solves the problem that the existing technology cannot efficiently and accurately segment breast ultrasound images in the case of small data sets.

[0007] (2) Technical Solutions

[0008] To achieve the above object, the present invention is realized through the following technical solutions:

[0009] In the first aspect, the present invention first proposes a method for segmenting breast mass images based on a pruned U-Net++. The method includes:

[0010] Obtain the original breast ultrasound image and preprocess the original breast ultrasound image;

[0011] Construct a pruned U-Net++ breast ultrasound image segmentation model based on the U-Net++ network; the pruned U-Net++ breast ultrasound image segmentation model includes: fusing the feature expressions of each branch U-Net through skip connections, and introducing residual connections to replace the dense connections in the U-Net++ network;

[0012] Segment the original breast ultrasound image based on the pruned U-Net++ breast ultrasound image segmentation model.

[0013] Preferably, the preprocessing of the original breast ultrasound image includes:

[0014] S11. For the original breast ultrasound gray-scale image in DICOM format, convert the three-channel gray-scale image into a single-channel gray-scale image, and save the original breast ultrasound image in DICOM format as a PNG format breast ultrasound image without changing the image resolution;

[0015] S12. Uniformly crop the original breast ultrasound image in PNG format, and uniformly scale the cropped original breast ultrasound image in PNG format to the same size;

[0016] S13. Normalize the pixel value matrix of the original breast ultrasound image in PNG format with unified size;

[0017] S14. Randomly divide the breast ultrasound image data set after normalization processing into a training set and a test set according to a certain ratio.

[0018] Preferably, the method further includes: training the pruned U-Net++ breast ultrasound image segmentation model after the step S2 and before the step S3.

[0019] Preferably, the model training includes:

[0020] S21. Initialize the parameters of the pruned U-Net++ breast ultrasound image segmentation model;

[0021] S22. Organize the training set data to train the pruned U-Net++ breast ultrasound image segmentation model and obtain the model prediction value;

[0022] S23. Determine the loss value through the loss function based on the model prediction value and the true value;

[0023] S24. Based on the backpropagation method, optimize the parameters of the pruned U-Net++ breast ultrasound image segmentation model by the stochastic gradient descent method to obtain the optimal parameters.

[0024] Preferably, the loss function includes the BCEWithLogitsLoss function.

[0025] In a second aspect, the present invention also proposes a breast mass image segmentation system based on pruned U-Net++, and the system includes:

[0026] An image acquisition and processing module, configured to acquire an original breast ultrasound image and preprocess the original breast ultrasound image;

[0027] A model construction module, configured to construct a pruned U-Net++ breast ultrasound image segmentation model based on the U-Net++ network; the pruned U-Net++ breast ultrasound image segmentation model includes: fusing the feature expressions of each branch U-Net through skip connections, and at the same time introducing residual connections to replace the dense connections in the U-Net++ network;

[0028] An image segmentation module, configured to segment the original breast ultrasound image based on the pruned U-Net++ breast ultrasound image segmentation model.

[0029] Preferably, the preprocessing of the original breast ultrasound image by the image acquisition and processing module includes:

[0030] S11. For the original breast ultrasound grayscale image in DICOM format, convert the three-channel grayscale image into a single-channel grayscale image, and save the original breast ultrasound image in DICOM format as a PNG format breast ultrasound image without changing the image resolution;

[0031] S12. Uniformly crop the original breast ultrasound image in PNG format, and uniformly scale the cropped original breast ultrasound image in PNG format to the same size;

[0032] S13. Normalize the pixel value matrix of the original breast ultrasound image in PNG format with unified size;

[0033] S14. Randomly divide the breast ultrasound image dataset after normalization processing into a training set and a test set according to a certain ratio.

[0034] Preferably, the system further includes a model training module, configured to perform model training on the pruned U-Net++ breast ultrasound image segmentation model after the construction of the pruned U-Net++ breast ultrasound image segmentation model and before the breast ultrasound image segmentation.

[0035] Preferably, when the model training module conducts model training, it includes:

[0036] S21. Initialize the parameters of the pruned U-Net++ breast ultrasound image segmentation model;

[0037] S22. Use the training set data to train the pruned U-Net++ breast ultrasound image segmentation model and obtain the model prediction values;

[0038] S23. Determine the loss value based on the model prediction values and the true values through a loss function;

[0039] S24. Based on the backpropagation method, optimize the parameters of the pruned U-Net++ breast ultrasound image segmentation model through the stochastic gradient descent method to obtain the optimal parameters.

[0040] Preferably, the loss function includes the BCEWithLogitsLoss function.

[0041] (III) Beneficial effects

[0042] The present invention provides a breast mass image segmentation method and system based on pruned U-Net++. Compared with the prior art, it has the following beneficial effects:

[0043] 1. The breast mass image segmentation method and system based on pruned U-Net++ of the present invention are based on the U-Net++ network, use skip connections to fuse the feature expressions of each branch U-Net, and introduce residual connections to replace the dense connections in the U-Net++ method to construct a pruned U-Net++ breast ultrasound image segmentation model, and then use this model to segment the obtained original breast ultrasound images. The pruned U-Net++ breast ultrasound image segmentation model constructed in the technical solution of the present invention can avoid the problem of model overfitting caused by the small data set problem in medical image data when segmenting the masses in breast ultrasound images, and improve the generalization ability of the ultrasound image breast mass segmentation model; at the same time, the parameter scale of the pruned U-Net++ breast ultrasound image segmentation model constructed by this technical solution is much smaller than that of the prior art, which directly reduces the complexity of the network model, shortens the calculation time of network training to a certain extent, reduces the memory occupation of network training, and improves the model training efficiency.

[0044] 2. In the present invention, the skip connection in the form of feature superposition is used to realize the mapping from shallow features to deep features, fuse the shallow and deep features between the same level and different levels of each sub-network, make up for the defect of the semantic gap of feature connection existing in the U-Net method in the prior art, can better extract image features from the model, and improve the segmentation effect of breast masses in ultrasound images.

[0045] 3. In the present invention, residual connections are adopted in the intermediate layer of the sub-network, and the output feature map is only concatenated with the features of the previous layer at the channel dimension. Compared with the dense connections in the U-Net++ network, this process can effectively consider the context information of the feature map while reducing the redundant information in the channel dimension of the network feature map, so as to capture the deep features of the grabbed image while taking into account the pruning optimization of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0047] Figure 1 It is a flowchart of the method for segmenting breast mass images based on pruned U-Net++ in the embodiments of the present invention;

[0048] Figure 2 It is a network architecture diagram of the pruned U-Net++ breast ultrasound image segmentation model in the embodiments of the present invention;

[0049] Figure 3 It is a result diagram after segmentation by the pruned U-Net++ breast ultrasound image segmentation model in the embodiments of the present invention;

[0050] Figure 4 It is a comparison diagram of the segmentation results of the method for segmenting breast ultrasound images based on pruned U-Net++ in the embodiments of the present invention and two existing segmentation methods. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0052] By providing a method and system for segmenting breast mass images based on pruned U-Net++, the embodiments of the present application solve the problem that the prior art cannot efficiently and accurately segment breast ultrasound images in the case of a small data set, and realize the problem of improving the accuracy and efficiency of mass segmentation in breast ultrasound images in the case of a small data set.

[0053] The technical solutions in the embodiments of the present application for solving the above technical problems are generally as follows:

[0054] In order to overcome the problem that the existing breast ultrasound images have overfitting in the segmentation of breast mass images due to a small data set, resulting in low accuracy of the breast ultrasound image segmentation results and poor generalization ability of the segmentation model, and at the same time overcome the problems in the existing technology that the breast ultrasound image segmentation model network is complex, resulting in large computational amount, long time and low efficiency during segmentation, the present invention uses skip connections to fuse the feature expressions of each branch of the U-Net, and introduces residual connections to replace the dense connections in the U-Net++ method. Based on the deep supervision method, the training accuracy and speed at different network depths are balanced. Based on the network pruning method, the redundant information of the feature map channels between the U-Net++ network levels is reduced, thereby constructing a pruned U-Net++ breast ultrasound image segmentation model. Based on this model, the original breast ultrasound image is segmented, which is more efficient and accurate than the existing technology.

[0055] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0056] Example 1:

[0057] In the first aspect, referring to Figure 1 , the present invention first proposes a method for segmenting breast mass images based on pruned U-Net++, and the method includes:

[0058] S1. Obtain the original breast ultrasound image and preprocess the original breast ultrasound image;

[0059] S2. Construct a pruned U-Net++ breast ultrasound image segmentation model based on the U-Net++ network; the pruned U-Net++ breast ultrasound image segmentation model includes: fusing the feature expressions of each branch of the U-Net through skip connections, and at the same time introducing residual connections to replace the dense connections in the U-Net++ network;

[0060] S3. Segment the original breast ultrasound image based on the pruned U-Net++ breast ultrasound image segmentation model.

[0061] It can be seen that the breast mass image segmentation method and system based on pruned U-Net++ in this embodiment are based on the U-Net++ network, use skip connections to fuse the feature expressions of each branch U-Net, and introduce residual connections to replace the dense connections in the U-Net++ method to construct a pruned U-Net++ breast ultrasound image segmentation model, and then use this model to segment the acquired original breast ultrasound image. In the technical solution of the present invention, the constructed pruned U-Net++ breast ultrasound image segmentation model can avoid the problem of model overfitting caused by the small data set problem in medical image data when segmenting the mass in the breast ultrasound image, and improve the generalization ability of the ultrasound image breast mass segmentation model; at the same time, the parameter scale of the pruned U-Net++ breast ultrasound image segmentation model constructed by this technical solution is much smaller than that of the prior art, which directly reduces the complexity of the network model, shortens the calculation time of network training to a certain extent, reduces the memory occupation of network training, and improves the model training efficiency.

[0062] Next, we take the breast ultrasound image data set collected from a top-three hospital as an example. Refer to Figures 1-4 and combine the explanation of the specific steps to detail the implementation process of an embodiment of the present invention.

[0063] S1. Obtain the original breast ultrasound image and preprocess the original breast ultrasound image.

[0064] 1) Obtain the original breast ultrasound image.

[0065] To ensure the authenticity and reliability of the data set, and at the same time verify the universality and generalization ability of the pruned U-Net++ breast mass image segmentation model in this technical solution, this embodiment takes the breast ultrasound image data set collected from a top-three hospital in Hefei, Anhui, China as an example. The images in the data set are collected from different patients by different B-ultrasound machines. Each patient may correspond to multiple breast ultrasound images, including gray-scale images in different directions such as transverse and longitudinal cuts. For the breast ultrasound images collected by a given hospital, the contour of the breast mass is accurately outlined by experienced clinical doctors in the ultrasound department for model training and testing.

[0066] 2) Preprocess the obtained original breast ultrasound image.

[0067] To accelerate the network model training speed and improve the accuracy of the segmentation result, it is necessary to perform standardized normalization processing on the original breast ultrasound image data set with the DICOM format and a resolution as high as 1260×910 pixels. The specific operations are as follows:

[0068] S11. For the original DICOM - format breast ultrasound grayscale image, convert the three - channel grayscale image into a single - channel grayscale image, and save the original DICOM - format breast ultrasound image as a PNG - format breast ultrasound image without changing the image resolution.

[0069] Since the region of interest in the breast ultrasound image is a grayscale image, converting the three - channel grayscale image into a single - channel grayscale image will not lose image feature information and can reduce the image channel dimension. Therefore, for the original DICOM - format breast ultrasound grayscale image, to simplify the model training process, we convert the three - channel grayscale image into a single - channel grayscale image. At the same time, to reduce the image file size to speed up the network training speed, we save the original DICOM - format breast ultrasound image as a PNG image format while keeping the image resolution unchanged and retaining the image feature information.

[0070] S12. Uniformly crop the original PNG - format breast ultrasound image, and uniformly scale the cropped original PNG - format breast ultrasound image to the same size.

[0071] By uniformly cropping the original PNG - format breast ultrasound image, the influence of irrelevant noise regions is reduced, so that the target lesion region of interest can be focused, and at the same time, patient - sensitive information can be deleted. In addition, to facilitate the setting of training parameters for the network model, the cropped images are uniformly scaled to the same size.

[0072] S13. Normalize the pixel - value matrix of the original PNG - format breast ultrasound image.

[0073] Since the pixel - value matrix of the breast image takes values in the range of 0 - 255, it will reduce the learning efficiency of the network to a certain extent. To achieve unified probability calculation between [0,1] and speed up the learning speed of the network model, the pixel - value matrix of the breast ultrasound image is normalized and unified to the interval of [0,1].

[0074] S14. Randomly divide the normalized breast ultrasound image dataset into a training set and a test set according to a certain ratio.

[0075] Randomly divide the breast ultrasound image dataset into a training set and a test set according to a certain ratio. Among them, both the test set and the training set have labels for subsequent comparison of the segmentation accuracy during model testing. Generally, we randomly divide it into a training set and a test set according to a ratio of 4:1.

[0076] S2. Construct a pruned U-Net++ breast ultrasound image segmentation model based on the U-Net++ network; the pruned U-Net++ breast ultrasound image segmentation model includes: fusing the feature expressions of each branch U-Net through skip connections, and at the same time introducing residual connections to replace the dense connections in the U-Net++ network.

[0077] In this embodiment, a convolutional neural network (CNN) method is used to construct a pruned U-Net++ breast ultrasound image segmentation model based on the U-Net++ network. The pruned U-Net++ breast ultrasound image segmentation model constructed in this embodiment is based on the U-Net++ network. Through skip connections, the feature expressions of each branch U-Net are fused, and at the same time, residual connections are introduced to replace the dense connections in the U-Net++ network. Specifically, the network structure of the pruned U-Net++ breast ultrasound image segmentation model constructed in this embodiment is as Figure 2 shown, which mainly includes an input layer, a hidden layer, and an output layer.

[0078] 1) The backbone network of the pruned U-Net++ breast ultrasound image segmentation model.

[0079] The pruned U-Net++ breast ultrasound image segmentation model proposed in this embodiment is based on the core idea of the encoder-decoder structure. The backbone network follows the symmetric network architecture of the encoder-decoder of the U-Net method, taking into account the extraction of deep features and the recovery of shallow features. The pruned U-Net++ breast ultrasound image segmentation model in this embodiment provides four downsampling layers in the encoding path and four upsampling layers in the decoding path to fully extract the deep and shallow features of the image. Among them, each downsampling layer includes a convolutional layer and a pooling layer. Two cascaded 3×3 convolutional layers ensure fewer parameters under the premise of the same receptive field of the convolutional kernel, and a max pooling layer ensures retaining the most significant features of the image while reducing the model size; each upsampling layer consists of a 2×2 transposed convolutional layer, which is used to restore the feature map extracted by the downsampling layer to the resolution of the original image; after each layer of operation, the ReLU function with low computational complexity and good convergence effect is used as the activation function to realize the mapping of the input data from linear to non-linear.

[0080] 2) Perform pruning operations on the network.

[0081] Based on the ideas of the above backbone network and the U-Net++ method, in this embodiment, to retain the features extracted at each level, an upsampling operation is performed at each downsampling layer, which solves the problem of ignoring simple shallow features and overall grasps the deep and shallow features of the image data. The deep and shallow features at the same level are superimposed through skip connections, realizing the recovery of edge features lost due to the increase in the number of network layers. By comprehensively combining long connections and short connections, the features between different levels are superimposed, narrowing the semantic gap between directly combining deep and shallow features with large differences and improving the effect of restoring the original image size in the decoding path. Among them, in this embodiment, residual connections are adopted in the middle layer of the sub-network, and the output feature map is only concatenated with the features of the previous layer at the channel dimension. Compared with the dense connections in the U-Net++ network, this process can reduce the redundant information in the channel dimension of the network feature map while effectively considering the context information of the feature map, so as to achieve grasping the deep features of the image while taking into account the pruning optimization of the network.

[0082] To ensure that the accuracy and segmentation efficiency of the pruned U-Net++ breast ultrasound image segmentation model constructed in this embodiment can reach the expected effect during segmentation, it is necessary to train the constructed pruned U-Net++ breast ultrasound image segmentation model. When the pruned U-Net++ breast ultrasound image segmentation model is actually trained, it mainly includes the following steps:

[0083] S21. Initialize the parameters of the pruned U-Net++ breast ultrasound image segmentation model.

[0084] Randomly initialize the parameters of the pruned U-Net++ breast ultrasound image segmentation model.

[0085] S22. Organize the training set data to train the pruned U-Net++ breast ultrasound image segmentation model and obtain the model prediction values.

[0086] All the data (with labels) in the training set are divided into multiple batches and input into the constructed pruned U-Net++ breast ultrasound image segmentation model batch by batch, and the prediction values are calculated through forward propagation, that is, the pruned U-Net++ model performs binary classification on all pixel points shown in the image to obtain the prediction values.

[0087] S23. Determine the loss value based on the model prediction values and the true values through the selected loss function.

[0088] Considering that the essence of image semantic segmentation is a pixel classification problem, that is, a binary classification is performed on all pixel points shown in the image. Therefore, the identification of breast mass regions essentially belongs to a binary classification task, and the BCEWithLogitsLoss function can be used to calculate the binary cross-entropy. Specifically, the gap between the predicted value and the true value is compared, that is, the loss value of each pixel point is calculated by combining the label value and the predicted value, and the average value of the loss values of all pixel points is used as the loss value of the corresponding breast ultrasound image.

[0089] S24. Based on the backpropagation method, the parameters of the pruned U-Net++ breast ultrasound image segmentation model are optimized by the stochastic gradient descent method to obtain the optimal parameters.

[0090] During the training process, the function value of the loss function can be changed by adjusting the parameter values to obtain a model with better performance. The idea is to make the loss function value tend to the minimum value at the end of the model training. At this time, the model is considered to be the best-performing model. Therefore, the process of adjusting the parameters of network training is transformed into the process of minimizing the loss function. We determine the gradient vector through backpropagation and update the gradient, that is, adjust the parameters through the gradient vector, and determine the amplitude of each parameter update through the adaptive learning rate, so that the loss value tends to the minimum value. After multiple rounds of mini-batch training, the optimal parameters of the pruned U-Net++ breast ultrasound image segmentation model can be determined. Specifically,

[0091] This embodiment follows the RMSprop optimization algorithm, which is the adaptive learning rate algorithm used in the U-Net and U-Net++ methods. Assume that the batch processes m samples {x(1), …, x(m)} collected from the training set, and the corresponding segmentation results are y(i), and the gradient of the loss function is The cumulative squared gradient is r ← ρr + (1 - ρ)g ⊙ g, then the update of the sample value is as follows:

[0092]

[0093] where ε represents the global learning rate, ρ represents the decay rate, θ represents the initial parameter, δ = 10 -6 , and r represents the initialized cumulative variable.

[0094] Repeat each epoch until the average value of the loss function no longer decreases (or decreases to the lowest point), and then stop training. The parameters obtained at this time are the best parameters obtained after model training.

[0095] S3. Segment the original breast ultrasound image based on the pruned U-Net++ breast ultrasound image segmentation model.

[0096] Based on the pruning U-Net++ breast ultrasound image segmentation model constructed and trained according to the above steps, efficient and accurate segmentation of the original breast ultrasound images can be achieved. As Figure 3 shown, the segmentation results obtained by using the pruning U-Net++ breast ultrasound image segmentation model in this embodiment are presented. The figure shows the segmentation results of 3 breast ultrasound images processed by the pruning U-Net++ method. From Figure 3 (a)- Figure 3 (c) are respectively the breast ultrasound images after standard normalization, the annotation results of clinical ultrasound doctors for breast masses, and the segmentation results of the pruning U-Net++ method for breast masses.

[0097] To verify the effectiveness of the method proposed in this embodiment for segmenting breast ultrasound images, the segmentation results of the original U-Net-based method and U-Net++-based method are compared with the segmentation results of this method. The experimental results are as Figure 4 shown. It can be seen from the figure that the segmentation results of using the pruning U-Net++ method for breast ultrasound images are closer to the doctor's annotation results, and the outlined contour of the breast mass is smoother, and the processing of details is also more ideal, which qualitatively verifies the effectiveness of the method proposed in this embodiment.

[0098] Under the same training set and test set, the segmentation effects of the original two technical methods and the method proposed in this embodiment on the training set are not very different. To quantitatively illustrate the effectiveness of the method proposed in this patent, the segmentation results of the method proposed in this embodiment are compared with the original U-Net / U-Net++ method on the same test set.

[0099] The evaluation indicators of the breast ultrasound image segmentation model based on the pruning U-Net++ method mainly include quantitative accuracy, visual quality, segmentation efficiency (inference time), and model complexity (parameter scale), etc. Among them, the Mean Intersection over Union and Variance of Intersection over Union, which are used to quantify the accuracy of the model, are the most important.

[0100] The Intersection over Union (IoU) is a dedicated evaluation criterion for image semantic segmentation, which is used to measure the closeness between the model segmentation result and the doctor's annotation result. It means the ratio of the overlapping area between the model segmentation result and the doctor's annotation result to the union area of the model segmentation result and the doctor's annotation result. Specifically, the larger the value of the Intersection over Union, the better the segmentation result of the breast ultrasound image;

[0101] The mean Intersection over Union (mIoU) is the average of the Intersection over Union for all classes and is used to measure the overall performance of the model's segmentation. It is generally calculated for different classes, where the Intersection over Union for each class is calculated and then accumulated and averaged.

[0102] The variance of Intersection over Union is the variance of the Intersection over Union for all classes and is used to measure the stability of the model's segmentation. Among them, taking the arithmetic square root of the variance of Intersection over Union gives the standard deviation of Intersection over Union.

[0103] On the same test set, the Intersection over Union of the segmentation results of the method proposed in this embodiment and the original U-Net / U-Net++ methods were compared, and the experimental results are shown in Table 1.

[0104] Table 1 Comparison results of IoU with the prior art methods

[0105]

[0106] As can be seen from the table, the mean Intersection over Union of the image segmentation of the models trained using the original U-Net method and the improved U-Net method (U-Net++) is relatively low. After appropriately pruning the U-Net++ method by replacing the dense connection with a residual connection, the mean Intersection over Union of the image segmentation of the trained model is improved, and the distribution of the Intersection over Union of different test image data is relatively concentrated, indicating that the method proposed in this embodiment can improve the accuracy of image segmentation. In addition, the segmentation effect of the pruned U-Net++ method on the test set is better than the original two technical methods, indicating that the method proposed in this patent can more effectively address the problem of overfitting of the network model.

[0107] To further verify the advantages of the method proposed in this embodiment in terms of training efficiency, the parameter scales of the method proposed in this embodiment and the original method were compared under the same test set, as shown in Table 2. Among them, the parameter scale of the pruned U-Net++ method is much smaller than that of the prior art methods, indicating that the method proposed in this embodiment can effectively reduce the complexity of the network model, reduce the computational amount of model training, and improve the efficiency of model training.

[0108] Table 2 Comparison results of parameter scales with the prior art methods

[0109]

[0110] Thus, the entire process of a method for breast mass image segmentation based on pruned U-Net++ of the present invention is completed.

[0111] Embodiment 2:

[0112] Second aspect, the present invention also provides a breast mass image segmentation system based on pruned U-Net++. The system includes:

[0113] An image acquisition and processing module, configured to acquire an original breast ultrasound image and preprocess the original breast ultrasound image;

[0114] A model construction module, configured to construct a pruned U-Net++ breast ultrasound image segmentation model based on the U-Net++ network; the pruned U-Net++ breast ultrasound image segmentation model includes: fusing the feature expressions of each branch U-Net through skip connections, and introducing residual connections to replace the dense connections in the U-Net++ network;

[0115] An image segmentation module, configured to segment the original breast ultrasound image based on the pruned U-Net++ breast ultrasound image segmentation model.

[0116] Optionally, the preprocessing of the original breast ultrasound image by the image acquisition and processing module includes:

[0117] S11. For the original breast ultrasound gray-scale image in DICOM format, convert the three-channel gray-scale image into a single-channel gray-scale image, and save the original breast ultrasound image in DICOM format as a PNG format breast ultrasound image without changing the image resolution;

[0118] S12. Uniformly crop the original breast ultrasound image in PNG format, and uniformly scale the cropped original breast ultrasound image in PNG format to the same size;

[0119] S13. Normalize the pixel value matrix of the original breast ultrasound image in PNG format with unified size;

[0120] S14. Randomly divide the breast ultrasound image dataset after normalization processing into a training set and a test set according to a certain ratio.

[0121] Optionally, the system further includes a model training module, configured to perform model training on the pruned U-Net++ breast ultrasound image segmentation model after the construction of the pruned U-Net++ breast ultrasound image segmentation model and before the breast ultrasound image segmentation.

[0122] Optionally, when the model training module performs model training, it includes:

[0123] S21. Initialize the parameters of the pruned U-Net++ breast ultrasound image segmentation model;

[0124] S22. Organize the training set data to train the pruned U-Net++ breast ultrasound image segmentation model and obtain the model prediction value;

[0125] S23. Determine the loss value based on the model prediction value and the true value through a loss function;

[0126] S24. Based on the backpropagation method, optimize the parameters of the pruning U-Net++ breast ultrasound image segmentation model through the stochastic gradient descent method to obtain the optimal parameters.

[0127] Optionally, the loss function includes the BCEWithLogitsLoss function.

[0128] It can be understood that the breast mass image segmentation system based on pruning U-Net++ provided by the embodiments of the present invention corresponds to the above-mentioned breast mass image segmentation method based on pruning U-Net++. For the explanations, examples, beneficial effects, etc. of the relevant content, reference can be made to the corresponding content in the breast mass image segmentation method based on pruning U-Net++, and details will not be repeated here.

[0129] In summary, compared with the prior art, the following beneficial effects are achieved:

[0130] 1. The breast mass image segmentation method and system based on pruning U-Net++ of the present invention are based on the U-Net++ network, use skip connections to fuse the feature expressions of each branch U-Net, and introduce residual connections to replace the dense connections in the U-Net++ method to construct a pruning U-Net++ breast ultrasound image segmentation model, and then use this model to segment the obtained original breast ultrasound image. The pruning U-Net++ breast ultrasound image segmentation model constructed in the technical solution of the present invention can avoid the problem of model overfitting caused by the small dataset problem in medical image data when segmenting the mass in the breast ultrasound image, and improve the generalization ability of the ultrasound image breast mass segmentation model; at the same time, the parameter scale of the pruning U-Net++ breast ultrasound image segmentation model constructed by this technical solution is much smaller than that of the prior art, which directly reduces the complexity of the network model, shortens the calculation time of network training to a certain extent, reduces the memory occupation of network training, and improves the model training efficiency.

[0131] 2. In the present invention, the mapping from shallow features to deep features is realized through skip connections in the form of feature superposition, and the shallow and deep features at the same level and different levels of each sub-network are fused, making up for the defect of the semantic gap in feature connection existing in the U-Net method in the prior art, and can better extract image features from the model, improving the segmentation effect of the breast mass in the ultrasound image.

[0132] 3. In the present invention, a residual connection is adopted in the middle layer of the sub-network, and the output feature map is only concatenated with the features of the previous layer at the channel dimension. Compared with the dense connection in the U-Net++ network, this process can effectively consider the context information of the feature map while reducing the redundant information of the channel dimension of the network feature map, so as to achieve grasping the deep features of the image while taking into account the pruning optimization of the network.

[0133] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0134] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or equivalently replace some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A breast mass image segmentation method based on pruning U-Net++, characterized in that The method includes: S1. Obtain the original breast ultrasound image and preprocess the original breast ultrasound image; S2. Construct a pruned U-Net++ breast ultrasound image segmentation model based on the U-Net++ network; the pruned U-Net++ breast ultrasound image segmentation model includes: fusing the feature expressions of each branch U-Net through skip connections, introducing residual connections to replace the dense connections in the U-Net++ network, weighing the training accuracy and speed under different network features based on the deep supervision method, and reducing the redundant information of the feature map channels between the levels of the U-Net++ network based on the network pruning method; S3. Segment the original breast ultrasound image based on the pruned U-Net++ breast ultrasound image segmentation model; The pruned U-Net++ breast ultrasound image segmentation model provides four downsampling layers in the encoding path and four upsampling layers in the decoding path; among them, each downsampling layer includes a convolutional layer and a pooling layer, and two cascaded 3×3 convolutional layers ensure fewer parameters under the premise of the same receptive field of the convolutional kernel, and a max pooling layer ensures retaining the most significant features of the image while reducing the model scale; each upsampling layer consists of a 2×2 transposed convolutional layer, which is used to restore the feature map extracted by the downsampling layer to the resolution of the original image; The fusing of the feature expressions of each branch U-Net through skip connections includes: Realize the mapping from shallow features to deep features through skip connections in the form of feature superposition, and fuse the shallow and deep features at the same level and different levels of each sub-network; comprehensively stack the features between different levels by combining long connections and short connections; The introducing of residual connections to replace the dense connections in the U-Net++ network includes: Adopt residual connections in the middle layer of the sub-network, and cascade the output feature map only with the feature of the previous layer at the channel dimension, so as to consider the context information of the feature map while reducing the redundant information of the channel dimension of the network feature map, capture the deep features of the image, and take into account the pruning optimization of the network.

2. The method according to claim 1, characterized in that The preprocessing of the original breast ultrasound image includes: S11. For the original breast ultrasound grayscale image in DICOM format, convert the three-channel grayscale image into a single-channel grayscale image, and save the original breast ultrasound image in DICOM format as a PNG format breast ultrasound image without changing the image resolution; S12. Uniformly crop the original breast ultrasound image in PNG format, and uniformly scale the cropped original breast ultrasound image in PNG format to the same size; S13. Normalize the pixel value matrix of the original breast ultrasound image in PNG format with unified size; S14. Randomly divide the breast ultrasound image dataset after normalization processing into a training set and a test set according to a certain ratio.

3. The method according to claim 1, characterized in that, The method further includes: training the pruned U-Net++ breast ultrasound image segmentation model after the S2 step and before the S3 step.

4. The method according to claim 3, wherein The model training includes: S21. Initialize the parameters of the pruned U-Net++ breast ultrasound image segmentation model; S22. Train the pruned U-Net++ breast ultrasound image segmentation model with the training set data and obtain the model prediction values; S23. Determine the loss value based on the model prediction values and the ground truth values through a loss function; S24. Based on the backpropagation method, optimize the parameters of the pruned U-Net++ breast ultrasound image segmentation model by the stochastic gradient descent method to obtain the optimal parameters.

5. The method according to claim 4, wherein The loss function includes the BCEWithLogitsLoss function.

6. A breast mass image segmentation system based on pruning U-Net++, characterized in that, The system includes: An image acquisition and processing module for acquiring the original breast ultrasound image and preprocessing the original breast ultrasound image; A model construction module for constructing a pruned U-Net++ breast ultrasound image segmentation model based on the U-Net++ network; the pruned U-Net++ breast ultrasound image segmentation model includes: fusing the feature expressions of each branch U-Net through skip connections, and at the same time introducing residual connections to replace the dense connections in the U-Net++ network, weighing the training accuracy and speed under different network features based on the deep supervision method, and reducing the redundant information of the feature map channels between the levels of the U-Net++ network based on the network pruning method; An image segmentation module for segmenting the original breast ultrasound image based on the pruned U-Net++ breast ultrasound image segmentation model; The pruned U-Net++ breast ultrasound image segmentation model provides four downsampling layers in the encoding path and four upsampling layers in the decoding path; among them, each downsampling layer includes a convolutional layer and a pooling layer. Two cascaded 3×3 convolutional layers ensure fewer parameters under the premise of having the same receptive field of the convolutional kernel, and a max pooling layer ensures retaining the most significant features of the image while reducing the model scale; each upsampling layer consists of a 2×2 transposed convolutional layer for restoring the feature map extracted by the downsampling layer to the resolution of the original image; The fusing of the feature expressions of each branch U-Net through skip connections includes: Realizing the mapping from shallow features to deep features through skip connections in the form of feature stacking, fusing the shallow and deep features of the same level and different levels of each sub-network; comprehensively stacking the features between different levels through long connections and short connections; The introducing of residual connections to replace the dense connections in the U-Net++ network includes: Adopting residual connections in the middle layer of the sub-network, and concatenating the output feature map with the feature of only the previous layer at the channel dimension, so as to consider the context information of the feature map while reducing the redundant information of the channel dimension of the network feature map, grasping the deep features of the image, and taking into account the pruning optimization of the network.

7. The system according to claim 6, wherein The preprocessing of the original breast ultrasound image by the image acquisition and processing module includes: S11. For the original DICOM format breast ultrasound grayscale image, convert the three-channel grayscale image into a single-channel grayscale image, and save the original DICOM format breast ultrasound image as a PNG format breast ultrasound image without changing the image resolution; S12. Uniformly crop the original PNG format breast ultrasound image, and uniformly scale the cropped original PNG format breast ultrasound image to the same size; S13. Normalize the pixel value matrix of the original PNG format breast ultrasound image after size unification; S14. Randomly divide the breast ultrasound image dataset after normalization processing into a training set and a test set according to a certain ratio.

8. The system according to claim 6, wherein The system further includes a model training module for training the pruned U-Net++ breast ultrasound image segmentation model after the construction of the pruned U-Net++ breast ultrasound image segmentation model and before breast ultrasound image segmentation.

9. The system according to claim 8, wherein When the model training module conducts model training, it includes: S21. Initialize the parameters of the pruned U-Net++ breast ultrasound image segmentation model; S22. Organize the training set data to train the pruned U-Net++ breast ultrasound image segmentation model and obtain model prediction values; S23. Determine the loss value based on the model prediction values and the true values through a loss function; S24. Optimize the parameters of the pruned U-Net++ breast ultrasound image segmentation model based on the backpropagation method and the stochastic gradient descent method to obtain the optimal parameters.

10. The system according to claim 9, wherein, The loss function includes the BCEWithLogitsLoss function.

Images