Breast mass segmentation network and method, computing device, and storage medium
By fixing the number of feature map channels in the breast mass segmentation network and increasing regularization terms, and simplifying the feature fusion method, the problem of breast mass segmentation algorithm relies on artificial knowledge and deep learning models in the prior art is solved, and efficient and accurate breast mass segmentation is achieved.
Patent Information
- Application Number
- CN202111131635.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-26
- Publication Date
- 2025-05-06
- Estimated Expiration
- 2041-09-26
AI Technical Summary
The existing breast mass segmentation algorithm relies on artificial prior knowledge, has poor generalization ability, and is difficult to achieve efficient and automated segmentation. In addition, deep learning models such as UNet and UNet3+ have a long model inference time due to the increase in parameter quantity and calculation quantity.
A breast mass segmentation network is proposed. By fixing the number of channels of the feature map in the encoder and increasing he_normal and L2 regularization, the feature fusion method in the decoder is simplified, thereby reducing network complexity and parameter amount, improving model performance and alleviating overfitting.
It realizes the reduction of the complexity and parameter volume of segmented networks, improves network performance and inference speed, reduces the overfitting phenomenon of the model, and improves the efficiency and accuracy of breast mass segmentation.
Smart Images
Figure CN114170238B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a breast mass segmentation network, method, computer equipment, and storable medium. Background Art
[0002] There are many methods for early diagnosis of breast cancer, among which mammography (also known as breast molybdenum target X-ray) is considered to be the most reliable and effective method.
[0003] like Figure 1 The following are some X-ray images of breast masses. The highlighted outlines are gold standard outlines. However, it is not easy for radiologists to observe breast X-rays to diagnose patients. Manual diagnosis of X-rays causes radiologists to have too much work, which in turn leads to a decrease in diagnostic accuracy. Therefore, it is very meaningful to adopt low-latency and automated mass area segmentation for breast X-rays.
[0004] Traditional breast mass segmentation algorithms mainly include manual segmentation, semi-automatic segmentation, and automatic segmentation, etc. These algorithms rely heavily on human prior knowledge, have poor generalization capabilities, and are difficult to achieve satisfactory results.
[0005] In recent years, deep learning has developed rapidly, among which deep convolutional neural networks have a powerful ability to extract a large number of features and have developed rapidly in the application of computer vision tasks. The Fully Convolution Networks (FCN) proposed by some people classifies each pixel in the image and achieves a good segmentation effect, but the pooling process causes the loss of target edge information, and the segmentation results for fine-grained targets are not fine enough. The UNet network proposed by others has excellent performance in the field of medical image segmentation. Each downsampling of UNet will have a jump connection to cascade with the corresponding upsampling, and medical images have the characteristics of relatively simple semantics and relatively fixed structures, so high-level semantic information and low-level features are very important. UNet, a feature fusion of different scales, combines the underlying and high-level information, which is perfectly suitable for medical image segmentation.
[0006] However, the multiple feature fusions and upsampling by deconvolution in UNet lead to an increase in the number of parameters and computation. Therefore, someone proposed the UNet3+ network, which uses the UpSampling2D method for upsampling, which reduces many parameters compared to UNet. However, in order to achieve full-scale jump connections in the UNet3+ network, 3×3 convolutions are used to fix the number of feature map channels of different scales. These additional convolution operations also lead to an increase in the number of parameters. Although, overall, the number of parameters of UNet3+ is less than that of UNet, these 3×3 convolution operations lead to a significant increase in the amount of computation, thereby increasing the time required for model inference. Summary of the invention
[0007] In view of this, the present invention provides a breast mass segmentation network and method, a computer device and a storable medium to reduce the complexity of the segmentation network, improve network performance and alleviate the overfitting phenomenon of the network, thereby reducing the number of parameters and the amount of calculation of the network model.
[0008] On the one hand, an embodiment of the present application provides a breast mass segmentation network, the network comprising:
[0009] It consists of encoder and decoder parts;
[0010] The encoder fixes the number of channels of the feature map and adds he_normal and L2 regularization between the 3×3 convolution and BN layers;
[0011] The decoder changes the four upsampling and four same-scale feature fusions in the original UNet network into four upsampling and one multi-scale feature fusion.
[0012] Preferably, the multi-scale feature fusion includes:
[0013] The feature map whose size meets the first threshold range is reduced by maximum pooling;
[0014] The feature map whose size meets the second threshold range is upsampled by UpSampling2D, and then a 3×3 convolution operation is performed to fix the number of channels of the feature map. Finally, the final segmentation result is obtained by two 3×3 convolutions and one 1×1 convolution.
[0015] Preferably, the encoder processes the breast image as follows:
[0016] The first feature map with a channel number of 1 is passed through two convolution layers with the same structure and 64 convolution kernels to obtain a second map with 64 channels. The two convolution layers with the same structure have the following structure: 3×3 convolution + he_normal + L2 regularization + BN + ReLU function;
[0017] The second atlas with 64 channels is subjected to a maximum pooling operation with a kernel size of 2×2 to obtain a third atlas with 64 channels but half the size;
[0018] The third atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain a fourth atlas with 64 channels.
[0019] The fourth atlas with 64 channels is subjected to a maximum pooling operation with a kernel size of 2×2 to obtain a fifth atlas with 64 channels but half the size;
[0020] The fifth atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain a sixth atlas with 64 channels.
[0021] The sixth atlas with 64 channels is subjected to a maximum pooling operation with a kernel size of 2×2 to obtain a seventh atlas with 64 channels but half the size;
[0022] The seventh atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain an eighth atlas with 64 channels.
[0023] The eighth atlas with 64 channels is subjected to a maximum pooling convolution with a kernel size of 2×2 to obtain a ninth atlas with 64 channels but reduced spatial dimension;
[0024] The ninth atlas is passed through two convolutional layers with the same structure and 64 convolution kernels to obtain the tenth atlas with 64 channels.
[0025] Preferably, the decoder processes the characteristic spectrum of the breast image as follows:
[0026] The tenth spectrum is processed by 16 times UpSampling2D,
[0027] The eighth atlas is processed 8 times UpSampling2D,
[0028] The sixth spectrum is processed by 4 times UpSampling2D,
[0029] After the fourth spectrum is subjected to 2 times UpSampling2D, it is stacked with the second spectrum to obtain an eleventh spectrum with a channel number of 64×5;
[0030] The eleventh atlas is subjected to two 3×3 convolution and BN+ReLU operations and one 1×1 convolution+Lambda+Softmax operation to obtain the final breast mass segmentation image.
[0031] In a second aspect, an embodiment of the present invention provides a breast mass segmentation method of a breast mass segmentation network, the method comprising:
[0032] Acquiring a breast image to be segmented, wherein the breast image to be segmented includes a breast mass to be segmented and extracted;
[0033] Preprocessing the breast image to be segmented to obtain a data-enhanced input image;
[0034] The input image is processed by the trained breast segmentation network to obtain a segmentation result of the breast mass in the breast image to be segmented.
[0035] Preferably, in the third aspect, the method further comprises:
[0036] The area containing the mass in the original mammographic image is cropped to obtain the mammographic ROI as the input image;
[0037] Before training the breast segmentation network, dividing the input image into a training set and a test set;
[0038] The training set images were rotated clockwise every 45° for a total of 7 times;
[0039] The original breast X-ray images were flipped horizontally and vertically once to increase the number of images in the training set to 10 times the original number. The validation set ratio was set to 0.2, that is, 20% of the images in the training set were automatically randomly selected as the validation set at the beginning of training.
[0040] On the other hand, an embodiment of the present invention provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the breast mass segmentation method as described above.
[0041] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the breast mass segmentation method as described above.
[0042] The embodiment of the present invention retains the divide-and-conquer strategy of the original UNet and simplifies the feature fusion method: the number of feature map channels of each step of the UNet encoder is fixed to reduce the network complexity while facilitating the feature fusion of the decoder, and he_normal and L2 regularization are added to the convolution operation in the encoder, thereby improving the network performance and alleviating the overfitting phenomenon of the network. Secondly, the network structure of the UNet decoder is simplified, that is, while ensuring that the model segmentation ability remains unchanged, the training speed of the network model is improved and the number of network parameters and the amount of calculation are reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The specific implementation modes of the present invention will be described below with reference to the accompanying drawings.
[0044] Figure 1 This is an X-ray image of a breast mass;
[0045] Figure 2 A schematic diagram of a Half-UNet breast mass segmentation network model provided in an embodiment of the present invention;
[0046] Figure 3 A schematic diagram of an encoder for a Half-UNet breast mass segmentation network model provided in an embodiment of the present invention;
[0047] Figure 4 The ReLU function image provided by the embodiment of the present invention;
[0048] Figure 5 A schematic diagram of a multi-scale feature fusion method provided in an embodiment of the present invention;
[0049] Figure 6 A schematic diagram of sliding a 3×3 convolution kernel provided in an embodiment of the present invention on the original image and performing corresponding element multiplication and summation to obtain the pixel value of each point in the next layer feature map;
[0050] Figure 7 A schematic diagram of 2×2 maximum pooling provided in an embodiment of the present invention;
[0051] Figure 8 A schematic diagram of a decoder of the Half-UNet breast mass segmentation network model provided in an embodiment of the present invention;
[0052] Fig. 9 A schematic diagram of an UpSampling2D operation provided in an embodiment of the present invention;
[0053] Fig.10 A schematic diagram of the Half-UNet breast mass segmentation network provided in an embodiment of the present invention for segmenting breast masses;
[0054] Fig.11 A schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work. In order to make the drawings concise, only the parts related to the present invention are schematically shown in each figure, and they do not represent the actual structure of the product.
[0056] In order to achieve the purpose of the present invention, an embodiment of the present invention provides a breast mass segmentation network, the network comprising:
[0057] It consists of encoder and decoder parts;
[0058] The encoder fixes the number of channels of the feature map and adds he_normal and L2 regularization between the 3×3 convolution and BN layers;
[0059] The decoder changes the four upsampling and four same-scale feature fusions in the original UNet network into four upsampling and one multi-scale feature fusion.
[0060] like Figure 2 As shown, the embodiment of the present invention proposes a Half-UNet breast mass segmentation network model, and the network of the Half-UNet breast mass segmentation network model consists of an encoder and a decoder. Figure 2 In the figure, rectangles represent feature maps and arrows represent corresponding operations ( Figure 2 There is an explanation in ), where the length of the rectangle represents the number of channels of the feature map (such as 3 channels for RGB images), and the width of the rectangle represents the image size (all images have the same length and width).
[0061] like Figure 2 As shown, the encoder has two improvements compared to the original UNet network:
[0062] First, the number of channels of the feature map is fixed, thereby reducing the number of network parameters and calculations, and facilitating decoding by the decoder;
[0063] The second is the convolution process of the encoder, such as Figure 2 As shown by the white right arrow in the figure, he_normal and L2 regularization are added between the 3×3 convolution and BN layers to promote model convergence and alleviate the overfitting problem.
[0064] exist Figure 2In the figure, on the right side of the Half-UNet network, the decoder part is simpler than the original UNet decoder network structure, simplifying the four upsampling and four same-scale feature fusion methods in UNet to four upsampling and one multi-scale feature fusion.
[0065] Multi-scale feature fusion methods such as Figure 5 As shown in the figure: the large-sized feature map is reduced by maximum pooling, the small-sized feature map is upsampled by UpSampling2D, and then a 3×3 convolution operation is performed to fix the number of channels of the feature map;
[0066] Preferably, the multi-scale feature fusion includes:
[0067] The feature map whose size meets the first threshold range is reduced by maximum pooling;
[0068] The feature map whose size meets the second threshold range is upsampled by UpSampling2D, and then a 3×3 convolution operation is performed to fix the number of channels of the feature map. Finally, the final segmentation result is obtained by two 3×3 convolutions and one 1×1 convolution.
[0069] Different from the existing technology, since the number of channels of the feature map has been fixed in the Half-UNet encoder, here we only need to unify the size of the multi-scale feature maps through UpSampling2D to achieve feature fusion. Finally, the final segmentation result is obtained by two 3×3 convolutions and one 1×1 convolution. The convolution operation in the decoder is the same as UNet.
[0070] From the perspective of the network model, the Half-UNet network model of the embodiment of the present invention retains the divide-and-conquer strategy of the original UNet, while simplifying the feature fusion part in the decoder, from four feature fusions in the original UNet to one feature fusion.
[0071] Experimental data show that Half-UNet not only reduces the number of model parameters but also speeds up the model's reasoning speed while maintaining similar segmentation accuracy compared to the original UNet.
[0072] like Figure 3 As shown, the encoder part of Half-UNet in an embodiment of the present invention.
[0073] The processing flow of the encoder part is as follows:
[0074] S11. The original image (128×128×1) undergoes two convolution operations (3×3 convolution + he_normal + L2 regularization + BN + ReLU function) to obtain a 128×128×64 feature map C1.
[0075] S111, 3×3 convolution
[0076] 3×3 is the size of the convolution kernel. Slide the 3×3 convolution kernel on the original image, multiply and sum the corresponding elements, and get the pixel value of each point in the next layer of feature map (such as Figure 6 As shown, Figure 6 The original image in the previous layer is convolved with a 3×3 convolution kernel to obtain the pixel values of the shadow points in the next layer of image).
[0077] Preferably, padding can be added before the 3×3 convolution of the embodiment of the present invention, that is, the edge of the original image is expanded before the convolution, and the pixel value is padded to 0, that is, the 128×128 original image is padded to 129×129, and then a 3×3 convolution is performed to obtain a 128×128 feature map, so that the image size will not change due to the convolution operation.
[0078] That is, the original image (128×128×1)->padding->(129×129×1)->64 3×3 convolution kernels slide to multiply the corresponding elements and sum them->(128×128×64).
[0079] S112.he_normal+L2 regularization
[0080] he_normal is the initialization formula for the ReLU activation network:
[0081] Among them, nl is the number of neurons in the lth layer.
[0082] L2 regularization directly adds the sum of squares of weight parameters to the original loss function:
[0083]
[0084] Among them, E in is the training sample error without regularization term, and λ is the regularization parameter, which is adjustable.
[0085] S113.BN layer
[0086] The BN layer is a normalization layer, which means that a normalization process is performed first (normalized to: mean 0, variance 1) before entering the next layer of the network.
[0087] S114.ReLU function
[0088] The ReLU function is used as the activation function of the neuron, which is the linear transformation of the neuron w T The nonlinear output result after x+b. For the input vector x from the previous layer of neural network entering the neuron, the neuron using the ReLU function will output max(0, wT x+b) to the next layer of neurons or as the output of the entire neural network (depending on the position of the current neuron in the network structure). The corresponding function graph is as follows Figure 4 shown.
[0089] In summary, after two rounds of (3×3 convolution + he_normal + L2 regularization + BN + ReLU function), a 128×128×64 feature map is obtained, denoted as C1.
[0090] like Figure 3 As shown in the figure, the encoder obtains a 128×128×64 feature map after two (3×3 convolution+he_normal+L2 regularization+BN+ReLU function), which is denoted as C1.
[0091] S12 and C1 first undergo a 2×2 maximum pooling to obtain a 64×64×64 feature map, that is, the number of channels remains unchanged and the image size is halved.
[0092] S1212×2 max pooling
[0093] like Figure 7 As shown in the figure, the 2×2 maximum pooling scans the entire image in a 2×2 area with a step size of 2 and takes the maximum value of each 2×2 area.
[0094] The feature map S122 and 64×64×64 undergoes two more operations (3×3 convolution + he_normal + L2 regularization + BN + ReLU function) to obtain the feature map C2.
[0095] S13. Repeat the process in S12 to obtain C4, C8, and C16.
[0096] like Figure 8 As shown in the figure, it is the decoder part in the Half-UNet network model. The process of the decoder part is as follows:
[0097] S21. Except for C1, C2, C4, C8, and C16 are upsampled to an image size of 128×128, that is, upsampled by 2, 4, 8, and 16 times respectively. The UpSampling2D method is used here for upsampling, that is, a value in the input feature map is mapped and filled into a corresponding area of the output upsampled feature map, and all are filled with the same value. The detailed process of the UpSampling2D method is as follows: Fig. 9 As shown;
[0098] S22, concatenate the five groups of 128×128×64 feature maps obtained after processing in S21, that is, concatenate the number of channels into a whole to obtain a 128×128×320 feature map;
[0099] S23, the feature map obtained in S22 is subjected to two (3×3 convolution + BN + ReLU) and one (1×1 convolution + Lambda + Softmax) steps to obtain the final breast mass segmentation image;
[0100] Lambda: Application of Lambda expressions.
[0101] Softmax: The output of multiple neurons is mapped to the interval (0,1), which can be understood as probability. The closer the Softmax result is to 1, the more the model believes that this pixel is a breast mass area.
[0102] It can be seen from the above process of the embodiment of the present invention that the decoder part is simpler than the original UNet decoder network structure, and the four upsampling and four same-scale feature fusion methods in UNet are simplified to four upsampling and one multi-scale feature fusion. The multi-scale feature fusion idea is as follows: Figure 4 As shown in the figure, the large-scale feature map is reduced by maximum pooling, the small-scale feature map is upsampled by UpSampling2D, and then a 3×3 convolution operation is performed to fix the number of channels of the feature map. Since the encoder in the Half-UNet network has fixed the number of channels of the feature map, the decoder only needs to unify the size of the multi-scale feature map by UpSampling2D to achieve feature fusion. Finally, the final segmentation result is obtained by two 3×3 convolutions and one 1×1 convolution. Here, the convolution operation in the decoder can be the same as the original UNet. Experimental data shows that unifying the number of feature map channels has almost no effect on the segmentation performance of the model, and can significantly reduce the number of network parameters and training time, reducing the complexity of the network.
[0103] like Fig.10 As shown, in this embodiment of the present invention, in order to use the Half-UNet network as a network for breast mass segmentation, it is assumed that the input layer of the Half-UNet network is a breast image to be segmented, the number of channels is 1, and the width and height are both 64.
[0104] The first graph with a channel number of 1 (i.e., the input layer) is passed through two convolutional layers with the same structure and 64 convolution kernels to obtain a second graph with a channel number of 64, where the two convolutional layers with the same structure have the following structure: 3×3 convolution + he_normal + L2 regularization + BN + ReLU function. The implementation of the encoder is explained in detail in the previous embodiment, which will not be repeated here.
[0105] The second atlas with 64 channels is subjected to a maximum pooling operation with a kernel size of 2×2 to obtain a third atlas with 64 channels but half the size;
[0106] The third atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain a fourth atlas with 64 channels.
[0107] The fourth atlas with 64 channels is subjected to a maximum pooling operation with a kernel size of 2×2 to obtain a fifth atlas with 64 channels but half the size;
[0108] The fifth atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain a sixth atlas with 64 channels.
[0109] The sixth atlas with 64 channels is subjected to a maximum pooling operation with a kernel size of 2×2 to obtain a seventh atlas with 64 channels but half the size;
[0110] The seventh atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain an eighth atlas with 64 channels.
[0111] The eighth atlas with 64 channels is subjected to a maximum pooling convolution with a kernel size of 2×2 to obtain a ninth atlas with 64 channels but reduced spatial dimension;
[0112] The ninth atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain a tenth atlas with 64 channels.
[0113] The tenth spectrum (corresponding to the attached Figure 3 , Figure 8 C16) after 16 times UpSampling2D (such as Fig. 9 ), the eighth spectrum (corresponding to the attached Figure 3 , Figure 8 (same as C8 in the figure) after 8 times UpSampling2D, the sixth spectrum (corresponding to the attached Figure 3 , Figure 8 The fourth spectrum (corresponding to the attached Figure 3 , Figure 8 The eleventh atlas with the second atlas is obtained after 2 times UpSampling2D (i.e., concatenated) to obtain a channel number of 64×5 (i.e., 320).
[0114] The eleventh atlas is processed twice (3×3 convolution + BN + ReLU) and once (1×1 convolution + Lambda + Softmax) to obtain the final breast mass segmentation image.
[0115] An embodiment of the present invention provides a breast mass segmentation method using a breast mass segmentation network, the method comprising:
[0116] Acquiring a breast image to be segmented, wherein the breast image to be segmented includes a breast mass to be segmented and extracted;
[0117] Preprocessing the breast image to be segmented to obtain a data-enhanced input image;
[0118] The input image is processed by the trained breast segmentation network to obtain a segmentation result of the breast mass in the breast image to be segmented.
[0119] Preferably, the method further comprises:
[0120] The area containing the mass in the original mammographic image is cropped to obtain the mammographic ROI as the input image;
[0121] Before training the breast segmentation network, dividing the input image into a training set and a test set;
[0122] The training set images were rotated clockwise every 45° for a total of 7 times;
[0123] The original breast X-ray images were flipped horizontally and vertically once to increase the number of images in the training set to 10 times the original number. The validation set ratio was set to 0.2, that is, 20% of the images in the training set were automatically randomly selected as the validation set at the beginning of training.
[0124] The data set of the embodiment of the present invention can be selected from the Digital Database for Screening Mammography (DDSM) database of the University of South Florida, from which 483 breast X-ray film ROIs containing masses are sorted and selected, of which 400 images are used as training sets and 83 images are used as test sets. In order to reduce the overfitting phenomenon of the model and enhance the generalization ability of the model, the embodiment of the present invention rotates the training set images clockwise every 45° for a total of 7 times, and on this basis, performs a horizontal flip and a vertical flip, so that the number of training set images is expanded to 10 times the original, that is, 4000 images.
[0125] The Half-UNet network of the embodiment of the present invention retains the divide-and-conquer part of the original UNet and simplifies the feature fusion method: first, the number of feature map channels of each step of the UNet encoder is fixed to reduce the network complexity, which is beneficial to the feature fusion of the decoder, and he_normal and L2 regularization are added to the convolution operation in the encoder, thereby improving the network performance and alleviating the overfitting phenomenon of the network. Secondly, the network structure of the UNet decoder is simplified to reduce the number of parameters and calculation amount of the network model.
[0126] Please refer to Fig.11, which shows a schematic diagram of the structure of a computer device 1500 provided in one embodiment of the present application. The computer device 1500 can be used to implement the breast mass segmentation method in the image provided in the above embodiment.
[0127] Specifically:
[0128] The computer device 1500 includes a central processing unit (CPU) 1501, a system memory 1504 including a random access memory (RAM) 1502 and a read-only memory (ROM) 1503, and a system bus 1505 connecting the system memory 1504 and the central processing unit 1501. The computer device 1500 also includes a basic input / output system (I / O system) 1506 that helps transfer information between various components in the computer, and a large-capacity storage device 1507 for storing an operating system 1513, application programs 1514, and other program modules 1515.
[0129] The basic input / output system 1506 includes a display 1508 for displaying information and an input device 1509 such as a mouse and a keyboard for user inputting information. The display 1508 and the input device 1509 are connected to the central processing unit 1501 through an input / output controller 1510 connected to the system bus 1505. The basic input / output system 1506 may also include an input / output controller 1510 for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 1510 also provides output to a display screen, a printer, or other types of output devices.
[0130] The mass storage device 1507 is connected to the central processing unit 1501 through a mass storage controller (not shown) connected to the system bus 1505. The mass storage device 1507 and its associated computer readable media provide non-volatile storage for the computer device 1500. That is, the mass storage device 1507 may include a computer readable medium (not shown) such as a hard disk or a CD-ROM drive.
[0131] Without loss of generality, the computer readable medium may include computer storage media and communication media. Computer storage media include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technology, CD-ROM, DVD or other optical storage, tape cassettes, magnetic tapes, disk storage or other magnetic storage devices.
[0132] Of course, those skilled in the art will appreciate that the computer storage media are not limited to the above mentioned ones. The above mentioned system memory 1504 and large capacity storage device 1507 can be collectively referred to as memory.
[0133] According to various embodiments of the present application, the computer device 1500 can also be connected to a remote computer on the network through a network such as the Internet. That is, the computer device 1500 can be connected to the network 1512 through the network interface unit 1511 connected to the system bus 1505, or the network interface unit 1511 can be used to connect to other types of networks or remote computer systems (not shown).
[0134] The memory further comprises one or more programs, which are stored in the memory and configured to be executed by one or more processors. The one or more programs include a method for implementing the breast mass segmentation method in the image.
[0135] In an exemplary embodiment, a computer device is also provided, the computer device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, at least one program, code set or instruction set is configured to be executed by the processor to implement the above-mentioned method for segmenting breast masses in images.
[0136] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein at least one instruction, at least one program, code set or instruction set is stored in the storage medium, and the at least one instruction, at least one program, code set or instruction set implements the above-mentioned breast mass segmentation method in the image when executed by the processor of the terminal. Optionally, the above-mentioned computer-readable storage medium can be ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device, etc.
[0137] In an exemplary embodiment, a computer program product is also provided. When the computer program product is executed, it is used to implement the above-mentioned breast mass segmentation method in the image.
[0138] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.
[0139] In addition, the step numbers described in this document only illustrate a possible execution order between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to that shown in the figure. The embodiments of the present application are not limited to this.
[0140] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
[0141] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A breast mass segmentation network, characterized in that: The network includes: It consists of encoder and decoder parts; The encoder fixes the number of channels of the feature map and adds he_normal and L2 regularization between the 3×3 convolution and BN layers; The decoder changes the four upsampling and four same-scale feature fusions in the original UNet network into four upsampling and one multi-scale feature fusion, and the multi-scale feature fusion includes: The feature map whose size meets the first threshold range is reduced by maximum pooling; The feature map whose size meets the second threshold range is upsampled by UpSampling2D, and then a 3×3 convolution operation is performed to fix the number of channels of the feature map. Finally, the final segmentation result is obtained by two 3×3 convolutions and one 1×1 convolution.
2. The breast mass segmentation network according to claim 1, characterized in that: The encoder processes the breast image as follows: The first feature map with a channel number of 1 is passed through two convolution layers with the same structure and 64 convolution kernels to obtain a second map with 64 channels. The two convolution layers with the same structure have the following structure: 3×3 convolution + he_normal + L2 regularization + BN + ReLU function; The second atlas with 64 channels is subjected to a maximum pooling operation with a kernel size of 2×2 to obtain a third atlas with 64 channels but half the size; The third atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain a fourth atlas with 64 channels. The fourth atlas with 64 channels is subjected to a maximum pooling operation with a kernel size of 2×2 to obtain a fifth atlas with 64 channels but half the size; The fifth atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain a sixth atlas with 64 channels. The sixth atlas with 64 channels is subjected to a maximum pooling operation with a kernel size of 2×2 to obtain a seventh atlas with 64 channels but half the size; The seventh atlas is passed through two convolution layers with the same structure and 64 convolution kernels to obtain an eighth atlas with 64 channels. The eighth atlas with 64 channels is subjected to a maximum pooling convolution with a kernel size of 2×2 to obtain a ninth atlas with 64 channels but reduced spatial dimension; The ninth atlas is passed through two convolutional layers with the same structure and 64 convolution kernels to obtain the tenth atlas with 64 channels.
3. The breast mass segmentation network according to claim 2, characterized in that: The decoder processes the characteristic map of the breast image as follows: The tenth atlas is processed by 16 times UpSampling2D, The eighth atlas is processed 8 times UpSampling2D, The sixth spectrum is processed by 4 times UpSampling2D, After the fourth spectrum is subjected to 2 times UpSampling2D, it is stacked with the second spectrum to obtain an eleventh spectrum with a channel number of 64×5; The eleventh atlas is subjected to two 3×3 convolution and BN+ReLU operations and one 1×1 convolution+Lambda+Softmax operation to obtain the final breast mass segmentation image.
4. A breast mass segmentation method based on the breast mass segmentation network according to any one of claims 1 to 3, characterized in that: The method comprises: Acquiring a breast image to be segmented, wherein the breast image to be segmented includes a breast mass to be segmented and extracted; Preprocessing the breast image to be segmented to obtain a data-enhanced input image; The input image is processed by the trained breast mass segmentation network to obtain a segmentation result of the breast mass in the breast image to be segmented.
5. The breast mass segmentation method according to claim 4, characterized in that: The method further comprises: The area containing the mass in the original mammographic image is cropped to obtain the mammographic ROI as the input image; Before training the breast mass segmentation network, dividing the input image into a training set and a test set; The training set images were rotated clockwise every 45° for a total of 7 times; The original breast X-ray image is flipped horizontally and vertically once, so that the number of images in the training set is expanded to 10 times of the original; the validation set ratio is set to 0.2, that is, 20% of the images in the training set are automatically randomly selected as the validation set at the beginning of training.
6. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the breast mass segmentation method as described in claim 4.
7. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the breast mass segmentation method as described in claim 4.