Breast ultrasonic image segmentation method based on multistage feature integration network
By adopting a multi-level feature integration network method in breast ultrasonic image segmentation, a new encoding and decoding structure is constructed, which solves the problems of insufficient boundary feature extraction capabilities and loss of context information, and significantly improves segmentation performance and detail capture capabilities.
Patent Information
- Application Number
- CN202411922015.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-25
- Publication Date
- 2025-05-30
AI Technical Summary
The prior art has problems in breast tumor ultrasound image segmentation, which are insufficient boundary feature extraction capabilities and prone to loss of context information during feature extraction.
Using a breast ultrasound image segmentation method based on multi-level feature integration network, a new encoding and decoding structure, including FEB modules and multiple sets of MRAB modules, improves the ability to extract boundary feature and capture and integrate context information.
It improves the overall segmentation performance of the network, especially in capturing the details of the tumor edge, significantly improving the accuracy of ultrasonic image segmentation of breast tumors.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer image processing, and particularly relates to a method for segmenting breast ultrasound images based on a multi-level feature integration network. Background Art
[0002] Accurately and automatically segmenting breast tumor ultrasound images is of great significance for the early diagnosis and treatment of breast cancer. Deep learning technology has the ability of autonomous learning and segmentation. Therefore, in the field of medical image segmentation, deep learning methods are the current mainstream research direction. Some convolutional neural networks, such as FCN, SegNet, U-Net, etc., have been widely favored by researchers due to their efficient convolutional operations, good structural designs, and excellent performance. U-Net occupies an important position in the field of medical image segmentation with its unique U-shaped encoder-decoder architecture and skip connections.
[0003] However, the current existing technologies still need to further improve the ability to extract the boundary features of breast tumors, and there is also a problem that context information is easily lost during feature extraction. Summary of the Invention
[0004] The present invention aims to provide a method for segmenting breast ultrasound images based on a multi-level feature integration network. This method constructs a new encoding structure and decoding structure, improves the ability to extract breast tumor boundary features and capture and integrate context information, enhances the overall segmentation performance of the network, especially the ability to capture details at the tumor edge.
[0005] The technical solution of the present invention is as follows:
[0006] The method for segmenting breast ultrasound images based on a multi-level feature integration network includes the following steps:
[0007] A. Construct a neural network, and the specific neural network structure is as follows:
[0008] It includes an encoding network and a decoding network;
[0009] The encoding network is provided with a FEB module and multiple groups of MRAB modules;
[0010] B. Input the original image into the encoding network. In the encoding network:
[0011] The original image is processed by the FEB module to obtain four output results, namely G 1 Output result, G 2 Output result, G 3 Output result, G 4 Output result;
[0012] The original image is convolved with a 7×7 filter and then combined with G 1 After the output results are added and fused, the F output result is obtained;
[0013] After the F output result is successively processed by max pooling and two residual modules, the resulting residual processing result and the F output result are respectively input into the first MRAB module for processing to obtain the first MRAB processing result; The first MRAB processing result is combined with G 2 After the output results are added and fused, an F 1 output result with doubled number of channels and halved size is obtained;
[0014] F 1 After the output result is successively processed by three residual modules, the resulting residual processing result and the F 1 output result are respectively input into the second MRAB module for processing to obtain the second MRAB processing result; The second MRAB processing result is combined with G 3 After the output results are added and fused, an F 2 output result with doubled number of channels and halved size is obtained;
[0015] F 2 After the output result is successively processed by five residual modules, the resulting residual processing result and the F 2 output result are respectively input into the third MRAB module for processing to obtain the third MRAB processing result; The third MRAB processing result is combined with G 4 After the output results are added and fused, an F 3 output result with doubled number of channels and halved size is obtained;
[0016] F 3 After the output result is successively processed by two residual modules, the resulting residual processing result and the F 3 output result are respectively input into the fourth MRAB module for processing to obtain an F 4 output result with unchanged number of channels and halved size;
[0017] The described F output result, F 1 output result, F 2 output result, F 3 output result, F 4 output results are respectively input into the decoding network;
[0018] C. After the decoding network fuses all the input results, the final segmentation result is obtained.
[0019] The processing in the FEB module is as follows
[0020] After the original image is processed by a 1×1 convolution to change the number of channels, it is split into four parts by the Split function, namely G1 Splitting part, G 2 Splitting part, G 3 Splitting part, G 4 Splitting part;
[0021] G 1 The splitting part is divided into two paths. The first path is sequentially processed by central difference convolution, BN function, and ReLU function to obtain the result of the first path; the second path is sequentially processed by 3×3 depthwise separable convolution, BN function, and ReLU function to obtain the result of the second path; the results of the first path and the second path are concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling to obtain G 1 Intermediate result, G 1 The intermediate result is processed by 1×1 convolution to obtain G 1 Output result;
[0022] G 2 The splitting part is divided into two paths. The first path is sequentially processed by central difference convolution, BN function, and ReLU function to obtain the result of the first path; the second path is sequentially processed by 3×3 depthwise separable convolution, BN function, and ReLU function to obtain the result of the second path; the results of the first path and the second path are concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling, and the obtained result is combined with G 1 The intermediate result is concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling to obtain G 2 Intermediate result, G 2 The intermediate result is processed by 1×1 convolution and then combined with the upsampled G 1 The output results are added and fused, and then processed by 1×1 convolution to obtain G 2 Output result;
[0023] G 3 The splitting part is divided into two paths. The first path is sequentially processed by central difference convolution, BN function, and ReLU function to obtain the result of the first path; the second path is sequentially processed by 3×3 depthwise separable convolution, BN function, and ReLU function to obtain the result of the second path; the results of the first path and the second path are concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, max pooling, 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling, and the obtained result is combined with G 2 The intermediate result is concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling to obtain G 3 Intermediate result, G3 After the intermediate result is processed by a 1×1 convolution, it is added and fused with the upsampled G 2 The output results are added and fused, and then processed by a 1×1 convolution to obtain the G 3 output result;
[0024] G 4 The splitting part is divided into two paths. The first path is sequentially processed by a central difference convolution, a BN function, and a ReLU function to obtain the first path result; the second path is sequentially processed by a 3×3 depthwise separable convolution, a BN function, and a ReLU function to obtain the second path result; after the first path result and the second path result are concatenated by a Concat function, they are sequentially processed by a 3×3 depthwise separable convolution, a BN function, a ReLU function, a max pooling, a 3×3 depthwise separable convolution, a BN function, a ReLU function, a max pooling, a 3×3 depthwise separable convolution, a BN function, a ReLU function, and a max pooling. The obtained result is concatenated with the G 3 After the intermediate result is concatenated by a Concat function, it is sequentially processed by a 3×3 depthwise separable convolution, a BN function, a ReLU function, and a max pooling to obtain the G 4 intermediate result, G 4 After the intermediate result is processed by a 1×1 convolution, it is added and fused with the upsampled G 3 The output results are added and fused, and then processed by a 1×1 convolution to obtain the G 4 output result.
[0025] The processing process in the residual module is as follows:
[0026] The input result is sequentially processed by two 3×3 convolutions, and the obtained result is added and fused with the input result to obtain the output result.
[0027] The first MRAB module, the second MRAB module, the third MRAB module, and the fourth MRAB module have the same structure, and the processing process is as follows:
[0028] The input residual processing result is divided into two paths. The first path is processed by a 3×3 convolution to obtain the first convolution result, and the second path is processed by a 5×5 convolution to obtain the second convolution result;
[0029] The input upper-level F output result is sequentially processed by downsampling and a 3×3 convolution, and the obtained result is added and fused with the first convolution result and the second convolution result to obtain the added and fused result;
[0030] The added and fused result is divided into three paths. The first path is processed by a 1×1 convolution to obtain the first path result;
[0031] The second path undergoes 1×1 convolution processing to obtain the second path result; the third path undergoes 1×1 convolution and Transpose function processing in sequence to obtain the third path result;
[0032] After the second path result and the third path result are multiplied and fused, the obtained result is multiplied and fused with the first path result, and then added and fused with the input residual processing result to obtain the output result.
[0033] The processing process in the encoding network is as follows:
[0034] F 4 After the output result is upsampled, it is combined with F 3 After the output result is concatenated by the Concat function, the obtained result undergoes 3×3 convolution, BN function, ReLU function, 3×3 convolution, BN function, and ReLU function processing in sequence to obtain the first layer result;
[0035] The first layer result and F 2 The output result is respectively input into the first CIB module for processing to obtain the first CIB processing result. After the first CIB processing result and the first layer result are concatenated by the Concat function, the obtained result undergoes 3×3 convolution, BN function, ReLU function, 3×3 convolution, BN function, and ReLU function processing in sequence to obtain the second layer result;
[0036] The second layer result and F 1 The output result is respectively input into the second CIB module for processing to obtain the second CIB processing result. After the second CIB processing result and the second layer result are concatenated by the Concat function, the obtained result undergoes 3×3 convolution, BN function, ReLU function, 3×3 convolution, BN function, and ReLU function processing in sequence to obtain the third layer result;
[0037] The third layer result and the F output result are respectively input into the third CIB module for processing to obtain the third CIB processing result. After the third CIB processing result and the third layer result are concatenated by the Concat function, the obtained result undergoes 3×3 convolution, BN function, ReLU function, 3×3 convolution, BN function, and ReLU function processing in sequence to obtain the fourth layer result;
[0038] The fourth layer result undergoes upsampling and 1×1 convolution in sequence to obtain the output result.
[0039] The first CIB module, the second CIB module, and the third CIB module have the same structure, and the processing process is as follows:
[0040] The results of the encoded network input are successively processed by a 1×1 convolution and a Sigmoid function to obtain a first intermediate result. After the first intermediate result is processed by a Transpose function, the obtained result is dot-product processed with the encoded network input result to obtain a first dot-product result;
[0041] The input result of the previous layer is divided into three branches. The first branch, after upsampling, is dot-product processed with the first intermediate result to obtain a second dot-product result; the second branch is successively processed by a 1×1 convolution and a Sigmoid function. After the obtained result is added and fused with the first intermediate result, the obtained result is dot-product processed with the encoded network input result and then processed by a 3×3 dilated convolution with a dilation coefficient of 2 to obtain a second branch result; the third branch is successively processed by a 3×3 convolution and a 1×1 convolution to obtain a third branch result;
[0042] After the first dot-product result and the second dot-product result are added and fused, they are dot-product processed with the encoded network input result and then added and fused with the encoded network input result to obtain a second intermediate result;
[0043] The second intermediate result is divided into two branches. The first branch is processed by a 1×1 convolution to obtain a first branch result; the second branch is processed by a 3×3 convolution to obtain a second branch result;
[0044] After the first branch result, the second branch result, the second branch result, and the third branch result are added and fused, the obtained result is added and fused with the encoded network input result and the input result of the previous layer. The obtained result is successively processed by a 1×1 convolution, a BN function, a 3×3 convolution, and a ReLU function to obtain an output result.
[0045] In the neural network of the present invention, the encoding structure is improved based on the ResNet34 network, which can effectively enhance the ability of the encoding network to extract boundary features and at the same time enhance the detailed features of the boundary. And, aiming at the problem that the existing technology is prone to ignore details when the encoding network extracts image boundary features, resulting in incomplete segmentation boundaries, the FEB module is designed for feature enhancement, which can improve the tumor segmentation accuracy.
[0046] The neural network for breast ultrasound image segmentation proposed by the present invention designs a new encoding structure and a decoding structure, which improves the ability to extract boundary features and capture and integrate context information of breast tumors, enhances the overall segmentation performance of the network, especially in capturing the details of the tumor edge. At the same time, experiments can show that the neural network model and segmentation method are effective for breast ultrasound image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 It is a schematic structural diagram of the neural network according to Embodiment 1 of the present invention;
[0048] Figure 2 Schematic diagram of the FEB module of Embodiment 1 of the present invention;
[0049] Figure 3 Schematic diagram of the MRAB module of Embodiment 1 of the present invention;
[0050] Figure 4 Schematic diagram of the CIB module of Embodiment 1 of the present invention;
[0051] Figure 5 Comparison chart of the effects of the segmentation method provided in Embodiment 1 and the segmentation method in Document 1. Detailed implementation manners
[0052] The present invention will be specifically described below in conjunction with the accompanying drawings and embodiments.
[0053] Embodiment 1
[0054] A breast ultrasound image segmentation method based on a multi-level feature integration network includes the following steps:
[0055] A. Construct a neural network, and the specific structure of the neural network is as follows:
[0056] As Figure 1 shown, it includes an encoding network and a decoding network;
[0057] The encoding network is provided with an FEB module and multiple groups of MRAB modules;
[0058] B. Input the original image into the encoding network. In the encoding network:
[0059] The original image is processed by the FEB module to obtain four output results, namely G 1 output result, G 2 output result, G 3 output result, G 4 output result;
[0060] As Figure 2 shown, the processing process in the FEB module is as follows
[0061] After the original image is processed by 1×1 convolution to change the number of channels, it is split into four parts by the Split function, namely G 1 split part, G 2 split part, G 3 split part, G 4 split part;
[0062] G 1The splitting part is divided into two paths. The first path is sequentially processed by central difference convolution, BN function, and ReLU function to obtain the result of the first path; the second path is sequentially processed by 3×3 depthwise separable convolution, BN function, and ReLU function to obtain the result of the second path; the results of the first path and the second path are concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling to obtain G 1 Intermediate result, G 1 After the intermediate result is processed by 1×1 convolution, G is obtained 1 Output result;
[0063] G 2 The splitting part is divided into two paths. The first path is sequentially processed by central difference convolution, BN function, and ReLU function to obtain the result of the first path; the second path is sequentially processed by 3×3 depthwise separable convolution, BN function, and ReLU function to obtain the result of the second path; the results of the first path and the second path are concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling, and the obtained result is combined with G 1 After the intermediate result is concatenated by the Concat function, it is sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling to obtain G 2 Intermediate result, G 2 After the intermediate result is processed by 1×1 convolution, it is combined with the upsampled G 1 The output results are added and fused, and then processed by 1×1 convolution to obtain G 2 Output result;
[0064] G 3 The splitting part is divided into two paths. The first path is sequentially processed by central difference convolution, BN function, and ReLU function to obtain the result of the first path; the second path is sequentially processed by 3×3 depthwise separable convolution, BN function, and ReLU function to obtain the result of the second path; the results of the first path and the second path are concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, max pooling, 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling, and the obtained result is combined with G 2 After the intermediate result is concatenated by the Concat function, it is sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling to obtain G 3 Intermediate result, G 3 After the intermediate result is processed by 1×1 convolution, it is combined with the upsampled G 2 The output results are added and fused, and then processed by 1×1 convolution to obtain G 3 Output result;
[0065] G 4 The split part is divided into two paths. The first path is sequentially processed by central difference convolution, BN function, and ReLU function to obtain the result of the first path; the second path is sequentially processed by 3×3 depthwise separable convolution, BN function, and ReLU function to obtain the result of the second path; the results of the first path and the second path are concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, max pooling, 3×3 depthwise separable convolution, BN function, ReLU function, max pooling, 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling. The obtained result is combined with G 3 The intermediate result is concatenated by the Concat function and then sequentially processed by 3×3 depthwise separable convolution, BN function, ReLU function, and max pooling to obtain G 4 Intermediate result, G 4 The intermediate result is processed by 1×1 convolution and then combined with the upsampled G 3 The output results are added and fused, and then processed by 1×1 convolution to obtain G 4 Output result;
[0066] The original image is convolved by 7×7 and then combined with G 1 The output results are added and fused to obtain the F output result;
[0067] The F output result is sequentially processed by max pooling and two residual modules. The obtained residual processing result and the F output result are respectively input into the first MRAB module for processing to obtain the first MRAB processing result; the first MRAB processing result is combined with G 2 The output results are added and fused to obtain F with the number of channels doubled and the size halved 1 Output result;
[0068] F 1 The F output result is sequentially processed by three residual modules. The obtained residual processing result and the F 1 Output result are respectively input into the second MRAB module for processing to obtain the second MRAB processing result; the second MRAB processing result is combined with G 3 The output results are added and fused to obtain F with the number of channels doubled and the size halved 2 Output result;
[0069] F 2 The F output result is sequentially processed by five residual modules. The obtained residual processing result and the F 2 Output result are respectively input into the third MRAB module for processing to obtain the third MRAB processing result; the third MRAB processing result is combined with G 4 The output results are added and fused to obtain F with the number of channels doubled and the size halved3 Output result;
[0070] F 3 After the output result is processed by 2 residual modules in sequence, the obtained residual processing result and F 3 The output results are respectively input into the fourth MRAB module for processing, and F with the same number of channels but half the size is obtained 4 Output result;
[0071] The said F output result, F 1 Output result, F 2 Output result, F 3 Output result, F 4 The output results are respectively input into the decoding network;
[0072] The processing process in the said residual module is as follows:
[0073] After the input result is processed by two 3×3 convolutions in sequence, the obtained result is added and fused with the input result to obtain the output result.
[0074] Such as Figure 3 As shown, the structures of the first MRAB module, the second MRAB module, the third MRAB module, and the fourth MRAB module are the same, and the processing process is as follows:
[0075] The input residual processing result is divided into two paths. The first path is processed by a 3×3 convolution to obtain the first convolution result, and the second path is processed by a 5×5 convolution to obtain the second convolution result;
[0076] The input F output result of the previous level is processed by downsampling and 3×3 convolution in sequence, and the obtained result is added and fused with the first convolution result and the second convolution result to obtain the added fusion result;
[0077] The added fusion result is divided into three paths. The first path is processed by a 1×1 convolution to obtain the first path result;
[0078] The second path is processed by a 1×1 convolution to obtain the second path result; the third path is processed by a 1×1 convolution and the Transpose function in sequence to obtain the third path result;
[0079] After the second path result and the third path result are multiplied and fused, the obtained result is multiplied and fused with the first path result, and then added and fused with the input residual processing result to obtain the output result.
[0080] C. After the decoding network fuses all the input results, the final segmentation result is obtained.
[0081] The processing process in the said encoding network is as follows:
[0082] F 4After the output result is upsampled, it is combined with F 3 After the output results are concatenated by the Concat function, the resulting results are successively processed by a 3×3 convolution, a BN function, a ReLU function, a 3×3 convolution, a BN function, and a ReLU function to obtain the first-layer result;
[0083] The first-layer result and F 2 The output results are respectively input into the first CIB module for processing to obtain the first CIB processing result. After the first CIB processing result and the first-layer result are concatenated by the Concat function, the resulting results are successively processed by a 3×3 convolution, a BN function, a ReLU function, a 3×3 convolution, a BN function, and a ReLU function to obtain the second-layer result;
[0084] The second-layer result and F 1 The output results are respectively input into the second CIB module for processing to obtain the second CIB processing result. After the second CIB processing result and the second-layer result are concatenated by the Concat function, the resulting results are successively processed by a 3×3 convolution, a BN function, a ReLU function, a 3×3 convolution, a BN function, and a ReLU function to obtain the third-layer result;
[0085] The third-layer result and the F output result are respectively input into the third CIB module for processing to obtain the third CIB processing result. After the third CIB processing result and the third-layer result are concatenated by the Concat function, the resulting results are successively processed by a 3×3 convolution, a BN function, a ReLU function, a 3×3 convolution, a BN function, and a ReLU function to obtain the fourth-layer result;
[0086] The fourth-layer result is successively upsampled and then processed by a 1×1 convolution to obtain the output result.
[0087] As Figure 4 shown, the first CIB module, the second CIB module, and the third CIB module have the same structure, and the processing process is as follows:
[0088] The input result of the encoding network is successively processed by a 1×1 convolution and a Sigmoid function to obtain the first intermediate result. After the first intermediate result is processed by the Transpose function, the resulting result is dot-product processed with the input result of the encoding network to obtain the first dot-product result;
[0089] The input result of the previous layer is divided into three branches. After the first branch undergoes upsampling, it is dot - product processed with the first intermediate result to obtain the second dot - product result; the second branch sequentially undergoes 1×1 convolution and Sigmoid function processing. After the obtained result is added and fused with the first intermediate result, the resulting result is dot - product processed with the input result of the encoding network, and then undergoes 3×3 convolution processing to obtain the result of the second branch; the third branch sequentially undergoes 3×3 convolution and 1×1 convolution processing to obtain the result of the third branch.
[0090] After the first dot - product result and the second dot - product result are added and fused, they are dot - product processed with the input result of the encoding network, and then added and fused with the input result of the encoding network to obtain the second intermediate result.
[0091] The second intermediate result is divided into two branches. The first branch undergoes 1×1 convolution processing to obtain the result of the first branch; the second branch undergoes 3×3 convolution processing to obtain the result of the second branch.
[0092] After the result of the first branch, the result of the second branch, the result of the second branch, and the result of the third branch are added and fused, the resulting result is added and fused with the input result of the encoding network and the input result of the previous layer. Then, the resulting result sequentially undergoes 1×1 convolution, BN function, 3×3 convolution, and ReLU function processing to obtain the output result.
[0093] Example 2
[0094] For the quantitative performance evaluation of the final breast tumor ultrasound image segmentation map, the performance measurement criteria we adopted are specifically evaluated as shown in the formula.
[0095]
[0096] Among them, TN, TP, FN, and FP respectively represent the number of true negative, true positive, false negative, and false positive pixels. DiceCoefficient (Dice) is used to measure the overlap degree between the model segmentation result and the actual annotation. Recall evaluates the ability of the model to detect actual positive samples. Intersection over Union (IoU) evaluates the overlap degree between the predicted region and the actual region. F1 Score (F1) comprehensively evaluates the precision and recall of the model, and Accuracy (Acc) is used to evaluate the correctness of the overall prediction of the model.
[0097] Reference 1: Jieneng Chen, Jieru Mei, et al. Transunet: Rethinking the u-net architecture design for medical image segmentation through the lens of transformers. Medical Image Analysis, 97:103280, 2024.
[0098] The segmentation method of Example 1 was compared with the image segmentation method of Reference 1. For fairness, both methods used exactly the same training conditions, including hyperparameter settings, loss functions, data segmentation strategies, and data augmentation methods, to avoid performance biases, and all were ensured to be the optimal parameters of the model.
[0099] Figure 5 Shown are two breast ultrasound images selected from the BUSI dataset, the corresponding ground truth segmentation maps, the optimal segmentation map of the method in Reference 1, and the optimal segmentation image of the method in Example 1.
[0100] From the experimental results, the detection method of Example 1 is superior to that of Reference 1.
[0101] Table 1 summarizes the experimental data of Reference 1 and Example 1 on the BUSI dataset. From the experimental results, the detection method of Example 1 is superior to that of Reference 1.
[0102] Table 1 Quantitative data
[0103]
Claims
1. A breast ultrasound image segmentation method based on a multi-level feature integration network, characterized in that: The following steps are involved: A. Construct a neural network. The structure of the neural network is as follows: Including encoding network and decoding network; The encoding network is provided with a FEB module and multiple groups of MRAB modules; B. The original image is input into the encoding network. In the encoding network: The original image is processed by the FEB module to obtain four output results, namely G 1 Output result, G 2 Output result, G 3 Output result, G 4 Output the result; The original image is convolved with G after 7×7 convolution. 1 After the output results are added and fused, the F output result is obtained; After the output of F is processed by the maximum pooling and two residual modules in sequence, the residual processing result and the output of F are respectively input into the first MRAB module for processing to obtain the first MRAB processing result; the first MRAB processing result and G 2 After the output results are added and fused, we get F with double the number of channels and half the size. 1 Output the result; F 1 After the output results are processed by three residual modules in sequence, the residual processing results and F 1 The output results are respectively input into the second MRAB module for processing to obtain the second MRAB processing results; The second MRAB treatment results and G 3 After the output results are added and fused, we get F with double the number of channels and half the size. 2 Output the result; F 2 After the output results are processed by five residual modules in sequence, the residual processing results and F 2 The output results are respectively input into the third MRAB module for processing to obtain the third MRAB processing results; the third MRAB processing results and G 4 After the output results are added and fused, we get F with double the number of channels and half the size. 3 Output the result; F 3 After the output results are processed by two residual modules in turn, the residual processing results and F 3 The output results are input into the fourth MRAB module for processing, and the F 4 Output the result; The F output result, F 1 Output result, F 2 Output result, F 3 Output result, F 4 The output results are input into the decoding network respectively; C. The decoding network fuses all input results to obtain the final segmentation result.
2. The breast ultrasound image segmentation method based on a multi-level feature integration network as claimed in claim 1, characterized in that: The processing process in the FEB module is as follows After the original image is processed by 1×1 convolution to change the number of channels, it is split into four parts by the Split function, namely G 1 Division, G 2 Division, G 3 Division, G 4 Division; G 1 The segmentation part is divided into two paths. The first path is processed by the center difference convolution, BN function, and ReLU function in sequence to obtain the first path result; the second path is processed by 3×3 depth separable convolution, BN function, and ReLU function in sequence to obtain the second path result; the first path result and the second path result are concatenated by the Concat function, and then processed by 3×3 depth separable convolution, BN function, ReLU function, and maximum pooling in sequence to obtain G 1 Intermediate result, G 1 After the intermediate result is processed by 1×1 convolution, G is obtained. 1 Output the result; G 2 The segmentation part is divided into two paths. The first path is processed by the center difference convolution, BN function, and ReLU function in sequence to obtain the first path result; the second path is processed by 3×3 depth separable convolution, BN function, and ReLU function in sequence to obtain the second path result; the first path result and the second path result are concatenated by the Concat function, and then processed by 3×3 depth separable convolution, BN function, ReLU function, and maximum pooling in sequence. The result is the same as G 1 After the intermediate results are concatenated by the Concat function, they are processed by 3×3 depth-separable convolution, BN function, ReLU function, and maximum pooling in sequence to obtain G 2 Intermediate result, G 2 The intermediate result is processed by 1×1 convolution and then compared with the upsampled G 1 The output results are added and fused, and then processed by 1×1 convolution to obtain G 2 Output the result; G 3 The segmentation part is divided into two paths. The first path is processed by central difference convolution, BN function, and ReLU function in sequence to obtain the first path result; the second path is processed by 3×3 depth separable convolution, BN function, and ReLU function in sequence to obtain the second path result; the first path result and the second path result are concatenated by Concat function, and then processed by 3×3 depth separable convolution, BN function, ReLU function, maximum pooling, 3×3 depth separable convolution, BN function, ReLU function, and maximum pooling in sequence. The result is the same as G 2 After the intermediate results are concatenated by the Concat function, they are processed by 3×3 depth-separable convolution, BN function, ReLU function, and maximum pooling in sequence to obtain G 3 Intermediate result, G 3 The intermediate result is processed by 1×1 convolution and then compared with the upsampled G 2 The output results are added and fused, and then processed by 1×1 convolution to obtain G 3 Output the result; G 4 The segmentation part is divided into two paths. The first path is processed by central difference convolution, BN function, and ReLU function in sequence to obtain the first path result; the second path is processed by 3×3 depth separable convolution, BN function, and ReLU function in sequence to obtain the second path result; the first path result and the second path result are concatenated by Concat function, and then processed by 3×3 depth separable convolution, BN function, ReLU function, maximum pooling, 3×3 depth separable convolution, BN function, ReLU function, maximum pooling, 3×3 depth separable convolution, BN function, ReLU function, maximum pooling, and the result is the same as G 3 After the intermediate results are concatenated by the Concat function, they are processed by 3×3 depth-separable convolution, BN function, ReLU function, and maximum pooling in sequence to obtain G 4 Intermediate result, G 4 The intermediate result is processed by 1×1 convolution and then compared with the upsampled G 3 The output results are added and fused, and then processed by 1×1 convolution to obtain G 4 Output the result.
3. The breast ultrasound image segmentation method based on a multi-level feature integration network as claimed in claim 1, characterized in that: The processing process in the residual module is as follows: The input result is processed by two 3×3 convolutions in sequence, and the obtained result is added and fused with the input result to obtain the output result.
4. The breast ultrasound image segmentation method based on a multi-level feature integration network as claimed in claim 1, characterized in that: The first MRAB module, the second MRAB module, the third MRAB module, and the fourth MRAB module have the same structure, wherein the processing process is as follows: The input residual processing result is divided into two paths. The first path is processed by 3×3 convolution to obtain the first convolution result, and the second path is processed by 5×5 convolution to obtain the second convolution result. The output result of the previous level F is downsampled and processed by 3×3 convolution in sequence, and the obtained result is added and fused with the first convolution result and the second convolution result to obtain the added fusion result; The addition fusion results are divided into three paths. The first path is processed by 1×1 convolution to obtain the first path result; The second path is processed by 1×1 convolution to obtain the second path result; the third path is processed by 1×1 convolution and Transpose function in turn to obtain the third path result; After the second result is multiplied and fused with the third result, the result is multiplied and fused with the first result, and then added and fused with the input residual processing result to obtain the output result.
5. The breast ultrasound image segmentation method based on a multi-level feature integration network as claimed in claim 1, characterized in that: The processing process in the encoding network is as follows: F 4 After upsampling, the output result is compared with F 3 After the output results are concatenated by the Concat function, they are processed by 3×3 convolution, BN function, ReLU function, 3×3 convolution, BN function, and ReLU function in sequence to obtain the first layer result; The first layer results are similar to F 2 The output results are respectively input into the first CIB module for processing to obtain the first CIB processing results. After the first CIB processing results and the first layer results are concatenated by the Concat function, the obtained results are processed by 3×3 convolution, BN function, ReLU function, 3×3 convolution, BN function, and ReLU function in sequence to obtain the second layer results. The second layer results are similar to F 1 The output results are respectively input into the second CIB module for processing to obtain the second CIB processing results. After the second CIB processing results and the second layer results are concatenated by the Concat function, the obtained results are sequentially processed by 3×3 convolution, BN function, ReLU function, 3×3 convolution, BN function, and ReLU function to obtain the third layer results. The third layer result and the F output result are respectively input into the third CIB module for processing to obtain the third CIB processing result. The third CIB processing result and the third layer result are concatenated by the Concat function. The obtained result is processed by 3×3 convolution, BN function, ReLU function, 3×3 convolution, BN function, and ReLU function in sequence to obtain the fourth layer result. The fourth layer result is up-sampled and 1×1 convolution to obtain the output result.
6. The breast ultrasound image segmentation method based on a multi-level feature integration network as claimed in claim 5, characterized in that: The first CIB module, the second CIB module, and the third CIB module have the same structure, wherein the processing process is as follows: The encoding network input result is processed by 1×1 convolution and Sigmoid function in sequence to obtain a first intermediate result. After the first intermediate result is processed by Transpose function, the obtained result and the encoding network input result are processed by dot product to obtain a first dot product result. The input result of the previous layer is divided into three branches. After upsampling, the first branch is dot-product processed with the first intermediate result to obtain the second dot-product result; The second branch is processed by 1×1 convolution and Sigmoid function in sequence. The result is added and fused with the first intermediate result. The result is then processed by dot product with the encoding network input result. Then, it is processed by 3×3 dilated convolution with a dilation factor of 2 to obtain the result of the second branch. The third branch is processed by 3×3 convolution and 1×1 convolution in sequence to obtain the result of the third branch; After the first dot product result and the second dot product result are added and fused, the result and the encoding network input result are subjected to dot product processing, and then added and fused with the encoding network input result to obtain a second intermediate result; The second intermediate result is divided into two branches. The first branch is processed by 1×1 convolution to obtain the first branch result. The second branch is processed by 3×3 convolution to obtain the result of the second branch; After the results of the first branch, the second branch, the second branch, and the third branch are added and fused, the results are added and fused with the encoding network input results and the previous layer input results. The results are processed by 1×1 convolution, BN function, 3×3 convolution, and ReLU function in sequence to obtain the output result.