A dual-view breast ultrasound classification method based on lesion mask guidance

Abstract

Invention Title: A Lesion Mask-Guided Dual-View Breast Ultrasound Classification Method. This invention discloses a breast ultrasound image grading method based on lesion prior guidance and dual-view fusion, belonging to the field of medical image processing and artificial intelligence technology. The method includes: S1, acquiring axial and sagittal images of the case to be processed and corresponding lesion masks; S2, spatially aligning and channel-overlaying the images and lesion masks of each view to construct a four-channel input; S3, feeding the four-channel dual-view input into a dual-branch feature extraction network to obtain lesion-specific features for each view; S4, performing cross-view fusion of the features and outputting the BI-RADS grading result. This invention improves the accuracy and stability of breast ultrasound image grading by introducing lesion region prior information and combining complementary features from both views.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence technology, specifically to a dual-view breast ultrasound classification method guided by lesion masking. Background Technology

[0002] Breast ultrasound is widely used for screening and auxiliary assessment of breast lesions due to its non-invasive, convenient, and real-time characteristics. For the BI-RADS classification task of breast ultrasound images, most existing studies directly base classification modeling on single-view images. However, breast ultrasound images typically suffer from strong speckle noise, complex background tissue, blurred lesion boundaries, and uneven grayscale distribution, making the classification model susceptible to interference from non-lesion areas, thus affecting the accuracy and stability of the classification results.

[0003] During clinical examinations, multiple views of breast ultrasound images are typically acquired for the same case. Axial and sagittal images can reflect the morphology, boundaries, and internal structure of lesions from different directions, and the two are highly complementary. Current methods for classifying breast ultrasound images often focus on single-view analysis or simply stitch together multi-view information, failing to effectively utilize the complementary features between different views.

[0004] Furthermore, lesion masks can provide prior information such as the location, extent, and morphology of lesions, guiding classification models to focus on lesion areas and reducing interference from background tissue. Existing methods typically do not represent lesion masks and original images uniformly at the input level, resulting in insufficient utilization of prior lesion knowledge during the classification process.

[0005] Therefore, it is necessary to propose a breast ultrasound classification method based on lesion masking and dual-view fusion to fully integrate axial and sagittal image information and introduce prior knowledge of lesion regions, thereby improving the accuracy and stability of BI-RADS classification of breast ultrasound images. Summary of the Invention

[0006] The purpose of this invention is to provide a dual-view breast ultrasound classification method based on lesion masking, in order to solve the problems of insufficient attention to lesion areas, inadequate utilization of complementary information in dual views, and significant interference from complex backgrounds in the existing single-view breast ultrasound image classification.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A dual-view breast ultrasound classification method guided by lesion masking includes the following steps:

[0009] S1: Obtain axial breast ultrasound images, sagittal breast ultrasound images, and binary masks of lesions corresponding to the axial and sagittal breast ultrasound images of the case to be processed.

[0010] S2: Spatial alignment of the breast ultrasound images in each view with the corresponding binary lesion mask, and the alignment binary lesion mask as the fourth channel is stitched with the corresponding three-channel breast ultrasound images to construct axial four-channel input tensors and sagittal four-channel input tensors respectively.

[0011] S3: Input the axial four-channel input tensor and the sagittal four-channel input tensor into the dual-branch feature extraction model respectively to extract the lesion-specific features of each view;

[0012] S4: Perform cross-view fusion of lesion-specific features from each view to obtain fused features;

[0013] S5: Output the BI-RADS classification results of the case to be processed based on the fusion features.

[0014] Furthermore, the binary mask for the lesion is generated by manual annotation, semi-automatic annotation, or segmentation model.

[0015] Furthermore, the three-channel breast ultrasound image is a three-channel image copied from the original grayscale breast ultrasound image, or a three-channel image formed after image enhancement.

[0016] Furthermore, the dual-branch feature extraction model employs a shared weight design.

[0017] Furthermore, the dual-branch feature extraction model is EfficientNet-B3.

[0018] Furthermore, the cross-view fusion employs an attention-weighted fusion module to dynamically learn the fusion weights of features from each view.

[0019] Furthermore, the BI-RADS classification results include any one of BI-RADS 2, BI-RADS 3, BI-RADS 4A, BI-RADS 4B, BI-RADS 4C, and BI-RADS 5.

[0020] Beneficial effects

[0021] Compared with the prior art, the present invention has at least the following beneficial effects:

[0022] By using a binary mask of the lesion as the fourth channel and stitching it with the breast ultrasound image, prior information about the lesion region can be introduced at the input level, enhancing the classification model's ability to focus on the lesion region.

[0023] By simultaneously utilizing axial and sagittal images for dual-branch feature extraction, the complementary information between the two views can be fully explored.

[0024] By jointly modeling features from different views through a cross-view fusion mechanism, the accuracy and stability of BI-RADS classification of breast ultrasound images can be improved.

[0025] By using a dual-branch feature extraction structure with shared weights, the number of model parameters can be reduced while ensuring consistency in feature extraction across different views.

[0026] This invention is applicable to various prior sources of lesions, such as manually labeled masks and automatically generated masks, and has good adaptability and application value. Attached Figure Description

[0027] Figure 1 This is a flowchart of a dual-view breast ultrasound classification method guided by lesion masking in an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of the construction of a four-channel input tensor in an embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram of the network structure for dual-branch feature extraction and cross-view fusion in an embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram of the overall framework of the breast ultrasound classification method in an embodiment of the present invention. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited to the following embodiments.

[0032] Example 1: A dual-view breast ultrasound classification method guided by lesion masking

[0033] like Figure 1 As shown in the figure, this embodiment provides a dual-view breast ultrasound classification method based on lesion masking, which includes the following steps.

[0034] Step S1: Obtain dual-view breast ultrasound images and binary masks of lesions.

[0035] Axial and sagittal breast ultrasound images of the case to be processed are acquired, and binary masks of lesions corresponding to each view are obtained. These binary masks can be obtained through manual annotation, semi-automatic interactive methods, or automatically generated using an existing segmentation model. The binary masks are used to characterize the location and extent of the lesion region, providing prior information about the lesion region for subsequent classification models.

[0036] Step S2: Construct a four-channel input tensor.

[0037] Spatially align the breast ultrasound images of each view with the corresponding binary masks of the lesions to ensure that the binary masks of the lesions are consistent with the corresponding breast ultrasound images in terms of size and spatial location. Let the original grayscale breast ultrasound image in the axial view be... The original grayscale breast ultrasound image in the sagittal plane is The corresponding binary masks for the lesions are as follows: and

[0038] For any view Let the spatial alignment transformation be... Then we have:

[0039] in, Represents the space-aligned first... One view of breast ultrasound images, Represents the space-aligned first... A binary mask for each view of the lesion.

[0040] After spatial alignment, the grayscale breast ultrasound image is copied into a three-channel image. Let the three-channel copying transformation be... Then we have:

[0041]

[0042] Subsequently, the binary mask of the lesion is used as the fourth color channel and stitched together with the corresponding three-channel breast ultrasound image along the channel dimension to construct a dual-view image. Figure 4 Channel input tensor:

[0043] ,in, This represents an axial four-channel input tensor. This represents the four-channel input tensor in the sagittal position. This indicates a channel dimension splicing operation. Figure 2 The construction of a four-channel input tensor is shown, where the first three channels are breast ultrasound image channels and the fourth color channel is a lesion binary mask channel.

[0044] In one implementation, the three-channel breast ultrasound images are standardized. Let the unstandardized images be... The pixel value of each channel is The standardization result is:

[0045] in, and They represent the first The mean and standard deviation of each channel. In one implementation, , .

[0046] Step S3: Dual-branch feature extraction.

[0047] Input the axial four-channel tensor and sagittal four-channel input tensor The data are input into a dual-branch feature extraction model to extract lesion-specific features for each view. In this embodiment, the dual-branch feature extraction model includes an axial feature extraction branch and a sagittal feature extraction branch, and the two branches adopt a shared weight design. Let the feature extraction network with shared weights be... ,in To share network parameters, we have:

[0048]

[0049]

[0050] in, This indicates the specific characteristics of axial lesions. This represents the specific characteristics of lesions in the sagittal plane. Furthermore, the feature extraction network can use EfficientNet-B3 as its backbone to extract deep feature information related to BI-RADS classification.

[0051] Step S4: Cross-view merging.

[0052] Specific features of axial lesions obtained in step S3 Sagittal lesion specific characteristics Cross-view fusion is performed to form a fusion feature that comprehensively reflects the information from both views. In this embodiment, the cross-view fusion employs an attention-weighted fusion module. Let the attention score function be... The attention scores corresponding to each view feature are as follows:

[0053]

[0054]

[0055] In one implementation, the attention scoring function can be expressed as:

[0056]

[0057] in, For learnable parameters, This represents a non-linear activation function.

[0058] Calculate normalized fusion weights based on the attention scores of each view feature:

[0059]

[0060]

[0061] in, and Let represent the fusion weights of axial and sagittal features, respectively, and satisfy . .

[0062] Based on the fusion weights, the features of the two views are weighted and fused to obtain the fused features:

[0063] in, This indicates a fusion feature that comprehensively reflects complementary information from axial and sagittal positions.

[0064] Step S5: Output the BI-RADS classification results.

[0065] The fused features obtained in step S4 are input into the classification module, which outputs the BI-RADS classification results for the cases to be processed. Let the classifier be... Then the predicted probability vector output by the classifier is:

[0066]

[0067] in, and These are the learnable parameters for the classification module. This represents the predicted probability distribution for each BI-RADS category.

[0068] In one implementation, if the BI-RADS classification results include BI-RADS 2, BI-RADS 3, BI-RADS 4A, BI-RADS 4B, BI-RADS 4C, and BI-RADS 5, then the predicted probability vector can be expressed as:

[0069]

[0070] in, These represent the predicted probability that a sample belongs to the corresponding BI-RADS category.

[0071] The final classification result can be represented as:

[0072]

[0073] in, This indicates the final BI-RADS classification result for the case to be processed.

[0074] During the model training phase, training samples labeled with BI-RADS can be used to jointly train the dual-branch feature extraction model, the cross-view fusion module, and the classification module. Let the true label vector be... The predicted probability vector is The classification loss function can then be expressed as:

[0075]

[0076] in, This indicates the total number of classification categories. During the model inference phase, the input consists of a dual-view breast ultrasound image of the case to be processed and the corresponding binary mask of the lesion. After processing by the dual-branch feature extraction, cross-view fusion, and classification modules, the corresponding BI-RADS classification results can be output.

Claims

1. A dual-view breast ultrasound classification method based on lesion mask guidance, characterized in that, Includes the following steps: S1. Obtain axial breast ultrasound images, sagittal breast ultrasound images, and binary masks of lesions corresponding to the axial and sagittal breast ultrasound images of the case to be processed. S2. Spatial alignment of the breast ultrasound images in each view with the corresponding binary lesion mask, and the alignment binary lesion mask as the fourth channel is stitched with the corresponding three-channel breast ultrasound images to construct axial four-channel input tensors and sagittal four-channel input tensors respectively. S3. Input the axial four-channel input tensor and the sagittal four-channel input tensor into the dual-branch feature extraction model respectively to extract the lesion-specific features of each view; S4. Perform cross-view fusion of lesion-specific features from each view to obtain fused features; S5. Output the BI-RADS classification results of the case to be processed based on the fusion features.

2. The breast ultrasound image grading method of claim 1, wherein, The binary mask for the lesion is generated by manual annotation, semi-automatic annotation, or segmentation model.

3. The breast ultrasound image grading method of claim 1, wherein, The three-channel breast ultrasound image in step S2 is either a three-channel image copied from the original grayscale breast ultrasound image, or a three-channel image formed after image enhancement.

4. The breast ultrasound image grading method of claim 1, wherein, The dual-branch feature extraction model in step S3 adopts a shared weight design.

5. The breast ultrasound image grading method of claim 1 or 4, wherein, The dual-branch feature extraction model is EfficientNet-B3.

6. The breast ultrasound image grading method of claim 1, wherein, The cross-view fusion in step S4 employs an attention-weighted fusion module to dynamically learn the fusion weights of features for each view.

7. The method of claim 1, wherein, The cross-view fusion method in step S4 includes one or more of feature stitching, weighted summation, or attention fusion.

8. The method of claim 1 or 6, wherein, The cross-view fusion includes calculating corresponding fusion weights based on the features of each view, and performing weighted fusion of the features of each view based on the fusion weights.

Images