Breast ultrasonic image classification method and device based on deep learning and medium
By introducing an attention mechanism and combining BI-RADS labels into the classification of breast ultrasound images, an evaluation and validation model was constructed, which solved the problem that classification results could not be automatically verified in existing technologies, achieving higher accuracy and reducing the burden of expert review.
Patent Information
- Application Number
- CN202511835408.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-08
- Publication Date
- 2026-02-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing breast ultrasound image classification methods cannot automatically verify the accuracy of classification results after training, which requires expert review and increases the workload of experts.
A deep learning-based breast ultrasound image classification method was adopted. An evaluation model and a validation model were constructed by introducing two attention mechanisms. Data quality was detected and enhanced by combining BI-RADS labels. The comparison between the malignancy probability assessment value and the BI-RADS classification prediction value was used to reduce the need for expert review.
It improves the accuracy of breast ultrasound image classification, reduces the burden of expert review, and enhances the reliability and efficiency of classification results.
Smart Images
Figure CN121459073A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of ultrasound image classification technology, and in particular to a method, device and medium for classifying breast ultrasound images based on deep learning. Background Technology
[0002] Breast ultrasound imaging is a non-invasive imaging technique that uses high-frequency sound waves to generate images of the internal structure of breast tissue. Its principle is based on the differences in the propagation speed and reflection characteristics of sound waves between different tissues. When ultrasound waves pass through breast tissue, they encounter tissues of different densities and elasticities (such as fat, glands, and lesions), resulting in reflection and attenuation, thus forming an image. The ultrasound device converts the reflected sound wave signals into electrical signals, which are then processed by a computer to generate the image. Breast ultrasound can not only display the morphological characteristics of the breast but also assess blood flow using Doppler technology, helping to identify the nature of lesions (such as benign or malignant). Due to its advantages of being radiation-free, providing real-time imaging, and clearly displaying the internal structure of the breast, breast ultrasound has become an important tool for screening and diagnosing breast diseases. To standardize and systematically assess the benign and malignant risk of breast lesions, breast ultrasound image classification is needed, providing crucial information for clinical diagnosis and treatment decisions. Existing breast ultrasound image classification methods often incorporate deep learning technology. These methods process the acquired breast ultrasound images, dividing them into training, testing, and validation sets for direct model training. The trained model then assists in classifying the breast ultrasound images. However, this approach relies solely on training results, making it impossible to verify the accuracy of the classification after the results are obtained. Therefore, expert review is required to ensure accuracy, placing a burden on expert reviewers. In light of this, this proposal suggests a deep learning-based breast ultrasound image classification method, equipment, and media to address these issues. Summary of the Invention
[0003] The purpose of this invention is to provide a method, device, and medium for classifying breast ultrasound images based on deep learning, so as to solve the problems mentioned in the background art.
[0004] To achieve the above objectives, the present invention provides the following technical solution: a breast ultrasound image classification method based on deep learning, wherein the classification method includes the following steps: S1. First, breast ultrasound image data is collected. After the dataset is established, the ultrasound physician delineates and marks the lesion areas in the images. The markings include benign labels, malignant labels, and BI-RADS labels. S2, two attention mechanisms are introduced into the convolutional neural network to construct two sets of modified models. Then, the image data with benign and malignant labels are input into one set of modified models for training to obtain the evaluation model, and the image data with BI-RADS labels are input into the other set of modified models for training to obtain the validation model. S3: Perform quality inspection on the image data to be classified, filter out information with clear and accurate location and directly input it into the evaluation model to obtain the malignancy probability evaluation value, perform data augmentation on unclear image data and then re-inspect the quality, and record the number of times the same set of data is inspected. Data that fails more than twice is directly deleted. S4. Input the image that has passed the quality inspection into the verification model to obtain the BI-RADS classification prediction value, and determine whether the malignancy probability assessment value is consistent with the BI-RADS classification prediction value. If they are consistent, output the BI-RADS classification prediction value. If they are inconsistent, mark it as an expert consultation. S5 summarizes the model classification and expert consultation results and outputs a classification report.
[0005] Preferably, S1 includes the following specific operations: S101, firstly, breast ultrasound images are obtained from the publicly available datasets BUSI and Dataset B; S102, the ultrasound doctor outlines a closed area along the edge of the lesion; S103. After delineating the lesion area, the doctor assigns one or more semantic labels to the area based on medical knowledge, including benign labels, malignant labels, and BI-RADS labels.
[0006] Preferably, the breast ultrasound image data is collected from breast ultrasound images from different hospitals and different ultrasound equipment.
[0007] Preferably, the two sets of modified models are established by first selecting a convolutional neural network architecture, then inserting attention modules after the convolutional layers of the convolutional neural network architecture, and finally using a deep learning framework to build two independent models with different internal structures. After training, they are respectively designated as the evaluation model and the validation model.
[0008] Preferably, a clinical rule mapping table is set between the assessment model and the validation model to associate BI-RADS categories with the expected range of malignancy probability.
[0009] Preferably, in the image data quality monitoring, image data with clear lesion areas, good probe contact, no significant shadows or artifacts, and containing complete lesions are considered qualified data, while image data with blurred images, excessive or insufficient probe pressure, shadows covering key areas, and lesions not being completely scanned are considered unqualified data.
[0010] Preferably, the enhancement of the defective data includes random rotation of the image, horizontal / vertical flipping, brightness / contrast adjustment, elastic deformation, and addition of Gaussian noise.
[0011] Preferably, the classification report includes BI-RADS classified cases and their information, a list of cases marked as requiring expert consultation, inconsistent results between the two models, and overall diagnostic efficiency and case triage statistics.
[0012] A deep learning-based breast ultrasound image classification device is used to implement the deep learning-based breast ultrasound image classification method. The classification device consists of a data acquisition module, a data processing and storage module, a deep learning framework, a data quality detection module, a data augmentation module, and a result output module. The data acquisition module is used to collect breast ultrasound imaging data; The data processing and storage module is used to store image data and its tagging information; The deep learning framework is used to provide APIs and tools for building and training deep learning models; The data quality detection module is used to detect the quality of image data; The data enhancement module is used to enhance image data; The results output module is used to summarize and output model evaluation results, expert consultation opinions, and other statistical information.
[0013] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the deep learning-based breast ultrasound image classification method.
[0014] The technical effects and advantages of this invention are as follows: This invention first labels the collected data with benign, malignant, and BI-RADS labels. Then, it constructs two independent modified models using two attention mechanisms. These models are then trained separately on the benign, malignant, and BI-RADS labels to obtain evaluation and validation models. In the classification of breast ultrasound images, the accuracy of the classification is increased by comparing the consistency between the malignancy probability assessment value and the BI-RADS classification prediction value, thereby reducing the burden of expert review. Attached Figure Description
[0015] Figure 1 This is an operation flowchart illustrating the steps of the classification method of the present invention. Detailed Implementation
[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0017] Example 1: The present invention provides as follows Figure 1 The deep learning-based breast ultrasound image classification method shown includes the following steps: S1. First, breast ultrasound image data is collected. After the dataset is established, the ultrasound physician delineates and marks the lesion areas in the images. The markings include benign labels, malignant labels, and BI-RADS labels. Specifically, S1 includes the following operations: S101 First, breast ultrasound images are obtained from the publicly available datasets BUSI and Dataset B. The breast ultrasound image data is collected from breast ultrasound images from different hospitals and different ultrasound equipment. After the ultrasound images are collected, the image size is adjusted to fit the network input and the pixel value range is unified. S102, the ultrasound doctor outlines a closed area along the edge of the lesion; S103. After delineating the lesion area, the doctor assigns one or more semantic labels to the area based on medical knowledge, including benign labels, malignant labels, and BI-RADS labels.
[0018] It should be noted that the imaging content of the lesion includes mass, calcification, structural distortion, and lesion-related features. Among them, the mass content includes shape: oval, round, and irregular; orientation: parallel (parallel to the skin, usually more benign) or non-parallel (vertical growth, more suspicious); margin: smooth (clear and smooth) or irregular (blurred, angular, spiculated), with spiculated margins being a typical feature of malignancy; boundary: clear or unclear; internal echo: anechoic (simple cyst), hypoechoic, isoechoic, hyperechoic, and mixed echo; calcification content: ultrasound is less sensitive to calcification than mammography, but coarse calcifications are seen in benign lesions, while clustered microcalcifications suggest malignancy; structural distortion refers to the interruption and disorder of normal breast structure layers, but without obvious mass; lesion-related features include changes in surrounding tissue: thickened and dimpled skin, disordered echoes in the subcutaneous fat layer; blood flow signal: blood flow within and around the lesion is assessed using color Doppler ultrasound. Malignant lesions often exhibit abundant blood flow and disordered course. Elastography assesses tissue stiffness; malignant tumors are typically harder and appear blue (hard) on elastography images.
[0019] S2, two attention mechanisms are introduced into the convolutional neural network to construct two sets of modified models. Then, the image data with benign and malignant labels are input into one set of modified models for training to obtain the evaluation model, and the image data with BI-RADS labels are input into the other set of modified models for training to obtain the validation model. It should be noted that the chosen convolutional neural networks utilize pre-trained models on large natural image datasets, including VGG, ResNet, DenseNet, and EfficientNet, transferring their knowledge to the specific domain of breast ultrasound. The convolutional bases of the pre-trained model are retained for feature extraction, while the top fully connected classification layer is replaced to accommodate the number of classification categories in breast ultrasound. The model is then fine-tuned using breast ultrasound data. Typically, the convolutional bases are frozen first, training only the newly added classifier, and then some or all of the convolutional layers are unfrozen for fine-tuning. In this approach, the attention mechanism includes the introduction of channel attention and spatial attention modules, enabling the model to focus on the lesion area and suppress background interference. This includes embedding SE-Net and CBAM modules into the pre-trained model.
[0020] Specifically, the two sets of modified models are established by first selecting a convolutional neural network architecture, then inserting attention modules after the convolutional layers of the convolutional neural network architecture, and finally using a deep learning framework to build two independent models with different internal structures. After training, they are designated as the evaluation model and the validation model, respectively.
[0021] It should be noted that attention mechanisms include: Channel attention is achieved by calculating the importance of each feature channel and adjusting the weights of different channels using the Squeeze-and-Excitation (SE) module. Spatial attention: By calculating the importance of different spatial locations in an image, the influence of pixels is adjusted, using non-local attention or convolutional attention; Self-attention: Establishes relationships between different regions of an image. It is often used in Transformer models and is also combined with CNNs.
[0022] Furthermore, a clinical rule mapping table is set up between the assessment model and the validation model to associate BI-RADS categories with the expected range of malignancy probabilities.
[0023] S3: Perform quality inspection on the image data to be classified, filter out information with clear and accurate location and directly input it into the evaluation model to obtain the malignancy probability evaluation value, perform data augmentation on unclear image data and then re-inspect the quality, and record the number of times the same set of data is inspected. Data that fails more than twice is directly deleted. It should be noted that the evaluation model will output a Logit (unnormalized score) or Softmax probability. For binary classification, the output is a probability value P1 (probability of malignancy) between 0 and 1. When labeling the classification, P1 > 0.5 is defined as the prediction result being malignant, and P1 ≤ 0.5 is defined as the prediction result being benign.
[0024] Specifically, in image data quality monitoring, image data with clear lesion areas, good probe contact, no significant shadows or artifacts, and containing complete lesions are considered qualified data, while image data with blurry images, excessive or insufficient probe pressure, shadows covering key areas, and lesions not being completely scanned are considered unqualified data.
[0025] Further enhancements to substandard data include random image rotation (±30°), horizontal / vertical flipping, brightness / contrast adjustment, elastic deformation, and the addition of Gaussian noise to increase data diversity and prevent model overfitting.
[0026] It should be noted that random rotation refers to randomly rotating the image content within a range of ±30°. For breast ultrasound images, a rotation of ±30° can simulate shooting from different angles, helping the model learn lesion features from different directions. This method improves the model's robustness to changes in image orientation and reduces bias caused by specific orientations. Horizontal / vertical flipping is a simple yet effective enhancement technique that can generate a mirrored version of the image. Since lesions in breast ultrasound images do not always have directional characteristics, the flipping operation can effectively increase the number of training samples without changing the essential characteristics of the lesions in the image. Adjusting brightness and / or contrast can simulate image changes under different imaging devices or shooting conditions. This enhancement method enables the model to adapt to different lighting conditions and image quality, which helps to improve the model's performance in real environments. By randomly changing brightness and contrast, the model can better identify lesion areas under different conditions. Elastic deformation is a more complex transformation method that simulates the natural variations in tissue structures in an image by locally distorting it. This method can increase the diversity of training data, enabling the model to better learn the morphological changes and complex structures of breast tissue and improve the ability to detect subtle lesions. Adding Gaussian noise can simulate random interference in images, noise that may occur during the imaging process. This enhancement method can improve the model's robustness to noise and help the model maintain high accuracy when processing imperfect image data.
[0027] S4. Input the image that has passed the quality inspection into the verification model to obtain the BI-RADS classification prediction value, and determine whether the malignancy probability assessment value is consistent with the BI-RADS classification prediction value. If they are consistent, output the BI-RADS classification prediction value. If they are inconsistent, mark it as an expert consultation. It should be noted that the BI-RADS classification is divided into multiple levels, typically from 0 to 6, as detailed below: BI-RADS0: Incomplete assessment. Description: Imaging findings are incomplete and further imaging (additional mammography, ultrasound, or MRI) is needed to obtain more information. Action recommendation: Perform follow-up examinations.
[0028] BI-RADS1: Negative. Description: No abnormalities detected; breast tissue normal. Action Recommendation: Follow up according to the routine screening program.
[0029] BI-RADS2: Benign. Description: Benign lesions (cysts, fibroadenomas) were found. These lesions usually do not require further intervention. Action Recommendation: Follow up according to the routine screening program.
[0030] BI-RADS3: Possibly Benign. Description: The lesion found has benign characteristics and the probability of malignancy is very low (<2%). Action Recommendation: Follow-up imaging is recommended in the short term (usually 6 months) to confirm the stability of the lesion.
[0031] BI-RADS4: Suspicious for Malignancy. Description: The lesion found has malignant characteristics and carries an intermediate risk (2% - 95%). Action Recommendation: A tissue biopsy is recommended to confirm the nature of the lesion.
[0032] BI-RADS5: Highly Suspicious for Malignancy. Description: The lesion exhibits clear malignant characteristics, with a high probability of malignancy (>95%). Action Recommendation: A tissue biopsy is recommended, and further treatment is usually required as soon as possible.
[0033] BI-RADS6: Known Malignancy. Description: Malignant lesion confirmed by biopsy. Action Recommendation: Develop a treatment plan based on pathology results.
[0034] S5 summarizes the model classification and expert consultation results and outputs a classification report. The classification report includes BI-RADS classified cases and their information, a list of cases marked as requiring expert consultation, inconsistent results between the two models, and overall diagnostic efficiency and case triage statistics.
[0035] Example 2: A breast ultrasound image classification device based on deep learning, used to implement the breast ultrasound image classification method based on deep learning in Example 1. The classification device consists of a data acquisition module, a data processing and storage module, a deep learning framework, a data quality detection module, a data augmentation module, and a result output module. The data acquisition module is used to collect breast ultrasound image data. The data acquisition module retrieves breast image data from the public datasets BUSI and Dataset B in the cloud database. The data processing and storage module is used to securely store image data and its tagging information, and supports fast access and retrieval of the data; Deep learning frameworks provide APIs and tools for building and training deep learning models, supporting custom model architectures and training processes, including the implementation of attention mechanisms; The data quality inspection module is used to inspect the quality of image data, automating the quality inspection and determining whether the image meets the standards for sharpness and integrity. The data augmentation module is used to enhance image data, performing enhancement processing on substandard image data to increase the diversity of training samples; The results output module is used to summarize and output model evaluation results, expert consultation opinions, and other statistical information to support doctors' clinical decision-making.
[0036] Example 3: A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the deep learning-based breast ultrasound image classification method of Example 1.
[0037] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A deep learning-based breast ultrasound image classification method, characterized in that, The classification method includes the following steps: S1. First, breast ultrasound image data is collected. After the dataset is established, the ultrasound physician delineates and marks the lesion areas in the images. The markings include benign labels, malignant labels, and BI-RADS labels. S2, two attention mechanisms are introduced into the convolutional neural network to construct two sets of modified models. Then, the image data with benign and malignant labels are input into one set of modified models for training to obtain the evaluation model, and the image data with BI-RADS labels are input into the other set of modified models for training to obtain the validation model. S3: Perform quality inspection on the image data to be classified, filter out information with clear and accurate location and directly input it into the evaluation model to obtain the malignancy probability evaluation value, perform data augmentation on unclear image data and then re-inspect the quality, and record the number of times the same set of data is inspected. Data that fails more than twice is directly deleted. S4. Input the image that has passed the quality inspection into the verification model to obtain the BI-RADS classification prediction value, and determine whether the malignancy probability assessment value is consistent with the BI-RADS classification prediction value. If they are consistent, output the BI-RADS classification prediction value. If they are inconsistent, mark it as an expert consultation. S5 summarizes the model classification and expert consultation results and outputs a classification report.
2. The breast ultrasound image classification method based on deep learning according to claim 1, characterized in that, S1 includes the following specific operations: S101, firstly, breast ultrasound images are obtained from the publicly available datasets BUSI and Dataset B; S102, the ultrasound doctor outlines a closed area along the edge of the lesion; S103. After delineating the lesion area, the doctor assigns one or more semantic labels to the area based on medical knowledge, including benign labels, malignant labels, and BI-RADS labels.
3. The breast ultrasound image classification method based on deep learning according to claim 2, characterized in that, The breast ultrasound image data was collected from breast ultrasound images from different hospitals and different ultrasound equipment.
4. The breast ultrasound image classification method based on deep learning according to claim 1, characterized in that, The two sets of modified models are established by first selecting a convolutional neural network architecture, then inserting attention modules after the convolutional layers of the convolutional neural network architecture, and finally using a deep learning framework to build two independent models with different internal structures. After training, they are respectively designated as the evaluation model and the validation model.
5. The breast ultrasound image classification method based on deep learning according to claim 4, characterized in that, A clinical rule mapping table is set up between the assessment model and the validation model to associate BI-RADS categories with the expected range of malignancy probabilities.
6. The breast ultrasound image classification method based on deep learning according to claim 1, characterized in that, In the image data quality monitoring, image data with clear lesion areas, good probe contact, no significant shadows or artifacts, and containing complete lesions are considered qualified data. Image data with blurry images, excessive or insufficient probe pressure, shadows covering key areas, and lesions not being completely scanned are considered unqualified data.
7. The breast ultrasound image classification method based on deep learning according to claim 6, characterized in that, The enhancement of the defective data includes random rotation of the image, horizontal / vertical flipping, brightness / contrast adjustment, elastic deformation, and addition of Gaussian noise.
8. The breast ultrasound image classification method based on deep learning according to claim 1, characterized in that, The classification report includes BI-RADS classification of cases and their information, a list of cases marked as requiring expert consultation, inconsistencies between the two models, and overall diagnostic efficiency and case triage statistics.
9. A deep learning-based breast ultrasound image classification device, used to implement the deep learning-based breast ultrasound image classification method according to claims 1-8, characterized in that, The classification device consists of a data acquisition module, a data processing and storage module, a deep learning framework, a data quality detection module, a data augmentation module, and a result output module. The data acquisition module is used to collect breast ultrasound imaging data; The data processing and storage module is used to store image data and its tagging information; The deep learning framework is used to provide APIs and tools for building and training deep learning models; The data quality detection module is used to detect the quality of image data; The data enhancement module is used to enhance image data; The results output module is used to summarize and output model evaluation results, expert consultation opinions, and other statistical information.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the deep learning-based breast ultrasound image classification method according to any one of claims 1 to 8.