BI-RADS Feature Scoring Method Based on Multi-Task and Single-Task Joint Learning

The BI-RADS feature scoring method, which employs joint learning of multi-task and single-task approaches, addresses the issues of high manual workload and low diagnostic accuracy in existing systems. It achieves higher scoring accuracy and semantic consistency while reducing reliance on image preprocessing.

CN117475196BActive Publication Date: 2026-11-17ANHUI PROVINCIAL HOSPITAL
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311235618.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-22
Publication Date
2026-11-17
Estimated Expiration
2043-09-22

AI Technical Summary

Technical Problem

Existing computer-aided diagnostic systems for breast cancer based on BI-RADS features suffer from problems such as high manual workload, strong dependence on image preprocessing, and low diagnostic accuracy and credibility due to the inability of low-level feature extraction to meet clinical semantic requirements.

Method used

We employ a multi-task and single-task joint learning approach. By constructing a multi-task learning framework, we utilize the backbone network and subnetworks to learn the latent connections and differences between BI-RADS features. We combine an attention module for feature selection and scoring, group highly correlated features for unified scoring, and use the standard cross-entropy loss function to optimize the model.

Benefits of technology

It improves the accuracy and credibility of BI-RADS feature scoring, reduces reliance on image preprocessing, enhances the semantic consistency of feature extraction, and reduces diagnostic discrepancies among doctors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117475196B_ABST
    Figure CN117475196B_ABST
Patent Text Reader

Abstract

This invention relates to the field of feature analysis technology for breast imaging reporting and data systems, and discloses a BI-RADS feature scoring method based on multi-task and single-task joint learning. The method includes: acquiring breast ultrasound images; preprocessing the breast ultrasound images; and selecting BI-RADS features; constructing a multi-task learning framework, which includes a backbone network for learning global features and K sub-networks for different BI-RADS features; setting different loss functions for different tasks; training the multi-task learning framework using a training dataset and the loss functions; and inputting breast ultrasound images from a test dataset into the multi-task learning framework to obtain BI-RADS feature scores. This invention can not only learn the potential correlations between different BI-RADS features, but also learn the unique specificity of each feature, allowing them to complement each other and jointly improve scoring accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of feature analysis technology for breast imaging reporting and data systems, specifically to a BI-RADS feature scoring method based on multi-task and single-task joint learning. Background Technology

[0002] Breast cancer is one of the most common cancers among women. The characterization of breast tumors using the Breast Imaging Reporting and Data System (BI-RADS) has effectively improved the accuracy and sensitivity of breast tumor detection. In recent years, many experts have been researching how to improve the accuracy of early breast cancer diagnosis. Currently, commonly used techniques for breast cancer detection include biopsy, mammography, magnetic resonance imaging (MRI), and ultrasound. Among these, ultrasound (US) is relatively safe, economical, and convenient, and has been widely used in the early detection of breast cancer in recent years. Related studies have shown that ultrasound is more sensitive than X-rays for cysts, effectively improving diagnostic specificity and avoiding unnecessary biopsies. Although the advantages of ultrasound images are obvious, it is difficult to diagnose breast lesions based solely on ultrasound data in practice. This is mainly because some ultrasound images have low contrast and signal-to-noise ratios, making them difficult to interpret and requiring experienced physicians.

[0003] Given the high cost of training ultrasound imaging physicians and the potential for biases in physicians' understanding of ultrasound images, computer-aided diagnostic (CAD) technology is gaining popularity. CAD provides physicians with diagnostic suggestions, assists in diagnosis, shortens diagnostic time, and reduces physician workload. However, several problems remain in practice. Traditional CAD methods extract low-level image information, such as morphological and textural features. The extraction of morphological features often depends on the image preprocessing and is highly influenced by parameters. Furthermore, textural features are based on the image's grayscale information, which differs significantly from semantic features in clinical diagnosis. This leads to a prevalent problem known as the semantic gap. Physicians typically differentiate between benign and malignant tumors based on high-level semantics. Final decisions based on these low-level features are difficult to interpret and unconvincing.

[0004] To standardize diagnostic criteria for ultrasound images and reduce diagnostic discrepancies among physicians, the Chinese Society of Radiology proposed the Breast Imaging Reporting and Data System (BI-RADS). However, existing computer-aided diagnostic systems for breast cancer classification based on BI-RADS features suffer from several serious problems. These methods first require segmenting the tumor region and then extracting morphological and textural features from the segmentation results. After quantifying the extracted features into different BI-RADS feature values, these values ​​are input into a classifier to distinguish between benign and malignant tumors. This process requires significant manual work to mark the tumor edges. Furthermore, the extraction of these low-level features largely depends on image preprocessing and parameter fine-tuning. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a BI-RADS feature scoring method based on multi-task and single-task joint learning.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A BI-RADS feature scoring method based on multi-task and single-task joint learning includes the following steps:

[0008] Step 1: Acquire breast ultrasound images, preprocess the images, and select BI-RADS features; divide the preprocessed breast ultrasound images into training and testing datasets.

[0009] Step 2, construct a multi-task learning framework: The multi-task learning framework includes a backbone network for learning global features, and K subnetworks for different BI-RADS features; the backbone network can learn the latent connections between all BI-RADS features, and each subnetwork learns the differences between BI-RADS features by applying an attention module to a specific layer of the backbone network.

[0010] Step 3: Treat each BI-RADS feature rating as a separate task. Set different loss functions for different tasks, and train the multi-task learning framework using the training dataset and loss functions. During training, BI-RADS features with relevance greater than a set threshold are grouped into the same multi-task learning framework for rating; BI-RADS features with relevance less than or equal to the set threshold are rated using different multi-task learning frameworks.

[0011] Step 4: Input the breast ultrasound images from the test dataset into the multi-task learning framework to obtain the scores of the BI-RADS features.

[0012] Furthermore, step one specifically includes:

[0013] S11, Preprocessing of breast ultrasound images: The region of interest is cropped from the original breast ultrasound image to obtain the cropped breast ultrasound image;

[0014] S12, Select BI-RADS features: BI-RADS features include shape, location, edge, ambiguity, angle, differential lobes, needle-like, echo, calcification, distortion, and repetition.

[0015] Furthermore, the backbone network includes four residual modules, and each subnetwork includes four attention modules. Each residual module in the backbone network corresponds to one attention module in each subnetwork. Each attention module learns a soft attention mask based on the features of the corresponding residual module in the backbone network.

[0016] The attention module obtains specific information about the corresponding task from the backbone network. That is, each attention module selects features from the backbone network through a soft attention mask. Each feature channel of each task has a soft attention mask.

[0017] The shared feature of the i-th residual module in the backbone network is denoted as r1. (i) and r2 (i) ; and These consist of convolutional layers, batch normalization layers, and nonlinear activation layers; each subnetwork has a pooling layer.

[0018] The first attention module in each subnet only uses the corresponding shared feature r1 from the backbone network. (1) r2 (1) As input, the input to the i-th attention module includes shared features r1 from the backbone network. (1) r2 (1) and the BI-RADS features output by the previous attention module The BI-RADS features are concatenated; the first attention module output in the subnet corresponding to task k, 1≤k≤k. for:

[0019]

[0020] Here, ⊙ represents element-wise multiplication. and These represent the convolutional layer feature extraction module and the batch processing normalization layer for different BI-RADS features, respectively. This represents a non-linear activation layer targeting different BI-RADS features; This is a pooling layer used to match the resolution of the next attention module;

[0021] The BI-RADS features output by the i-th attention module for:

[0022]

[0023] in This represents element-wise addition. This indicates a pooling operation.

[0024] Furthermore, step three specifically includes: using the Pearson correlation coefficient to calculate the correlation between different BI-RADS features; grouping BI-RADS features with a correlation greater than a set threshold into the same multi-task learning framework for scoring; and using different multi-task learning frameworks to score BI-RADS features with a correlation less than or equal to a set threshold.

[0025] The input for each task is the same training dataset;

[0026] Applying standard cross-entropy loss to all BI-RADS feature scoring tasks:

[0027]

[0028] L i Let n be the loss function for task k, i = 1, 2, ..., K, n i Let i represent the number of categories of task i, and define... The value is 0 or 1, indicating whether category n is the true classification; if category n is the true classification, then If category n is not a true category This represents the probability of predicting category n.

[0029] Compared with the prior art, the beneficial technical effects of the present invention are:

[0030] This invention designs a multi-task learning (MTL) framework that can directly output scores for different BI-RADS features from raw ultrasound images. The MTL framework consists of a shared network that learns global features and K soft attention networks targeting different BI-RADS features. It can learn not only the potential correlations between different BI-RADS features but also the unique specificity of each feature, working together to improve scoring accuracy. Furthermore, this invention groups different BI-RADS features based on the correlation between tasks, constructing a joint multi-task and single-task framework. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the multi-task learning framework of the present invention. Detailed Implementation

[0032] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0033] This invention proposes a deep learning framework based on Multi-Task Learning (MTL). US images are input into the MTL framework, which outputs scores for different BI-RADS features. Then, features are grouped according to the correlation between tasks, with highly correlated features placed in the same task learning framework. Different STL scoring models are used to solve for weakly correlated features. Finally, this invention obtains scores for all 11 BI-RADS features. The MTL framework consists of two parts: a shared network (backbone network) that learns global features and K soft attention networks (subnetworks) for different BI-RADS features. The backbone network is used to learn the latent connections between all BI-RADS features; each subnetwork learns the differences between features by applying an attention module to a specific layer of the backbone network. Therefore, the attention module can be viewed as a feature selector of the backbone network, performing end-to-end automatic learning. This invention solves the technical problems of low accuracy and low reliability in existing methods for US image diagnosis.

[0034] This invention first randomly divides the patient data into ten equal parts, grouping US images belonging to the same patient into the same part. Nine parts are used as the training dataset, and one part is used as the test dataset. Then, this invention uses the training dataset to construct the BI-RADS feature scoring model and uses the test dataset to evaluate the model. The dataset segmentation remains the same for different tasks and different models. To accelerate training, the size of all training and test images is adjusted to 72×72.

[0035] A BI-RADS feature scoring method based on multi-task and single-task joint learning includes the following steps:

[0036] Step 1: Acquire breast ultrasound images, perform preprocessing and feature selection; specifically including the following steps:

[0037] S11: Data preprocessing. The Region of Interest (ROI) occupies only a small portion of the original image, and some irrelevant information exists in the original data. Therefore, this invention requires doctors to use LabelMe software to crop the ROI. First, the doctor determines two diagonal points and draws a relatively small rectangle from the original ultrasound image. The rectangle should contain the tumor area, occupying 30%-50% of the entire ROI. Next, the image is cropped into a square according to the long and short sides of the rectangle: this invention takes the long side of the rectangle as the side length of the square. If the long side length is less than 400 pixels, it is cropped to (400×400) pixels according to the center pixel of the marked rectangle; if the long side length m is greater than 400 pixels, the original image is cropped to (m×m) pixels.

[0038] S12: Determining Selected Features: Based on the BI-RADS feature scoring table and the physician's prior knowledge, terminological descriptions of 25 BI-RADS features can be constructed. Given an image containing a tumor, the physician can score the features according to clinical manifestations and scoring rules; the higher the score, the higher the probability of malignancy. It is important to note that the BI-RADS feature scores obtained in this invention are not the final diagnostic results, but rather intermediate results. The information obtained based on existing medical knowledge and the content disclosed in this patent application cannot directly lead to a diagnosis or health status; further complex computer processing is required to obtain a diagnostic result. The quantitative scoring results are used as labels in the multi-task and single-task combined framework of this invention. Because blurred edges and microlobed features have high diagnostic value for ultrasound breast tumor classification, skin retraction, cyst type, and foreign bodies have little diagnostic value for breast ultrasound tumor classification. This invention ultimately selected 11 features as scoring criteria, as shown in Table 1.

[0039] Table 1: BI-RADS Feature Scoring Criteria

[0040] ID Feature 0 1 2 3 4 5 0 shape oval round irregular 1 Location parallel Non-parallel 2 edge Restricted Unrestricted 3 Ambiguity clear Vague 4 angle none have 5 Differential lobes none have 6 acicular none have 7 echo No echo High Echo Waiting for the echo Uneven echo Complex and sharp echo low echo 8 Calcification powerful none common shadow 9 distort none thick dispersion coarse + dispersed in groups 10 repeat none have

[0041] Step 2: Construct a multi-task learning framework; such as Figure 1 As shown, the MTL framework consists of two parts: a shared network (backbone) that learns global features, and K soft attention networks (multiple subnetworks) targeting different BI-RADS features. The backbone is used to learn the latent connections between all BI-RADS features; each subnetwork learns the differences between features by applying an attention module to a specific layer of the backbone. Therefore, the attention module can be viewed as a feature selector of the backbone, enabling end-to-end automatic learning. The specific network architecture is as follows... Figure 1 As shown.

[0042] Specifically, in this multi-task learning framework, the backbone network consists of four residual modules, each corresponding to a focus module in each subnetwork. Therefore, each subnetwork has four attention modules, and each attention module learns a soft attention mask based on the features of the corresponding layer in the backbone network. Thus, the multi-task learning framework can jointly learn features from the backbone network and the soft attention mask, improving the generalization ability of shared features among multiple BI-RADS features. Simultaneously, the attention mask can be used to mine high-level semantic information for each specific feature in BI-RADS.

[0043] The attention module obtains specific information about the corresponding task from the backbone network. That is, each attention module selects features from the backbone network through a soft attention mask, and each feature channel for each task has a soft attention mask.

[0044] like Figure 1 As shown, r1 (i) and r2 (i) This refers to the shared characteristics of the i-th residual module in the backbone network. and These represent convolutional layers (conv), batch normalization (bn), and non-linear activations (ReLU and Sigmoid), respectively. This indicates a pooling operation. The first attention module only takes features from the backbone network as input. Subsequent attention modules take shared features r1 from the backbone network as input. (1) and the BI-RADS features of the previous attention module. Concatenated. The output of the first attention module in task k. for:

[0045]

[0046] Here, ⊙ represents element-wise multiplication. and It consists of convolution operations with a kernel size of (1×1), representing a task-specific feature extractor for different BI-RADS features; It consists of convolution operations with a kernel size of (3×3), representing a shared feature extractor for different BI-RADS features. Finally, the pooling layer matches the resolution of the next interest module.

[0047] The outputs of the last three attention modules are as follows, where This represents element-wise addition:

[0048]

[0049] Step 3: Set different loss functions for different tasks, and train the multi-task learning framework using the training dataset and the loss functions.

[0050] Step S3 above: Since BI-RADS feature scoring is a multi-task image classification problem, this invention treats each BI-RADS feature scoring as a separate task. The input to each task is the same dataset.

[0051] This invention applies standard cross-entropy loss to all BI-RADS feature scoring tasks.

[0052] Specifically, the loss function is broken down into the following steps: S31 to S32.

[0053] S31: In multi-task learning frameworks, overall performance degrades as task objectives compete. Therefore, a crucial issue is selecting which tasks are cooperative and which are competitive. Cooperative (strongly correlated) tasks are solved by the same network, while competitive tasks should be solved by different networks. Therefore, this invention uses the Pearson correlation coefficient to discover the correlation between different BI-RADS features, and then groups highly correlated tasks into the same multi-task learning framework. For tasks with weak correlation, different learning frameworks are used. Finally, this invention extracts 7 highly correlated features from 11 features and groups them together to obtain a network that maximizes the accuracy of BI-RADS feature scoring.

[0054] S32: Since BI-RADS feature scoring is a multi-task image classification problem, this invention treats each BI-RADS feature scoring as a separate task. The input to each task is the same dataset.

[0055] This invention applies standard cross-entropy loss to all BI-RADS feature scoring tasks, as shown in the following formula:

[0056]

[0057] L i Let n be the loss for task i, where i = 1, 2, ..., k. Since some BI-RADS feature scores are binary classification problems (e.g., orientation features, which describe the sign as parallel or non-parallel), and some BI-RADS feature scores are multi-class classification problems (e.g., shape features, which describe the sign as elliptical, circular, and irregular), this invention uses n... i Let i represent the number of categories of task i, and define... The value is 0 or 1, indicating whether category n is a true classification. If the category is n, then... otherwise, This represents the probability of predicting category n.

[0058] Step 4: Input the breast ultrasound images from the test dataset into the multi-task learning framework to obtain the scores of the BI-RADS features.

[0059] Evaluation of multi-task learning frameworks:

[0060] For all 11 BI-RADS features, this invention uses average precision to measure the performance of the entire multi-task learning framework. Where TN is the number of true negative samples in real benign breast tumors, TP is the number of true positive samples in malignant breast tumors, FN is the number of false negative samples in real benign breast tumors, and FP is the number of false positive samples in malignant breast tumors, then:

[0061]

[0062]

[0063]

[0064] Different tasks were grouped appropriately. Among the seven strongly correlated features, which should be placed in the same MTL framework, and which should be addressed by different MTL frameworks? Therefore, this invention conducted a series of exploratory experiments to obtain a network that maximizes the accuracy of BI-RADS feature scoring. The exploratory experiments were divided into six groups. The first group selected two features from the seven strongly correlated features to train the MTL framework; the second group selected three features from the seven strongly correlated features, and so on. Simultaneously, this invention explored the performance of the seven tasks as single-task models. This invention only selects the results of the primary task as the BI-RADS scoring result, and does not use the results of the auxiliary tasks as the basis for judgment. Specific results are shown in Table 2.

[0065] Table 2

[0066]

[0067] As shown in Table 2, training two tasks together or training one task alone often yields better results. For example, this invention combines shape features and angle features for training, achieving optimal performance. However, training multiple features within the same MTL framework may not optimize the performance of each task. When shape features and blur features are trained simultaneously, the scoring accuracy of shape features is lower than that trained with angle features. However, this means that shape features help improve the scoring accuracy of blur. Therefore, shape features are the auxiliary task, and blur features are the primary task. When six features—shape, edge, blur, angle, spikes, and distortion—are trained together, only the distortion feature achieves the highest scoring accuracy. This demonstrates that when the other five tasks simultaneously assist a primary task, the performance of the primary task is ultimately improved.

[0068] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0069] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A BI-RADS feature scoring method based on multi-task and single-task joint learning, comprising the following steps: Step 1: Acquire breast ultrasound images, preprocess the breast ultrasound images, and select BI-RADS features; The preprocessed breast ultrasound images were divided into training and testing datasets. Step 2: Construct a multi-task learning framework: The multi-task learning framework includes a backbone network for learning global features and K subnetworks for different BI-RADS features. The backbone network can learn the latent connections between all BI-RADS features, and each subnetwork learns the differences between BI-RADS features by applying attention modules to specific layers of the backbone network. The backbone network includes four residual modules, and each subnetwork includes four attention modules. Each residual module in the backbone network corresponds to one attention module in each subnetwork. Each attention module learns a soft attention mask based on the features of the corresponding residual module in the backbone network. The attention modules obtain specific information about the corresponding task from the backbone network, that is, each attention module selects features from the backbone network through a soft attention mask, and each feature channel of each task has a soft attention mask. The shared feature of the i-th residual module of the backbone network is denoted as... and In each subnet , and These represent convolutional layers, batch normalization layers, and nonlinear activation layers, respectively; each subnetwork has a pooling layer. The first attention module in each subnet only considers the corresponding shared features from the backbone network. , As input, the input to the i-th attention module includes shared features from the backbone network. , and the BI-RADS features output by the previous attention module Pieced together; task BI-RADS features output by the first attention module in the corresponding subnet for: ; This represents element-wise multiplication. and These represent the convolutional layer feature extraction module and the batch processing normalization layer for different BI-RADS features, respectively. This represents a non-linear activation layer targeting different BI-RADS features; This is a pooling layer used to match the resolution of the next attention module; the... BI-RADS features output by each attention module for: ; ; This represents element-wise addition. Indicates pooling operation; Step 3: Treat each BI-RADS feature score as a separate task. Set different loss functions for different tasks, and train the multi-task learning framework using the training dataset and loss functions. During training, BI-RADS features with a relevance greater than a set threshold are grouped into the same multi-task learning framework for scoring; BI-RADS features with a relevance less than or equal to the set threshold are scored using different multi-task learning frameworks. Step 4: Input the breast ultrasound images from the test dataset into the multi-task learning framework to obtain the scores of the BI-RADS features.

2. The BI-RADS feature scoring method based on multi-task and single-task joint learning according to claim 1, characterized in that, Step one specifically includes: S11, Preprocessing of breast ultrasound images: The region of interest is cropped from the original breast ultrasound image to obtain the cropped breast ultrasound image; S12, Select BI-RADS features: BI-RADS features include shape, location, edge, ambiguity, angle, differential lobes, needle-like, echo, calcification, distortion, and repetition.

3. The BI-RADS feature scoring method based on multi-task and single-task joint learning according to claim 1, characterized in that, Step 3 specifically includes: using the Pearson correlation coefficient to calculate the correlation between different BI-RADS features; grouping BI-RADS features with a correlation greater than the set threshold into the same multi-task learning framework for scoring; and using different multi-task learning frameworks to score BI-RADS features with a correlation less than or equal to the set threshold. The input for each task is the same training dataset; Applying standard cross-entropy loss to all BI-RADS feature scoring tasks: ; Let k be the loss function for task k. , Let i represent the number of categories of task i, and define... The value is 0 or 1, indicating whether category n is the true classification; if category n is the true classification, then If category n is not a true category, ; This represents the probability of predicting category n.

Citation Information

Patent Citations

  • Breast cancer histology grading prediction method and device fused with multi-task learning

    CN116152180A

  • Breast ultrasonic image target area identification method and system based on multi-task learning

    CN116309585A