Breast tumor benign and malignant identification system based on double-branch attention distillation network

Abstract

The invention discloses a breast tumor benign and malignant identification system based on a double-branch attention distillation network, belongs to the field of medical image processing, and particularly relates to the breast tumor benign and malignant identification system based on the double-branch attention distillation network. The invention aims to solve the problem that the existing method based on deep learning is insufficient in feature extraction capability and cannot accurately focus a lesion area. The system comprises an ultrasonic image preprocessing main module used for preprocessing collected breast ultrasonic images, obtaining ultrasonic imaging areas, unifying the image size of the obtained ultrasonic imaging areas, obtaining preprocessed breast ultrasonic images, and taking the preprocessed breast ultrasonic images as a training set; the double-branch convolutional neural network main module is used for building a double-branch attention distillation network model; the model training main module is used for obtaining a trained double-branch attention distillation network model; and the model test main module is used for loading the trained double-branch attention distillation network model and classifying breast ultrasound images to be diagnosed.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and in particular relates to a benign and malignant breast tumor recognition system based on a dual-branch attention distillation network. Background Art

[0002] Breast cancer is one of the most common malignant tumors in women worldwide and the leading cause of cancer death in women. The latest data from the World Health Organization shows that in 2022, there will be approximately 2.3 million new cases of breast cancer and approximately 665,000 related deaths worldwide. Early diagnosis and timely treatment are important means to improve patient prognosis. Therefore, it is particularly important to use more accurate and sensitive medical imaging technologies for early diagnosis of breast cancer. Among various medical imaging technologies, breast ultrasound has become the most commonly used clinical diagnostic tool for breast cancer due to its real-time, non-invasive, radiation-free and low-cost advantages, especially in dense breasts, which significantly improves the accuracy of early diagnosis of breast cancer. The Breast Imaging Reporting and Data System (BI-RADS) summarizes the characteristic descriptions of breast tumors in ultrasound, including tumor margins, shapes and directions, to assist radiologists in identifying malignant tumors. However, the diagnostic process based on the BI-RADS standard is time-consuming, labor-intensive and heavily dependent on the rich experience of doctors, which can easily lead to missed or misdiagnoses, especially in areas with scarce medical resources. Therefore, it is crucial to develop a computer-aided diagnosis system for breast cancer diagnosis using ultrasound images.

[0003] With the rapid development of artificial intelligence, computer-aided diagnosis systems have become a current research hotspot. However, existing diagnostic systems still face many challenges. First, the existing convolutional neural network has insufficient feature extraction capabilities and cannot effectively focus on the target lesion area. Secondly, although the introduction of spatial and channel attention mechanisms can enhance the network's feature extraction capabilities and thus focus on the lesion area more accurately, the complex attention module will significantly increase the model's computational complexity and parameter scale, thereby increasing the risk of overfitting. Finally, the use of knowledge distillation strategies can enable the student network to learn important information from the teacher network to improve the overall performance of the model, but building a reliable teacher network itself is challenging. Summary of the invention

[0004] In order to solve the problem that the existing deep learning method has insufficient feature extraction ability and cannot accurately focus on the lesion area, the present invention proposes a benign and malignant breast tumor identification system based on a dual-branch attention distillation network.

[0005] A breast tumor benign and malignant identification system based on a dual-branch attention distillation network includes:

[0006] Ultrasound image preprocessing main module, dual-branch convolutional neural network main module, model training main module, model testing main module;

[0007] The ultrasound image preprocessing main module is used to preprocess the collected breast ultrasound images, obtain the ultrasound imaging area, unify the image size of the obtained ultrasound imaging area, and obtain the preprocessed breast ultrasound images as a training set;

[0008] The dual-branch convolutional neural network main module is used to build a dual-branch attention distillation network model;

[0009] The model training main module uses the preprocessed breast ultrasound image to train the constructed dual-branch attention distillation network model, and obtains the trained dual-branch attention distillation network model;

[0010] The model test main module is used to load the trained dual-branch attention distillation network model to classify the breast ultrasound images to be diagnosed.

[0011] The beneficial effects of the present invention are:

[0012] 1. The present invention proposes a breast tumor benign and malignant identification system based on a dual-branch attention distillation network. This method uses a dual-branch convolutional neural network as the overall framework, and combines the attention module and knowledge distillation strategy to improve the network's ability to extract important spatial and channel information.

[0013] 2. The present invention proposes a dynamic spatial attention module and a gated channel attention module. The dynamic spatial attention module is used to consider the dependencies between pixels in the feature map to capture its spatial features, and the gated channel attention module is used to learn the dependencies between feature map channels to extract their channel features, thereby improving the feature representation capability of the network.

[0014] 3. The present invention integrates feature distillation into a two-branch convolutional neural network, uses feature distillation to enhance the information capture capability of the attention mechanism, and refines and self-adjusts the extracted spatial and channel features, thereby improving the accuracy of the model in clinical identification of benign and malignant breast tumors. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the system of the present invention;

[0016] Figure 2 Schematic diagram of the architecture of the dual-branch attention distillation network in the system of the present invention;

[0017] Figure 3 Schematic diagram of the gated channel attention module in the system of the present invention. DETAILED DESCRIPTION

[0018] Specific implementation method 1: Implementing a breast tumor benign and malignant identification system based on a dual-branch attention distillation network includes:

[0019] Ultrasound image preprocessing main module, dual-branch convolutional neural network main module, model training main module, model testing main module;

[0020] The ultrasound image preprocessing main module is used to preprocess the collected breast ultrasound images, thereby reducing noise interference, obtaining the ultrasound imaging area, unifying the image size of the obtained ultrasound imaging area, and obtaining the preprocessed breast ultrasound images as a training set;

[0021] The dual-branch convolutional neural network main module is used to build a dual-branch attention distillation network model;

[0022] The model training main module uses the preprocessed breast ultrasound image to train the constructed dual-branch attention distillation network model, and obtains the trained dual-branch attention distillation network model;

[0023] The model test main module is used to load the trained dual-branch attention distillation network model, classify the breast ultrasound images to be diagnosed, and identify the benign and malignant breast tumors in the ultrasound images.

[0024] Specific implementation method 2: This implementation method is different from specific implementation method 1 in that: the ultrasound image preprocessing main module is used to preprocess the collected breast ultrasound images, thereby reducing noise interference, obtaining the ultrasound imaging area, unifying the image size of the obtained ultrasound imaging area, and obtaining the preprocessed breast ultrasound images as a training set;

[0025] The specific process is:

[0026] Step A1, collecting a breast ultrasound image, cutting out a region of interest in the breast ultrasound image, and cutting out an ultrasound imaging area;

[0027] Aiming at the problem that the contrast between the breast tumor and the surrounding tissue in the ultrasound imaging area is not obvious enough, the cropped ultrasound imaging area is subjected to histogram equalization to obtain an image after histogram equalization;

[0028] Enhance the local contrast of the image without affecting its overall contrast, highlighting the edge and shape of the breast tumor.

[0029] Step A2: In order to enable the trained model to effectively process breast ultrasound images of different sizes, the image after histogram equalization is scaled so that the length of the short side of the image is 224 pixels;

[0030] Step A3, cropping the central area of ​​the image processed in step A2 by means of center cropping to obtain a target image of size 224×224;

[0031] The target image of size 224×224 is used as the final preprocessed breast ultrasound image and is used as the training set.

[0032] The breast tumors in the training set are labeled as benign and malignant.

[0033] The preprocessed breast ultrasound images are used as input to the two-branch attention distillation network model.

[0034] The other steps and parameters are the same as those in the first embodiment.

[0035] Specific implementation method three: This implementation method is different from specific implementation method one or two in that: the dual-branch convolutional neural network main module is used to build a dual-branch attention distillation network model; the specific process is:

[0036] The dual-branch attention distillation network model includes branch one and branch two;

[0037] Branch 1 includes residual convolution block 1, residual convolution block 2, dynamic spatial attention module 1, dynamic spatial attention module 2, gated channel attention module 1, gated channel attention module 2, gated channel attention module 3, global average pooling layer, and fully connected layer in sequence;

[0038] Branch 2 includes residual convolution block 1, residual convolution block 2, dynamic spatial attention module 1, dynamic spatial attention module 2, gated channel attention module 1, gated channel attention module 2, gated channel attention module 3, global average pooling layer, and fully connected layer.

[0039] It is necessary to classify breast ultrasound images into benign and malignant categories. The features output by the gated channel attention module are compressed through the global average pooling layer, and then the fully connected layer and Softmax function are used to obtain the probability value of the network classifying the image into each category.

[0040] The other steps and parameters are the same as those in the first or second embodiment.

[0041] Specific implementation method 4: This implementation method is different from one of the specific implementation methods 1 to 3 in that the working process of the dual-branch attention distillation network model is:

[0042] The image is sequentially input into residual convolution block 1 and residual convolution block 2 in branch one, and residual convolution block 2 outputs feature A; feature A is sequentially input into dynamic spatial attention module 1 and dynamic spatial attention module 2 in branch one, and dynamic spatial attention module 2 outputs feature B; feature B is sequentially input into gated channel attention module 1, gated channel attention module 2, and gated channel attention module 3 in branch one, and gated channel attention module 3 outputs feature D; feature D is sequentially input into the global average pooling layer and the fully connected layer in branch one, and the output feature of the fully connected layer is used as the output feature of branch one;

[0043] The image is sequentially input into residual convolution block 1 and residual convolution block 2 in branch 2, and residual convolution block 2 outputs feature A′; feature A′ is added element by element with feature A output by residual convolution block 2 in branch 1 to obtain feature A″; feature A″ is sequentially input into dynamic spatial attention module 1 and dynamic spatial attention module 2 in branch 2, and dynamic spatial attention module 2 outputs feature B′; feature B′ is added element by element with feature B output by dynamic spatial attention module 2 in branch 1 to obtain feature B″; feature B″ is sequentially input into gated channel attention module 1, gated channel attention module 2, and gated channel attention module 3 in branch 2, and gated channel attention module 3 outputs feature D′; feature D′ is added element by element with feature D output by gated channel attention module 3 in branch 1 to obtain feature D″; feature D″ is sequentially input into global average pooling layer and fully connected layer in branch 2, and the output feature of fully connected layer is used as the output feature of branch 2;

[0044] The output features of branch one and branch two are added element by element to obtain feature E, feature E is divided by 2 to obtain feature G, and feature G is passed through the softmax function to obtain the predicted probability value of the dual-branch attention distillation network model.

[0045] The output features of branch one are spliced ​​into branch two through residual connection to achieve feature interaction of the two-branch network and improve the diagnostic performance of the two-branch network.

[0046] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0047] Specific implementation mode five: This implementation mode is different from any one of specific implementation modes one to four in that the residual convolution block 1 includes in sequence: a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, a batch normalization layer, a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, a batch normalization layer, a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, and a batch normalization layer;

[0048] The residual convolution block 2 includes: a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, a batch normalization layer, a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, a batch normalization layer, a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, and a batch normalization layer.

[0049] Residual convolution block 1 and residual convolution block 2 are residual convolution modules with different numbers of channels; in residual convolution block 1, the number of output channels of the convolution layer is 64, and the number of output channels of the convolution layer in residual convolution block 2 is 128.

[0050] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0051] Specific implementation example 6: This implementation example is different from any one of specific implementation examples 1 to 5 in that each of the dynamic spatial attention modules 1 and 2 includes dilated group convolution, upsampling, and linear mapping in sequence;

[0052] The dilated group convolution includes a dilated convolution layer with a dilation rate of 1, a dilated convolution layer with a dilation rate of 2, and a dilated convolution layer with a dilation rate of 3;

[0053] The working process of each dynamic spatial attention module is:

[0054] The features are input into the dilated convolution layer with a dilation rate of 1, the dilated convolution layer with a dilation rate of 2, and the dilated convolution layer with a dilation rate of 3, respectively, to obtain three features;

[0055] The three features obtained are upsampled respectively;

[0056] The three upsampled features are linearly mapped to obtain the output features of each dynamic spatial attention module.

[0057] The dilated group convolution output and input are upsampled; the upsampled output and input are linearly mapped;

[0058] Construct a dynamic spatial attention module as a shallow feature extraction network to enhance the network's spatial feature extraction capability;

[0059] The dynamic spatial attention module consists of three stages: dilated group convolution, upsampling, and linear mapping;

[0060] First, adjacent pixels in shallow feature maps have small information differences, and the small receptive field cannot fully consider their spatial features. Therefore, grouped convolutions with different expansion sizes can increase the receptive field while maintaining the number of network parameters.

[0061] Subsequently, the grouped feature maps are restored to their original size by upsampling. Different regions of the feature maps are assigned different weights to adapt to the regional clustering characteristics exhibited by the lesion area.

[0062] Finally, the feature maps are merged and linearly mapped to obtain the spatial attention weights. The spatial attention weights are used to perform a weighted transformation on the original feature maps, and the lesion area is accurately focused by comprehensively considering the spatial dimension information.

[0063] In addition, a hypernetwork is used to learn the convolution kernel parameters in this module. This method makes the convolution kernel parameters dependent on the input image, realizes dynamic convolution kernels, and improves the module's adaptability.

[0064] The dynamic spatial attention module is calculated as follows:

[0065] F=Linear(Up(GroupConv(u)))⊙u

[0066] Among them, u is the input feature of the dynamic spatial attention module, Linear is the linear mapping layer, GroupConv is the dilated group convolution (the dilated group convolution is a dilated convolution layer, but the feature channels are divided into three groups by grouping, and each group uses a dilated convolution layer with a dilation rate of 1, 2, and 3, respectively), Up is upsampling, and ⊙ represents the Hadamard product;

[0067] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0068] Specific implementation example 7: This implementation example is different from any one of specific implementation examples 1 to 6 in that each of the gated channel attention modules 1, 2 and 3 includes: global information encoding, channel normalization and adaptive gating mechanism in sequence;

[0069] Construct a gated channel attention module to enhance the channel feature extraction capability of deep networks;

[0070] Global information encoding uses a larger receptive field to avoid local confusion problems; global information encoding aggregates feature maps and extracts global information for each channel. The calculation formula for global information encoding is as follows:

[0071]

[0072] Among them, x represents the input feature map of the gated channel attention module, s g represents the output of global information encoding, m and n are the height and width of the feature map, respectively, x i,j Represents the eigenvalue of the i-th row and j-th column on the feature graph x;

[0073] Channel normalization supports competition between channels, enhancing channels with significant responses and suppressing other channels. The channel normalization calculation formula is as follows:

[0074]

[0075] Among them, s n represents the normalized result, σ represents the minimum constant, C represents the total number of channels, Represents the square of the mean of the eigenvalues ​​of the kth channel on the feature map x;

[0076] Shallow networks are used to learn low-level features of images, such as texture and edges. Therefore, channel cooperation is required to obtain fused features. In contrast, high-level features in deep networks are highly abstract. The differences between channel information are large and directly related to the task, and channel competition is required to extract meaningful information.

[0077] The adaptive gating mechanism realizes competition and cooperation between channels through learnable parameters. The calculation process of the adaptive gating mechanism is as follows:

[0078] s o =x×[1+tanh(α×s n +β)]

[0079] Among them, s o represents the output of the gated channel attention module;

[0080] x represents the input feature map of the gated channel attention module (x is a feature map, including three dimensions: length, width, and channel, including the feature value x in the i-th row and j-th column of the feature map i,j And the square of the eigenvalue of the kth channel on the feature map x );

[0081] α and β are learnable parameters, which are used to adjust the weight and bias of the channel normalization output respectively;

[0082] It should be noted that if the average value of α is less than 0, the module will increase the diversity of channel information. On the contrary, if the average value of α exceeds 0, the module will enhance the competition between channels;

[0083] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0084] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that the loss function acquisition process of the dual-branch attention distillation network model is:

[0085] 1) The loss function of the dynamic spatial attention module (local feature distillation) is as follows:

[0086]

[0087] in, Represents the weight value of the i-th row and j-th column on the spatial attention weight map of the dynamic spatial attention module of branch one; Represents the weight value of the i-th row and j-th column on the spatial attention weight map of the dynamic spatial attention module of branch 2;

[0088] The knowledge distillation strategy is used to distill the attention weights extracted by the dual-branch network to further enhance the extracted spatial and channel features. The feature distillation used is divided into local feature distillation and global feature distillation.

[0089] Shallow feature maps are usually used to extract spatial information such as textures and edges, but the importance of pixel information in different areas of the feature map varies. By utilizing local feature distillation, the network focuses on key pixels and improves its ability to extract spatial features. At the same time, since the two branches of the network learn different spatial information, local feature distillation is used to fully fuse the spatial information from the two branches. The dynamic spatial attention module obtains the spatial attention map of the two-branch network to highlight the tumor area, and the corresponding spatial attention weights can obtain better feature extraction results after feature distillation.

[0090] 2) The loss function of the gated channel attention module (global feature distillation) is as follows:

[0091]

[0092] in, represents the weight of the kth channel of the channel attention weight map of branch one, Represents the weight of the kth channel of the channel attention weight map of branch 2;

[0093] The extracted deep features are used to learn semantic information related to the task, and the importance of the contained information varies from channel to channel. Global feature distillation forces the network to strengthen important channels and suppress irrelevant channels to enhance global features. In addition, the two branch networks learn different semantic information, and using global feature distillation to refine semantic context knowledge can further improve the performance of the network. The channel attention map of the two-branch network is extracted using the gated channel attention module, and global feature distillation is performed by adopting the loss function shown below;

[0094] 3) The classification loss function is defined as follows:

[0095]

[0096] in, is the predicted probability value, y i is the real category;

[0097] The predicted probability value The acquisition process is:

[0098] The output of the fully connected layer of branch one and the output of the fully connected layer of branch two are added element by element to obtain feature E. Feature E is divided by 2 to obtain feature G. Feature G is passed through the softmax function to obtain the predicted probability value.

[0099] 4) The loss function of the dual-branch attention distillation network model is as follows:

[0100] L=L LF +L GF +L C .

[0101] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0102] Specific implementation method 9: This implementation method is different from any one of specific implementation methods 1 to 8 in that the model training main module uses the preprocessed breast ultrasound image to train the built dual-branch attention distillation network model to obtain the trained dual-branch attention distillation network model; the specific process is:

[0103] Step C1, initialize the parameters of the dual-branch attention distillation network model;

[0104] The optimizer uses the Adam optimizer;

[0105] Set weight decay parameters;

[0106] Set the initial learning rate and learning rate decay rule;

[0107] Set the maximum number of iterations;

[0108] Step C2: The two-branch attention distillation network model reaches the set maximum number of iterations or the loss function of the two-branch attention distillation network model converges, obtains the trained two-branch attention distillation network model, and saves the trained two-branch attention distillation network model weights.

[0109] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0110] Specific implementation method ten: This implementation method is different from the first to ninth specific implementation methods in that the model test main module is used to load the trained two-branch attention distillation network model to classify the breast ultrasound images to be diagnosed and identify the benign and malignant breast tumors in the ultrasound images; the specific process is:

[0111] The test set data is processed by the ultrasound image preprocessing main module to obtain the processed breast ultrasound image;

[0112] The processed breast ultrasound images are input into the trained two-branch attention distillation network model, and the two-branch attention distillation network model outputs the classification results to identify the benign and malignant breast tumors.

[0113] Calculate the test results’ accuracy, recall, precision and other performance indicators.

[0114] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0115] The following examples are used to verify the beneficial effects of the present invention:

[0116] Embodiment 1:

[0117] The specific implementation of the present invention is described below in conjunction with specific embodiments and drawings. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0118] The breast ultrasound image dataset comes from actual hospital cases, including 248 breast ultrasound images of 248 patients, including 145 benign cases and 103 malignant cases. These patients all underwent biopsy or surgery after ultrasound examination, and obtained histopathological results as the basic facts for lesion classification. Benign tumors include breast fibroadenoma, breast adenosis, breast cysts and fibroepithelial breast tumors, among which breast fibroadenoma is the most common. Malignant tumors include invasive lobular carcinoma, invasive ductal carcinoma, mucinous carcinoma and invasive breast cancer, among which invasive ductal carcinoma is the most common. The maximum diameter of breast tumors ranges from 5 to 72 mm. It is randomly divided into training set and test set in a ratio of 8:2. Among them, 198 images are used as training set to update model parameters; 50 images are used as test set to test the final model.

[0119] Perform step 1:

[0120] Execute step A1, divide the data set at the patient level, read the breast ultrasound image of the training set, crop the region of interest of the read breast ultrasound image, and perform histogram equalization on the cropped image;

[0121] Execute step A2, scale the image after histogram equalization so that the length of the short side of the image is 224 pixels;

[0122] Execute step A3, crop the central area of ​​the image processed by step A2 by center cropping to obtain a target image of size 224×224 as a training set;

[0123] Execute step 2:

[0124] Execute step B1 and build a two-branch attention distillation network;

[0125] Construct residual convolution block 1, residual convolution block 2, dynamic spatial attention module 1, dynamic spatial attention module 2, gated channel attention module 1, gated channel attention module 2, gated channel attention module 3, global average pooling layer, and fully connected layer in branch 1;

[0126] Construct residual convolution block 1, residual convolution block 2, dynamic spatial attention module 1, dynamic spatial attention module 2, gated channel attention module 1, gated channel attention module 2, gated channel attention module 3, global average pooling layer, and fully connected layer in branch 2;

[0127] Execute step B2.

[0128] The preprocessed image is sequentially input into residual convolution block 1 and residual convolution block 2 in branch one, and residual convolution block 2 outputs feature A; feature A is sequentially input into dynamic spatial attention module 1 and dynamic spatial attention module 2 in branch one, and dynamic spatial attention module 2 outputs feature B; feature B is sequentially input into gated channel attention module 1, gated channel attention module 2, and gated channel attention module 3 in branch one, and gated channel attention module 3 outputs feature D; feature D is sequentially input into the global average pooling layer and the fully connected layer in branch one, and the output feature of the fully connected layer is used as the output feature of branch one;

[0129] The preprocessed image is sequentially input into residual convolution block 1 and residual convolution block 2 in branch two, and residual convolution block 2 outputs feature A′; feature A′ is element-wise added with feature A output by residual convolution block 2 in branch one to obtain feature A″; feature A″ is sequentially input into dynamic spatial attention module 1 and dynamic spatial attention module 2 in branch two, and dynamic spatial attention module 2 outputs feature B′; feature B′ is element-wise added with feature B output by dynamic spatial attention module 2 in branch one to obtain feature B″; feature B″ is sequentially input into gated channel attention module 1, gated channel attention module 2, and gated channel attention module 3 in branch two, and gated channel attention module 3 outputs feature D′; feature D′ is element-wise added with feature D output by gated channel attention module 3 in branch one to obtain feature D″; feature D″ is sequentially input into global average pooling layer and fully connected layer in branch two, and the output feature of fully connected layer is used as the output feature of branch two.

[0130] Execute step B3 and use the knowledge distillation strategy to distill the attention weight map extracted by the dual-branch network. The loss function of local feature distillation is as follows:

[0131]

[0132] in, Represents the weight value of the i-th row and j-th column on the spatial attention weight map of the dynamic spatial attention module of branch one; Represents the weight value of the i-th row and j-th column on the spatial attention weight map of the dynamic spatial attention module of branch 2;

[0133] At the same time, the gated channel attention module is used to extract the channel attention weight map of the two-branch network, and global feature distillation is performed by adopting the loss function as shown below:

[0134]

[0135] in, represents the weight of the kth channel of the channel attention weight map of branch one, Represents the weight of the kth channel of the channel attention weight map of branch 2;

[0136] Then, the output of the fully connected layer of branch one and the output of the fully connected layer of branch two are added element by element to obtain feature E, feature E is divided by 2 to obtain feature G, and feature G is passed through the softmax function to obtain the predicted probability value

[0137] The classification loss function is defined as follows:

[0138]

[0139] in, is the predicted probability value, y i is the real category;

[0140] Finally, the loss function for executing the two-branch attention distillation network model is as follows:

[0141] L=L LF +L GF +L C

[0142] Perform step three:

[0143] Execute step C1, initialize the network parameters, set the hyperparameters in the training process, use the Adam optimizer as the optimizer, set the weight decay parameter to 0.0001, set the batch size to 64, and set the maximum number of iterations to 150. The learning rate is set to decrease gradually according to the number of rounds. The learning rate from the 1st to the 80th round is set to 0.001, the learning rate from the 80th to the 125th round is set to 0.0005, and the learning rate for the last 25 rounds is set to 0.0001.

[0144] Execute step C2. If the network reaches the set maximum number of training rounds or the network loss does not change significantly within 10 rounds, the network is considered to have converged and the trained network parameters are saved.

[0145] Execute step 4:

[0146] Execute step D1, select the ultrasound image in the test set to execute the image preprocessing process of step 1, and obtain a processed breast ultrasound image.

[0147] Execute step D2, input the processed ultrasound image into the trained network for testing, and output the diagnosis result. This process does not update the network parameters. Calculate the performance indicators such as the accuracy, recall rate, and precision of the test results.

[0148] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A breast tumor benign and malignant identification system based on a dual-branch attention distillation network, characterized by: The system comprises: Ultrasound image preprocessing main module, dual-branch convolutional neural network main module, model training main module, model testing main module; The ultrasound image preprocessing main module is used to preprocess the collected breast ultrasound images, obtain the ultrasound imaging area, unify the image size of the obtained ultrasound imaging area, and obtain the preprocessed breast ultrasound images as a training set; The dual-branch convolutional neural network main module is used to build a dual-branch attention distillation network model; The model training main module uses the preprocessed breast ultrasound image to train the constructed dual-branch attention distillation network model, and obtains the trained dual-branch attention distillation network model; The model test main module is used to load the trained dual-branch attention distillation network model to classify the breast ultrasound images to be diagnosed.

2. According to claim 1, a breast tumor benign and malignant identification system based on a dual-branch attention distillation network is characterized by: The ultrasound image preprocessing main module is used to preprocess the collected breast ultrasound images, obtain the ultrasound imaging area, unify the image size of the obtained ultrasound imaging area, and obtain the preprocessed breast ultrasound images as a training set; the specific process is: Step A1, collecting a breast ultrasound image, cutting out a region of interest in the breast ultrasound image, and cutting out an ultrasound imaging area; Performing histogram equalization on the cropped ultrasound imaging area to obtain an image after histogram equalization; Step A2, scaling the image after histogram equalization so that the length of the short side of the image is 224 pixels; Step A3, cropping the central area of ​​the image processed in step A2 by means of center cropping to obtain a target image of size 224×224; The target image of size 224×224 is used as the final preprocessed breast ultrasound image and is used as the training set.

3. According to claim 2, a breast tumor benign and malignant tumor identification system based on a dual-branch attention distillation network is characterized in that: The dual-branch convolutional neural network main module is used to build a dual-branch attention distillation network model; the specific process is: The dual-branch attention distillation network model includes branch one and branch two; Branch 1 includes residual convolution block 1, residual convolution block 2, dynamic spatial attention module 1, dynamic spatial attention module 2, gated channel attention module 1, gated channel attention module 2, gated channel attention module 3, global average pooling layer, and fully connected layer in sequence; Branch 2 includes residual convolution block 1, residual convolution block 2, dynamic spatial attention module 1, dynamic spatial attention module 2, gated channel attention module 1, gated channel attention module 2, gated channel attention module 3, global average pooling layer, and fully connected layer.

4. According to claim 3, a breast tumor benign and malignant identification system based on a dual-branch attention distillation network is characterized in that: The working process of the dual-branch attention distillation network model is: The image is sequentially input into residual convolution block 1 and residual convolution block 2 in branch one, and residual convolution block 2 outputs feature A; feature A is sequentially input into dynamic spatial attention module 1 and dynamic spatial attention module 2 in branch one, and dynamic spatial attention module 2 outputs feature B; feature B is sequentially input into gated channel attention module 1, gated channel attention module 2, and gated channel attention module 3 in branch one, and gated channel attention module 3 outputs feature D; Feature D is sequentially input into the global average pooling layer and the fully connected layer in branch one, and the output feature of the fully connected layer is used as the output feature of branch one; The image is sequentially input into residual convolution block 1 and residual convolution block 2 in branch 2, and residual convolution block 2 outputs feature A′; feature A′ is added element by element with feature A output by residual convolution block 2 in branch 1 to obtain feature A″; Feature A″ is sequentially input into the dynamic spatial attention module 1 and the dynamic spatial attention module 2 in branch 2, and the dynamic spatial attention module 2 outputs feature B′; Add feature B′ and feature B output by dynamic spatial attention module 2 in branch 1 element by element to obtain feature B″; Feature B″ is sequentially input into the gated channel attention module 1, gated channel attention module 2, and gated channel attention module 3 in branch 2, and the gated channel attention module 3 outputs feature D′; Add feature D′ and the output feature D of gated channel attention module 3 in branch 1 element by element to obtain feature D″; The feature D″ is sequentially input into the global average pooling layer and the fully connected layer in branch two, and the output feature of the fully connected layer is used as the output feature of branch two. The output features of branch one and branch two are added element by element to obtain feature E, feature E is divided by 2 to obtain feature G, and feature G is passed through the softmax function to obtain the predicted probability value of the dual-branch attention distillation network model.

5. According to claim 4, a breast tumor benign and malignant identification system based on a dual-branch attention distillation network is characterized in that: The residual convolution block 1 includes, in sequence: a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, a batch normalization layer, a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, a batch normalization layer, a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, and a batch normalization layer; The residual convolution block 2 includes: a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, a batch normalization layer, a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, a batch normalization layer, a convolution layer with a convolution kernel size of 3×3, a ReLU activation function, and a batch normalization layer.

6. According to claim 5, a breast tumor benign and malignant tumor identification system based on a dual-branch attention distillation network is characterized in that: Each of the dynamic spatial attention modules in the dynamic spatial attention module 1 and the dynamic spatial attention module 2 includes a dilated group convolution, an upsampling, and a linear mapping in sequence; The dilated group convolution includes a dilated convolution layer with a dilation rate of 1, a dilated convolution layer with a dilation rate of 2, and a dilated convolution layer with a dilation rate of 3; The working process of each dynamic spatial attention module is: The features are input into the dilated convolution layer with a dilation rate of 1, the dilated convolution layer with a dilation rate of 2, and the dilated convolution layer with a dilation rate of 3, respectively, to obtain three features; The three features obtained are upsampled respectively; The three upsampled features are linearly mapped to obtain the output features of each dynamic spatial attention module.

7. The breast tumor benign and malignant identification system based on a dual-branch attention distillation network according to claim 6 is characterized by: Each of the gated channel attention modules 1, 2 and 3 includes: global information encoding, channel normalization and adaptive gating mechanism; The global information encoding calculation formula is as follows: Among them, x represents the input feature map of the gated channel attention module, s g represents the output of global information encoding, m and n are the height and width of the feature map, respectively, x i,j Represents the eigenvalue of the i-th row and j-th column on the feature graph x; The channel normalization calculation formula is as follows: Among them, s n represents the normalized result, σ represents the minimum constant, C represents the total number of channels, Represents the square of the mean of the eigenvalues ​​of the kth channel on the feature map x; The calculation process of the adaptive gating mechanism is as follows: s o =x×[1+tanh(α×s n +β)] Among them, s o represents the output of the gated channel attention module; x represents the input feature map of the gated channel attention module; Both α and β represent learnable parameters.

8. According to claim 7, a breast tumor benign and malignant identification system based on a dual-branch attention distillation network is characterized in that: The loss function acquisition process of the dual-branch attention distillation network model is: 1) The loss function of the dynamic spatial attention module is as follows: in, Represents the weight value of the i-th row and j-th column on the spatial attention weight map of the dynamic spatial attention module of branch one; Represents the weight value of the i-th row and j-th column on the spatial attention weight map of the dynamic spatial attention module of branch 2; 2) The loss function of the gated channel attention module is as follows: in, Represents the weight of the kth channel of the channel attention weight map of branch one; Represents the weight of the kth channel of the channel attention weight map of branch 2; 3) The classification loss function is defined as follows: in, is the predicted probability value, y i is the real category; 4) The loss function of the dual-branch attention distillation network model is as follows: L=L LF +L GF +L C .

9. The breast tumor benign and malignant identification system based on a dual-branch attention distillation network according to claim 8, characterized in that: The model training main module uses the preprocessed breast ultrasound images to train the built dual-branch attention distillation network model to obtain the trained dual-branch attention distillation network model; the specific process is: Step C1, initialize the parameters of the dual-branch attention distillation network model; The optimizer uses the Adam optimizer; Set weight decay parameters; Set the initial learning rate and learning rate decay rule; Set the maximum number of iterations; Step C2: The two-branch attention distillation network model reaches the set maximum number of iterations or the loss function of the two-branch attention distillation network model converges, obtains the trained two-branch attention distillation network model, and saves the trained two-branch attention distillation network model weights.

10. The breast tumor benign and malignant identification system based on a dual-branch attention distillation network according to claim 9, characterized in that: The model test main module is used to load the trained dual-branch attention distillation network model, classify the breast ultrasound images to be diagnosed, and identify the benign and malignant breast tumors in the ultrasound images; the specific process is: The test set data is processed by the ultrasound image preprocessing main module to obtain the processed breast ultrasound image; The processed breast ultrasound images are input into the trained two-branch attention distillation network model, and the two-branch attention distillation network model outputs the classification results.

Images