Artificial intelligence-based breast cancer ultrasound image diagnosis assistance method

Abstract

The application discloses an artificial intelligence-based breast cancer ultrasonic image diagnosis auxiliary method, and aims to provide a high-precision classification model to assist doctors in diagnosis.The method comprises image preprocessing and training of a convolutional neural network model.Firstly, in the data preprocessing part, the application adopts a Laplacian operator to perform edge extraction and sharpening on the image, enhances the image definition, and simultaneously performs data enhancement and standardization processing on the image.Secondly, in the model training part, the application adopts RepVGG as a baseline model, combines an attention mechanism module, embeds position information into channel attention, replaces the last layer of the network model with a self-defined multi-branch convolution structure, adopts a weighted sum result of cross-entropy loss and focal loss as a loss function of the model, and finally obtains an optimal model of the network through an ablation experiment.The application has important significance for automatic analysis and diagnosis of breast ultrasonic images.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary application of medicine and artificial intelligence, and specifically relates to an artificial intelligence-based method for assisting in the diagnosis of breast cancer using ultrasound images. Background Technology

[0002] Currently, breast cancer is one of the most common cancers among women, and ultrasound imaging is a crucial diagnostic tool. However, due to the complex structure of breast tissue, many interfering factors exist in ultrasound images, making it easy for doctors to misdiagnose or miss diagnoses when faced with a large amount of image data. Traditional ultrasound imaging methods for breast cancer diagnosis rely heavily on the doctor's experience and judgment, which suffers from subjectivity and instability, and still requires significant manual analysis and interpretation, resulting in low efficiency and difficulty in meeting clinical needs.

[0003] Therefore, AI-based methods for assisting in the diagnosis of breast cancer using ultrasound images have become a research hotspot. Convolutional neural network models can automatically extract image features, automatically select image features, and classify images without relying on manual feature extraction, thus avoiding the influence of subjective factors. Furthermore, they can better assist doctors in diagnosis, improving performance. The advantage of this method is that it can quickly and accurately identify and classify breast cancer ultrasound images, greatly improving diagnostic accuracy and efficiency while reducing the workload of doctors. However, existing AI-based methods for breast cancer ultrasound image diagnosis still have some problems, such as low model training efficiency and insufficient generalization ability. Therefore, establishing a fast and accurate model for analyzing breast cancer ultrasound images and correctly classifying breast diseases is crucial. Summary of the Invention

[0004] To address the problems of incomplete feature extraction, low inference speed, and low classification accuracy in existing classification models, this invention designs an artificial intelligence-based ultrasound image-assisted method for breast cancer diagnosis.

[0005] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution:

[0006] An AI-based ultrasound image-assisted diagnostic method for breast cancer includes the following steps:

[0007] Step (1): Divide the acquired breast ultrasound image data into training set and test set according to the ratio.

[0008] Step (2): Preprocess the ultrasound image dataset as follows:

[0009] First, data augmentation is performed on the divided training and test sets, specifically through random cropping, random flipping, random rotation, and random contrast adjustments. Second, the Laplacian operator is used to extract edge pairs and sharpen the augmented data. The Laplacian operator is fundamental to calculating pixel gray-level differences within an image's neighborhood; a neighborhood enhancement algorithm can be derived using second-order differentiation. Its basic idea is that when the gray level of the center pixel in a neighborhood is lower than the average gray level of other pixels in that neighborhood, the gray level of the center pixel should be further reduced; when it is higher, the gray level of the center pixel should be further increased, thus achieving image sharpening. In the algorithm implementation, gradients are calculated in four or eight directions around the center pixel, and the gradients are summed to determine the relationship between the center pixel's gray level and the gray levels of other pixels in the neighborhood. The results of the gradient calculations are used to adjust the pixel gray levels, thereby improving image clarity. Finally, the sharpened data is standardized. Standardization involves centering the data using a mean-removing method. Data centering conforms to data distribution patterns and helps improve the model's generalization ability.

[0010] Step (3): Construction of the RCM (RepVGG-CA-MB)-Net network model

[0011] The RepVGG network model was used as the baseline network.

[0012] Add an attention mechanism module (Coordinate Attention, CA) to the RepVGG network model. Specifically, add it after the last basic block of the network and embed the position information into the channel attention.

[0013] The last stage of the RepVGG network model is replaced with a custom multi-branch module. This multi-branch module is a structure formed by concatenating a Conv-BN base module and a multi-scale convolutional fusion module using the `concat` function. The Conv-BN base module consists of a 3×3 convolution and a BN layer in sequence, while the multi-scale convolutional module is formed by adding three convolutional branches: 1×1, 1×3, and 3×1. In this invention, the input feature layers of both the Conv-BN base module and the multi-scale convolutional fusion module have 256 channels, and the output is identical to the input. The two output feature layers are concatenated using the `concat` function to form a feature layer with 512 channels before proceeding to the next stage. The multi-branch module contains a spatially separable structure, significantly reducing the model's parameters and computational cost while maintaining the receptive field size.

[0014] Step (4): Use the weighted sum of cross-entropy loss and focus loss as the loss function for the established RCM-Net network model;

[0015] Step (5): Set the output category of the network model to 3 to make it suitable for the breast ultrasound image classification task. The three categories are normal, benign and malignant.

[0016] Step (6): Obtain the optimal model of the RCM-Net network through ablation experiments;

[0017] Step (7): Train the constructed RCM-Net network model using the preprocessed training set, calculate the model performance index on the test set, and select the model with the best performance based on the index to achieve the goal of high-precision prediction of breast ultrasound image properties.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0019] (1) The artificial intelligence-based ultrasound image diagnostic auxiliary method for breast cancer proposed in this invention can extract more complete features of the image compared with the traditional machine learning model. At the same time, due to the structural reparameter function, it can greatly reduce the inference speed.

[0020] (2) The model proposed in this invention incorporates an attention focus mechanism module, which embeds location information into channel attention. Without adding extra computational burden, the model can focus more on key areas and feature location information, which is of great significance for the identification of tumor edge features.

[0021] (3) The method proposed in this invention combines a custom multi-branch convolution module, which includes a spatially separable structure and has a structure reparameterization function, making the convolution process easier and more efficient. Compared with the standard convolution operation, it reduces the number of parameters and computational cost. Attached Figure Description

[0022] Figure 1 This is a basic flowchart of an AI-based ultrasound image-assisted diagnostic method for breast cancer.

[0023] Figure 2(a) and Figure 2(b) are schematic diagrams of the RepBlock basic module and the backbone structure in the RepVGG baseline network, which are structurally reparameterizable.

[0024] Figure 3 This is a schematic diagram of the CA attention mechanism module framework;

[0025] Figure 4 This is a schematic diagram of a custom multi-branch convolution module. Detailed Implementation

[0026] The application principle of the present invention will be described in detail below with reference to the accompanying drawings.

[0027] This invention provides an artificial intelligence-based ultrasound image-assisted method for breast cancer diagnosis, such as... Figure 1 As shown, it includes the following steps:

[0028] Step (1): Obtain the breast ultrasound image dataset. Divide the acquired breast ultrasound image dataset into a training set and a test set at a ratio of 3:1. Dividing the dataset into training and test sets is the first step in classifying a deep convolutional neural network. The dataset in this embodiment is the open-source breast ultrasound image dataset BUSI downloaded from the Kaggle platform. It contains 780 breast ultrasound images from 600 female patients aged 25 to 75 years. The average image size is 500*500 pixels, and the images are saved in PNG format. These images are divided into three categories: normal (no tumor), benign, and malignant.

[0029] Step (2): First, perform data augmentation on the divided training set. Specific measures include random cropping, random flipping (horizontal / vertical), random rotation (-40° to 40°), and random contrast adjustment. The augmented training set data is expanded by a factor of 2. Second, perform sharpening and standardization on all data. Sharpening involves calculating the gradients in the four directions of the center pixel in the neighborhood, summing the gradients to determine the relationship between the grayscale of the center pixel and the grayscale of other pixels in the neighborhood, and adjusting the pixel grayscale using the gradient calculation results to improve image clarity. Standardization is a process that centers the data by removing the mean, making the data conform to the data distribution pattern, which helps improve the model's generalization ability. The formula is as follows:

[0030]

[0031] Where μ is the mean of the image, and X represents the image matrix. σ represents the standard deviation, and N represents the number of pixels in image X.

[0032] Finally, the breast ultrasound images were standardized to a size of 224×224 to meet the model's requirements for input data size.

[0033] Step (3): Construct the RCM-Net network model

[0034] (3.1) Using the RepVGG network model as the baseline network, as shown in Figures 2(a) and 2(b), this network can be divided into four parts: an input layer, a RepBlock layer, a global average pooling layer, and a fully connected layer. The input layer receives the input image, resizes it to a uniform size, and passes it to the first RepBlock. RepVGG-A2 contains 20 RepBlocks, each containing two convolutional layers and one reparameterized layer. The first RepBlock has 64 input channels and 64 output channels, while the last RepBlock has 512 input channels and 1024 output channels. The global average pooling layer transforms the output tensors of the 20 RepBlocks into one-dimensional vectors. Finally, the fully connected layer receives the output vector from the global average pooling layer and transforms it into a 10-dimensional vector, representing the probability distribution of 10 categories. In this invention, the final transformation into a 3-dimensional vector makes it suitable for breast ultrasound image classification tasks.

[0035] When a network employs a large number of multi-branch structures, different branches apply different convolutional kernels, resulting in different receptive fields and a higher likelihood of achieving a high-performance network. However, multi-branch structures require storing intermediate results, significantly increasing memory usage. Too many parameters can also slow down model inference. RepVGG's reparameterization feature addresses this issue. Through reparameterization, the multi-path structure of the training network (advantage of high performance during multi-branch model training) is transformed into a single-path structure for the inference network (advantage of fast speed and memory saving during model inference). As shown in Figure 2(a), the basic block in RepVGG can be transformed into a 3×3 convolution through reparameterization, thereby improving inference speed while maintaining model accuracy.

[0036] (3.2) Integrating the attention mechanism module (CA) into the network model, specifically added after the last block of the baseline model, can embed the position information into the channel attention, combining the channel main attention with spatial attention, so that the system can choose to process the important information of the input image in space and channel without putting extra burden on computing resources, and can make the network pay more attention to the effective features and position information of the input image.

[0037] The implementation of CA attention is as follows: Figure 3As shown, the process can be considered as two parallel stages. In the first stage, global average pooling is performed on the input feature layer in both the width and height directions. Assuming the shape of the input feature layer is [C,H,W], where C is the number of channels in the feature layer, and H and W represent the height and width of the feature layer, respectively, after global average pooling, two feature layers with dimensions [C,H,1] and [C,1,W] are obtained. The width and height of these two feature layers are transposed to the same dimension and then stacked to merge the width and height features, resulting in a feature layer with dimensions [C,1,H+W]. Features are then obtained using convolution, normalization, and activation functions. In the second stage, the feature layer obtained in the first stage is again divided into two parallel stages, separating the width and height into [C,1,H] and [C,1,W], and then transposing them. This yields two feature layers [C,H,1] and [C,1,W]. Then, after adjusting the number of channels using 1x1 convolution, the sigmoid function is used to obtain the attention results in the width and height dimensions. Multiplying this by the original features gives the final result of the CA attention mechanism.

[0038] (3.3) Add a custom multi-branch module, replacing the last stage of the baseline model with a custom multi-branch module. This module is a multi-branch structure composed of the Conv-BN basic module and the multi-scale convolution fusion module. For example... Figure 4 As shown, the Conv-BN branch consists of a 3×3 convolution and a BN layer in sequence. The BN layer accelerates the training and convergence of the network, avoids gradient explosion or vanishing gradients, and prevents overfitting. The multi-scale convolutional fusion branch consists of three convolutional branches: 1×1, 1×3, and 3×1. The results from these three branches are added together, resulting in a spatially separable structure that significantly reduces the model's parameters and computational cost while maintaining the receptive field size. The two branches are then concatenated to fuse the extracted features. This module significantly reduces the model's parameters and computational cost while ensuring the quantity and completeness of extracted features. After training, the entire custom multi-branch module can be replaced with a single 3×3 convolutional structure through structural reparameterization, further simplifying the model structure and improving inference speed.

[0039] Step (4): Use the weighted sum of cross-entropy loss and focus loss as the loss function, the specific expression of which is as follows:

[0040]

[0041] Where, p i Let q represent the true distribution. i Let p represent the predicted distribution, n represent the number of categories, and p represent the number of categories. t α represents the probability that the model predicts a positive sample. tThe value represents the sample class weight, and γ is an adjustment factor. This parameter is used to adjust the difficulty level. When γ is greater than 0, it will increase the penalty for difficult samples, thereby increasing the model's attention to difficult samples.

[0042] The focus loss function is based on the binary cross-entropy loss function. It is a dynamically scaled cross-entropy loss. Through a dynamic scaling factor, the weight of easily distinguishable samples during training can be dynamically reduced, thereby quickly focusing the attention on those difficult-to-distinguish samples.

[0043] Step (5): Change the output dimension of the network to suit the breast ultrasound image classification task. The original baseline network has an output dimension of 1000. To adapt to the ultrasound image classification task of this dataset, this implementation can set the output dimension of the network to 3.

[0044] Step (6): Through ablation experiments, the optimal network model is obtained. The custom multi-branch module of the invention is used to replace a single block in the last layer of the baseline network. The preprocessed breast ultrasound images in the test set are input into the model to obtain the classification results and the image prediction results. At the same time, the weight file saved during training can be used to calculate the network performance on the test set. The present invention uses common classification model performance evaluation indicators, namely accuracy, precision, recall and F1-score, to evaluate the model performance. Accuracy, precision, recall and F1-score are defined as shown in the following formulas (3)-(6), where TP (True Positive) means that the classifier predicts that the sample is positive and it is actually positive, that is, the number of positive samples correctly identified; FP (False Positive) means that the classifier predicts that the sample is positive and it is actually negative, that is, the number of negative samples falsely reported. TN (True Negative) means that the classifier predicts that the sample is negative and it is actually negative, that is, the number of negative samples correctly identified. FN (False Negative) represents the number of positive samples that the classifier predicts as negative but are actually positive.

[0045] accuracy

[0046] Accuracy

[0047] Recall rate

[0048]

[0049] To make the experimental results more concise and easier to understand, a confusion matrix can be used to display the classification results for each class of data.

[0050] Furthermore, the method of this invention can also be used to analyze different categories of medical image data, such as predicting the response to neoadjuvant chemotherapy in axillary lymph nodes. The use of artificial intelligence methods to predict the nature of breast tumors shows great promise, with reliable, accurate, and specific results, thus playing a significant role in the early screening and diagnosis of breast cancer.

Claims

1. An artificial intelligence-based ultrasound image-assisted method for breast cancer diagnosis, characterized in that, Includes the following steps: Step (1): Divide the acquired breast ultrasound image data into training set and test set according to the ratio; Step (2): Preprocess the ultrasound image dataset as follows: First, data augmentation is performed on the divided training and test sets, specifically through random cropping, random flipping, random rotation, and random contrast adjustment. Second, the Laplacian operator is used to extract edges and sharpen the augmented image data. Finally, the sharpened data is standardized. Step (3): Construction of the RCM-Net network model; The RepVGG network model was used as the baseline network. An attention mechanism module is added to the RepVGG network model, specifically after the last basic block of the network, and the position information is embedded into the channel attention. Replace the last stage in the RepVGG network model with a custom multi-branch module: the multi-branch module is a multi-branch structure concatenated from the Conv-BN base module and the multi-scale convolutional fusion module. The Conv-BN base module consists of a... The convolutional layer and a batch normalization (BN) layer are sequentially combined, and the multi-scale convolutional fusion module consists of... The three convolutional branches are added together; the input Conv-BN base module and the multi-scale convolutional fusion module both have 256 feature layer channels, and the output is the same as the input; the two output feature layers are concatenated into a feature layer with 512 channels by the concat function and then enter the next stage; Step (4): Use the weighted sum of cross-entropy loss and focus loss as the loss function for the established RCM-Net network model; Step (5): Set the output category of the network model to 3 to make it suitable for the breast ultrasound image classification task. The three categories are normal, benign and malignant. Step (6): Obtain the optimal model of the RCM-Net network through ablation experiments; Step (7): Train the constructed RCM-Net network model using the preprocessed training set, calculate the model performance index on the test set, and select the model with the best performance based on the index.

Images