A multi-scale adaptive lesion detection method based on breast ultrasound

Abstract

The present invention discloses a multi-scale adaptive lesion detection method based on breast ultrasound, comprising the following steps: image pre-processing; input module: extracting features through convolution blocks to clearly display boundaries, and then refining these boundary features through deconvolution blocks to complete image expression; backbone module; neck network: CA generates feature maps in two directions, splices the feature maps in the two directions, extracts features through convolution operations, and further generates attention weights in the two directions; detection head network: positioning prediction, classification prediction, loss calculation, and finally output results. The present invention improves the accuracy and richness of feature extraction, enhances the model's ability to clearly identify the boundaries of different tissues; improves the ability to capture multi-scale lesion features, better captures lesion characteristics; can highlight important areas such as smaller calcifications, changes in cyst walls, and the boundaries of fibroadenomas, thereby improving the accuracy and specificity of detection.

Description

Technical Field

[0001] The present invention relates to the technical field of breast ultrasound lesion detection, and in particular to a multi-scale adaptive lesion detection method based on breast ultrasound. Background Art

[0002] Many breast lesions may not show obvious symptoms in their early stages. Early detection can buy valuable time for treatment and improve cure rates. This is especially true for malignant lesions like breast cancer, where early diagnosis and treatment can significantly improve patient survival. Breast lesions detected early are generally easier to treat and relatively inexpensive. As one of the most common problems faced by women, the detection of breast lesions is a crucial topic in clinical practice.

[0003] Currently, breast lesion detection is mostly based on breast ultrasound, mammography, and breast magnetic resonance imaging, using waveform feature rules, machine learning, or deep learning. The main methods are as follows:

[0004] Waveform feature-based rule-based methods utilize technologies such as expert systems to classify and diagnose breast lesions according to predefined rules. These rules are often based on medical knowledge and clinical experience. Some studies use rules such as mass shape, margin characteristics, and calcification distribution in mammographic images to determine the nature of lesions. This approach has the advantage of strong interpretability, but the characteristics of breast lesions are complex, and fixed rules are difficult to adapt to all situations. This is especially true for atypical lesions, which are difficult to automatically adjust and optimize based on new data or cases.

[0005] This machine learning-based approach first uses ultrasound elastography to assess tissue stiffness. Malignant tumors are typically stiffer than surrounding normal tissue. Machine learning algorithms are then applied to analyze the ultrasound elastography data, helping doctors more accurately determine the nature of the lesion. Manual feature extraction is required, which not only increases the preprocessing workload but can also lead to the omission of key features or inaccurate feature selection. Furthermore, the performance of machine learning models is highly dependent on the quality and quantity of training data. Insufficient or unbalanced training data can limit the model's generalization capabilities.

[0006] Deep learning approaches often use deep learning networks, such as convolutional neural networks, to analyze image data, automatically extract features from images, and detect breast lesions. For example, one study used CNNs to identify benign and malignant masses in breast ultrasound images, achieving 90% accuracy, with diagnostic sensitivity and specificity of 86% and 96%, respectively. CNNs can also be combined with other techniques, such as transfer learning and multi-instance learning, to improve model performance. In mammography, CNNs are trained on large amounts of labeled image data to automatically identify tiny calcifications and masses. However, these methods often fail to account for the wide range of lesion scales, resulting in inaccurate feature extraction and limited ability to capture key features such as smaller calcifications, cyst wall changes, and fibroadenoma boundaries.

[0007] Therefore, a multi-scale adaptive lesion detection method based on breast ultrasound is urgently needed to solve the above problems. Summary of the Invention

[0008] The purpose of the present invention is to solve the problems of poor flexibility, insufficient generalization ability, and deficiencies in multi-scale lesion capture and key area identification in the existing technology, and to propose a multi-scale adaptive lesion detection method based on breast ultrasound. An artificial intelligence algorithm is developed based on non-invasive, radiation-free and highly adaptable breast ultrasound to achieve accurate judgment of the four common different types of breast lesions: cysts, fibroadenomas, suspicious tumors, and calcifications, thereby timely alerting these lesions.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A multi-scale adaptive lesion detection method based on breast ultrasound includes the following specific steps:

[0011] S1: Image preprocessing: First, based on the letterbox method, the breast ultrasound image is converted to a size of 1280×1280×3. This method can simultaneously maintain the original aspect ratio of the image and the integrity of the breast ultrasound content. The scaling ratio is determined based on the original length and width of the breast ultrasound image and the target length and width, and the image size is adjusted. The scaling ratio is the smaller ratio of the target size to the original size to ensure that the image does not exceed the target size after scaling. 0 pixels are added to the short edges of the top, bottom, left, and right sides of the image to ensure that the long and short edges match the target size.

[0012] S2: Input module: Receives the pre-processed image and extracts features such as edges and textures through the convolution block to clearly display the boundaries of fat, glands, and fibrous tissue. The deconvolution block then refines these boundary features to complete the image representation. After passing through the input module, the image is converted to a size of 640×640×3 that is most suitable for processing by the backbone module.

[0013] S3: Backbone module: The ultrasound image is input into the backbone module of the YOLOv12 model improved based on the MSCA module. Its core is the residual efficient layer aggregation network. It first passes through the first and second convolutional layers, then through the first C3k2 module, and then through the third convolutional layer. The output passes through the second C3k2 module and the fourth convolutional layer in sequence, and then inputs into the first A2C2f module. After passing through the fifth convolutional layer, it is input into the A2C2f module and finally into the MSCA module.

[0014] S4: Neck network: CA aggregates features in the horizontal and vertical directions through 1D pooling operations to generate feature maps in two directions. Specifically, the input feature map is processed using average pooling with pooling kernel sizes of sum to obtain two direction-aware feature maps. The feature maps in the two directions are concatenated, and then features are extracted through convolution operations. Attention weights in the two directions are further generated. The h-swish nonlinear activation function is used to increase the expressive power of the model. The generated attention weights are applied to the horizontal and vertical directions of the input feature map, respectively, to achieve direction-aware and position-sensitive attention enhancement.

[0015] S5: Detection head network: The detection head network receives multi-scale feature maps from the neck network. These feature maps are fused and adjusted to contain semantic information and spatial details at different levels. They perform positioning prediction, classification prediction, loss calculation, and finally output the results.

[0016] As a further technical solution of the present invention, in S2, the convolution block consists of a convolution layer, a normalization layer and an activation function. The convolution layer adopts a 3×3 convolution kernel and the stride is set to 2. Then the convolution output is further normalized and activated to improve its expression ability; the low-level features in breast ultrasound are extracted by the first convolution block, wherein the first convolution block selects the GELU activation function, adaptively adjusts the output according to the input data, and enhances the feature expression; then the second convolution block is used to extract more abstract features, wherein the activation function selects SiLU to further enhance the nonlinearity of feature expression.

[0017] As a further technical solution of the present invention, in S3, the MSCA module is suitable for detecting targets with diverse sizes. First, features of different scales are extracted from the feature map through the multi-scale convolution branch, and multiple feature maps with different scale information are adaptively obtained; then, the inter-channel weights of these feature maps are calculated through the attention mechanism, and the feature maps are weightedly fused according to the importance of the channels; the fused feature maps are then passed through the spatial attention mechanism to generate a spatial attention weight map, and multiplied with the original feature map, so that the model pays more attention to the key feature areas.

[0018] As a further technical solution of the present invention, in S3, the improved YOLOv12 model adopts a new convolution block category, which is more in line with the requirements of medical scenarios for lightweight operations and high parallelism. The Yolov12 model uses a series of smaller kernels, generally expressed as: ,in: is the output feature, are input features, is the weight of the i-th convolution kernel, is the bias of the i-th convolution kernel.

[0019] As a further technical solution of the present invention, S3 specifically includes:

[0020] S31: The first convolutional layer uses a 3×3 convolution kernel with a stride of 2 and a padding of 1 to extract basic features of the input image. After this convolution operation, the model initially captures basic features such as edges and textures of the image.

[0021] S32: The output features are input to the second convolutional layer. This layer still uses the 3×3 convolution kernel, but the stride is changed to 1 and the padding is still 1. After the convolution operation, the size of the feature map remains unchanged, and the number of channels is doubled. The second convolutional layer further mines deeper feature information and enriches the feature expression while maintaining the stability of the feature map size.

[0022] S33: The output of the second convolutional layer then enters the first C3k2 module. The input feature map is divided into two parts. One part is directly passed to retain shallow features, and the other part is processed through the standard Bottleneck to process deep features. The final parts are spliced ​​and fused. The C3k2 module assists in feature extraction. The C3k2 module retains multiple bottleneck layers, similar to CSPNet. By stacking multiple bottleneck layers, it gradually extracts and fuses features, improving the feature expression capability.

[0023] S34: The output feature map of the first C3k2 module flows into the third convolutional layer, which uses the filters of the first convolutional layer. The main function of the third convolutional layer is to further downsample, compress the size of the feature map, extract deeper and more abstract feature information, and expand the number of channels to meet the needs of subsequent complex feature extraction;

[0024] S35: The output feature map of the third convolutional layer enters the second C3k2 module and the fourth convolutional layer. This layer follows the structure of the first C3k2 module and the second convolutional layer.

[0025] S36: The output feature map of the fourth convolutional layer is then input to the first A2C2f module. The A2C2f module uses adaptive channels and spatial attention mechanisms to enhance the network's perception and improve the efficiency of capturing objects of different scales.

[0026] S37: The output feature map of the first A2C2f module is sent to the fifth convolutional layer, which uses a 3×3 convolution kernel with a stride of 2 and a padding of 1. The fifth convolutional layer is used to further downsample and compress the feature map size, extracting deeper and more semantic feature information, while increasing the number of channels to meet the requirements of subsequent feature extraction.

[0027] S38: The output feature map of the fifth convolutional layer is input to the second A2C2f module. This module follows the architecture of the first A2C2f module. After the feature map is divided into regions, a self-attention mechanism is applied to each region. Dynamic adjustments are made based on the correlation between regions to highlight key regions and weaken interference. The processed features of each region are fused, superimposed on the input feature map, and output through the ReLU activation function. The size of the output feature map of the second A2C2f module remains unchanged. The feature map is optimized through attention, and the features of key regions are further strengthened, thereby improving the accuracy of feature expression.

[0028] S39: The second A2C2f module outputs the feature map and inputs it into the MSCA module, which is mainly composed of a multi-scale convolution module and an attention module. The multi-scale convolution module uses convolution kernels of different sizes to convolve the feature map, thereby capturing different scale features from fine-grained to coarse-grained, and integrating the features extracted by multi-scale convolution to generate attention weights; local information is aggregated through deep convolution, and then multi-branch deep convolution is used to capture multi-scale contextual information. Finally, 1×1 convolution is used to simulate the relationship between different channels in the feature to generate the weight of convolution attention.

[0029] As a further technical solution of the present invention, in the S36, the A2C2f module uses two convolutional layers cv1 and cv2 to process the input features to obtain two feature maps with different channel numbers. cv1 reduces the number of input channels by one half, while cv2 keeps the number of input channels unchanged. Each convolutional layer is composed of a Conv2d layer, a BatchNorm2d layer, and an activation function SiLU or Identity, which is used to perform convolution operations, normalization, and nonlinear transformations on the input features. In each ABlock module, an attention mechanism module AAttn and a multi-layer perceptron MLP module are included. First, the query, key, and value vectors are generated through the qkv layer in AAttn, and Flash is used to generate the query, key, and value vectors. The Attention mechanism calculates the attention weight, implements the spatial attention mechanism for features, and highlights the features of important spatial areas; at the same time, position encoding is performed through the pe convolution layer to enhance the model's perception of position information; then, the attention result is projected back to the original dimension through the proj convolution layer, and residual connection is performed with the input feature to achieve feature fusion and enhancement; then, the fused features are further nonlinearly transformed through the MLP module to further extract and fuse features; among them, qkv is a convolution layer used to generate query query, key key and value value vectors, and the number of output channels is three times that of the input; proj is another convolution layer used to project the attention result back to the original dimension; pe is position encoding, which is achieved through a depth-wise separable convolution with a kernel size of 7x7; by stacking two ABlock modules, multiple attention calculations and feature fusion can be performed on the features, thereby enhancing the model's ability to capture objects of different scales.

[0030] As a further technical solution of the present invention, in S4, the neck network is improved based on the CA attention mechanism, which improves the model's ability to express feature maps by combining the channel attention and spatial position information contained in the previous feature map.

[0031] As a further technical solution of the present invention, the S4 specifically includes:

[0032] S41: The MSCA output feature map of the backbone module is upsampled and fused with the first A2C2f layer, and then passed to the third A2C2f layer. The structure of the A2C2f layer is the same as that of the first A2C2f layer. The output feature map is upsampled and then fused with the output features of the second A2C2f layer, and output to the fourth A2C2f layer.

[0033] S42: After the feature input processed by the fourth A2C2f layer is embedded in the CA attention module, the multi-scale module is output to the detection module;

[0034] S43: The features processed by the fourth A2C2f layer are also output to the sixth convolutional layer, and the features processed by the sixth convolutional layer are fused with the output of the third A2C2f layer;

[0035] S44: After fusion, it is input into the fifth A2C2f layer, the output feature map is input into the CA attention module for processing, and the multi-scale feature map is output to the detection module;

[0036] S45: The feature map output by the fifth A2C2f layer is simultaneously input into the seventh convolutional layer, and the output result is fused with the output of the MSCA module, and then output to the third C3k2 layer, whose output feature map flows into the CA attention module, and then the multi-scale module three is output to the detection module.

[0037] As a further technical solution of the present invention, in S5, the detection head network of the YOLOv12 model is mainly responsible for performing the target detection tasks of cysts, fibroadenomas, suspicious tumor groups, and calcification detection.

[0038] As a further technical solution of the present invention, the S5 specifically includes:

[0039] S51: Positioning Prediction: Use the anchor box mechanism to predict the bounding box. Predefine anchor boxes of different scales and aspect ratios. Then predict the offset and confidence of each anchor box to determine the location and size of the target. Through a combination of fully connected layers and convolutional layers, predict the coordinates of the bounding box for each feature point, usually including the center coordinates (x, y), width and height of the bounding box.

[0040] S52: Classification prediction: Perform classification prediction on each feature point or anchor box, and output the probability distribution of whether the location belongs to the cyst, fibroadenoma, suspicious group tumor, or calcification category. Use the softmax activation function to convert the classification prediction output into a probability value to make the result more interpretable.

[0041] S53: Loss calculation: The cross entropy loss function is used to measure the difference between the classification prediction result and the true label. The difference between the predicted bounding box and the true bounding box is calculated based on the mean square error and CIoU loss. The bounding box coordinates after non-maximum suppression, the corresponding category label and confidence score are output as the final detection result.

[0042] The beneficial effects of the present invention are:

[0043] 1. The input layer can capture the boundary features of different tissue components in breast ultrasound images, including fat, glandular, and fibrous tissue. This not only improves the accuracy and richness of feature extraction, but also enhances the model's ability to clearly identify different tissue boundaries, solving the problem of insufficient feature extraction accuracy in existing technologies.

[0044] 2. The YOLOv12 backbone network is improved based on the MSCA module, which enhances the ability to capture multi-scale lesion features. The improved backbone network can adaptively extract feature information at different scales, thereby better capturing lesion features from fine-grained to coarse-grained. Compared with existing technologies, it improves the feature extraction capability of multi-scale targets.

[0045] 3. The improved neck network combined with the CA attention mechanism enhances the model's ability to capture key feature areas. The CA attention mechanism, by combining channel attention and spatial position information, can highlight important areas such as smaller calcifications, changes in cyst walls, and the boundaries of fibroadenomas, thereby improving detection accuracy and specificity and addressing the shortcomings of existing technologies in identifying key areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of a multi-scale adaptive lesion detection method based on breast ultrasound proposed by the present invention.

[0047] Figure 2 Schematic diagram of the MSCA attention module.

[0048] Figure 3 Schematic diagram of steps S3-S5. DETAILED DESCRIPTION

[0049] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0050] Please see the attached Figure 1 A multi-scale adaptive lesion detection method based on breast ultrasound includes the following specific steps:

[0051] S1: Image preprocessing: First, based on the letterbox method, the breast ultrasound image is converted to a size of 1280×1280×3. This method can simultaneously maintain the original aspect ratio of the image and the integrity of the breast ultrasound content. The scaling ratio is determined based on the original length and width of the breast ultrasound image and the target length and width, and the image size is adjusted. The scaling ratio is the smaller ratio of the target size to the original size to ensure that the image does not exceed the target size after scaling. 0 pixels are added to the short edges of the top, bottom, left, and right sides of the image to ensure that the long and short edges match the target size.

[0052] S2: Input module: Receives the pre-processed image and extracts features such as edges and textures through the convolution block to clearly display the boundaries of fat, glands, and fibrous tissue. The deconvolution block then refines these boundary features to complete the image representation. After passing through the input module, the image is converted to a size of 640×640×3 that is most suitable for processing by the backbone module.

[0053] The convolution block consists of a convolution layer, a normalization layer, and an activation function. The convolution layer uses a 3×3 convolution kernel with a stride of 2. The convolution output is then further normalized and activated to improve its expressiveness. The first convolution block extracts low-level features from breast ultrasound. The first convolution block uses the GELU activation function to adaptively adjust the output based on the input data to enhance feature expression. The second convolution block extracts more abstract features. The SiLU activation function is used to further enhance the nonlinearity of feature expression.

[0054] This input module can retain important features while reducing computational complexity, and the deconvolution block can also help detect subtle lesions such as microcalcifications.

[0055] S3: Backbone module: The ultrasound image is input into the backbone module of the YOLOv12 model improved based on the MSCA module. Its core is the residual efficient layer aggregation network. It first passes through the first and second convolutional layers, then through the first C3k2 module, and then through the third convolutional layer. The output passes through the second C3k2 module and the fourth convolutional layer in sequence, and then inputs into the first A2C2f module. After passing through the fifth convolutional layer, it is input into the A2C2f module and finally into the MSCA module.

[0056] The MSCA module is suitable for detecting targets of diverse sizes. First, it extracts features of different scales from feature maps through multi-scale convolution branches, adaptively obtaining multiple feature maps with information of different scales. Then, it calculates inter-channel weights for these feature maps through an attention mechanism, and performs weighted fusion of the feature maps based on the importance of the channels. The fused feature maps are then passed through a spatial attention mechanism to generate a spatial attention weight map, which is multiplied with the original feature map, allowing the model to focus more on key feature areas.

[0057] The improved YOLOv12 model uses a new convolutional block category that is more in line with the medical scenario's requirements for lightweight operations and high parallelism. The Yolov12 model utilizes a series of smaller kernels, generally represented as: ,in: is the output feature, are input features, is the weight of the i-th convolution kernel, is the bias of the i-th convolution kernel;

[0058] S31: The first convolutional layer uses a 3×3 convolution kernel with a stride of 2 and a padding of 1 to extract basic features of the input image. After this convolution operation, the model initially captures basic features such as edges and textures of the image.

[0059] S32: The output features are input to the second convolutional layer. This layer still uses the 3×3 convolution kernel, but the stride is changed to 1 and the padding is still 1. After the convolution operation, the size of the feature map remains unchanged, and the number of channels is doubled. The second convolutional layer further mines deeper feature information and enriches the feature expression while maintaining the stability of the feature map size.

[0060] S33: The output of the second convolutional layer then enters the first C3k2 module. The input feature map is divided into two parts. One part is directly passed to retain shallow features, and the other part is processed through the standard Bottleneck to process deep features. The final parts are spliced ​​and fused. The C3k2 module assists in feature extraction. The C3k2 module retains multiple bottleneck layers, similar to CSPNet. By stacking multiple bottleneck layers, it gradually extracts and fuses features, improving the feature expression capability.

[0061] S34: The output feature map of the first C3k2 module flows into the third convolutional layer, which uses the filters of the first convolutional layer. The main function of the third convolutional layer is to further downsample, compress the size of the feature map, extract deeper and more abstract feature information, and expand the number of channels to meet the needs of subsequent complex feature extraction;

[0062] S35: The output feature map of the third convolutional layer enters the second C3k2 module and the fourth convolutional layer. This layer follows the structure of the first C3k2 module and the second convolutional layer.

[0063] S36: The output feature map of the fourth convolutional layer is then input to the first A2C2f module. The A2C2f module uses adaptive channels and spatial attention mechanisms to enhance the network's perception and improve the efficiency of capturing objects of different scales.

[0064] The A2C2f module uses two convolutional layers cv1 and cv2 to process the input features and obtain two feature maps with different channel numbers. cv1 reduces the number of input channels by half, while cv2 keeps the number of input channels unchanged. Each convolutional layer consists of a Conv2d layer, a BatchNorm2d layer, and an activation function SiLU or Identity, which are used to perform convolution operations, normalization, and nonlinear transformations on the input features. In each ABlock module, there is an attention mechanism module AAttn and a multi-layer perceptron MLP module. First, the query, key, and value vectors are generated through the qkv layer in AAttn, and Flash is used to generate the query, key, and value vectors. The Attention mechanism calculates the attention weights to implement the spatial attention mechanism for features and highlight the features of important spatial areas. At the same time, the pe convolution layer performs position encoding to enhance the model's perception of position information. The attention results are then projected back to the original dimension through the proj convolution layer and residually connected with the input features to achieve feature fusion and enhancement. The fused features are then further nonlinearly transformed through the MLP module to further extract and fuse features. Among them, qkv is a convolution layer used to generate query, key, and value vectors, with the output channel number being three times that of the input. Proj is another convolution layer used to project the attention results back to the original dimension. Pe is position encoding, which is achieved through a depth-wise separable convolution with a kernel size of 7x7. By stacking two ABlock modules, multiple attention calculations and feature fusion can be performed on features, thereby enhancing the model's ability to capture objects of different scales.

[0065] S37: The output feature map of the first A2C2f module is sent to the fifth convolutional layer, which uses a 3×3 convolution kernel with a stride of 2 and a padding of 1. The fifth convolutional layer is used to further downsample and compress the feature map size, extracting deeper and more semantic feature information, while increasing the number of channels to meet the requirements of subsequent feature extraction.

[0066] S38: The output feature map of the fifth convolutional layer is input to the second A2C2f module. This module follows the architecture of the first A2C2f module. After the feature map is divided into regions, a self-attention mechanism is applied to each region. Dynamic adjustments are made based on the correlation between regions to highlight key regions and weaken interference. The processed features of each region are fused, superimposed on the input feature map, and output through the ReLU activation function. The size of the output feature map of the second A2C2f module remains unchanged. The feature map is optimized through attention, and the features of key regions are further strengthened, thereby improving the accuracy of feature expression.

[0067] S39: The second A2C2f module outputs the feature map and inputs it into the MSCA module, which is mainly composed of a multi-scale convolution module and an attention module. The multi-scale convolution module uses convolution kernels of different sizes to convolve the feature map, thereby capturing different scale features from fine-grained to coarse-grained, and integrating the features extracted by multi-scale convolution to generate attention weights; local information is aggregated through deep convolution, and then multi-branch deep convolution is used to capture multi-scale contextual information. Finally, 1×1 convolution is used to simulate the relationship between different channels in the feature to generate the weight of convolution attention.

[0068] S4: Neck network: CA aggregates features in the horizontal and vertical directions through 1D pooling operations to generate feature maps in two directions. Specifically, it processes the input feature map using average pooling with kernel sizes of sum to obtain two direction-aware feature maps. The feature maps in the two directions are concatenated, and then features are extracted through convolution operations. Attention weights in the two directions are further generated. The h-swish nonlinear activation function is used to increase the expressive power of the model. The generated attention weights are applied to the horizontal and vertical directions of the input feature map, respectively, to achieve direction-aware and position-sensitive attention enhancement.

[0069] The neck network is improved based on the CA attention mechanism, which combines the channel attention and spatial position information contained in the previous feature map to improve the model's ability to express feature maps;

[0070] S41: The MSCA output feature map of the backbone module is upsampled and fused with the first A2C2f layer, and then passed to the third A2C2f layer. The structure of the A2C2f layer is the same as that of the first A2C2f layer. The output feature map is upsampled and then fused with the output features of the second A2C2f layer, and output to the fourth A2C2f layer.

[0071] S42: After the feature input processed by the fourth A2C2f layer is embedded in the CA attention module, the multi-scale module is output to the detection module;

[0072] S43: The features processed by the fourth A2C2f layer are also output to the sixth convolutional layer, and the features processed by the sixth convolutional layer are fused with the output of the third A2C2f layer;

[0073] S44: After fusion, it is input into the fifth A2C2f layer, the output feature map is input into the CA attention module for processing, and the multi-scale feature map is output to the detection module;

[0074] S45: The feature map output by the fifth A2C2f layer is simultaneously input into the seventh convolutional layer, and the output result is fused with the output of the MSCA module, and then output to the third C3k2 layer, whose output feature map flows into the CA attention module, and then the multi-scale module three is output to the detection module.

[0075] S5: Detection head network: The detection head network receives multi-scale feature maps from the neck network. These feature maps are fused and adjusted to contain semantic information and spatial details at different levels. It performs positioning prediction, classification prediction, loss calculation, and finally outputs the results.

[0076] The detection head network of the YOLOv12 model is mainly responsible for performing target detection tasks such as cysts, fibroadenomas, suspicious tumor clusters, and calcification detection;

[0077] S51: Positioning Prediction: Use the anchor box mechanism to predict the bounding box. Predefine anchor boxes of different scales and aspect ratios. Then predict the offset and confidence of each anchor box to determine the location and size of the target. Through a combination of fully connected layers and convolutional layers, predict the coordinates of the bounding box for each feature point, usually including the center coordinates (x, y), width and height of the bounding box.

[0078] S52: Classification prediction: Perform classification prediction on each feature point or anchor box, and output the probability distribution of whether the location belongs to the cyst, fibroadenoma, suspicious group tumor, or calcification category. Use the softmax activation function to convert the classification prediction output into a probability value to make the result more interpretable.

[0079] S53: Loss calculation: The cross entropy loss function is used to measure the difference between the classification prediction result and the true label. The difference between the predicted bounding box and the true bounding box is calculated based on the mean square error and CIoU loss. The bounding box coordinates after non-maximum suppression, the corresponding category label and confidence score are output as the final detection result.

[0080] From the above description, it can be seen that the above-mentioned embodiments of the present invention achieve the following technical effects: a new input layer is designed, and its structure can extract features such as edges and textures for the boundaries of different tissue components, and clearly define the boundaries of fat, glands, fibrous tissue, etc. For cases with limited resolution of ultrasound images, the input layer reduces the computational complexity while retaining important features, thereby improving the accuracy and richness of feature extraction.

[0081] In breast ultrasound images, breast lesions such as cysts, fibroadenomas, and calcifications present in a variety of sizes and forms. Based on the MSCA module, the backbone network of YOLOv12 is improved. Feature maps of information at different scales are adaptively extracted through multi-scale convolution features, so that the improved YOLOv12 model can better capture lesion features from fine-grained to coarse-grained. The input features are weighted using simple element-by-element multiplication operations, achieving low computational complexity while efficiently extracting spatial information encoding of breast ultrasound.

[0082] The CA attention mechanism is combined with the improved neck network. By combining channel attention and spatial position information, irrelevant background information is suppressed, interference is reduced, and the model's ability to capture key feature areas is enhanced. It highlights areas that are critical for detecting lesions, including smaller calcifications, changes in cyst walls, and the boundaries of fibroadenomas.

[0083] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention is limited to these examples. Within the scope of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity.

Claims

1. A multi-scale adaptive lesion detection method based on breast ultrasound, characterized in that: The specific steps include: S1: Image preprocessing: Based on the letterbox method, the breast ultrasound image is converted to a size of 1280×1280×3, the scaling ratio is determined according to the original length and width of the breast ultrasound image and the target length and width, and the image size is adjusted; S2: Input module: Receives the pre-processed image, extracts features through the convolution block, clearly displays the boundaries, and then refines these boundary features through the deconvolution block to complete the image representation; S3: Backbone module: The ultrasound image is input into the backbone module of the YOLOv12 model improved based on the MSCA module, passes through the first convolutional layer and the second convolutional layer, then passes through the first C3k2 module, and then passes through the third convolutional layer. The output passes through the second C3k2 module and the fourth convolutional layer in sequence, and then inputs into the first A2C2f module, and then passes through the fifth convolutional layer. The A2C2f module is input and finally inputs into the MSCA module; Among them, the MSCA module is suitable for detecting targets with diverse sizes. First, the feature maps are extracted at different scales through the multi-scale convolution branch, and multiple feature maps with different scale information are adaptively obtained; then these feature maps are passed through the attention mechanism. The weight calculation between channels is used to perform weighted fusion of feature maps according to the importance of the channels. The fused feature maps are then passed through the spatial attention mechanism to generate a spatial attention weight map, which is multiplied with the original feature map to make the model pay more attention to the key feature areas. S4: Neck network: CA aggregates features in the horizontal and vertical directions through 1D pooling operations to generate feature maps in two directions. The feature maps in the two directions are spliced ​​together, and then features are extracted through convolution operations to further generate attention weights in the two directions. Among them, the neck network is improved based on the CA attention mechanism. It combines the channel attention and spatial position information contained in the previous feature map to improve the model's ability to express feature maps. S5: Detection head network: The detection head network receives the multi-scale feature map from the neck network, performs positioning prediction, classification prediction, loss calculation, and finally outputs the result.

2. The multi-scale adaptive lesion detection method based on breast ultrasound according to claim 1, characterized in that: In S2, the convolution block consists of a convolution layer, a normalization layer, and an activation function. The convolution layer uses a 3×3 convolution kernel and a stride of 2, and then the convolution output is further normalized and activated. The low-level features in breast ultrasound are extracted by the first convolution block, wherein the first convolution block selects the GELU activation function and adaptively adjusts the output according to the input data. Then, more abstract features are extracted by the second convolution block, wherein the activation function selects SiLU.

3. The multi-scale adaptive lesion detection method based on breast ultrasound according to claim 2, characterized in that: In S3, the improved YOLOv12 model adopts a new convolution block category. The Yolov12 model uses a series of smaller kernels, which are expressed as: ,in: is the output feature, are input features, is the weight of the i-th convolution kernel, is the bias of the i-th convolution kernel.

4. The multi-scale adaptive lesion detection method based on breast ultrasound according to claim 3, characterized in that: The S3 specifically includes: S31: The first convolutional layer uses a 3×3 convolution kernel with a stride of 2 and a padding of 1 to extract basic features from the input image. S32: The output features are input to the second convolutional layer. This layer still uses the 3×3 convolution kernel, but the stride is changed to 1 and the padding is still 1. After the convolution operation, the size of the feature map remains unchanged, and the number of channels is doubled. S33: The output of the second convolutional layer then enters the first C3k2 module. The input feature map is divided into two parts. One part is directly passed to retain shallow features, and the other part is processed by standard Bottleneck to process deep features, and finally spliced ​​and fused. S34: The output feature map of the first C3k2 module flows into the third convolutional layer, which uses the filters of the first convolutional layer. The main function of the third convolutional layer is to further downsample, compress the size of the feature map, extract deeper and more abstract feature information, and expand the number of channels. S35: The output feature map of the third convolutional layer enters the second C3k2 module and the fourth convolutional layer. This layer follows the structure of the first C3k2 module and the second convolutional layer. S36: The output feature map of the fourth convolutional layer is then input to the first A2C2f module. The A2C2f module uses adaptive channels and spatial attention mechanisms to enhance the network's perception and improve the efficiency of capturing objects of different scales. S37: The output feature map of the first A2C2f module is sent to the fifth convolutional layer, which uses a 3×3 convolution kernel with a stride of 2 and a padding of 1. The fifth convolutional layer is used to further downsample and compress the feature map size, extracting deeper and more semantic feature information while increasing the number of channels. S38: The output feature map of the fifth convolutional layer is input to the second A2C2f module. This module follows the architecture of the first A2C2f module. After the feature map is divided into regions, a self-attention mechanism is applied to each region. Dynamic adjustments are made based on the correlation between regions to highlight key regions and weaken interference. The processed features of each region are fused, superimposed on the input feature map, and output through the ReLU activation function. The size of the output feature map of the second A2C2f module remains unchanged. The feature map is optimized through attention, and the features of key regions are further strengthened, thereby improving the accuracy of feature expression. S39: The second A2C2f module outputs the feature map and inputs it into the MSCA module, which is mainly composed of a multi-scale convolution module and an attention module. The multi-scale convolution module uses convolution kernels of different sizes to convolve the feature map, thereby capturing different scale features from fine-grained to coarse-grained, and integrating the features extracted by multi-scale convolution to generate attention weights; local information is aggregated through deep convolution, and then multi-branch deep convolution is used to capture multi-scale contextual information. Finally, 1×1 convolution is used to simulate the relationship between different channels in the feature to generate the weight of convolution attention.

5. The multi-scale adaptive lesion detection method based on breast ultrasound according to claim 4, characterized in that: In the S36, the A2C2f module uses two convolutional layers cv1 and cv2 to process the input features to obtain two feature maps with different numbers of channels. cv1 reduces the number of input channels by one half, while cv2 keeps the number of input channels unchanged. Each convolutional layer is composed of a Conv2d layer, a BatchNorm2d layer, and an activation function SiLU or Identity. In each ABlock module, an attention mechanism module AAttn and a multi-layer perceptron MLP module are included. First, the qkv layer in AAttn is used to generate query, key, and value vectors, and the attention weight is calculated using the Flash Attention mechanism to implement a spatial attention mechanism for the features and highlight the features of important spatial areas. At the same time, the pe convolution layer is used to perform position encoding to enhance the model's perception of position information. Then, the attention result is projected back to the original dimension through the proj convolution layer and residually connected with the input feature to achieve feature fusion and enhancement. Then, the fused features are further nonlinearly transformed through the MLP module to further extract and fuse the features. By stacking two ABlock modules, multiple attention calculations and feature fusion can be performed on features, thereby enhancing the model's ability to capture objects of different scales.

6. The multi-scale adaptive lesion detection method based on breast ultrasound according to claim 5, characterized in that: The S4 specifically includes: S41: The MSCA output feature map of the backbone module is upsampled and fused with the first A2C2f layer, and then passed to the third A2C2f layer. The structure of the A2C2f layer is the same as that of the first A2C2f layer. The output feature map is upsampled and then fused with the output features of the second A2C2f layer, and output to the fourth A2C2f layer. S42: After the feature input processed by the fourth A2C2f layer is embedded in the CA attention module, the multi-scale module is output to the detection module; S43: The features processed by the fourth A2C2f layer are also output to the sixth convolutional layer, and the features processed by the sixth convolutional layer are fused with the output of the third A2C2f layer; S44: After fusion, it is input into the fifth A2C2f layer, the output feature map is input into the CA attention module for processing, and the multi-scale feature map is output to the detection module; S45: The feature map output by the fifth A2C2f layer is simultaneously input into the seventh convolutional layer, and the output result is fused with the output of the MSCA module, and then output to the third C3k2 layer, whose output feature map flows into the CA attention module, and then the multi-scale module three is output to the detection module.

7. The multi-scale adaptive lesion detection method based on breast ultrasound according to claim 1, characterized in that: In S5, the detection head network of the YOLOv12 model is mainly responsible for performing target detection tasks such as cysts, fibroadenomas, suspicious tumor clusters, and calcification detection.

8. The multi-scale adaptive lesion detection method based on breast ultrasound according to claim 7, characterized in that: The S5 specifically includes: S51: Positioning Prediction: Use the anchor box mechanism to predict the bounding box. Predefine anchor boxes of different scales and aspect ratios. Then predict the offset and confidence of each anchor box to determine the location and size of the target. Through a combination of fully connected layers and convolutional layers, predict the coordinates of the bounding box for each feature point. S52: Classification prediction: Perform classification prediction on each feature point or anchor box, and output the probability distribution of whether the location belongs to the cyst, fibroadenoma, suspicious group tumor, or calcification category. Use the softmax activation function to convert the classification prediction output into a probability value to make the result more interpretable. S53: Loss calculation: The cross entropy loss function is used to measure the difference between the classification prediction result and the true label. The difference between the predicted bounding box and the true bounding box is calculated based on the mean square error and CIoU loss. The bounding box coordinates after non-maximum suppression, the corresponding category label and confidence score are output as the final detection result.

Images