A Method and System for Classifying Ultrasound Breast Images Based on Frequency Domain Self-Attention Mechanism

Abstract

This invention provides a method and system for classifying ultrasound breast images based on a frequency domain self-attention mechanism. The classification method includes: S1, acquiring an ultrasound breast image as input, and performing preliminary processing on the input ultrasound breast image through a stem module to extract low-level features; S2, inputting the low-level features into four backbone network blocks to further extract deep features; each backbone network block includes multiple ConvNeXt blocks, and each ConvNeXt block integrates an FDSA module for enhancing frequency domain feature representation; S3, performing global average pooling on the enhanced depth feature maps output by the four backbone network blocks to obtain pooled feature maps; inputting the pooled feature maps into a classification output module, which includes fully connected layers and a Softmax function for classifying the enhanced depth feature maps to achieve breast lesion detection; the classification method provided by this invention has high computational efficiency, high classification accuracy, low resource consumption, and can be flexibly adapted to multiple scenarios.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for classifying ultrasound breast images based on a frequency domain self-attention mechanism. Background Technology

[0002] Medical ultrasound imaging is widely used in the clinical diagnosis of breast diseases due to its advantages of being radiation-free, real-time, and cost-effective. However, ultrasound images are susceptible to speckle noise, which can degrade image quality and affect the accuracy of deep learning-based classification.

[0003] Existing technical solutions are mainly divided into two categories: First, traditional convolutional neural networks (such as ConvNeXtV2 and MobileNetV4), which can process large-scale data, but have problems such as high computational complexity, slow processing speed (difficult to meet the needs of real-time diagnosis), low classification accuracy of noisy ultrasound images, and high resource consumption (increasing hardware costs and energy consumption). Moreover, the model structure is fixed, the versatility is poor, and the adaptability is insufficient. Second, networks based on self-attention mechanisms (such as InternImage and SeaFormer), which can capture global dependencies, still need to improve their classification performance in complex ultrasound scenarios.

[0004] In addition, frequency domain analysis techniques, such as converting feature maps to the frequency domain using Fast Fourier Transform (FFT), can analyze frequency distribution characteristics, and multi-head self-attention mechanism (MHSA) can achieve interactive fusion of features from different frequency bands to suppress noise. However, existing technologies still have drawbacks such as low computational efficiency, high resource consumption, and poor scalability, making it difficult to meet the requirements of real-time performance, accuracy, and universality.

[0005] Therefore, there is an urgent need for an ultrasound breast image classification method and system based on the frequency domain self-attention mechanism, which has high computational efficiency, high classification accuracy, low resource consumption, and can be flexibly adapted to multiple scenarios. Summary of the Invention

[0006] To address the aforementioned shortcomings of the existing technology, the purpose of this invention is to provide a method and system for classifying ultrasound breast images based on a frequency domain self-attention mechanism, aiming to solve the technical problems of slow computation, low accuracy, high resource consumption, and poor scalability in traditional ultrasound breast image classification methods.

[0007] To achieve the above objectives, this invention provides a method for classifying ultrasound breast images based on a frequency domain self-attention mechanism, the steps of which include:

[0008] S1. Acquire an ultrasound breast image as input, and perform preliminary processing on the input ultrasound breast image through the stem module to extract low-level features of the ultrasound breast image;

[0009] S2. The low-level features are input into four backbone network blocks for further extraction to obtain deep features. Each backbone network block includes multiple ConvNeXt blocks, and each ConvNeXt block integrates a frequency-domain self-attention (FDSA) module to enhance the expression of frequency domain features.

[0010] S3. Perform global average pooling on the enhanced deep feature maps output from the four backbone network blocks to obtain pooled feature maps; input the pooled feature maps into the classification output module, which includes fully connected layers and a Softmax function, to classify the enhanced deep feature maps to complete the detection of breast lesions.

[0011] As a further improvement to the above scheme, step S0 is added before inputting the ultrasound breast images into the STEM module to preprocess the obtained ultrasound breast images to ensure data consistency and model training efficiency; the specific preprocessing steps include:

[0012] S01. Adjust the resolution of the original image to 512×512 pixels, and use bilinear interpolation or nearest neighbor interpolation to achieve scaling in order to unify the image size;

[0013] S02. Randomly crop a 256×256 pixel sub-region from the adjusted image to increase data diversity;

[0014] S03. Apply a horizontal flip operation to the cropped image to improve the model's ability to learn image symmetry.

[0015] S04. Normalize the processed image by dividing the pixel values ​​by 255 and mapping them to the [0,1] range to generate an image. As input data for the entire network, C represents the number of channels, and for grayscale images, C=1.

[0016] As a further improvement to the above scheme, in step S1, the stem module includes a convolutional layer with a kernel size of 4, an output channel number of 96, and a stride of 4, used to extract low-level features from the original ultrasound breast image. The low-level features include the edge, texture, and color information of the ultrasound breast image.

[0017] As a further improvement to the above scheme, in step S2, the processing steps for each ConvNeXt block include:

[0018] S211. Perform a 2x downsampling process on the feature map input to the current ConvNeXt block to obtain a downsampled feature map;

[0019] S212. The downsampled feature map is processed sequentially through depthwise convolution, layer normalization, 1×1 convolution, GELU activation function and global response normalization (GRN) operations.

[0020] S213. Convert the number of channels from C to 64 by performing a 1×1 convolution on the feature map after GRN operation to obtain the first intermediate feature map;

[0021] S214. Input the first intermediate feature map into the FDSA module;

[0022] S215. The feature map processed by the FDSA module is converted from 64 channels to C by 1×1 convolution to obtain the second intermediate feature map.

[0023] S216. Add the second intermediate feature map to the downsampled feature map to obtain the output feature map of the current ConvNeXt block.

[0024] As a further improvement to the above scheme, in step S2, the processing steps of the FDSA module include:

[0025] S221. Receive the first intermediate feature map and extract the intermediate layer feature map through the convolutional neural network feature extraction module;

[0026] S222. The intermediate layer feature map is transformed into a frequency domain using Fast Fourier Transform (FFT) to generate a spectrum map;

[0027] S223, Based on the ratio of low-frequency to high-frequency components in the aforementioned spectrum diagram The feature channels are sorted and grouped to form feature groups with multiple frequency bands;

[0028] S224. Feature groups from different frequency bands interact and fuse information through the Multi-Head Self-Attention (MHSA) module to obtain enhanced feature maps;

[0029] S225. Restore the frequency band information calculated by the MHSA module, and complete the frequency domain feature enhancement through batch normalization operation.

[0030] As a further improvement to the above scheme, in step S222, the method for generating the spectrum is as follows:

[0031] The intermediate layer feature map F is fed into the frequency domain transformation module. The frequency domain transformation module performs a fast Fourier transform on the intermediate layer feature map F of each channel, transforming it from the spatial domain to the frequency domain to generate a spectrum map, so as to extract information of different frequency components in the feature map.

[0032] Spectrum The specific generation process is shown in the following formula:

[0033] ;

[0034] Where h and w represent the height and width of the intermediate layer feature map, respectively, d represents the number of channels in the intermediate layer feature map; u and v represent the frequency coordinates in the spectrum graph, and i is the imaginary unit; This represents the value of a channel at position (x, y) in the original intermediate layer feature map; This represents the complex value of a certain channel at the frequency coordinate (u,v) in the frequency domain after transformation;

[0035] This transformation decomposes the feature map into multiple frequency components, facilitating subsequent feature grouping by frequency band.

[0036] As a further improvement to the above scheme, in step S223, the method for forming feature groups of multiple frequency bands is as follows:

[0037] In the spectrum diagram, a square region with side length r is defined as the low-frequency region, with the center point as the origin, and the remaining part is the high-frequency region; then the amplitude of the low-frequency region is calculated separately. and the amplitude in the high-frequency region Obtain the ratio of low-frequency to high-frequency components in channel i. ;

[0038] Then, the channels are sorted from largest to smallest based on the ratio of low-frequency to high-frequency components, and the feature maps are grouped according to the sorting results to obtain the grouped features G={g1,g2,…,gn}, where, Each g i This is called a single-group feature, where n represents the number of groups.

[0039] As a further improvement to the above scheme, the amplitude of the low-frequency region The calculation formula is as follows:

[0040] ;

[0041] The amplitude of the high-frequency region The calculation formula is as follows:

[0042] ;

[0043] in, The first of the spectrum diagrams row element, The first of the spectrum diagrams Column elements, Let be the i-th spectrum.

[0044] As a further improvement to the above scheme, in step S224, the method for obtaining the enhanced feature map is as follows:

[0045] The grouped feature group G is input into the self-attention calculation module and processed on a single feature-by-feature basis.

[0046] A multi-head self-attention module is used to calculate the correlation of features in different frequency bands, where the query Q=GW Q Key K=GW K Value V=GW V W Q W K W V The learnable parameter matrix;

[0047] Calculate attention output d k For query vector dimensions;

[0048] An enhanced feature map is obtained by concatenating and linearly transforming the outputs of multiple attention heads using a multi-head mechanism. .

[0049] As a further improvement to the above scheme, in step S3, when the classification output module classifies the enhanced deep feature map, the fully connected layer maps the feature map into a category vector, and then the output is transformed into a probability distribution of each category through the Softmax function to achieve the classification of benign and malignant breast lesions.

[0050] The classification results are represented as Where N is the number of samples and M is the number of categories, the classification process can be represented as:

[0051] ;

[0052] Where FC represents fully connected layer operations, Softmax This is the Softmax activation function.

[0053] Secondly, the present invention also provides an ultrasound breast image classification system based on a frequency domain self-attention mechanism, comprising:

[0054] The preprocessing module is used to preprocess ultrasound breast images;

[0055] The stem module is used to perform preliminary processing on the preprocessed image to extract low-level features;

[0056] The backbone network module consists of four backbone network blocks for extracting deep features. Each backbone network block includes multiple ConvNeXt blocks, and each ConvNeXt block integrates an FDSA module. The ConvNeXt block performs a 2x downsampling, depthwise convolution, layer normalization, 1×1 convolution, GELU activation function, and GRN operation on the input feature map to convert the number of channels to 64 before inputting it into the FDSA module. The FDSA module processes the data, restores the number of channels, and adds it to the downsampled output.

[0057] The FDSA module is used to receive intermediate feature maps, generate spectrum maps through fast Fourier transform, sort and group feature channels according to the ratio of low frequency to high frequency, perform frequency band interaction fusion through the MHSA module, restore and batch normalize to enhance features.

[0058] The global average pooling module is used to perform global average pooling on deep feature maps to obtain pooled feature maps.

[0059] The classification output module, including a fully connected layer and a Softmax function, is used to classify pooled feature maps to achieve breast lesion detection.

[0060] Because the present invention adopts the above technical solutions, the beneficial effects of this application are as follows:

[0061] This invention provides a frequency domain self-attention mechanism-based ultrasound breast image classification method. By introducing a frequency domain self-attention (FDSA) module, the traditional spatial domain attention calculation is transformed into frequency domain processing. Due to the computational sparsity of frequency domain transformations (such as FFT), only key frequency bands need to be calculated, avoiding the high complexity of full-image attention calculation in the spatial domain. This significantly reduces the amount of computation, reduces GPU memory usage, improves inference speed, and is more suitable for the real-time requirements of medical scenarios. The Stem module, combined with the ConvNeXt backbone, forms a multi-scale progressive feature extraction structure. First, low-level features (such as edges and textures) are extracted through a lightweight Stem module, and then the feature abstraction is deepened step by step through ConvNeXt blocks, avoiding the loss of shallow information. The FDSA module transforms the features to the frequency domain through FFT, utilizing the natural suppression characteristics of the spectrum for high-frequency noise (such as ultrasound artifacts). Combined with frequency band grouping and sorting, it highlights the frequency domain components related to lesions (such as high-frequency signals of microcalcifications), enhancing the model's sensitivity to key features. Multi-band feature interaction fusion (through the MHSA mechanism) enables the model to capture local details (such as tumor boundaries) and global context (such as gland distribution) simultaneously, significantly improving classification accuracy compared to traditional pure spatial attention or CNN.

[0062] Furthermore, the backbone network adopts a modular design, that is, four stacked ConvNeXt blocks integrate FDSA modules, and the model depth can be flexibly adjusted by adding or removing blocks to balance accuracy and speed; FDSA's frequency domain processing and frequency band grouping strategy are not sensitive to the input image resolution, and can maintain stable performance on images acquired by different ultrasound devices (such as different probe frequencies), which is convenient for clinical deployment and promotion.

[0063] Furthermore, ultrasound images are susceptible to speckle noise, while frequency domain transformation naturally separates noise frequency bands (usually high-frequency disordered components). FDSA suppresses irrelevant frequency domain signals through frequency band filtering and retains effective spatial information by combining the local convolutional structure of ConvNeXt. The final output feature map has lower noise sensitivity, reducing the rate of missed diagnoses / false diagnoses. This invention, through the synergistic design of frequency domain feature enhancement and progressive convolution extraction, solves the problems of slow computation, high resource consumption, and poor generalization of traditional methods while ensuring accuracy, directly improving the clinical applicability of ultrasound breast lesion detection. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0065] Figure 1 This is a schematic diagram of the framework of an ultrasound breast image classification method based on a frequency domain self-attention mechanism disclosed in this invention, wherein... Figure 1 Figure (a) in the diagram shows a schematic of the overall network framework. Figure 1 Figure (b) in the diagram shows a schematic diagram of the ConvNeXt module structure. Figure 1 Figure (c) in the figure shows a schematic diagram of the FDSA module structure;

[0066] Figure 2 This is a schematic diagram of obtaining the spectrum map of each channel's feature map through Fast Fourier Transform (FFT) as disclosed in this invention;

[0067] Figure 3 This is a schematic diagram of the feature map after sorting and grouping the spectrum map according to the ratio, as disclosed in this invention;

[0068] Figure 4 This is a schematic diagram of self-attention calculation for different frequency groups disclosed in this invention.

[0069] The realization of the objective, functional characteristics and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] It should be noted that the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0072] Example 1

[0073] See Figures 1-4 This invention relates to the field of medical image processing technology, specifically to a classification method for ultrasound breast images based on a frequency domain self-attention mechanism. This method is suitable for the detection and classification of lesions in ultrasound breast images, aiming to address the shortcomings of traditional methods in terms of computational efficiency, feature representation accuracy, resource consumption, and model scalability. Specifically, the classification method includes the following steps:

[0074] S1. Ultrasound breast image input and low-level feature extraction:

[0075] Input data: Raw ultrasound breast images are exported directly from a medical image storage system (such as PACS) or ultrasound equipment. The format is usually DICOM or PNG, with a resolution ranging from 512×512 to 1024×1024 pixels and 8-bit grayscale images with pixel values ​​of 0-255.

[0076] Stem Module Structure and Processing: The Stem module is a lightweight front-end network used for initial downsampling and low-level feature extraction of the original image. In this embodiment, the Stem module contains a single convolutional layer with the following parameters:

[0077] kernel size: 4×4;

[0078] Number of output channels: 96;

[0079] Step size: 4

[0080] The input image size is downsampled by 1 / 4, i.e., 256×256 input is downsampled to 64×64. This convolutional layer uses a large convolutional kernel and a high stride design to quickly extract low-level features such as basic texture (e.g., regular / irregular arrangement of glandular tissue), edges (e.g., the boundary between the mass and surrounding tissue), and color contrast (e.g., grayscale difference between benign and malignant regions) in the first layer, and outputs a feature map with size H×W×96 (H / W is the height / width after downsampling, and the number of channels is 96).

[0081] S2, Deep Feature Extraction (Integration of FDSA Module with Backbone Network Block and ConvNeXt Block):

[0082] Backbone network block structure: The low-level features output by the Stem module are input into 4 stacked backbone network blocks. Each backbone network block includes N ConvNeXt blocks. Each ConvNeXt block deepens the feature representation step by step through depthwise convolution and frequency domain self-attention (FDSA) modules.

[0083] S21. The processing steps for a single ConvNeXt block are as follows:

[0084] S211. The feature map input to the current ConvNeXt block (from the output of the previous block or the Stem module) is downsampled by a convolution or pooling operation with a stride of 2 (in this embodiment, a depthwise convolution with a stride of 2 is preferred) to reduce the spatial size to 1 / 2 (e.g., 256×256→128×128) to obtain the downsampled feature map.

[0085] S212. Pass the downsampled feature map through the following sub-modules in sequence:

[0086] Depthwise Convolution: The kernel size is 3×3 and the stride is 1. It performs independent convolution on each input channel to extract local spatial features.

[0087] Layer Normalization (LayerNorm): Normalizes the channel dimension of the depthwise convolution output to stabilize the training process;

[0088] 1×1 convolution: Adjusts the number of channels and fuses local features;

[0089] GELU activation function: Introduces a nonlinear transformation to replace the traditional ReLU, resulting in smoother gradient characteristics;

[0090] Global Response Normalization (GRN): Normalizes the responses between channels to enhance feature discrimination.

[0091] S213. The feature map processed by GRN is converted from the original value C to a fixed value of 64 through 1×1 convolution to obtain the first intermediate feature map in order to reduce the computational load of the subsequent FDSA module.

[0092] S214. Frequency domain self-attention processing: Input the first intermediate feature map (64 channels) into the FDSA module (see steps S221-S225 below for details) to extract frequency domain enhancement features.

[0093] S215. The feature map (64 channels) output by the FDSA module is processed by 1×1 convolution to restore the number of channels to the original value C, and a second intermediate feature map is obtained to match the subsequent residual connection.

[0094] S216. Add the second intermediate feature map to the downsampled feature map (original number of channels C) output in step S211, and fuse the low-level spatial information with the high-level semantic information enhanced in the frequency domain to obtain the final output feature map of the current ConvNeXt block.

[0095] Processing flow of S22 and FDSA modules:

[0096] The FDSA module is integrated into each ConvNeXt block and is used to transform spatial domain features to the frequency domain and enhance key frequency band information. The specific steps are as follows:

[0097] S221, Intermediate layer feature extraction: Receive the first intermediate feature map, and extract a more compact intermediate layer feature map through a lightweight convolutional neural network, such as a 1-layer 3×3 convolution + ReLU, to preserve the core frequency domain correlation information;

[0098] S222: Frequency domain transformation: The intermediate layer feature map is transformed to the frequency domain through Fast Fourier Transform (FFT) to generate a complex form spectrum map, which includes real and imaginary parts and reflects the energy distribution of different frequency components;

[0099] S223: Frequency band grouping: Based on the ratio of low-frequency to high-frequency components in the spectrum, the characteristic channels are sorted (e.g., by frequency domain energy from high to low) and grouped (e.g., divided into 4 frequency band groups: low frequency group, mid-low frequency group, mid-high frequency group, and high frequency group), each group containing several continuous or energy-filtered channels.

[0100] S224: Frequency domain feature interaction: Feature groups from different frequency bands interact through the multi-head self-attention (MHSA) module: Each frequency band group acts as an independent "head", calculating the attention weights between features within the group (through query-key-value projection) to achieve the fusion of cross-frequency band information, such as the correlation between high-frequency details and low-frequency structures, and enhancing the sensitivity to lesion-related frequency domain components;

[0101] S225, Frequency Domain-Spatial Restoration and Normalization: The frequency band information processed by the MHSA module is restored to the original frequency domain layout. The correspondence of the feature space is maintained through inverse operations (such as the equivalent processing of inverse FFT). Finally, the statistical properties of the frequency domain features are stabilized through batch normalization (BatchNorm), thus completing the frequency domain enhancement.

[0102] S3, Classification Output (Global Pooling and Softmax Classification):

[0103] Global Average Pooling (GAP): The enhanced depth feature maps output from the four backbone network blocks (with the size progressively downsampled to 1 / 8 to 1 / 16 of the original image, e.g., 128×128 → 16×16 or smaller, with the number of channels C = 192 / 384, etc.) are input into the global average pooling layer. The average value is taken for all spatial locations of each channel, and the feature map is compressed into a 1×1×C pooled feature map to preserve global semantic information and remove redundant spatial details.

[0104] Classification output module: The pooled feature map is passed through a fully connected layer (the output dimension is the preset number of categories, such as 2 for benign / malignant biclass classification, or the corresponding number of subtypes for multiclass classification), which maps the number of channels C to the category probability dimension; then the Softmax function is used to calculate the probability distribution of each category, and the category with the highest probability value is the type of breast lesion predicted by the model, such as "benign" or "malignant", and the probability of each category can be output to assist doctors in decision-making.

[0105] This invention introduces a Frequency Domain Self-Attention (FDSA) module to transform traditional spatial domain attention computation into frequency domain processing. Due to the computational sparsity of frequency domain transformations (such as FFT), only key frequency bands need to be calculated, avoiding the high complexity of full-image attention computation in the spatial domain. This significantly reduces computational load, GPU memory usage, and inference speed, making it more suitable for the real-time requirements of medical scenarios. The Stem module, combined with the ConvNeXt backbone, forms a multi-scale progressive feature extraction structure. Low-level features (such as edges and textures) are first extracted using a lightweight Stem module, and then the ConvNeXt blocks progressively deepen the feature abstraction, avoiding the loss of shallow information. The FDSA module transforms features to the frequency domain using FFT, leveraging the natural suppression of high-frequency noise (such as ultrasound artifacts) by the spectrum. Combined with frequency band grouping and sorting, it highlights lesion-related frequency domain components (such as high-frequency signals of microcalcifications), enhancing the model's sensitivity to key features. Multi-band feature interaction fusion (through the MHSA mechanism) enables the model to capture local details (such as tumor boundaries) and global context (such as gland distribution) simultaneously, significantly improving classification accuracy compared to traditional pure spatial attention or CNN.

[0106] Furthermore, the backbone network adopts a modular design, that is, four stacked ConvNeXt blocks integrate FDSA modules, and the model depth can be flexibly adjusted by adding or removing blocks to balance accuracy and speed; FDSA's frequency domain processing and frequency band grouping strategy are not sensitive to the input image resolution, and can maintain stable performance on images acquired by different ultrasound devices (such as different probe frequencies), which is convenient for clinical deployment and promotion.

[0107] Furthermore, ultrasound images are susceptible to speckle noise, while frequency domain transformation naturally separates noise frequency bands (usually high-frequency disordered components). FDSA suppresses irrelevant frequency domain signals through frequency band filtering and retains effective spatial information by combining the local convolutional structure of ConvNeXt. The final output feature map has lower noise sensitivity, reducing the rate of missed diagnoses / false diagnoses. This invention, through the synergistic design of frequency domain feature enhancement and progressive convolution extraction, solves the problems of slow computation, high resource consumption, and poor generalization of traditional methods while ensuring accuracy, directly improving the clinical applicability of ultrasound breast lesion detection.

[0108] In a preferred embodiment, before inputting the ultrasound breast images into the STEM module, a step S0 is included to preprocess the obtained ultrasound breast images to ensure data consistency and efficient model training; the specific preprocessing steps include:

[0109] S01. First, the resolution of all input images is uniformly adjusted to 512×512 pixels. This process ensures that images from different sources and of different sizes have a consistent spatial resolution when entering subsequent processing steps, thus avoiding performance degradation due to differences in image size. Assuming the original image resolution is H×W, the adjusted image resolution is 512×512 pixels. Through interpolation algorithms, such as bilinear interpolation or nearest neighbor interpolation, image scaling can be performed to unify the image size.

[0110] S02. To increase the diversity of the training set and prevent overfitting, after adjusting the image resolution, each image is randomly cropped. Specifically, a smaller region, such as a 256×256 pixel sub-region, is randomly selected from each 512×512 pixel image and used as the final input image to the model. This method not only increases the diversity of training samples but also improves the model's ability to learn features from different locations. Assume the cropped image is represented as I. crop Then its size remains 256×256 pixels, but the content is a sub-region randomly selected from the original image.

[0111] S03. To further enhance the diversity of the dataset, after random cropping, a horizontal flip operation is performed on some images; that is, a mirror transformation is performed along the vertical axis of the image. This step helps the model learn the symmetry characteristics of objects in the image and reduces bias caused by specific directional features. Assume the horizontally flipped image is represented as I. flip If the size remains 256×256 pixels, the content is a horizontally flipped version of the original image.

[0112] S04. Finally, the image processed above is normalized; specifically, each pixel value is divided by 255, mapping the pixel value range from [0, 255] to [0, 1]. This operation eliminates the grayscale differences between different images, making the images have similar scales, thereby accelerating the convergence speed during training.

[0113] Let the original image data be I. raw , where each pixel value I x,y This represents the gray value of the image at coordinates (x, y). The normalized image I′ can be represented as:

[0114] ;

[0115] in, This represents the normalized pixel value.

[0116] Normalized image As input data for the entire network, C represents the number of channels, and for grayscale images C=1; this standardization operation not only eliminates the difference in dimensions, but also significantly improves the convergence speed and stability of the model during training.

[0117] In a preferred embodiment, in step S222, the generation of the spectrum map involves processing the intermediate layer feature map through a frequency domain transformation module. The specific operation flow is as follows:

[0118] The intermediate layer feature map F output in step S221 is used as input data. It is a three-dimensional tensor, whose dimensions are represented as h×w×d, where h represents the height of the feature map, w represents the width of the feature map, and d represents the number of channels of the feature map.

[0119] The frequency domain transformation module independently performs Fast Fourier Transform (FFT) processing on each channel of the intermediate layer feature map F. Specifically, for each channel, the two-dimensional spatial domain feature map of that channel, with dimensions h×w, is transformed to the frequency domain using Fast Fourier Transform to generate a spectrum map, as shown below. Figure 2 As shown, this operation can effectively extract information from different frequency components in the feature map, thus providing a basis for subsequent frequency band grouping.

[0120] After FFT transformation, the spatial domain feature map of each channel is converted into a corresponding spectrogram, ultimately generating a complete set of spectrograms. The set of spectrograms contains d independent spectrograms, each corresponding to one channel of the original feature map, and its frequency coordinates are represented by u and v, where u and v represent the horizontal and vertical frequency coordinates in the spectrogram, respectively.

[0121] Spectrum The generation process is based on the Fast Fourier Transform, and its mathematical expression is as follows:

[0122] ;

[0123] Where h and w represent the height and width of the intermediate layer feature map, respectively, d represents the number of channels in the intermediate layer feature map; u and v represent the frequency coordinates in the spectrum graph, and i is the imaginary unit; This represents the value of a channel at position (x, y) in the original intermediate layer feature map; This represents the complex value of a certain channel at the frequency coordinate (u,v) in the frequency domain after transformation.

[0124] This transformation decomposes the feature map into multiple frequency components, facilitating subsequent feature grouping by frequency band.

[0125] In a preferred embodiment, the method for forming the feature group of multiple frequency bands in step S223 is as follows:

[0126] Step 1: Spectrum Map Region Division: Using the center point of the spectrum map as the origin coordinate (0,0), define a square region with side length r as the low-frequency region. This square region covers the components with lower absolute frequency values ​​in the spectrum map. The remaining part of the spectrum map excluding the square region is defined as the high-frequency region. Here, r is a preset positive integer parameter, determined according to the actual spectrum map size and characteristic distribution requirements. It typically ranges from 1 / 4 to 1 / 2 of the total side length of the spectrum map to ensure that the low-frequency region covers the main low-frequency components.

[0127] Step 2, Regional Amplitude Calculation: For each channel i, calculate the amplitude in the low-frequency region. and the amplitude in the high-frequency region Specifically,

[0128] The amplitude of the low-frequency region The calculation formula is as follows:

[0129] ;

[0130] The amplitude of the high-frequency region The calculation formula is as follows:

[0131] ;

[0132] in, The first of the spectrum diagrams row element, The first of the spectrum diagrams Column elements, This is the i-th spectrum;

[0133] Step 3: Ratio Calculation: Based on the above calculation results, for each channel i, obtain the ratio of low-frequency to high-frequency components of channel i. This ratio reflects the frequency characteristics of channel i. i The larger the value, the higher the proportion of low-frequency components in the channel, indicating that the feature channel represents more of the low-frequency characteristics of the original image; conversely, the smaller the value, the higher the proportion of low-frequency components in the channel, indicating that the channel represents more of the high-frequency characteristics of the original image.

[0134] Step 4, Channel Sort: See Figure 3 All d channels are sorted in descending order according to the calculated ratio of low-frequency to high-frequency components, forming a channel sequence, with channels of higher ratios listed first and channels of lower ratios listed last.

[0135] Step 5: Feature Grouping: Based on the sorting results, divide the sorted channels into n groups, where n is the preset number of groups, usually an integer from 2 to 8, determined according to actual application requirements, forming a grouped feature set G={g1,g2,…,gn}, where… Each g i This is called a single-group feature, which includes several consecutively ordered channels, where n represents the number of groups. Different single-group features actually correspond to different frequency bands, while different channel features within the same single-group feature represent similar frequency representations. As the group number i increases, the single-group feature g... i The higher the corresponding frequency, the better. When grouping, channels can be assigned to each group in the order of sorting. For example, when n=4, the sorted channel sequence can be divided into 4 groups, each containing d / 4 channels, or the number of channels in each group can be adjusted appropriately according to the total number of channels. Other reasonable grouping strategies can also be adopted, such as dividing according to a proportional threshold or dynamically adjusting the number of channels in each group to ensure that each group has relatively consistent frequency domain characteristics.

[0136] In a preferred embodiment, the method for obtaining the enhanced feature map in step S224 is as follows:

[0137] Step 1: Input Data Preparation: The feature set G={g1,g2,…,gn} obtained after grouping in step S223 is input into the self-attention calculation module, where each single feature g iThis represents a set of channels with similar frequency domain characteristics. In this embodiment, the self-attention calculation module processes each frequency band feature group independently, on a single feature basis.

[0138] Step 2, Query-Key-Value Projection: For each single feature g i Through the learnable parameter matrix W Q W K W V Calculate the query vector Q, key vector K, and value vector V separately, as follows:

[0139] Query Q=g i ×W Q ;

[0140] Key K=g i ×W K ;

[0141] Value V=g i ×W V ;

[0142] Among them, W Q W K W V A learnable parameter matrix used to map input features to query, key, and value spaces; g i This represents the single feature currently being processed, whose dimension matches the number of input feature channels.

[0143] Step 3, Attention Score Calculation: Based on the query Q, key K, and value V, calculate the attention score. Specifically, the correlation between features is measured by the dot product of the query vector and the key vector. The calculation formula is as follows:

[0144] ;

[0145] Where d k For query vector dimensions.

[0146] Step 4: Attention weight generation: Apply the softmax function to the attention score to convert it into normalized attention weights, which are used to represent the degree of correlation between different feature locations.

[0147] Step 5, Weighted Summation: Multiply the attention weights by the value vector V and sum them to obtain the output feature at each position. The output feature integrates information from relevant positions in the input features.

[0148] Step 6, Multi-head mechanism processing: Set up multiple independent attention heads (usually 4 to 8, the specific number can be adjusted according to actual needs), each attention head uses a different learnable parameter matrix W. Q W K WV The correlation between features is learned from different subspaces; each attention head independently executes steps 2-5 above to obtain multiple independent attention outputs.

[0149] Step 7, Output Concatenation and Linear Transformation: The multiple attention outputs obtained through the multi-head mechanism are concatenated to restore the original feature dimensions; then, a linear transformation layer is used to integrate the concatenated features to obtain the final single-set enhanced features.

[0150] Step 8, Group Feature Enhancement: For each individual feature g in the group set G, i Perform steps 2-7 above to obtain the enhanced features for each group; combine all the enhanced single-group features in the original grouping order to form a complete enhanced feature map. .

[0151] Through the learnable parameter matrix W Q W K W V Achieving flexible mapping in the feature space allows the model to adaptively focus on other locational information related to the current frequency band features. The multi-head mechanism captures complex relationships between features from different representation subspaces through multiple parallel attention heads, enhancing the model's ability to model multi-dimensional correlation patterns in frequency domain features. The multiple attention outputs after multi-head processing are integrated through concatenation and linear transformation, preserving the specialization of each attention head while achieving a unified representation of global information through a linear layer. The resulting enhanced feature map, while retaining the original frequency domain features, further highlights key frequency domain information related to breast lesions, improving the targeting and effectiveness of feature expression. This processing, based on feature grouping after frequency domain transformation, specifically enhances feature interactions within a specific frequency range, avoiding the complexity of full-image computation inherent in traditional spatial attention mechanisms. The computational objective is more clearly defined, aligning with the medical characteristic that lesion features in ultrasound breast images are typically related to specific frequency domain components.

[0152] In a preferred embodiment, in step S3, the classification output module performs mapping and probability transformation on the enhanced depth feature map to classify the benign and malignant aspects of the ultrasound breast image. The specific operation process is as follows:

[0153] Input data preparation: The enhanced depth feature maps output by the four backbone network blocks are subjected to global average pooling to obtain pooled feature maps. The pooled feature maps obtained after global average pooling are then input into the classification output module. The feature maps have been processed step by step by the backbone network blocks and the FDSA module, and contain the global semantic information of the image and the enhanced frequency domain features.

[0154] Fully Connected Layer Mapping: The pooling feature map undergoes dimensionality transformation through a fully connected layer (FC), mapping the number of input feature channels C to the dimension corresponding to the preset number of categories M (e.g., M=2 in a binary classification task, corresponding to benign and malignant categories; M is the number of specific subtype categories in a multi-class classification task). The mathematical expression of the fully connected layer is:

[0155] Z = FC(X);

[0156] Where X represents the input pooling feature map, Z represents the class vector output by the fully connected layer, and FC represents the linear transformation operation of the fully connected layer.

[0157] Probability distribution transformation: The class vector Z output by the fully connected layer is transformed by the Softmax activation function, which maps the original output value to the probability distribution of each class.

[0158] The classification results are represented as a matrix, showing the probability distribution of each category in the final output. Its dimensions are N×M, where N is the number of samples and M is the number of categories. The classification process can be represented as:

[0159] ;

[0160] Where FC represents fully connected layer operations, Softmax This is the Softmax activation function.

[0161] For each sample, the category with the highest probability value is the type of breast lesion predicted by the model, such as "benign" or "malignant". At the same time, the specific probability value of each category can provide doctors with auxiliary decision-making information.

[0162] To further illustrate the inventive concept of this invention, in this embodiment, the datasets selected are the POCUS and BLU ultrasound image datasets. The hardware platform is an AMD 5950X CPU@3.4GHz and a single Nvidia RTX 3090 GPU. The software environment uses the PyTorch framework, the optimizer is Adam, the learning rate is 0.001, the weight decay is 0.0001, and the batch sizes are 64 (POCUS) and 128 (BLU), respectively.

[0163] First, let's take the BLU dataset as an example for a detailed explanation. The BLU dataset contains 25,272 breast ultrasound images for a binary classification task (malignant vs. benign). All images were resized to 512×512 pixels and underwent random cropping, horizontal flipping, and normalization (pixel values ​​divided by 255) to ensure data consistency and diversity. Then, a convolutional neural network (STEM) was used to perform initial low-level feature extraction on the input images. Next, ConvNeXt was used as the backbone of the convolutional neural network to extract intermediate layer feature maps from the low-level features. Subsequently, a Fast Fourier Transform (FFT) was applied to transform the feature maps to the frequency domain, generating a spectrogram. The feature channels were then sorted and grouped according to frequency distribution, forming feature groups for different frequency bands. A Multi-Head Self-Attention (MHSA) mechanism was used to achieve interaction and fusion between features from different frequency bands, generating enhanced feature maps. The information of each frequency band after MHSA calculation was restored, and batch normalization was used to enhance the expression of frequency domain features. Finally, global average pooling, fully connected layers, and the Softmax function were used for classification output. Experimental results show that ConvNeXt (denoted as ConvNeXt_FA) integrating the FDSA module performs excellently on the BLU dataset. As shown in Table 1, its average F1-score reaches 0.8060, which is significantly better than other models such as ConvNeXt V2 and MobileNet V4.

[0164]

[0165] To further illustrate the effectiveness of this invention, the POCUS dataset was used for lung cancer ultrasound image classification. The POCUS dataset contains 2116 lung ultrasound images for a three-class classification task (COVID-19, bacterial pneumonia, and healthy controls). Similarly, all images underwent the same preprocessing steps, and feature maps were extracted using stem and ConvNeXt. Then, FFT was applied to generate spectrograms and feature grouping was performed, followed by feature fusion using MHSA. The frequency band information after MHSA calculation was restored, and the frequency domain feature representation was further optimized through batch normalization. Finally, classification was completed using global average pooling, fully connected layers, and the Softmax function. Despite the small size of the POCUS dataset and the complexity of the task, ConvNeXt_FA performed excellently, as shown in Table 2, with an F1-score of 0.9428 and a standard deviation of 0.0435, higher than other models (such as ConvNeXt V2's 0.9269 and MobileNet V4's 0.0382). This demonstrates that the FDSA module is not only suitable for large-scale datasets, but also performs well when dealing with complex small-scale datasets.

[0166]

[0167] Validation on the POCUS and BLU datasets demonstrates that the frequency-domain self-attention-based ultrasound breast image classification method proposed in this invention exhibits superior performance in multiple ultrasound image classification tasks, maintaining high accuracy and stability even when dealing with noisy images and small sample sizes. The experimental results fully validate the advanced nature and practicality of this method, providing strong support for medical ultrasound image analysis.

[0168] Example 2

[0169] This invention also provides an ultrasound breast image classification system based on a frequency domain self-attention mechanism, comprising:

[0170] The preprocessing module, as the system's front-end input processing unit, performs basic preprocessing operations on the raw ultrasound breast images. Specific preprocessing operations include, but are not limited to: adjusting the input image to 512×512 pixels, and increasing data diversity and consistency through random cropping, horizontal flipping, and normalization.

[0171] The stem module is a lightweight front-end feature extraction unit used to perform preliminary processing on preprocessed ultrasound breast images, extracting low-level features such as edges, textures, and basic color contrast information.

[0172] The backbone network module, the core feature extraction unit, is used to extract deep features. It consists of four stacked backbone network blocks. Each backbone network block includes multiple ConvNeXt blocks. Each ConvNeXt block deepens the feature representation through step-by-step processing and integrates an FDSA module to achieve frequency domain feature enhancement. The ConvNeXt block performs a 2x downsampling, depthwise convolution, layer normalization, 1×1 convolution, GELU activation function, and GRN operation on the input feature map to convert the number of channels to 64 before inputting it into the FDSA module. The FDSA module processes the data to restore the number of channels and adds it to the downsampled output.

[0173] The FDSA module, integrated in each ConvNeXt block, is used to perform frequency domain transformation and frequency band feature interaction fusion on the input intermediate feature map. It generates a spectrum map through fast Fourier transform, sorts and groups feature channels according to the ratio of low frequency to high frequency, performs frequency band interaction fusion through the MHSA module, restores and batch normalizes to enhance features.

[0174] The global average pooling module is a feature compression unit used to aggregate global information from the deep feature maps output by the backbone network modules. This module takes the average value of all spatial locations in each channel and compresses the feature map into a 1×1×C pooled feature map to retain global semantic information and remove redundant spatial details.

[0175] The classification output module, as the final decision-making unit, is used to perform category mapping and probability transformation on the pooled feature map to achieve breast lesion detection. The classification output module includes a fully connected layer and a Softmax function. The pooled feature map is mapped to the number of channels C as a preset number of categories M through the fully connected layer (e.g., M=2 in binary classification tasks, corresponding to benign and malignant; M is the number of specific subtypes in multi-class tasks), generating a category vector. Then, the Softmax function is used to convert the category vector into the probability distribution of each category. The category with the highest probability value is the type of breast lesion predicted by the model, and the probability of each category is output to assist doctors in decision-making.

[0176] In this invention, the stem module extracts low-level features, such as edges and textures. The backbone network module deepens feature abstraction step by step through depthwise convolution of ConvNeXt blocks and conventional operations, avoiding the loss of shallow information. The FDSA module further highlights key frequency domain components related to lesions through frequency domain transformation and frequency band grouping, enhancing the model's sensitivity to key features. The frequency domain self-attention mechanism achieves interactive fusion between features of different frequency bands through multi-head self-attention (MHSA). Combined with the spatial convolution characteristics of ConvNeXt blocks, it captures both local details (such as tumor boundaries) and global context (such as glandular distribution), improving the accuracy of lesion detection. This invention transforms traditional spatial domain attention calculation into frequency domain processing through the FDSA module. By utilizing the sparsity of Fast Fourier Transform (FFT) (which only requires calculation on key frequency bands), it reduces the high complexity of full-image attention calculation, lowers GPU memory usage and computation time, and improves model inference speed while maintaining accuracy, making it more suitable for the real-time requirements of medical scenarios.

[0177] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct or indirect applications in other related technical fields, are within the patent protection scope of the present invention.

Claims

1. A method for classifying ultrasound breast images based on a frequency domain self-attention mechanism, characterized in that, The steps and methods include: S1. Acquire an ultrasound breast image as input, and perform preliminary processing on the input ultrasound breast image through the stem module to extract low-level features of the ultrasound breast image; S2. The low-level features are input into four backbone network blocks for further extraction to obtain deep features; each backbone network block includes multiple ConvNeXt blocks, and each ConvNeXt block integrates an FDSA module for enhancing frequency domain feature representation; the processing steps for each ConvNeXt block include: S211. Perform a 2x downsampling process on the feature map input to the current ConvNeXt block to obtain a downsampled feature map; S212. The downsampled feature map is processed sequentially through depthwise convolution, layer normalization, 1×1 convolution, GELU activation function and global response normalization (GRN) operations. S213. Convert the number of channels from C to 64 by performing a 1×1 convolution on the feature map after GRN operation to obtain the first intermediate feature map; S214. Input the first intermediate feature map into the FDSA module; S215. The feature map processed by the FDSA module is converted from 64 channels to C by 1×1 convolution to obtain the second intermediate feature map. S216. Add the second intermediate feature map to the downsampled feature map to obtain the output feature map of the current ConvNeXt block; The processing steps of the FDSA module include: S221. Receive the first intermediate feature map and extract the intermediate layer feature map through a convolutional neural network; S222. The intermediate layer feature map is converted to the frequency domain using a fast Fourier transform to generate a spectrum map; S223, Based on the ratio of low-frequency to high-frequency components in the aforementioned spectrum diagram The feature channels are sorted and grouped to form feature groups with multiple frequency bands; S224. Feature groups from different frequency bands interact and fuse information through a multi-head self-attention module to obtain enhanced feature maps; S225. Restore the frequency band information calculated by the MHSA module, and complete the frequency domain feature enhancement through batch normalization operation. S3. Perform global average pooling on the enhanced depth feature maps output by the four backbone network blocks to obtain pooled feature maps; input the pooled feature maps into the classification output module, which includes fully connected layers and a Softmax function, to classify the enhanced depth feature maps and realize breast lesion detection.

2. The ultrasound breast image classification method based on frequency domain self-attention mechanism according to claim 1, characterized in that, Before inputting the ultrasound breast image into the STEM module, step S0 is included, which preprocesses the obtained ultrasound breast image. The specific preprocessing steps and methods include: S01. Use bilinear interpolation or nearest neighbor interpolation to adjust the resolution of the original image to 512×512 pixels to unify the image size; S02. Randomly crop a 256×256 pixel sub-region from the adjusted image to increase data diversity; S03. Apply a horizontal flip operation to the cropped image to improve the model's ability to learn image symmetry. S04. Normalize the processed image by dividing the pixel values ​​by 255 and mapping them to the [0,1] range to generate an image. As network input, where C is the number of channels, and for grayscale images C=1.

3. A method for classifying ultrasound breast images based on a frequency domain self-attention mechanism according to claim 1 or 2, characterized in that, In step S1, the stem module includes a convolutional layer with a kernel size of 4, an output channel number of 96, and a stride of 4, which is used to extract the edge, texture, and color information of the ultrasound breast image as low-level features.

4. The ultrasound breast image classification method based on frequency domain self-attention mechanism according to claim 1, characterized in that, In step S222, the method for generating the spectrum is as follows: The intermediate layer feature map F is fed into the frequency domain transformation module, and a fast Fourier transform is performed on each channel to transform it from the spatial domain to the frequency domain, generating a spectrum map to extract information of different frequency components in the feature map. Spectrum The specific generation process is shown in the following formula: ； Where h and w represent the height and width of the intermediate layer feature map, respectively, d represents the number of channels in the intermediate layer feature map, u and v represent the frequency coordinates in the spectrum, and i is the imaginary unit; This represents the value of a channel at position (x, y) in the original intermediate layer feature map; This represents the complex value of a certain channel at the frequency coordinate (u,v) in the frequency domain after transformation; Through this transformation, the feature map is decomposed into multiple frequency components to facilitate subsequent grouping.

5. The ultrasound breast image classification method based on frequency domain self-attention mechanism according to claim 1, characterized in that, In step S223, the method for forming a feature group of multiple frequency bands is as follows: In the spectrum diagram, a square region with side length r is defined as the low-frequency region, with the center point as the origin, and the remaining part is the high-frequency region; then the amplitude of the low-frequency region is calculated separately. and the amplitude in the high-frequency region Obtain the ratio of low-frequency to high-frequency components in channel i. ; Then, the channels are sorted from largest to smallest based on their ratios, and the feature maps are grouped according to the sorting results to obtain the grouped features G={g1,g2,…,gn}, where, Each g i This is called a single-group feature, where n represents the number of groups.

6. The ultrasound breast image classification method based on frequency domain self-attention mechanism according to claim 1, characterized in that, In step S224, the method for obtaining the enhanced feature map is as follows: The grouped feature group G is input into the self-attention calculation module and processed on a single feature-by-feature basis. A multi-head self-attention module is used to calculate the correlation of features in different frequency bands, where the query Q=GW Q Key K=GW K Value V=GW V W Q W K W V The learnable parameter matrix; Calculate attention output d k For query vector dimensions; An enhanced feature map is obtained by concatenating and linearly transforming the outputs of multiple attention heads using a multi-head mechanism. .

7. A method for classifying ultrasound breast images based on a frequency domain self-attention mechanism according to claim 1 or 2, characterized in that, In step S3, when the classification output module classifies the enhanced deep feature map, the fully connected layer maps the feature map into a category vector, and then the output is transformed into a probability distribution of each category through the Softmax function to achieve the classification of benign and malignant breast lesions. The classification results are represented as Where N is the number of samples and M is the number of categories, the classification process can be represented as: ; Where FC represents fully connected layer operations, Softmax This is the Softmax activation function.

8. A breast ultrasound image classification system based on a frequency domain self-attention mechanism, characterized in that, include: The preprocessing module is used to preprocess ultrasound breast images; The stem module is used to perform preliminary processing on the preprocessed image to extract low-level features; The backbone network module consists of four backbone network blocks for extracting deep features. Each backbone network block includes multiple ConvNeXt blocks, and each ConvNeXt block integrates an FDSA module. The ConvNeXt block performs a 2x downsampling, depthwise convolution, layer normalization, 1×1 convolution, GELU activation function, and GRN operation on the input feature map to convert the number of channels to 64 before inputting it into the FDSA module. The FDSA module processes the data, restores the number of channels, and adds it to the downsampled output. The FDSA module is used to receive intermediate feature maps, generate spectrum maps through fast Fourier transform, sort and group feature channels according to the ratio of low frequency to high frequency, perform frequency band interaction fusion through the MHSA module, restore and batch normalize to enhance features. The global average pooling module is used to perform global average pooling on deep feature maps to obtain pooled feature maps. The classification output module, including a fully connected layer and a Softmax function, is used to classify pooled feature maps to achieve breast lesion detection.

Images