Breast cancer ultrasound image automatic segmentation method based on double token attention mechanism

Abstract

The application provides a breast cancer ultrasound image automatic segmentation method based on a double token attention mechanism, and relates to the technical field of medical image computer-aided diagnosis. The method first uses a public breast cancer data set to establish a training set and a test set: including cutting the image corresponding label file, and dividing it into a training set, a test set and a verification set according to a proportion; then a double token attention layer and a multi-scale skip connection are used to establish a breast cancer segmentation network model; the breast cancer segmentation network is trained using the training set to obtain an optimal breast cancer segmentation network model; finally, the breast cancer segmentation network model is verified in the test set to obtain breast cancer lesion information, and the performance thereof is evaluated using multiple evaluation indexes. The method uses a double token attention mechanism and a multi-scale skip connection to establish a segmentation network capable of effectively extracting and fusing global and local features in an ultrasound image, and the efficiency of the segmentation algorithm is significantly higher than that of existing segmentation algorithms.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image computer-aided diagnosis, and in particular to an automatic segmentation method for breast cancer ultrasound images based on a dual-token attention mechanism. Background Art

[0002] According to the 2020 Global Cancer Data released by the World Health Organization's International Agency for Research on Cancer, breast cancer is the most common cancer in women worldwide and the leading cause of cancer death in women. Breast cancer screening can effectively increase the rate of early diagnosis and reduce mortality.

[0003] Ultrasound is an important imaging method for breast cancer screening, offering advantages such as low cost, non-invasiveness, no radiation, and ease of use. For Chinese women, ultrasound screening also surpasses X-rays in sensitivity and accuracy. The "Guidelines for Breast Cancer Screening in Chinese Women (2022 Edition)" recommends breast ultrasound as the preferred screening method for breast cancer in Chinese women.

[0004] Breast ultrasound examinations require ultrasound physicians with extensive clinical diagnostic experience, but many hospitals lack sufficient experienced ultrasound physicians. Furthermore, long, repetitive assessments can easily lead to misdiagnosis or missed diagnoses. With the development of artificial intelligence (AI) technology, computer-aided diagnosis (CAD) systems are helping to address these issues. Accurate segmentation of breast tumor regions plays a fundamental and crucial role in breast ultrasound CAD systems. In recent years, deep learning technology has achieved tremendous success in computer vision, developing a series of semantic segmentation models based on convolutional neural networks (CNNs). In 2015, Long et al. proposed the fully convolutional network (FCN) architecture, replacing the fully connected layers in traditional CNNs with convolutional layers. This eliminates the need for fixed-scale image inputs and enables semantic segmentation in an end-to-end manner. Since then, a variety of fully convolutional semantic segmentation models have been proposed, such as SegNet, UNet, and DeepLab. UNet is widely used in medical image segmentation due to its good performance even with small sample sizes. It consists of an encoder and a decoder. The encoder downsamples the image four times to 1 / 16 of its original size, while the decoder upsamples the image four times to restore the feature map to the original resolution. Using skip connections on feature maps of the same scale enables the fusion of high-level and low-level semantic features, resulting in finer edges in segmented objects.

[0005] However, the existing deep learning breast cancer ultrasound image segmentation algorithm still has the following shortcomings:

[0006] 1) Since breast cancer ultrasound images are relatively blurry, they are a difficult segmentation task in medical image processing. Existing segmentation algorithms still do not perform very well in breast cancer ultrasound image segmentation. Some researchers have adopted a global attention mechanism to increase the receptive field, which has improved the effect. However, the use of the global attention mechanism ignores the spatial inductive bias in the local receptive field in the feature map.

[0007] 2) Existing segmentation algorithms that fuse feature maps of different scales do not fully utilize the global feature map and local feature map in attention calculation to achieve efficient multi-scale attention calculation in which every position of the feature map can participate in local and global attention calculation.

[0008] 3) Most existing segmentation algorithms establish jump connections between the encoding and decoding parts based on feature maps of the same scale, but fail to establish connections between feature maps at different levels of the encoding and decoding parts of the network. As a result, the overall representation capability of the network is insufficient. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned existing technologies and provide a method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism. The breast cancer segmentation network is trained using a training set to obtain a trained detection model, thereby realizing automatic segmentation of breast cancer ultrasound images.

[0010] To solve the above technical problems, the technical solution adopted by the present invention is: a method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism, comprising the following steps:

[0011] Step 1: Use the public breast cancer dataset to create a training set and a test set: This includes cropping the image-corresponding label files and dividing them into training, test, and validation sets in proportion.

[0012] The open-source breast cancer ultrasound image dataset was read with ultrasound image files and their corresponding label files. The breast cancer images and their corresponding label images were resized from the irregular sizes of the original dataset to a uniform size. The dataset was then randomly divided into training, test, and validation sets in proportion.

[0013] Step 2: Use dual-token attention layer and multi-scale skip connection to build a breast cancer segmentation network model;

[0014] Step 2.1: Establish a segmentation network to perform two-stage convolution and downsampling on the breast ultrasound image to extract a feature map with a size of H×W and a number of channels C.

[0015] Step 2.2: Use the dual-token attention module to the feature map Finput Perform feature extraction, generate global tokens and local tokens through feature maps, concatenate them with the serialized local feature maps, perform attention calculation, and use the global tokens and local tokens after attention calculation to weight the feature maps;

[0016] Step 2.2.1: Input feature map Initialize the global token to get the global token

[0017] The average pooling method is used to initialize the global token of the global feature map;

[0018] Step 2.2.2: Input feature map F input Divided into 4 lengths and widths as input features Figure 1 Half of the local feature map

[0019] Step 2.2.3: Initialize the local feature map into a local token to obtain a local token The average pooling method is used to initialize local tokens on the local feature map;

[0020] Step 2.2.4: Local feature map Rearrange into a set of local feature map sequences And concatenate the global token GT and the local token LT by channel i Get a double token sequence

[0021] Step 2.2.5: The obtained double token sequence Y i Input attention layer to obtain the processed double token sequence Y′ after attention calculation i ;

[0022] Step 2.2.6: The obtained double token sequence Y′ i , divided into a local feature map sequence X′ along the channel dimension i , local token LT′ i and the global token GT′ i , GT′ i Represents the i-th global token obtained by participating in the attention calculation of the i-th local feature map sequence;

[0023] Step 2.2.7: Get the i-th global token GT′ i The global token weight is obtained through the global token upsampling module

[0024] First, the i-th global token GT′ i Use the fully connected layer to get the global token column weight

[0025] The global token row weight is obtained by multiplying the global token column weight with the global token row weight matrix

[0026] The global token weight is obtained by multiplying the global token column weight with the global token row weight matrix

[0027] The four global token weights are spliced together along the channel dimension, and the final global token weight is obtained by dimension reduction using convolution calculation

[0028] Step 2.2.9: The obtained local token LT' i The local token weight is obtained by passing the local token through the local token up-sampling module

[0029] Specifically, the local token LT' i The local token column weight is obtained by using a fully connected layer The local token row weight is obtained by using a fully connected layer The local token weight is obtained by multiplying the local token column weight with the local token row weight matrix

[0030] Step 2.2.10: The local feature map sequence X' i is rearranged into a local feature map The local token weight obtained in step 2.2.9 is After passing through the sigmoid activation function, it is multiplied with the local feature map respectively to obtain the local token weighted local feature map

[0031] Step 2.2.11: The local token weighted local feature maps obtained in step 2.2.10 are spliced together along the spatial dimension to obtain the local token weighted feature map

[0032] Step 2.2.12: The global token weight obtained in step 2.2.8 is After passing through the sigmoid activation function, it is multiplied with the local token weighted feature map F'(W LT ) to obtain the global and local double token weighted output feature map

[0033] Step 2.3: After obtaining different scale feature maps at different levels of the encoding part and the decoding part of the segmentation network and establishing a skip connection, the multi-scale skip fusion module is used to fuse the encoding part feature map with the decoding part feature map together;

[0034] Step 2.3.1 encode part feature map F encoder The number of channels is doubled by convolution to obtain the up-sampled encoded part feature map

[0035] Step 2.3.2: The up-sampled encoded part feature map F' encoder The average pooling down-sampling is multiplied by the transfer coefficient a of the encoded part feature map to obtain the encoded part feature map score

[0036] Step 2.3.3: The encoded part feature map score Score is multiplied by the decoding part feature map after random inactivation operation, and the product is added to the decoding part feature map to obtain the multi-scale skip connection output feature map after convolution

[0037] Step 2.4: On the basis of step 2.2 and step 2.3, the double token attention module and convolution are used to form a double token attention layer to extract features from the feature map; down-sampling and deconvolution are used to change the scale of the feature map; the same scale skip connection and multi-scale skip connection are used for feature interaction between the encoding part and the decoding part of the segmentation network, wherein the same scale skip connection is used to concatenate the same scale encoded part feature map and the decoding part feature map in the channel dimension, and the multi-scale skip connection is used to fuse the different scale encoded part feature map and the decoding part feature map together by using the multi-scale skip connection fusion module; finally, the convolution is used to obtain the final output image;

[0038] Step 3: Train the breast cancer segmentation network using the training set to obtain the best breast cancer segmentation network model;

[0039] Step 3.1: Place the images and labels of the training set, test set and validation set one by one in the corresponding folders, and then put them into a large folder;

[0040] Step 3.2: Use the breast cancer segmentation network obtained in step 2 to train the training set multiple times, and perform hyperparameter optimization to obtain the best breast cancer segmentation network model;

[0041] Step 4: Verify the breast cancer segmentation network model in the test set to obtain the breast cancer lesion information, and evaluate its performance using multiple evaluation indexes.

[0042] The beneficial effects of the above technical solution are as follows: the automatic segmentation method of breast cancer ultrasound images based on the dual-token attention mechanism provided by the present invention, (1) adopts the dual-token attention mechanism and multi-scale jump connection to establish a segmentation network that effectively extracts and fuses global and local features in ultrasound images, and the efficiency of the segmentation algorithm is significantly higher than the existing segmentation algorithm. (2) In the dual-token attention mechanism, the global features and local features of the feature map are tokenized separately, and the relationship between different tokens and between tokens and pixels is calculated using self-attention, thereby improving the network's ability to represent different scales and different positions of the feature map, further improving the segmentation effect. (3) The segmentation network adopts a combination of multi-scale jump connection and same-scale jump connection to establish a multi-scale connection between the feature maps of the encoding part and the decoding part, further improving the effect of the segmentation algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism provided by an embodiment of the present invention;

[0044] Figure 2 This is a breast cancer ultrasound image dataset provided by an embodiment of the present invention, where (a) is the original image and (b) is the labeled image;

[0045] Figure 3 This is a schematic diagram of the breast cancer segmentation network structure provided by an embodiment of the present invention.

[0046] Figure 4 Schematic diagram of a dual-token attention module provided for an embodiment of the present invention.

[0047] Figure 5 Schematic diagram of a token upsampling module provided in an embodiment of the present invention, wherein (a) is a global token upsampling module and (b) is a local token upsampling module;

[0048] Figure 6 Schematic diagram of the interaction relationship of the dual-token attention module provided in an embodiment of the present invention.

[0049] Figure 7 Schematic diagram of a multi-scale skip connection fusion module provided by an embodiment of the present invention.

[0050] Figure 8 This is a diagram of the segmentation experiment results provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0052] In this embodiment, a method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism is proposed. Figure 1 As shown, the specific steps include:

[0053] Step 1: Use the public breast cancer dataset to create a training set and a test set: This includes cropping the image-corresponding label files and dividing them into training, test, and validation sets in proportion.

[0054] The ultrasound image files and their corresponding label files of the open source breast cancer ultrasound image dataset are read, and the breast cancer images and their corresponding label images are uniformly scaled from the irregular size of the original dataset to 256×256 pixels. The processed breast cancer ultrasound images are as follows: Figure 2 As shown in a, the label image of breast cancer segmentation is as follows Figure 2 As shown in Figure 2, the dataset is then divided into training set, test set, and validation set in a ratio of 7:2:1 using a random allocation method.

[0055] Step 2: Use dual-token attention layer and multi-scale skip connection to build a breast cancer segmentation network model, such as Figure 3 As shown, it specifically includes the following steps:

[0056] Step 2.1: If Figure 3 As shown in the figure, a segmentation network is established to perform two-stage convolution and downsampling on the breast ultrasound image, and a feature map with a size of H×W pixels and a number of channels of C is extracted. In this embodiment, the feature map F input The size is 64×64 pixels and the number of channels is 256.

[0057] Step 2.2: If Figure 4 As shown, the dual-token attention module is used to input Perform feature extraction, generate global tokens and local tokens through feature maps, concatenate them with the serialized local feature maps, perform attention calculation, and use the global tokens and local tokens after attention calculation to weight the feature maps;

[0058] Step 2.2.1: Input feature map Initialize the global token to get the global token (Global token)

[0059]

[0060] This embodiment uses the average pooling method to initialize the global token of the global feature map, as shown in formula (1):

[0061] GT=Avgpool(F input ) (1)

[0062] Among them, GT represents global token, Avgpool represents global pooling, F input Represents the input feature map.

[0063] Step 2.2.2: Input feature map F input Divided into 4 lengths and widths as input features Figure 1 Half of the local feature map As shown in formula (2):

[0064]

[0065] in, Represents the i-th local feature map, i = 1, 2, 3, 4.

[0066] Step 2.2.3: Initialize the local feature map into a local token to obtain a local token Similar to the global token initialization, this embodiment also uses the average pooling method to initialize the local tokens of the local feature map, as shown in formula (3):

[0067]

[0068] Among them, LT i represents the i-th local token.

[0069] Step 2.2.4: Local feature map Rearrange into a set of local feature map sequences And concatenate the global token GT and the local token LT by channel i Get a double token sequence As shown in formula (4):

[0070] Y i =[X i ,LT i ,GT i ] (4)

[0071] Among them, Y i represents the i-th two-token sequence, X i Represents the i-th local feature map sequence, LT i represents the i-th local token, GT i represents the i-th global token corresponding to the i-th local token, and [] represents concatenation on the channel dimension.

[0072] Step 2.2.5: The obtained double token sequence Y i Input attention layer to obtain the processed double token sequence Y′ after attention calculation i , as shown in formula (5):

[0073] Y′ i =Attention(Y i ) (5)

[0074] Among them, Attention represents attention calculation.

[0075] Step 2.2.6: The obtained double token sequence Y′ i , divided into a local feature map sequence X′ along the channel dimension i , local token LT′ i and the global token GT′ i , GT′ i Represents the i-th global token obtained by participating in the attention calculation of the i-th local feature map sequence.

[0076] Step 2.2.7: Get the i-th global token GT′ i The global token weight is obtained through the global token upsampling module The global token upsampling module is as follows Figure 5 As shown in a.

[0077] Specifically, first the i-th global token GT′ i Use the fully connected layer to get the global token column weight As shown in formula (5):

[0078]

[0079] in, represents the column weight obtained by the i-th global token, and FC represents the fully connected calculation.

[0080] Similarly, the global token row weight is obtained using the fully connected layer As shown in formula (7):

[0081]

[0082] in, represents the row weight obtained by the i-th global token.

[0083] The global token weight is then multiplied by the global token row weight matrix to get the global token weight As shown in formula (8):

[0084]

[0085] in, represents the i-th global token weight, Represents matrix multiplication.

[0086] The four global token weights are spliced together along the channel dimension, and the final global token weight is obtained by dimension reduction using convolution calculation As shown in equation (9):

[0087]

[0088] where W GT represents the global token weight, and Conv represents the convolution calculation.

[0089] Step 2.2.9: Similarly, the obtained local token LT' i is obtained by the local token up-sampling module The local token up-sampling module is as shown in Figure 5 b. Specifically, the local token LT' i is obtained by using a fully connected layer is obtained by using a fully connected layer Then, the local token column weight is multiplied by the local token row weight matrix to obtain the local token weight

[0090] Step 2.2.10: The local feature map sequence X' i is rearranged into a local feature map The local token weight obtained in step 2.2.9 is After passing through the sigmoid activation function, it is multiplied by the local feature map respectively to obtain the local token weighted local feature map As shown in equation (10):

[0091]

[0092] where, represents the i-th local token weight, represents the i-th local feature map after attention calculation, represents the i-th local token weighted local feature map.

[0093] Step 2.2.11: The local token weighted local feature maps obtained in step 2.2.10 are spliced together along the spatial dimension to obtain the local token weighted feature map As shown in equation (11):

[0094]

[0095] where |..| represents splicing in the spatial dimension.

[0096] Step 2.2.12: Add the global token weight obtained in step 2.2.8 After the sigmoid activation function and the weighted feature map F′(W LT ) to obtain the global and local double-token weighted output feature map As shown in formula (12):

[0097] F output =sigmoid(W GT )×F′(W LT ) (12)

[0098] Among them, W GT represents the global token weight, F′(W LT ) represents the feature map after local token weighting, F output Represents the output feature map.

[0099] Figure 6 It is the interactive relationship established by the dual-token attention module in the present invention between different positions in the feature map. In the same local feature map (between pixel 3 and pixel 4), a connection is established through the local attention mechanism; the local feature map and the local token (between pixel 7 and local token 4) are connected through the local token attention mechanism; the local feature map and the global token (between pixel 6 and the global token) are connected through the global token attention mechanism; the local token and the global token (between local token 1 and global token 1) are connected through the global-local token attention mechanism; the global token and the global token (between global token 1 and global token 2) are connected through the global-global token convolution.

[0100] Step 2.3: Obtain feature maps of different scales at different levels of the encoding and decoding parts of the segmentation network and establish jump connections, then use the multi-scale jump fusion module to combine the feature maps of the encoding part and decoded feature maps Blend together, Figure 7 It is a schematic diagram of the multi-scale jump fusion module.

[0101] Step 2.3.1 Segment the network encoding part of the feature map F encoder The number of channels is doubled by convolution to obtain the feature map of the encoded part after dimensionality increase. As shown in formula (13):

[0102] F′ encoder =Conv(F encoder ) (13)

[0103] Among them, Conv represents convolution calculation, F′ encoder Represents the feature map of the encoded part after dimensionality increase.

[0104] Step 2.3.2: The feature map F′ of the coded part after dimensionality increase encoder After average pooling downsampling, the score of the encoded feature map is obtained by multiplying the transfer coefficient α of the encoded feature map As shown in formula (14):

[0105] Score = α × Downsample (F′ encoder ) (14)

[0106] Among them, Downsample represents average pooling downsampling, and α represents the transfer coefficient of the encoding part feature map.

[0107] Step 2.3.3: Multiply the encoding feature map score Score with the decoding feature map after random inactivation, add the product to the decoding feature map, and obtain the multi-scale jump connection output feature map after convolution. As shown in formula (15):

[0108] F multiscale =Conv(F decoder (1+Dropout(Score))) (15)

[0109] Among them, Conv represents convolution calculation and Dropout represents random inactivation.

[0110] Step 2.4: Based on steps 2.2 and 2.3, a dual-token attention layer consisting of a dual-token attention module and convolution is used to extract features from the feature map; downsampling and deconvolution are used to change the scale of the feature map; same-scale skip connections and multi-scale skip connections are used between the encoding and decoding parts of the segmentation network for feature interaction, where the same-scale skip connections concatenate the encoding feature map and the decoding feature map of the same scale in the channel dimension, and the multi-scale skip connections use the multi-scale skip connection fusion module to fuse the encoding feature map and the decoding feature map of different scales together; finally, the final output image is obtained through convolution;

[0111] Step 3: Use the training set to train the breast cancer segmentation network, which specifically includes the following steps:

[0112] Step 3.1: Place the images and labels of the training set, test set, and validation set into corresponding folders one by one, and then put them into a large folder;

[0113] Step 3.2: Use the breast cancer segmentation network obtained in step 2 to train the training set multiple times and perform hyperparameter tuning to obtain the optimal breast cancer segmentation network model;

[0114] Step 4: Verify the breast cancer segmentation network model in the test set to obtain breast cancer lesion information, and evaluate its performance using mIoU, acc, Dice, Precision, Recall, and F1-score evaluation indexes.

[0115] The detection results are compared with the experimental results. When evaluating the breast cancer segmentation algorithm, the algorithm performance needs to be judged according to reasonable evaluation criteria. Image segmentation is actually a collection of binary classification of all pixel points, so the performance indicators are related to the performance indicators of classification algorithms.

[0116] In image classification, there are four concepts of true positive, false positive, true negative, and false negative. The four concepts in image segmentation are similar but not exactly the same.

[0117] True positive (TP): The segmentation algorithm determines that the pixel point is a lesion area, and the true label is also a lesion area. At this time, the segmentation algorithm is consistent with the doctor's diagnosis.

[0118] False positive (FP): The segmentation algorithm determines that the pixel point is a lesion area, but the true label is a non-lesion area. At this time, the segmentation algorithm is not consistent with the doctor's diagnosis.

[0119] True negative (TN): The segmentation algorithm determines that the pixel point is a non-lesion area, and the true label is also a non-lesion area. At this time, the segmentation algorithm is consistent with the doctor's diagnosis.

[0120] False negative (FN): The segmentation algorithm determines that the pixel point is a non-lesion area, but the true label is a lesion area. At this time, the segmentation algorithm is not consistent with the doctor's diagnosis.

[0121] On this basis, the commonly used evaluation indicators of segmentation performance such as mIoU, acc, Dice, Precision, Recall, and F1-score are as follows:

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128] (1) Segmentation process experimental results.

[0129] Figure 8This embodiment uses the trained breast cancer segmentation network model to perform segmentation on multiple breast cancer ultrasound images. The test results show that the segmentation model of the present invention has a good segmentation effect on breast cancers of different locations, shapes and sizes.

[0130] (2) Analysis of image segmentation algorithm performance indicators.

[0131] This example uses mIoU, acc, Dice, Precision, Recall, and F1-score as six metrics to evaluate the performance of an image segmentation algorithm. Furthermore, this example partitions the 780-image breast cancer segmentation dataset into a ratio of 8:1:1, resulting in a test set of 156 images and a validation set of 78 images. Multiple tests are performed on both the test and validation sets, and the average values ​​are taken.

[0132] This example first adopts the strategy of dual-token attention and multi-scale skip connection to build a basic segmentation network framework, and conducts comparative experiments with existing advanced segmentation network algorithms such as UNet, UNet++, DeeplabV3+, SETR, AttUNet, and TransUNet. The segmentation network model of the present invention is much ahead of existing segmentation algorithms in terms of six performance indicators: mIoU, acc, Dice, Precision, Recall, and F1-score, as shown in Table 1:

[0133] Table 1 Comparative experimental results of different neural networks

[0134] UNet UNet++ DeeplabV3+ SETR AttUNet TransUNet The present invention mIoU 72.77％ 73.66％ 72.89％ 67.08％ 75.96％ 70.07％ 76.73％ acc 92.75％ 93.16％ 93.05％ 91.78％ 93.42％ 91.37％ 93.82％ Dice 64.68％ 64.55％ 64.23％ 53.92％ 69.65％ 62.20％ 71.52％ Precision 65.29％ 66.29％ 65.21％ 60.28％ 70.17％ 64.40％ 70.65％ Recall 66.56％ 66.79％ 64.96％ 62.48％ 69.51％ 63.98％ 71.88％ F1-score 65.92％ 66.54％ 65.08％ 61.36％ 69.84％ 64.19％ 71.26％

[0135] It is not difficult to see from the table that the mIoU index of the segmentation network model of the present invention reaches 76.73%, the acc index reaches 93.82%, the Dice index reaches 71.52%, the Precision index reaches 70.65%, the Recall index reaches 71.88%, and the F1-score index reaches 71.26%, all of which exceed the existing deep learning image segmentation algorithm.

[0136] Table 2 Comparative experimental results of improved dual-token attention layer and multi-scale skip connection

[0137] mIoU acc Dice Precision Recall F1-score Invention Foundation + Dual Token Attention Module 75.61％ 93.22％ 69.35％ 69.03％ 69.34％ 69.18％ The basis of the present invention + multi-scale jump connection 74.64％ 93.30％ 66.28％ 67.44％ 67.26％ 67.35％ Final model of the present invention 76.73％ 93.82％ 71.52％ 70.65％ 71.88％ 71.26％

[0138] Building upon the framework of the basic algorithm, our method incrementally improves upon it by incorporating a dual-token attention layer and multi-scale skip connections. We also conduct rigorous comparative experiments with controlled variables to explore the effectiveness of these improvements. Table 2 shows the experimental results, which demonstrate that the dual-token attention module and multi-scale skip connections further enhance the performance of our final algorithm compared to the basic algorithm.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism, characterized by: The following steps are involved: Step 1: Use the public breast cancer dataset to create a training set and a test set: This includes cropping the image-corresponding label files and dividing them into training, test, and validation sets in proportion; Step 2: Use dual-token attention layer and multi-scale skip connection to build a breast cancer segmentation network model; Step 2.1: Establish a segmentation network to perform two-stage convolution and downsampling on the breast ultrasound image to extract a feature map with a size of H×W and a number of channels C. Step 2.2: Use the dual-token attention module to the feature map F input Perform feature extraction, generate global tokens and local tokens through feature maps, concatenate them with the serialized local feature maps, perform attention calculation, and use the global tokens and local tokens after attention calculation to weight the feature maps; Step 2.3: Obtain feature maps of different scales at different levels of the encoding and decoding parts of the segmentation network and establish jump connections, then use the multi-scale jump fusion module to combine the feature maps of the encoding part and decoded feature maps blend together; Step 2.4: Based on steps 2.2 and 2.3, a dual-token attention layer consisting of a dual-token attention module and convolution is used to extract features from the feature map; downsampling and deconvolution are used to change the scale of the feature map; same-scale skip connections and multi-scale skip connections are used between the encoding and decoding parts of the segmentation network for feature interaction, where the same-scale skip connections concatenate the encoding feature map and the decoding feature map of the same scale in the channel dimension, and the multi-scale skip connections use the multi-scale skip connection fusion module to fuse the encoding feature map and the decoding feature map of different scales together; finally, the final output image is obtained through convolution; Step 3: Use the training set to train the breast cancer segmentation network to obtain the optimal breast cancer segmentation network model; Step 4: Verify the breast cancer segmentation network model in the test set, obtain breast cancer lesion information, and evaluate its performance using multiple evaluation indicators.

2. The automatic segmentation method for breast cancer ultrasound images based on the dual-token attention mechanism according to claim 1 is characterized in that: Step 1 reads ultrasound image files and their corresponding label files from an open-source breast cancer ultrasound image dataset, scales the breast cancer images and their corresponding label images from the irregular sizes of the original dataset to a uniform size, and then uses a random allocation method to divide the dataset into a training set, a test set, and a validation set in proportion.

3. The automatic segmentation method for breast cancer ultrasound images based on the dual-token attention mechanism according to claim 2 is characterized in that: The specific method of step 2.2 is: Step 2.2.1: Input feature map Initialize the global token to get the global token Step 2.2.2: Input feature map F input Divide into 4 local feature maps with half the length and width of the input feature map Step 2.2.3: Initialize the local feature map to obtain the local token Step 2.2.4: Local feature map Rearrange into a set of local feature map sequences And concatenate the global token GT and the local token LT by channel i Get a double token sequence Step 2.2.5: The obtained double token sequence Y i Input attention layer to obtain the processed double token sequence Y′ after attention calculation i ; Step 2.2.6: The obtained double token sequence Y′ i , divided into a local feature map sequence X′ along the channel dimension i , local token LT′ i and the global token GT′ i , GT′ i Represents the i-th global token obtained by participating in the attention calculation of the i-th local feature map sequence; Step 2.2.7: Get the i-th global token GT′ i The global token weight is obtained through the global token upsampling module Step 2.2.8: Concatenate the 4 global token weights along the channel dimension and use convolution to reduce the dimension to obtain the final global token weight. Step 2.2.9: The obtained local token LT′ i The local token weight is obtained through the local token upsampling module Step 2.2.10: The local feature map sequence X′ after attention calculation i Rearrange into local feature maps The local token weight obtained in step 2.2.9 After the sigmoid activation function, they are respectively combined with the local feature maps Multiply to get the local feature map after local token weighting Step 2.2.11: Concatenate the local feature maps weighted by local tokens obtained in step 2.2.10 according to the spatial dimension to obtain the feature map weighted by local tokens. Step 2.2.12: Add the global token weight obtained in step 2.2.8 After the sigmoid activation function and the weighted feature map F′(W LT ) to obtain the global and local double-token weighted output feature map 4. The automatic segmentation method for breast cancer ultrasound images based on the dual-token attention mechanism according to claim 3 is characterized by: The step 2.2.1 uses the average pooling method to perform global token initialization on the global feature map.

5. The method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism according to claim 3, characterized in that: The step 2.2.3 uses the average pooling method to initialize the local tokens of the local feature map.

6. The method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism according to claim 3, characterized in that: The step 2.2.7 obtains the global token weight The specific method is: First, the i-th global token GT′ i Use the fully connected layer to get the global token column weight Then use the fully connected layer to get the global token row weight The global token weight is then multiplied by the global token row weight matrix to get the global token weight 7. The method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism according to claim 6, characterized in that: The specific method of step 2.2.9 is: Specifically, first the local token LT′ i Use the fully connected layer to get the local token column weight Use the fully connected layer to get the local token row weight Then multiply the local token column weight by the local token row weight matrix to get the local token weight 8. The method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism according to claim 7, characterized in that: The specific method of step 2.3 is: Step 2.3.1 Segment the network encoding part of the feature map F encoder The number of channels is doubled by convolution to obtain the feature map of the encoded part after dimensionality increase. Step 2.3.2: The feature map F′ of the coded part after dimensionality increase encoder After average pooling downsampling, the score of the feature map of the encoded part is obtained by multiplying the transfer coefficient α of the feature map of the encoded part Step 2.3.3: Multiply the encoding feature map score Score with the decoding feature map after random inactivation, add the product to the decoding feature map, and obtain the multi-scale jump connection output feature map after convolution.

9. The method for automatic segmentation of breast cancer ultrasound images based on a dual-token attention mechanism according to claim 1, characterized in that: The specific method of step 3 is: Step 3.1: Place the images and labels of the training set, test set, and validation set into corresponding folders one by one, and then put them into a large folder; Step 3.2: Use the breast cancer segmentation network obtained in step 2 to train the training set multiple times, perform hyperparameter tuning, and obtain the optimal breast cancer segmentation network model.

Images