A thyroid nodule segmentation method based on edge perception attention
By combining edge-aware attention and multi-scale attention modules, the problems of difficulty in capturing edge details and insufficient multi-scale feature fusion in thyroid nodule segmentation are solved, achieving more efficient nodule segmentation results and improving the accuracy and efficiency of thyroid nodule segmentation.
Patent Information
- Application Number
- CN202610622646.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-08
- Publication Date
- 2026-07-31
AI Technical Summary
Existing thyroid nodule segmentation algorithms suffer from unclear boundaries and difficulty in accurately capturing edge details in ultrasound images. Furthermore, they lack multi-scale feature fusion, which limits the model's ability to distinguish between nodule boundaries and internal regions, resulting in insufficient overall representation capabilities.
A thyroid nodule segmentation method based on edge-aware attention is adopted. By combining an edge-aware attention module and a multi-scale attention fine segmentation module with an encoder and a decoder, feature information is fused using skip connections, and the representation ability of the feature map is improved by multi-head self-attention layer and cross-scale attention computation.
It significantly improves the accuracy and efficiency of thyroid nodule segmentation, better captures edge details, enhances the model's ability to distinguish between nodule boundaries and internal regions, and improves the overall performance of the segmentation algorithm.
Smart Images
Figure CN122492738A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical imaging technology, and in particular to a method for thyroid nodule segmentation based on edge-aware attention. Background Technology
[0002] The thyroid gland is an important organ of the human endocrine system, located in the front of the neck below the thyroid cartilage, and its overall shape is H-shaped. In recent years, due to unhealthy dietary habits, increasing mental stress, and gradually declining immunity, the incidence of various thyroid diseases has been rising rapidly. Currently, the prevalence of various thyroid diseases in my country exceeds 50%, with more than 200 million patients suffering from various thyroid diseases.
[0003] Ultrasound is an important imaging technique for screening thyroid nodules, with advantages such as low examination cost, non-invasiveness, no radiation, and convenient operation.
[0004] Thyroid ultrasound examinations require sonographers with extensive clinical diagnostic experience, but many hospitals lack a sufficient number of senior sonographers, and repetitive assessments over extended periods can easily lead to misdiagnosis or missed diagnosis. With the development of artificial intelligence technology, computer-aided diagnosis (CAD) systems help solve these problems, and accurate segmentation of nodule regions plays a fundamental and crucial role in thyroid ultrasound CAD systems. In recent years, deep learning technology has achieved great success in the field of computer vision, developing a series of semantic segmentation models based on convolutional neural networks (CNNs). In 2015, Long et al. proposed the fully convolutional network (FCN) structure, replacing the fully connected layers in traditional CNNs with convolutional layers, allowing for semantic segmentation on an end-to-end basis, regardless of the scale of the image input. Since then, various fully convolutional semantic segmentation models have been proposed, such as SegNet, UNet, and DeepLab. UNet, due to its good performance even with small sample sizes, is widely used in medical image segmentation. It consists of an encoder and a decoder. The encoder uses four downsampling operations to reduce the image scale to 1 / 16 of its original size, while the decoder uses four upsampling operations to restore the feature maps to the resolution of the original image. Skip connections are used on feature maps of the same scale to fuse high-level and low-level semantic features, resulting in finer edges on the segmented objects.
[0005] However, existing deep learning algorithms for thyroid nodule segmentation still have the following shortcomings:
[0006] (1) Due to the blurry ultrasound images and unclear boundaries of thyroid nodules, it is a difficult task in medical image segmentation. Existing segmentation algorithms still do not perform well in nodule segmentation. Some studies have adopted a global attention mechanism to expand the receptive field, which has improved the effect to a certain extent, but the computational complexity of global attention is high and it does not make sufficient use of local detail information, making it difficult to accurately capture edge detail information.
[0007] (2) Existing multi-scale feature fusion segmentation algorithms fail to make full use of feature information of different fine granularities in attention calculation at the same time, and cannot enable each position in the feature map to participate in multi-scale attention calculation and feature information interaction efficiently, resulting in the model's limited ability to distinguish between nodule boundaries and internal regions.
[0008] (3) Most existing segmentation algorithms only gradually increase the size of the feature map in the decoder, lacking modeling of cross-scale semantic relationships between feature maps at different levels, resulting in insufficient overall network representation ability and difficulty in taking into account the overall shape and local edge information of thyroid nodules. Summary of the Invention
[0009] To address the shortcomings of existing technologies, this invention provides a thyroid nodule segmentation method based on edge-aware attention. A training set is used to train the thyroid nodule segmentation network, resulting in a well-trained detection model. This invention can improve segmentation efficiency while maintaining segmentation effectiveness, helping to reduce the workload of doctors interpreting images and improve their diagnostic efficiency.
[0010] On one hand, the present invention provides a thyroid nodule segmentation method based on edge-aware attention, comprising the following steps:
[0011] Step 1: Establish training, testing, and validation sets using publicly available thyroid nodule datasets: This includes cropping the label files corresponding to thyroid ultrasound images and dividing them into training, testing, and validation sets according to proportions;
[0012] Specifically, the thyroid ultrasound images and their corresponding thyroid nodule labels were standardized into a common image format, and the image size was uniformly scaled from the original image to 256×256 pixels. Then, the publicly available thyroid ultrasound dataset was randomly divided into training set, test set and validation set in a 6:2:2 ratio.
[0013] Step 2: Establish a thyroid nodule segmentation network: This specifically includes using an edge-aware attention module and a multi-scale attention fine segmentation module;
[0014] The specific steps of step 2 are as follows:
[0015] Step 2.1: Construct the encoding and decoding parts of the thyroid nodule segmentation network; the encoding part uses convolution and pooling operations to extract and downsample features from thyroid ultrasound images, continuously reducing the spatial scale of the feature map and increasing the number of channels; the decoding part uses convolution and transposed convolution operations to fuse features and upsample the feature map, continuously reducing the number of channels and increasing the spatial size; the encoding and decoding parts are fused using skip connections.
[0016] Step 2.2: Input the feature map into multiple edge-aware attention modules respectively, use the edge-aware attention modules to extract features from the feature map input into the module, perceive the edge position in the feature map, perform high-resolution attention calculation on the edge features, and use different compressed features to model multi-scale information of the feature map;
[0017] Step 2.3: Using the multi-scale attention fine segmentation module, the feature maps F1, F2, and F3 output from the decoding part at different spatial scales are first unified into feature maps of the same scale, and then attention is calculated to obtain the multi-scale attention calculation result. The multi-scale attention calculation result is then upsampled and fused with the feature map F3 of the decoding part, and finally convolution is performed to obtain the segmentation result.
[0018] The edge-aware attention module specifically performs the following: input feature map The number of channels is reduced from C to d through convolutional layers, and the dimensionality-reduced feature map is divided into h×w local patches, where h and w are the length and width of the patch, as shown in equation (1):
[0019] (1);
[0020] Where Split represents the operation of dividing the map into patches along the spatial dimension, and A, B, C, D represent the fourth local feature map;
[0021] Initialize global compression feature T g The four local feature maps are average pooled and then stitched together along the channels, as shown in equation (2):
[0022] (2);
[0023] Where Concat represents concatenation along the channel dimension, and Avgpool represents average pooling. ;
[0024] Taking local feature map A as an example, the other branches are processed in the same way: A is further divided into 4 selection windows, each window being 1 / 2h × 1 / 2w in size, as shown in equation (3):
[0025] (3);
[0026] Initialize local compression feature T l The four selection windows are average pooled and then concatenated, as shown in equation (4):
[0027] (4);
[0028] in To detect edge regions, the range of pixel values within each selection window is calculated, which is the difference between the maximum and minimum values. The window A with the largest range is selected. i As the core region for edge perception, it is shown in equation (5):
[0029] (5);
[0030] Where j = 1, 2, 3, 4.
[0031] Flatten the selected selection window into sequence T. A and with global compression feature T g and local compression features T l By splicing along the channel dimension, we obtain the edge-aware sequence X. A As shown in equation (6):
[0032] (6);
[0033] Sequence X A The input is a multi-head self-attention layer, and the processed edge-aware sequence Y is obtained through residual connections. A As shown in equation (7):
[0034] (7);
[0035] MHSA stands for Multi-head Attention Calculation;
[0036] Subsequently, the edge-sensing sequence Y A Divide into processed sequences T' along the channel dimension A Global compression feature T' g and local compression feature T' l As shown in equation (8):
[0037] (8);
[0038] sequence T' A Reshape the spatial structure and merge it back into the corresponding local feature map A. Unselected windows remain unchanged. Compress the global feature T'. g and local compression feature T' l The system passes through two convolutional layers in sequence: the first convolutional layer halves the number of channels, and the second convolutional layer restores the original number of channels.
[0039] Then, a weighted calculation is performed: the processed local compressed feature T' l After activation, the result is multiplied by the corresponding four local selection windows to obtain a weighted window. These windows are then concatenated according to their spatial positions to obtain the local patch after local compression feature weighting, as shown in equation (9).
[0040] (9);
[0041] in This represents the activation function, and Merge represents stitching the tiles along the spatial dimension.
[0042] The processed global compressed feature T' g After activation, the feature map is multiplied by the weighted local patch to obtain the output feature map A of the current branch. output As shown in equation (10):
[0043] (10);
[0044] Perform the same operation on branches B, C, and D to obtain the output feature map of each branch. Concatenate the outputs of the four branches along the channel dimension, and restore the number of channels from 4d to the original number of channels C through a convolutional layer to obtain the final output feature map, as shown in Equation (11):
[0045] (11);
[0046] The multi-scale attention fine segmentation module specifically involves: using feature maps of different resolutions in the decoder as Q, K, and V respectively, and establishing contextual dependencies between features of different resolutions through cross-scale attention;
[0047] Multi-scale feature extraction: Three sets of feature maps are extracted from different layers of the network decoder: F1, F2, and F3. Among them, F1 is a high-level semantic feature map containing global semantic abstraction information; F3 is the feature map with the highest spatial resolution, restoring rich spatial detail information; and F2 is at an intermediate scale, combining spatial detail and semantic information.
[0048] Attention Input Transformation: Transform the three sets of feature maps respectively to generate the Q, K, and V matrices for attention calculation; convolve the F1 algorithm and then flatten it to obtain the Q matrix Q. MAR As shown in equation (12):
[0049] (12);
[0050] Flatten refers to the flattening operation;
[0051] Downsample F2 to the same size as F1, convolve and flatten to form matrix K. MAR As shown in equation (13):
[0052] (13);
[0053] Downsample is the downsampling operation;
[0054] After downsampling F3 and using the spatial dimensions of F1, the resulting flattened matrix V is generated. MAR As shown in equation (14):
[0055] (14);
[0056] Cross-scale attention computation: transforming Q MAR K MAR and V MAR Input attention computation, compute cross-scale attention output Att MAR As shown in equation (15):
[0057] (15);
[0058] Where d MAR For feature dimensions;
[0059] Feature fusion and segmentation output: The attention calculation result is upsampled to restore its spatial resolution, then concatenated with the feature map F3, and finally the final segmentation output is generated through a convolutional layer, as shown in Equation (16):
[0060] (16);
[0061] Upsample consists of two transposed convolutional layers and a random deactivation layer, F output This represents the final segmentation result of the network.
[0062] Step 3: Train the thyroid nodule segmentation network using the training set to obtain a trained segmentation model;
[0063] Step 3.1: Place the thyroid ultrasound images and labels of the training set, test set, and validation set into their respective folders, and then put them all into one folder;
[0064] Step 3.2: Use the thyroid nodule segmentation network obtained in Step 2 to train the thyroid nodules multiple times, perform hyperparameter tuning, and adjust the optimal learning rate to obtain the best segmentation model;
[0065] Step 4: Validate the segmentation model on the validation set, segment the thyroid nodules, and obtain their lesion information.
[0066] On the other hand, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the thyroid nodule segmentation method.
[0067] Thirdly, this application proposes a computer program product, including a computer program or instructions that, when executed by a processor, implement the thyroid nodule segmentation method described above.
[0068] The beneficial effects of adopting the above technical solution are as follows:
[0069] This invention provides a thyroid nodule segmentation method based on edge-aware attention, which specifically includes the following beneficial effects:
[0070] 1. This invention employs an edge-aware attention mechanism and multi-scale attention fine segmentation to establish a segmentation network that effectively extracts and fuses global and local features in ultrasound images. The efficiency of the segmentation algorithm in this invention is significantly higher than that of existing segmentation algorithms.
[0071] 2. In the edge-aware attention mechanism of this invention, global compressed features and local compressed features of the feature map are respectively used to perform attention calculation with fine-grained edge-aware features. By utilizing the relationship between different features, the network's ability to represent different scales and positions of the feature map is improved, thereby further enhancing the segmentation effect.
[0072] 3. This invention employs a multi-scale attention fine segmentation module in the network, establishing multi-scale connections between feature maps in the decoding part, which further improves the performance of the segmentation algorithm. Attached Figure Description
[0073] Figure 1 This is a flowchart of the ultrasound image segmentation method for thyroid nodules according to an embodiment of the present invention;
[0074] Figure 2 This is a specific data example diagram of the training set during network training in an embodiment of the present invention;
[0075] Among them, (a) - original images of the thyroid nodule ultrasound image dataset, and (b) - labeled images of the thyroid nodule ultrasound image dataset;
[0076] Figure 3 This is a schematic diagram of the algorithm network structure in an embodiment of the present invention;
[0077] Figure 4 This is a schematic diagram of the edge-aware attention module of the algorithm in an embodiment of the present invention;
[0078] Figure 5 These represent different fine-grained feature relationships in the algorithm of this invention's embodiments;
[0079] Figure 6 This is a schematic diagram of the multi-scale attention fine segmentation module of the algorithm in an embodiment of the present invention. Detailed Implementation
[0080] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.
[0081] Example 1:
[0082] On the one hand, this invention provides a thyroid nodule segmentation method based on edge-aware attention, such as... Figure 1 As shown, it includes the following steps:
[0083] Step 1: Establish training, testing, and validation sets using publicly available thyroid nodule datasets: This includes cropping the label files corresponding to thyroid ultrasound images and dividing them into training, testing, and validation sets according to proportions;
[0084] Specifically, the thyroid ultrasound images and their corresponding thyroid nodule labels were standardized into a common image format, and the image size was uniformly scaled from the original image to 256×256 pixels. Then, the publicly available thyroid ultrasound dataset was randomly divided into training set, test set and validation set in a 6:2:2 ratio.
[0085] The processed ultrasound image of the thyroid nodule in this embodiment is as follows: Figure 2 As shown in (a), the labeled image of the segmented thyroid nodules is as follows: Figure 2 As shown in (b), the dataset is then randomly divided into training, test and validation sets in a ratio of 6:2:2.
[0086] Step 2: Establish a thyroid nodule segmentation network: This specifically includes using an edge-aware attention module and a multi-scale attention fine segmentation module;
[0087] The specific steps of step 2 are as follows:
[0088] Step 2.1: As Figure 3 As shown, the encoding and decoding parts of the thyroid nodule segmentation network are constructed. The encoding part uses convolution and pooling operations to extract and downsample features from thyroid ultrasound images, continuously reducing the spatial scale of the feature map and increasing the number of channels. The decoding part uses convolution and transposed convolution operations to fuse features and upsample the feature map, continuously reducing the number of channels and increasing the spatial size. The encoding and decoding parts are fused using skip connections.
[0089] Step 2.2: Input the feature maps into multiple edge-aware attention modules, such as... Figure 4As shown, the edge-aware attention module is used to extract features from the feature map input to the module. By sensing the edge position in the feature map, high-resolution attention calculation is performed on the edge features, and multi-scale information modeling of the feature map is performed using different compressed features.
[0090] Step 2.3: Using the multi-scale attention fine segmentation module, the feature maps F1, F2, and F3 output from the decoding part at different spatial scales are first unified into feature maps of the same scale, and then attention is calculated to obtain the multi-scale attention calculation result. This result is then upsampled and fused with the feature map F3 from the decoding part, and finally convolution is performed to obtain the segmentation result. Figure 6 This is a schematic diagram of a multi-scale attention-based fine segmentation module.
[0091] The edge-aware attention module specifically performs the following: input feature map The number of channels is reduced from C to d by the convolutional layer, and the dimensionality-reduced feature map is divided into local patches of h×w, where h and w are the length and width of the patch. In this embodiment, h=1 / 2H and w=1 / 2W, as shown in equation (1):
[0092] (1);
[0093] Where Split represents the operation of dividing the map into patches along the spatial dimension, and A, B, C, D represent the fourth local feature map;
[0094] To extract global information across different local feature maps, the global compressed feature T is initialized. g The four local feature maps are average pooled and then stitched together along the channels, as shown in equation (2):
[0095] (2);
[0096] Where Concat represents concatenation along the channel dimension, and Avgpool represents average pooling. ;
[0097] Taking local feature map A as an example, the other branches are processed in the same way. A is further divided into 4 selection windows, each with a size of 1 / 2h × 1 / 2w, as shown in equation (3):
[0098] (3);
[0099] Similarly, initialize the local compression feature T l The four selection windows are average pooled and then concatenated, as shown in equation (4):
[0100] (4);
[0101] in To detect edge regions, the range of pixel values within each selection window is calculated, which is the difference between the maximum and minimum values. The window A with the largest range is selected. i As the core region for edge perception, it is shown in equation (5):
[0102] (5);
[0103] Where j = 1, 2, 3, 4.
[0104] Flatten the selected selection window into sequence T. A and with global compression feature T g and local compression features T l By splicing along the channel dimension, we obtain the edge-aware sequence X. A As shown in equation (6):
[0105] (6);
[0106] Sequence X A The input is a multi-head self-attention layer, and the processed edge-aware sequence Y is obtained through residual connections. A As shown in equation (7):
[0107] (7);
[0108] MHSA stands for Multi-head Attention Calculation;
[0109] Subsequently, the edge-sensing sequence Y A Divide into processed sequences T' along the channel dimension A Global compression feature T' g and local compression feature T' l As shown in equation (8):
[0110] (8);
[0111] sequence T' A Reshape the spatial structure and merge it back into the corresponding local feature map A. Unselected windows remain unchanged. To enhance feature fusion, compress the global feature T'. g and local compression feature T' l The system passes through two convolutional layers in sequence: the first convolutional layer halves the number of channels, and the second convolutional layer restores the original number of channels.
[0112] Then, a weighted calculation is performed: the processed local compressed feature T' lAfter activation, the result is multiplied by the corresponding four local selection windows to obtain a weighted window. These windows are then concatenated according to their spatial positions to obtain the local patch after local compression feature weighting, as shown in equation (9).
[0113] (9);
[0114] in This represents the activation function, and Merge represents stitching the tiles along the spatial dimension.
[0115] The processed global compressed feature T' g After activation, the feature map is multiplied by the weighted local patch to obtain the output feature map A of the current branch. output As shown in equation (10):
[0116] (10);
[0117] Perform the same operation on branches B, C, and D to obtain the output feature map of each branch. Concatenate the outputs of the four branches along the channel dimension, and restore the number of channels from 4d to the original number of channels C through a convolutional layer to obtain the final output feature map, as shown in Equation (11):
[0118] (11);
[0119] Figure 5 This invention describes the relationship between features and information transmission at different fine-grained levels in the edge-aware attention module. The module employs three types of features: edge-aware selection windows (red), other non-edge-aware selection windows (blue), and other tile features (yellow). The red portion corresponds to high-fine-grained detail features on the edge-aware selection window, preserving the highest resolution detail information; the blue portion represents the calculation of features from other selection windows in one-sixteenth of the feature map, carrying a medium range of local semantics; the yellow portion, through feature... Figure 4 Other tile features in a fraction of the region represent a more macroscopic global structure.
[0120] This invention concatenates these three types of features along the channel, enabling high-resolution edge details in the red edge-aware selection window features to be transferred to the blue and yellow features in a learnable manner. Through an attention mechanism, the red features guide the compressed features to focus on inconsistencies in the boundary region, while the compressed features provide contextual constraints for the red edge-aware selection window, thereby enhancing edge perception capabilities.
[0121] Following attention calculation, the module further employs global compressed features processed by convolutional layers to perform deep feature fusion. This operation not only smooths the information exchange between different compressed features but also aligns features of different granularities, laying the foundation for subsequent weighted calculations. In summary, this multi-granularity feature collaboration and fusion information transfer mechanism enables the model to accurately locate and enhance edge regions in the image while maintaining computational efficiency.
[0122] The multi-scale attention-based fine segmentation module specifically comprises:
[0123] To enhance spatial reconstruction capabilities during the decoding stage, a multi-scale attention-based fine segmentation module is proposed. This module utilizes feature maps of different resolutions in the decoder as Q, K, and V, respectively, and establishes contextual dependencies between features of different resolutions through cross-scale attention.
[0124] Multi-scale feature extraction: Three sets of feature maps are extracted from different layers of the network decoder: F1, F2, and F3. Among them, F1 is a high-level semantic feature map containing global semantic abstraction information; F3 is the feature map with the highest spatial resolution, restoring rich spatial detail information; and F2 is at an intermediate scale, combining spatial detail and semantic information.
[0125] Attention Input Transformation: Transform the three sets of feature maps respectively to generate the Q, K, and V matrices for attention calculation; convolve the F1 algorithm and then flatten it to obtain the Q matrix Q. MAR As shown in equation (12):
[0126] (12);
[0127] Flatten refers to the flattening operation;
[0128] Downsample F2 to the same size as F1, convolve and flatten to form matrix K. MAR As shown in equation (13):
[0129] (13);
[0130] Downsample is the downsampling operation;
[0131] After downsampling F3 and using the spatial dimensions of F1, the resulting flattened matrix V is generated. MAR As shown in equation (14):
[0132] (14);
[0133] High-level semantic features F1 provide condensed semantic information as Q; intermediate-scale features F2 provide contextual indexes as K; and high-resolution features F3 serve as V, allowing attention to be focused on more important spatial detail regions.
[0134] Cross-scale attention computation: transforming Q MAR K MAR and V MAR Input attention computation, compute cross-scale attention output Att MAR As shown in equation (15):
[0135] (15);
[0136] Where d MAR For feature dimensions.
[0137] This computation enables cross-scale fusion of spatial details and semantic context during the decoding phase.
[0138] Feature fusion and segmentation output: The attention calculation result is upsampled to restore its spatial resolution, then concatenated with the feature map F3, and finally the final segmentation output is generated through a convolutional layer, as shown in Equation (16):
[0139] (16);
[0140] Upsample consists of two transposed convolutional layers and a random deactivation layer, F output This represents the final segmentation result of the network.
[0141] The multi-scale attention-based fine segmentation module refines the segmentation results by utilizing multi-scale contextual information from the decoder. Its main functions include: using features at different resolutions as Q, K, and V respectively to establish cross-scale contextual dependencies and ensure effective integration of multi-scale information; simultaneously, high-resolution spatial details and high-level semantic information work together through an attention mechanism, enabling the model to consider both local fine structure and global semantic information; furthermore, the attention mechanism adaptively enhances feature regions corresponding to real structures, suppresses noise and artifacts, and improves segmentation robustness; finally, the upsampling of the attention output and concatenation with high-resolution features ensures accurate spatial alignment of cross-scale features.
[0142] Step 3: Train the thyroid nodule segmentation network using the training set to obtain a trained segmentation model;
[0143] Step 3.1: Place the thyroid ultrasound images and labels of the training set, test set, and validation set into their respective folders, and then put them into a large folder.
[0144] Step 3.2: Use the thyroid nodule segmentation network obtained in Step 2 to train the thyroid nodules multiple times, perform hyperparameter tuning, and adjust to find the optimal learning rate to obtain the best segmentation model.
[0145] Step 4: Validate the segmentation model on the validation set, segment the thyroid nodules, and obtain their lesion information.
[0146] Example 2:
[0147] In this embodiment, multiple evaluation metrics such as mIoU, acc, Dice, Precision, and Recall are used to evaluate its performance.
[0148] When evaluating thyroid nodule segmentation algorithms, it's crucial to compare the detection results with experimental results, using appropriate evaluation criteria to assess algorithm performance. Image segmentation is essentially a binary classification of all pixels, therefore, performance metrics are correlated with those of classification algorithms.
[0149] In image classification, there are four concepts: true positive, false positive, true negative, and false negative. These four concepts in image segmentation are similar but not entirely the same. True Positive (TP): The segmentation algorithm determines that the pixel is a lesion region, and the true label also indicates a lesion region; in this case, the segmentation algorithm matches the doctor's diagnosis. False Positive (FP): The segmentation algorithm determines that the pixel is a lesion region, but the true label indicates a non-lesion region; in this case, the segmentation algorithm does not match the doctor's diagnosis. True Negative (TN): The segmentation algorithm determines that the pixel is not a lesion region, and the true label also indicates a non-lesion region; in this case, the segmentation algorithm matches the doctor's diagnosis. False Negative (FN): The segmentation algorithm determines that the pixel is not a lesion region, but the true label indicates a lesion region; in this case, the segmentation algorithm does not match the doctor's diagnosis.
[0150] Based on this, commonly used metrics for evaluating cut performance, such as mIoU, acc, Dice, Precision, Recall, and F1-score, are defined by the following formulas:
[0151] (17);
[0152] (18);
[0153] (19);
[0154] (20);
[0155] (twenty one);
[0156] This invention uses mIoU, acc, Dice, Precision, Recall, and F1-score as six metrics to evaluate the performance of image segmentation algorithms.
[0157] Building upon the framework of the basic algorithm, this invention makes gradual improvements by adding an edge-aware attention module and multi-scale attention fine segmentation, and conducts rigorous controlled variable comparison experiments to explore the effects of these improvements.
[0158] Example 3
[0159] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
[0160] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the edge-aware attention-based thyroid nodule segmentation method described in the various embodiments of this application.
[0161] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes, on which computer programs are stored. When the computer program is executed by the processor, it can implement the various steps of the thyroid nodule segmentation method based on edge-aware attention described above.
[0162] Example 4:
[0163] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the thyroid nodule segmentation method based on edge-aware attention.
[0164] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.
[0165] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
[0166] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of the methods disclosed herein and their equivalents, then the intent of this disclosure also includes such modifications and variations.
Claims
1. A thyroid nodule segmentation method based on edge-aware attention, characterized in that, Includes the following steps: Step 1: Establish training, testing, and validation sets using publicly available thyroid nodule datasets: This includes cropping the label files corresponding to thyroid ultrasound images and dividing them into training, testing, and validation sets according to proportions; Step 2: Establish a thyroid nodule segmentation network: This specifically includes using an edge-aware attention module and a multi-scale attention fine segmentation module; Step 3: Train the thyroid nodule segmentation network using the training set to obtain a trained segmentation model; Step 4: Validate the segmentation model on the validation set, segment the thyroid nodules, and obtain their lesion information.
2. The thyroid nodule segmentation method based on edge-aware attention according to claim 1, characterized in that, Step 1 specifically involves: unifying the thyroid ultrasound images and their corresponding thyroid nodule labels into a common image format, scaling the image size from the original to 256×256 pixels, and then randomly allocating the publicly available thyroid ultrasound dataset into a training set, a test set, and a validation set in a 6:2:2 ratio.
3. The thyroid nodule segmentation method based on edge-aware attention according to claim 1, characterized in that, The specific steps of step 2 are as follows: Step 2.1: Construct the encoding and decoding parts of the thyroid nodule segmentation network; the encoding part uses convolution and pooling operations to extract and downsample features from thyroid ultrasound images, continuously reducing the spatial scale of the feature map and increasing the number of channels; the decoding part uses convolution and transposed convolution operations to fuse features and upsample the feature map, continuously reducing the number of channels and increasing the spatial size; the encoding and decoding parts are fused using skip connections. Step 2.2: Input the feature map into multiple edge-aware attention modules respectively, use the edge-aware attention modules to extract features from the feature map input into the module, perceive the edge position in the feature map, perform high-resolution attention calculation on the edge features, and use different compressed features to model multi-scale information of the feature map; Step 2.3: Using the multi-scale attention fine segmentation module, the feature maps F1, F2, and F3 output from the decoding part at different spatial scales are first unified into feature maps of the same scale, and then attention is calculated to obtain the multi-scale attention calculation result. The multi-scale attention calculation result is then upsampled and fused with the feature map F3 of the decoding part, and finally convolution is performed to obtain the segmentation result.
4. The thyroid nodule segmentation method based on edge-aware attention according to claim 3, characterized in that, The edge-aware attention module specifically performs the following: input feature map The number of channels is reduced from C to d through convolutional layers, and the dimensionality-reduced feature map is divided into h×w local patches, where h and w are the length and width of the patch, as shown in equation (1): （1）； Where Split represents the operation of dividing the map into patches along the spatial dimension, and A, B, C, D represent the fourth local feature map; Initialize global compression feature T g : 4 local feature maps are respectively averaged and pooled along the channel and spliced, as shown in equation (2): （2）； Where Concat represents concatenation along the channel dimension, and Avgpool represents average pooling. ; Taking local feature map A as an example, the processing of the other branches is the same: A is further divided into 4 selection windows, each window being 1 / 2h × 1 / 2w in size, as shown in equation (3): （3）； Initialize local compression feature T l : 4 selection windows are averaged and pooled, as shown in equation (4): （4）； in To detect edge regions, the range of pixel values within each selection window is calculated, which is the difference between the maximum and minimum values. The window A with the largest range is selected. i As the core region for edge perception, it is shown in equation (5): （5）； Where j = 1, 2, 3, 4; Flatten the selected selection window into sequence T A , and concatenate with global compression feature T g , and local compression feature T l along the channel dimension to obtain edge-aware sequence X A , as shown in equation (6): （6）； Sequence X A The input is a multi-head self-attention layer, and the processed edge-aware sequence Y is obtained through residual connections. A As shown in equation (7): （7）； MHSA stands for Multi-head Attention Calculation; Subsequently, the edge-sensing sequence Y A Divide into processed sequences T' along the channel dimension A Global compression feature T' g and local compression feature T' l As shown in equation (8): （8）； sequence T' A Reshape the spatial structure and merge it back into the corresponding local feature map A. Unselected windows remain unchanged. Compress the global feature T'. g and local compression feature T' l The channel count is halved in the first convolutional layer and restored to the original number of channels in the second convolutional layer. Then, a weighted calculation is performed: the processed local compressed feature T' l After activation, the result is multiplied by the corresponding four local selection windows to obtain a weighted window. These windows are then concatenated according to their spatial positions to obtain the local patch after local compression feature weighting, as shown in equation (9). （9）； in This represents the activation function, and Merge indicates stitching the tiles along the spatial dimension. The processed global compressed feature T' g After activation, the feature map is multiplied by the weighted local patch to obtain the output feature map A of the current branch. output As shown in equation (10): （10）； Perform the same operation on branches B, C, and D to obtain the output feature map of each branch; concatenate the outputs of the four branches along the channel dimension, and restore the number of channels from 4d to the original number of channels C through a convolutional layer to obtain the final output feature map, as shown in Equation (11): （11）。 5. A thyroid nodule segmentation method based on edge-aware attention according to claim 4, characterized in that, The multi-scale attention fine segmentation module specifically involves: using feature maps of different resolutions in the decoder as Q, K, and V respectively, and establishing contextual dependencies between features of different resolutions through cross-scale attention; Multi-scale feature extraction: Three sets of feature maps are extracted from different layers of the network decoder: F1, F2 and F3; where F1 is a high-level semantic feature map containing global semantic abstraction information; F3 is the feature map with the highest spatial resolution, restoring rich spatial detail information; and F2 is at an intermediate scale, containing both spatial detail and semantic information. Attention Input Transformation: Transform the three sets of feature maps respectively to generate the Q, K, and V matrices for attention calculation; convolve the F1 algorithm and then flatten it to obtain the Q matrix Q. MAR As shown in equation (12): （12）； Flatten refers to the flattening operation; Downsample F2 to the same size as F1, convolve and flatten to form matrix K. MAR As shown in equation (13): （13）； Downsample is the downsampling operation; After downsampling F3 and using the spatial dimensions of F1, the resulting flattened matrix V is generated. MAR As shown in equation (14): （14）； Cross-scale attention computation: transforming Q MAR K MAR and V MAR Input attention computation, compute cross-scale attention output Att MAR As shown in equation (15): (15)； Where d MAR For feature dimensions; Feature fusion and segmentation output: The attention calculation result is upsampled to restore its spatial resolution, then concatenated with the feature map F3, and finally the final segmentation output is generated through a convolutional layer, as shown in Equation (16): (16)； Upsample consists of two transposed convolutional layers and a random deactivation layer, F output This represents the final segmentation result of the network.
6. The thyroid nodule segmentation method based on edge-aware attention according to claim 1, characterized in that, Step 3 specifically includes the following steps: Step 3.1: Place the thyroid ultrasound images and labels of the training set, test set, and validation set into their respective folders, and then put them all into one folder; Step 3.2: Use the thyroid nodule segmentation network obtained in Step 2 to train the thyroid nodules multiple times, perform hyperparameter tuning, and adjust to find the optimal learning rate to obtain the best segmentation model.
7. A computer-readable storage medium storing executable instructions, characterized in that, When the instruction is executed, it causes the processor to perform a thyroid nodule segmentation method based on edge-aware attention as described in any one of claims 1-6.
8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the thyroid nodule segmentation method based on edge-aware attention as described in any one of claims 1-6.