A thyroid nodule image segmentation method, system, computer device and medium

Abstract

The application provides a thyroid nodule image segmentation method, system, computer device and medium, and belongs to the field of medical image processing, and comprises the following steps: acquiring a three-modal tensor of a thyroid nodule ultrasound image to be segmented; performing feature extraction of different resolutions on the modal tensor in sequence to obtain a feature extraction result; performing feature extraction on each resolution feature extraction result in parallel through different receptive fields to obtain a plurality of context features of different scales, fusing and optimizing based on a channel attention mechanism to obtain an optimized feature; optimizing based on a spatial attention mechanism to obtain a feature optimized based on spatial attention; fusing to obtain a feature extraction result optimized at the scale; fusing and performing scale reconstruction on the feature extraction result and scale reconstruction features of a lower scale to obtain scale reconstruction features at the scale; and determining a nodule probability map from the scale reconstruction features of the highest scale. The method can identify nodules with irregular edges and different sizes, and improves the segmentation accuracy.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, specifically relating to a method, system, computer equipment, and medium for thyroid nodule image segmentation. Background Technology

[0002] Thyroid nodules are a common clinical problem detected by ultrasound, with approximately 50% to 66% of adults having at least one nodule. About 5% of these nodules are malignant. Delayed diagnosis can lead to extracapsular invasion or metastasis, making timely identification of high-risk cases crucial. Ultrasound examination offers advantages such as real-time imaging, low cost, non-invasiveness, and no ionizing radiation, making it a commonly used and important imaging modality in clinical practice. In clinical practice, physicians typically assess malignancy risk based on the boundary characteristics of nodules in ultrasound images, such as manually annotating each frame of grayscale ultrasound images to identify the boundaries of thyroid nodules. However, this manual method is not only time-consuming but also susceptible to subjective factors, resulting in poor consistency in judgment.

[0003] To address the aforementioned shortcomings, existing technologies utilize computer-aided automatic boundary segmentation to achieve more objective volume measurement, reducing unnecessary punctures of stable benign nodules and providing a consistent quantitative benchmark for surgical planning and AI-driven risk stratification. For example, structural improvements such as dense skip connections and attention modules are introduced into variant models like U-Net, U-Net++, and Attention U-Net to enhance segmentation performance. However, this thyroid nodule image segmentation method fails to consider the significant differences in size and shape among actual human thyroid nodules, ranging from lesions smaller than one centimeter to highly irregular multinodular goiters. Such diverse nodules are difficult to accurately identify, resulting in low image segmentation accuracy. Summary of the Invention

[0004] To address the issue of low accuracy in existing thyroid nodule image segmentation methods, this invention provides a thyroid nodule image segmentation method, system, computer equipment, and medium.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for segmenting thyroid nodule images, comprising: Acquire ultrasound images of the thyroid nodules to be segmented, generate contrast-enhanced images and normalized gradient amplitude images from the ultrasound images of the thyroid nodules to be segmented, and determine the three-modal tensors based on the ultrasound images of the thyroid nodules to be segmented, the contrast-enhanced images and the normalized gradient amplitude images. The modal tensor is sequentially subjected to feature extraction at different resolutions to obtain feature extraction results. For each resolution, the feature extraction results are then processed in parallel through different receptive fields to obtain multiple context features at different scales. These multiple context features at different scales are fused, and the context features are optimized based on a channel attention mechanism to obtain channel attention-optimized features. Finally, the channel attention-optimized features are further optimized based on a spatial attention mechanism to obtain spatial attention-optimized features. The channel attention weights are determined by the channel descriptors corresponding to the context features at each scale based on a compression-excitation mechanism, while the spatial attention weights are determined by concatenating the channel attention-optimized features after average pooling and max pooling, respectively. The spatial attention-optimized features are fused with the feature extraction results to obtain the optimized feature extraction results; the optimized feature extraction results are used to determine the thyroid nodule probability map corresponding to the ultrasound image of the thyroid nodule to be segmented.

[0006] Optionally, the present invention provides a thyroid nodule image segmentation method that processes the ultrasound image of the thyroid nodule to be segmented through a trimodal edge-aware multi-scale network based on the U-net network architecture to obtain a thyroid nodule probability map. The skip connection layer in the trimodal edge-aware multi-scale network integrates a multi-focus cross-scale attention optimization module. The multi-focus cross-scale attention optimization module includes a parallel convolutional layer, a feature fusion layer, a dual-domain attention mechanism fusion layer, and a residual connection layer connected in sequence. The feature extraction results of each layer output by the encoder are processed by parallel convolutional layers to obtain feature maps of multiple different receptive fields of that layer. The feature fusion layer concatenates feature maps from multiple receptive fields along the channel dimension and fuses them using point convolution to obtain contextual features; The context features are calibrated by determining the corresponding channel weights from the context features through a dual-domain attention mechanism fusion layer, resulting in channel attention-optimized features. The spatial weights are then determined by the feature maps obtained from the channel attention-optimized features through average pooling and max pooling, respectively, to obtain spatial attention-optimized features. The feature extraction result optimized for this scale is obtained by fusing the spatial attention-optimized feature with the feature extraction result through the residual connection layer; the feature extraction result optimized for this scale is then passed to the decoder of the corresponding layer.

[0007] Optionally, the thyroid nodule image segmentation method provided by the present invention further includes: Block-limited histogram equalization was applied to redistribute gray levels in the ultrasound images of the thyroid nodules to be segmented, resulting in contrast-enhanced images. The gradient vector of each pixel is obtained by calculating the first-order finite difference in the ultrasound image of the thyroid nodule to be segmented, and the normalized gradient magnitude image is obtained by scaling the gradient vector of each pixel according to the Euclidean norm. The three-modal tensor is obtained by stacking the ultrasound image, contrast-enhanced image, and normalized gradient amplitude image of the thyroid nodule to be segmented.

[0008] Optionally, the trimodal tensor is an edge-enhanced trimodal tensor. The thyroid nodule image segmentation method provided by this invention further includes: The gray-level components are extracted from the three-modal tensor. The horizontal and vertical gradients are determined by the gray-level components using the Sobel kernel. The sum of the absolute values ​​of the horizontal and vertical gradients is used to determine the amplitude map of the salient candidate boundary. The candidate boundary magnitude map is convolved and activated by the Sigmoid function to obtain the mask; Multiply the mask by the trimodal tensor, and add the product to the trimodal tensor to obtain the edge-enhanced trimodal tensor.

[0009] Optionally, the thyroid nodule image segmentation method provided by the present invention further includes: Global average pooling is applied to the context features to obtain the channel descriptors; The corresponding channel weights are determined by the channel descriptor based on a compression-excitation mechanism. The context features are labeled based on the channel weights to obtain the channel attention-optimized features.

[0010] Optionally, the thyroid nodule image segmentation method provided by the present invention further includes: The channel attention-optimized features are then subjected to average pooling and max pooling in parallel to obtain average pooling feature maps and max pooling feature maps, respectively. The average pooling feature map and the max pooling feature map are concatenated and filtered to obtain the concatenated result; The spatial weights are obtained by processing the concatenation results using the Sigmoid function. The features optimized by channel attention are further optimized based on spatial weights to obtain features optimized by spatial attention.

[0011] Optionally, the thyroid nodule image segmentation method provided by the present invention further includes: Acquire training data for thyroid nodule ultrasound images; The training data of thyroid nodule ultrasound images were input into an untrained trimodal edge-aware multiscale network to obtain a predicted probability map. The binary cross-entropy loss function and the Dice loss function were calculated separately from the true probabilities corresponding to the predicted probability map and the ultrasound image training data of thyroid nodules to obtain the binary cross-entropy loss value and the Dice loss value; the total loss was determined by weighting the binary cross-entropy loss value and the Dice loss value. The parameters of the untrained trimodal edge-aware multi-scale network are optimized based on the total loss to obtain the trimodal edge-aware multi-scale network.

[0012] The present invention also provides a thyroid nodule image segmentation system, comprising: The modal tensor construction module is used to acquire ultrasound images of thyroid nodules to be segmented, generate contrast-enhanced images and normalized gradient amplitude images from the ultrasound images of thyroid nodules to be segmented, and determine the three-modal tensors based on the ultrasound images of thyroid nodules to be segmented, contrast-enhanced images and normalized gradient amplitude images. The attention enhancement module is used to sequentially extract features from the modal tensor at different resolutions to obtain feature extraction results. For each resolution, the feature extraction results are then extracted in parallel through different receptive fields to obtain multiple context features at different scales. These multiple context features at different scales are fused, and the context features are optimized based on a channel attention mechanism to obtain channel attention-optimized features. Finally, the channel attention-optimized features are further optimized based on a spatial attention mechanism to obtain spatial attention-optimized features. The channel attention weights are determined by the channel descriptors corresponding to the context features at each scale based on a compression-excitation mechanism, while the spatial attention weights are determined by concatenating the channel attention-optimized features after average pooling and max pooling, respectively. The probability map generation module is used to fuse the spatial attention-optimized features with the feature extraction results to obtain optimized feature extraction results; the optimized feature extraction results are used to determine the probability map of the thyroid nodule corresponding to the ultrasound image of the thyroid nodule to be segmented.

[0013] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in a thyroid nodule image segmentation method.

[0014] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any step of a thyroid nodule image segmentation method.

[0015] The thyroid nodule image segmentation method provided by this invention has the following beneficial effects: This invention acquires the ultrasound image to be segmented and converts it into a multimodal tensor. Then, it sequentially extracts features from this tensor at different resolutions to obtain a multi-level feature representation. For each resolution feature result, feature extraction operations with different receptive fields are performed in parallel to generate contextual features at multiple scales. These features are then fused to enrich the contextual information. Next, channel attention weights are calculated based on a compression-excitation mechanism to optimize the channel dimension of the fused features. Spatial attention weights are then generated by concatenating average pooling and max pooling to optimize the spatial dimension. The optimized features are fused with the original feature extraction results to form an enhanced feature representation. This enhanced representation is further merged with reconstructed features at lower resolution scales. Resolution is gradually restored through scale reconstruction operations, and finally, a nodule probability map is output from the highest-scale reconstructed features.

[0016] This invention introduces a trimodal input representation consisting of the original grayscale image, an enhanced contrast image, and a gradient magnitude image, providing complementary intensity, contrast, and gradient information to the network, thereby enriching the feature cues available during the encoding stage. Based on this, by sequentially extracting features at different resolutions, the method captures a complete information hierarchy from local details to global context. Furthermore, it uses different receptive fields in parallel for further feature extraction at each resolution, ensuring that the extracted features can adapt to the vast differences in nodule size from millimeters to centimeters, simultaneously perceiving the fine structure of small nodules and the overall morphology of large nodules. This avoids the limitations of existing image segmentation methods based on fixed receptive field models that struggle to handle multi-scale targets. Moreover, channel attention filters feature channels strongly correlated with nodules, while spatial attention focuses on key regions such as boundaries, further enhancing robustness to diverse nodules. This enables accurate identification of thyroid nodules with irregular edges and varying sizes, thereby improving the accuracy of image segmentation. Attached Figure Description

[0017] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of a thyroid nodule image segmentation method provided in an embodiment of the present invention; Figure 2 This is an example of the neural network model structure for thyroid nodule image segmentation provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the thyroid nodule image segmentation process provided in an embodiment of the present invention; Figure 4This is a schematic diagram of the edge guidance feature enhancement module processing flow provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the multi-focus cross-scale attention optimization module processing flow provided in an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0020] Early ultrasound nodule segmentation relied on machine learning methods based on hand-extracted texture features, such as degree co-occurrence matrix (GLCM), local binary pattern (LBP), and Gabor filters, combined with shallow classifiers or models like support vector machines (SVM), k-nearest neighbors (KNN), or active contour models for boundary delineation. For example, statistical descriptors like energy, contrast, and entropy were extracted from GLCM, and SVM was used to classify nodules as benign or malignant. However, these hand-extracted feature-based models are highly sensitive to speckle noise and scanner gain variations, leading to significant performance degradation when applied across different ultrasound devices. Furthermore, traditional workflows typically separate feature extraction from classification or segmentation tasks, failing to achieve end-to-end joint optimization. This lack of holistic learning capability makes these methods perform poorly in cases of ambiguous boundary definitions or low signal strength.

[0021] With the development of convolutional encoder-decoder networks, thyroid nodule segmentation has gradually shifted towards deep learning methods, achieving significant performance improvements. Researchers have proposed Transformer-based architectures that model long-distance dependencies through self-attention mechanisms. Hybrid frameworks such as Enhanced-TransUNet and UCTransNet integrate Transformer blocks into the backbone structure of U-Net, thereby utilizing both local and global features. Recent methods such as MLMSeg further enhance scale adaptability by combining convolutional features, Transformer-based global modeling, and graph structure information.

[0022] However, these deep learning-based ultrasound image thyroid nodule segmentation methods are hampered by the inherently low tissue contrast in ultrasound images, resulting in weak grayscale differences between the nodule and its surrounding parenchyma, and poor perception of intensity information by convolutional filters. Furthermore, thyroid nodule boundaries are often blurred, and coherent wave imaging introduces speckle noise. Malignant nodules often have serrated or invasive edges, which tend to gradually merge into the surrounding tissue, further reducing the reliability of the boundaries. In addition, there are significant differences in the size and shape of thyroid nodules, making it difficult for networks with fixed receptive fields to simultaneously take into account fine structural details and global contextual information.

[0023] To address the aforementioned shortcomings, the thyroid nodule image segmentation method provided by this invention introduces a trimodal input representation consisting of the original grayscale image, an enhanced contrast image, and a gradient magnitude image. This provides the network with complementary intensity, contrast, and gradient information, thereby enriching the feature cues available during the encoding stage and enhancing the network's ability to distinguish low-contrast regions. Furthermore, this invention applies an edge-guided feature enhancement module before the first downsampling to selectively enhance boundary responses, which is particularly suitable for regions with blurred or unclear boundaries. In addition, this invention embeds a multi-scale attention optimization module in each level of skip connections to promote the aggregation of contextual features at different spatial resolutions, further improving the accuracy of thyroid nodule image segmentation.

[0024] Example 1 This invention provides a method for thyroid nodule image segmentation, specifically as follows: Figure 1 As shown, it includes the following steps: Step 11: Obtain training data for thyroid nodule ultrasound images; input the training data for thyroid nodule ultrasound images into an untrained trimodal edge-aware multi-scale network to obtain a predicted probability map; calculate the binary cross-entropy loss function and the Dice loss function based on the predicted probability map and the true probabilities corresponding to the thyroid nodule ultrasound image training data, respectively, to obtain the binary cross-entropy loss value and the Dice loss value; weight the binary cross-entropy loss value and the Dice loss value to determine the total loss; optimize the parameters of the untrained trimodal edge-aware multi-scale network based on the total loss to obtain the trimodal edge-aware multi-scale network.

[0025] Specifically, during the model training phase of the trimodal edge-aware multi-scale network, this invention employs a composite loss function consisting of Binary Cross-Entropy (BCE) loss and Dice loss to calculate the loss function, thereby ensuring that the trimodal edge-aware multi-scale network achieves optimal pixel-level classification accuracy and region-level structural alignment. The BCE loss term promotes probability convergence for each pixel and is suitable for images where the background dominates, while the Dice loss term alleviates class imbalance and penalizes mismatches in overall region overlap.

[0026] For example, during model training, training data of thyroid nodule ultrasound images and the true probability of each pixel in the image belonging to a nodule are obtained. Then, the training data is input into the model to predict the probability, and the BCE loss term and Dice loss term are calculated from the predicted probability and the true probability. The BCE loss term and Dice loss term are shown in formula (1) and formula (2), respectively:

[0027] (1) (2) in, For BCE loss term, Here, i represents the pixel index, and N represents the total number of pixels after flattening the training sample mask. This represents the model's predicted probability that the i-th pixel belongs to the nodule region. Let i be the true binary mask label corresponding to the i-th pixel. For parameters used to ensure numerical stability, they can be set to, for example, as follows: The total loss is the weighted sum of the BCE loss term and the Dice loss term, as shown in formula (3):

[0028] (3) in, and All values ​​are fixed weights for the corresponding loss terms, for example, all are set to 1.0. Then, the network parameters are optimized based on this overall loss to obtain the trained trimodal edge-aware multi-scale network. Since the network can simultaneously optimize pixel-level classification accuracy and region-level overlap, it is suitable for thyroid ultrasound images where nodules occupy only a small portion of the field of view and often appear blurry or have indistinct boundaries. This ensures that the trained network can accurately identify the boundaries of thyroid nodules.

[0029] In addition, the training data for thyroid nodule ultrasound images can be self-constructed training datasets or publicly available benchmark datasets for thyroid nodule segmentation such as the TN3K dataset and the DDTI dataset. During training, a cosine annealing learning rate scheduler can be used to improve the convergence and generalization ability of the network model. Furthermore, in order to alleviate overfitting and enhance the robustness of the network model, data augmentation can be performed through random horizontal flipping, random vertical flipping, and random rotation.

[0030] Step 12: Obtain the ultrasound image of the thyroid nodule to be segmented. Generate a contrast-enhanced image and a normalized gradient amplitude image from the ultrasound image of the thyroid nodule to be segmented. Construct a three-modal tensor based on the ultrasound image, the contrast-enhanced image, and the normalized gradient amplitude image of the thyroid nodule to be segmented. For example... Figure 2 As shown, a trimodal edge-aware multi-scale network based on the U-net architecture is used to process ultrasound images of thyroid nodules to be segmented, resulting in a probability map of thyroid nodules. The skip connection layer in the trimodal edge-aware multi-scale network integrates a multi-focal cross-scale attention optimization module. This module comprises a parallel convolutional layer, a feature fusion layer, a dual-domain attention mechanism fusion layer, and a residual connection layer connected sequentially. The parallel convolutional layer processes the feature extraction results of each layer output by the encoder through parallel convolution, obtaining feature maps of multiple different receptive fields for that layer. The feature fusion layer concatenates these feature maps along the channel dimension and fuses them through point convolution to obtain contextual features. The dual-domain attention mechanism fusion layer determines the corresponding channel weights from the context features to calibrate the contextual features, resulting in channel attention-optimized features. The spatial weights determined by the feature maps after average pooling and max pooling of the channel attention-optimized features are used to calibrate the channel attention-optimized features, resulting in spatial attention-optimized features. The residual connection layer fuses the spatial attention-optimized features with the feature extraction results to obtain the scale-optimized feature extraction result. This scale-optimized feature extraction result is then passed to the decoder of the corresponding layer.

[0031] For example, a trimodal edge-aware multi-scale network includes an encoder-decoder backbone module, which comprises an encoder, a decoder, and a skip connection layer integrating a multifocal cross-scale attention optimization module. The encoder progressively downsamples the modal tensor to obtain feature extraction results at different resolutions after downsampling at each layer. The skip connection layer integrating the multifocal cross-scale attention optimization module processes the feature extraction results of each layer and passes the scale-optimized feature extraction results to the corresponding layer's decoder. The decoder of each layer merges the scale reconstruction features output by the previous layer's decoder with the scale-optimized feature extraction results and upsamples them to obtain the scale reconstruction features of that layer's decoder. The probability map of thyroid nodules is determined by the scale reconstruction features of the top layer.

[0032] Based on the aforementioned three-modal edge-aware multi-scale network, the image segmentation process is as follows: Figure 3 As shown, a network model based on the U-net network architecture is first constructed, and the network model is trained using a segmentation benchmark dataset to obtain a trained trimodal edge-aware multi-scale network. Then, for B-mode ultrasound images requiring image segmentation, a trimodal input tensor is generated after preprocessing. This trimodal input tensor is then input into the trained trimodal edge-aware multi-scale network to generate a nodule probability map. A binary segmentation mask is generated based on the nodule probability map, and the morphological features of the nodules are calculated.

[0033] The trimodal edge-aware multi-scale network further includes modules, and step 12 includes: Step 121: Apply block-limited histogram equalization to redistribute the grayscale of the ultrasound image of the thyroid nodule to be segmented, and obtain a contrast-enhanced image.

[0034] Step 122: Calculate the gradient vector of each pixel from the ultrasound image of the thyroid nodule to be segmented using the first-order finite difference method. Scale the gradient vector of each pixel using the Euclidean norm to obtain a normalized gradient magnitude image.

[0035] Step 123: Stack the ultrasound image, contrast-enhanced image, and normalized gradient amplitude image of the thyroid nodule to be segmented to obtain a three-modal tensor.

[0036] Specifically, using raw B-mode images of thyroid ultrasound. For example, a three-modal tensor or a three-channel tensor is first constructed, consisting of the original grayscale image, a contrast-enhanced image, and a gradient magnitude image. Furthermore, to ensure the uniformity of the model input, the original image can be normalized and resized, for example, to 512×512 pixels. Then, the original grayscale image reflecting the intrinsic intensity characteristics of the tissue, the contrast-enhanced image strengthening the brightness difference between low-echo regions and the background, and the gradient magnitude image providing edge structure and orientation information are determined.

[0037] For example, the first channel contains the raw ultrasound signal. This preserves all echo details; the second channel applies a block-limited histogram equalization method. For example, setting a limit value The size is 2.0, and the block size is... The value is set to 8, thereby redistributing gray levels within each local neighborhood, enhancing the brightness of low-echo areas and suppressing saturation, thus improving image contrast. As shown in formula (4): (4) The third channel encodes direction-independent edge energy. It generates a gradient magnitude map through first-order finite-difference calculations combined with Euclidean norm rescaling, thus covering the full dynamic range. As shown in formulas (5) and (6): (5) in, Represents the original grayscale image The gradient magnitude plot.

[0038] (6) and and They respectively represent the image domain Top Take the maximum and minimum values. Then, stack the three channels in an ordered manner to obtain the trimodal representation of the thyroid ultrasound image, i.e., the trimodal tensor, as shown in formula (7):

[0039] (7) Since the aforementioned three-modal tensors can simultaneously present the original intensity, locally enhanced contrast, and normalized gradient magnitude, they enrich the feature cues available for the first layer of convolution. On the one hand, this stabilizes the learning process of the early convolution kernel and accelerates model convergence without introducing additional trainable parameters. On the other hand, it improves the accuracy of thyroid nodule boundary recognition.

[0040] The trimodal edge-aware multi-scale network further includes modules, and step 12 includes: Step 124: Extract grayscale components from the trimodal tensor, determine the horizontal and vertical gradients from the grayscale components using the Sobel kernel, and determine the amplitude map of the salient candidate boundary by the sum of the absolute values ​​of the horizontal and vertical gradients.

[0041] Step 125: Convolve the amplitude map of the candidate boundary and activate it with the Sigmoid function to obtain the mask.

[0042] Step 126: Multiply the mask by the trimodal tensor, and add the product to the trimodal tensor to obtain the edge-enhanced trimodal tensor.

[0043] Specifically, such as Figure 4 As shown, after the trimodal tensor is constructed, the present invention can also perform edge-guided feature enhancement before the encoder's first downsampling, providing the encoder with noise-resistant boundary prior information.

[0044] First, for ultrasound images in the form of a three-modal tensor. Grayscale components are obtained by averaging the channels. and using a fixed Sobel core. and Extract the horizontal and vertical gradients, respectively, and... Perform convolution as shown in equations (8) and (9): (8) (9) in, For grayscale components, and These are the results in the horizontal and vertical directions after convolution, respectively. For two-dimensional convolution, use the Sobel kernel. , .

[0045] Then by and Calculate the magnitude map of prominent candidate boundaries As shown in formula (10): (10) Then, the candidate boundary magnitude map will be highlighted. The convolution process is performed using a 1×1 convolutional layer and then processed using the Sigmoid nonlinear activation function, as shown in Equation (11): (11) in, for Convolution kernel parameters of convolutional layers, The sigmoid function is used; the mask M is a continuous soft mask, with values ​​ranging from 1 to 10. .

[0046] Then, by the mask The trimodal tensor determines the trimodal tensor for edge-guided feature enhancement, as shown in Equation (12): (12) in, The Hadamard product or element-wise multiplication selectively amplifies activation values ​​located on strong edges while preserving the interior of the region almost unchanged, allowing subsequent convolutional filters to receive enhanced boundary features without introducing additional noise.

[0047] Step 13: Extract features from the modal tensor at different resolutions in sequence to obtain feature extraction results. For each resolution feature extraction result, extract features in parallel through different receptive fields to obtain multiple context features at different scales. Fuse the context features at multiple different scales and optimize them based on the channel attention mechanism to obtain channel attention-optimized features. Optimize the channel attention-optimized features based on the spatial attention mechanism to obtain spatial attention-optimized features. The channel attention weights are determined by the channel descriptors corresponding to the context features at each scale based on the compression-excitation mechanism. The spatial attention weights are determined by concatenating the channel attention-optimized features after average pooling and max pooling, respectively.

[0048] The optimization based on channel attention includes the following steps: Step 131: Perform global average pooling on the context features to obtain the channel descriptor.

[0049] Step 132: Determine the corresponding channel weights based on the channel descriptor using the compression-excitation mechanism.

[0050] Step 133: Label the context features according to the channel weights to obtain the channel attention-optimized features.

[0051] Step 134: Perform average pooling and max pooling on the channel attention-optimized features in parallel to obtain average pooling feature maps and max pooling feature maps, respectively.

[0052] Step 135: Concatenate the average pooling feature map and the max pooling feature map and filter them to obtain the concatenated result.

[0053] Step 136: Process the splicing results using the Sigmoid function to obtain the spatial weights.

[0054] Step 137: Optimize the channel attention-optimized features based on spatial weights to obtain spatial attention-optimized features.

[0055] Specifically, after edge enhancement, the encoder and decoder based on the U-net architecture perform downsampling and upsampling respectively. The feature extraction results of each downsampling layer or the jump connection tensor output by the encoder are passed to the decoder of the corresponding layer through the skip connection layer. The decoder of the corresponding layer then fuses the feature extraction results or encoder output results with the scale reconstruction features or decoder output results from the previous layer and performs upsampling.

[0056] Among them, such as Figure 5 As shown, the present invention performs multi-focus cross-scale attention optimization in each skip connection through a multi-focus cross-scale attention optimization module. It captures local and global contextual information through a series of expanded depthwise separable convolutions, and then adopts a dual-domain attention mechanism to treat each potential nodal region as an independent source of saliency for feature reweighting.

[0057] For example, the jump connection tensor output by the encoder Through a set of expansion rates Parallel applications Each depthwise separable convolutional branch is processed, as shown in formula (13): (13) in, For ReLU activation function, For having an expansion rate 3×3 depth separable convolution kernel, This is for context features. The various branches are then concatenated along the channel dimension and fused using a 1×1 point convolution to obtain the concatenated context features or context tensor. As shown in formula (14):

[0058] (14) Then, channel descriptors are obtained through global average pooling. As shown in formula (15): (15) in, For channel descriptor vectors, Let it be its c-th component; The context feature tensor output by equation (14), The element represents the position (i, j) in the channel c space.

[0059] Subsequently, in the Channel Attention module, the Squeeze-and-Excitation operation is used to calculate the channel weights. As shown in formula (16): (16) in, For GELU activation function, For the Sigmoid function, , These are the linear transformation weights for Squeeze and Excitation, respectively, where r is the compression ratio, for example, 8. Then, based on the channel weights... For feature maps Perform channel recalibration as shown in formula (17):

[0060] (17) in, The features are optimized for channel attention. Then, in the spatial attention module, the feature maps after average pooling and max pooling are concatenated and filtered through a convolutional layer with a large kernel, such as a 7×7 kernel, as shown in formula (18):

[0061] (18) in, The spatial attention weights are used. The optimized skip connection tensor is finally obtained, as shown in equation (19):

[0062] (19) in, The optimized jump connection tensor or the feature optimized by spatial attention can be represented in residual form.

[0063] For example, the skip connection tensor output by the encoder is processed in parallel by four depthwise separable convolutions (Dconv) with dilation rates d of 1, 4, 8 and 12, combined with batch normalization (BN) and ReLU activation functions. After that, each branch is processed by 1×1 convolution combined with BN and ReLU activation functions to obtain the features after processing each branch. These features are then concatenated and fused, and optimized sequentially based on channel attention and spatial attention mechanisms. Finally, they are added element-wise to the skip connection tensor to obtain the final optimized features.

[0064] Step 14: Fuse the spatial attention-optimized features with the feature extraction results to obtain the scale-optimized feature extraction results; fuse the scale-optimized feature extraction results with the scale reconstruction features of a lower scale and perform scale reconstruction to obtain the scale reconstruction features of that scale; determine the thyroid nodule probability map corresponding to the ultrasound image of the thyroid nodule to be segmented based on the highest scale reconstruction features.

[0065] Specifically, when the model outputs a nodule probability map of the same size as the input image, a binary segmentation mask can be obtained after thresholding. Based on this mask, key morphological features of the nodule, such as area, perimeter, aspect ratio, and edge sharpness, can be automatically calculated. Furthermore, the binary segmentation mask can be further smoothed, pseudo-color mapped, or overlaid with the original image for quick verification by doctors.

[0066] Example 2 The present invention also provides a thyroid nodule image segmentation system, comprising: The modal tensor construction module is used to acquire ultrasound images of thyroid nodules to be segmented, generate contrast-enhanced images and normalized gradient amplitude images from the ultrasound images of thyroid nodules to be segmented, and construct a three-modal tensor based on the ultrasound images of thyroid nodules to be segmented, contrast-enhanced images and normalized gradient amplitude images. The attention enhancement module is used to sequentially extract features from the modal tensor at different resolutions to obtain feature extraction results. The feature extraction results at each resolution are then extracted in parallel through different receptive fields to obtain multiple context features at different scales. These multiple context features at different scales are fused and optimized based on a channel attention mechanism to obtain channel attention-optimized features. The channel attention-optimized features are then optimized based on a spatial attention mechanism to obtain spatial attention-optimized features. The weights of the channel attention are determined by the channel descriptors corresponding to the context features at each scale based on a compression-excitation mechanism, while the weights of the spatial attention mechanism are determined by concatenating the channel attention-optimized features after average pooling and max pooling, respectively. The probability map generation module is used to fuse the spatial attention-optimized features with the feature extraction results to obtain optimized feature extraction results; the optimized feature extraction results are used to determine the probability map of the thyroid nodule corresponding to the ultrasound image of the thyroid nodule to be segmented.

[0067] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in an embodiment of a thyroid nodule image segmentation method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0068] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in an embodiment of a thyroid nodule image segmentation method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0069] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0071] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0072] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0073] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for segmenting thyroid nodule images, characterized in that, include: An ultrasound image of the thyroid nodule to be segmented is acquired, and a contrast-enhanced image and a normalized gradient amplitude image are generated from the ultrasound image of the thyroid nodule to be segmented. A three-modal tensor is constructed based on the ultrasound image of the thyroid nodule to be segmented, the contrast-enhanced image, and the normalized gradient amplitude image. The modal tensor is subjected to feature extraction at different resolutions in sequence to obtain feature extraction results; The feature extraction results at each resolution are extracted in parallel through different receptive fields to obtain multiple context features at different scales. The multiple context features at different scales are fused and optimized based on a channel attention mechanism to obtain channel attention-optimized features. The channel attention-optimized features are optimized based on the spatial attention mechanism to obtain spatial attention-optimized features. The weights of the channel attention are determined by the channel descriptors corresponding to the context features at each scale based on the compression-excitation mechanism. The weights of the spatial attention mechanism are determined by concatenating the channel attention-optimized features after average pooling and max pooling, respectively. The spatial attention-optimized features are fused with the feature extraction results to obtain optimized feature extraction results; the optimized feature extraction results are used to determine the thyroid nodule probability map corresponding to the ultrasound image of the thyroid nodule to be segmented.

2. The method for segmenting thyroid nodule images according to claim 1, characterized in that, The ultrasound image of the thyroid nodule to be segmented is processed by a trimodal edge-aware multi-scale network based on the U-net network architecture to obtain a probability map of the thyroid nodule. The skip connection layer in the trimodal edge-aware multi-scale network integrates a multi-focus cross-scale attention optimization module. The multi-focus cross-scale attention optimization module includes a parallel convolutional layer, a feature fusion layer, a dual-domain attention mechanism fusion layer, and a residual connection layer connected in sequence. The feature extraction results of each layer output by the encoder are processed by parallel convolution of the parallel convolutional layer to obtain feature maps of multiple different receptive fields of that layer. The feature fusion layer concatenates the feature maps of multiple different receptive fields along the channel dimension and fuses them through point convolution to obtain contextual features. The context features are calibrated by determining the corresponding channel weights from the context features through the dual-domain attention mechanism fusion layer, resulting in channel attention-optimized features; the spatial weights determined by the feature maps after average pooling and max pooling of the channel attention-optimized features are then used to calibrate the channel attention-optimized features, resulting in spatial attention-optimized features. The feature extraction result optimized for this scale is obtained by fusing the spatial attention-optimized features with the feature extraction result through the residual connection layer; The feature extraction result optimized at this scale is passed to the decoder of the corresponding layer.

3. The method for segmenting thyroid nodule images according to claim 2, characterized in that, A contrast-enhanced image and a normalized gradient amplitude image are generated from the ultrasound image of the thyroid nodule to be segmented. A three-modal tensor is constructed based on the ultrasound image, the contrast-enhanced image, and the normalized gradient amplitude image of the thyroid nodule to be segmented, including: The grayscale of the ultrasound image of the thyroid nodule to be segmented was redistributed by applying block-limited histogram equalization to obtain a contrast-enhanced image. The gradient vector of each pixel is obtained by calculating the first-order finite difference of the ultrasound image of the thyroid nodule to be segmented, and the normalized gradient magnitude image is obtained by scaling the gradient vector of the pixel using the Euclidean norm. The three-modal tensor is obtained by stacking the ultrasound image, contrast-enhanced image, and normalized gradient amplitude image of the thyroid nodule to be segmented.

4. The method for segmenting thyroid nodule images according to claim 3, characterized in that, The trimodal tensor is an edge-enhanced trimodal tensor. After stacking the ultrasound image, contrast-enhanced image, and normalized gradient amplitude image of the thyroid nodule to be segmented to obtain the trimodal tensor, the method further includes: Gray-level components are extracted from the three-modal tensor. The horizontal and vertical gradients are determined from the gray-level components using the Sobel kernel. The sum of the absolute values ​​of the horizontal and vertical gradients is used to determine the amplitude map of the salient candidate boundary. The prominent candidate boundary magnitude map is convolved and activated by the Sigmoid function to obtain the mask; Multiply the mask by the trimodal tensor, and add the product to the trimodal tensor to obtain the edge-enhanced trimodal tensor.

5. The method for segmenting thyroid nodule images according to claim 2, characterized in that, The context features are used to determine the corresponding channel weights, and the context features are then labeled to obtain the channel attention-optimized features, including: Global average pooling is applied to the context features to obtain the channel descriptors; The corresponding channel weights are determined by the channel descriptors based on a compression-excitation mechanism. The context features are labeled according to the channel weights to obtain the channel attention-optimized features.

6. The method for segmenting thyroid nodule images according to claim 2, characterized in that, The spatial weights determined by the feature maps after average pooling and max pooling of the channel attention-optimized features are used to calibrate the channel attention-optimized features to obtain spatial attention-optimized features; including: The channel attention-optimized features are then subjected to average pooling and max pooling in parallel to obtain average pooling feature maps and max pooling feature maps, respectively. The average pooling feature map and the max pooling feature map are concatenated and filtered to obtain the concatenated result; The splicing result is processed by the Sigmoid function to obtain the spatial weights; The channel attention-optimized features are then optimized based on spatial weights to obtain spatial attention-optimized features.

7. The method for segmenting thyroid nodule images according to claim 2, characterized in that, Before processing the ultrasound images of thyroid nodules to be segmented using a trimodal edge-aware multi-scale network based on the U-net network architecture, the following steps are also included: Acquire training data for thyroid nodule ultrasound images; The training data of the ultrasound images of the thyroid nodules were input into an untrained trimodal edge-aware multiscale network to obtain a predicted probability map. The binary cross-entropy loss function and the Dice loss function are calculated respectively from the true probabilities corresponding to the predicted probability map and the thyroid nodule ultrasound image training data to obtain the binary cross-entropy loss value and the Dice loss value; the total loss is determined by weighting the binary cross-entropy loss value and the Dice loss value. The parameters of the untrained trimodal edge-aware multi-scale network are optimized based on the total loss to obtain the trimodal edge-aware multi-scale network.

8. A thyroid nodule image segmentation system, characterized in that, include: The modal tensor construction module is used to acquire ultrasound images of thyroid nodules to be segmented, generate contrast-enhanced images and normalized gradient amplitude images from the ultrasound images of thyroid nodules to be segmented, and construct a three-modal tensor based on the ultrasound images of thyroid nodules to be segmented, contrast-enhanced images and normalized gradient amplitude images. An attention enhancement module is used to sequentially extract features from the modal tensor at different resolutions to obtain feature extraction results. The feature extraction results at each resolution are extracted in parallel through different receptive fields to obtain multiple context features at different scales. The multiple context features at different scales are fused and optimized based on a channel attention mechanism to obtain channel attention-optimized features. The channel attention-optimized features are optimized based on the spatial attention mechanism to obtain spatial attention-optimized features. The weights of the channel attention are determined by the channel descriptors corresponding to the context features at each scale based on the compression-excitation mechanism. The weights of the spatial attention mechanism are determined by concatenating the channel attention-optimized features after average pooling and max pooling, respectively. The probability map generation module is used to fuse the spatial attention-optimized features with the feature extraction results to obtain optimized feature extraction results; and to determine the thyroid nodule probability map corresponding to the ultrasound image of the thyroid nodule to be segmented based on the optimized feature extraction results.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the thyroid nodule image segmentation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the thyroid nodule image segmentation method according to any one of claims 1 to 7.

Images