Enhanced UNet + + nodule ultrasound image segmentation method based on double-path attention mechanism

Abstract

The invention provides an enhanced UNet + + nodule ultrasound image segmentation method based on a dual-path attention mechanism, the method is based on a DPAM-UNet + + network model, and the DPAM-UNet + + network model comprises an encoder, a decoder, a jump connection and a DPAM module integrated into the jump connection; the method comprises the following steps: step 1, an encoder extracts features in an ultrasonic image to obtain an input feature F1; step 2, obtaining an enhanced feature map after the input feature F1 passes through a DPAM module; 3, performing feature splicing on the enhanced feature map and the input feature F1 to obtain an initial fusion feature map; and step 4, a decoder performs feature recovery on the initial fusion feature map to obtain a recovered feature map, and performs feature splicing on the recovered feature map and the initial fusion feature map to form a final fusion feature map. According to the method, the dual-path attention module is introduced and integrated into the jump connection of the UNet + + network, and meanwhile, a novel integrated loss function is introduced, so that the nodule segmentation precision is improved.

Description

Technical Field

[0001] The present application relates to the technical field of image segmentation, and in particular to a nodule ultrasound image segmentation method based on a dual-path attention mechanism enhanced UNet++. Background Art

[0002] A nodule is a palpable localized solid lesion, usually including thyroid nodules, lung nodules, and breast nodules. The formation of nodules can be inflammatory or non-inflammatory and can involve the epidermis, dermis, or subcutaneous tissue. Although most nodules are benign, some nodules will develop into cancer. Therefore, early detection and treatment of nodules are crucial to reduce the potential risk of cancer and avoid unnecessary biopsies and surgeries.

[0003] Ultrasound examination is widely used for disease detection because of its simple operation, real-time efficiency and low cost. In ultrasound images, clinicians diagnose the benign or malignant nature of nodules based on solidity, echogenicity, calcification, shape and aspect ratio. Ultrasound images usually have low resolution and contrast, and are often plagued by speckle noise and echo interference. In addition, nodules are surrounded by structures such as trachea, blood vessels and muscles, and the boundaries between nodules and these adjacent tissues are more blurred due to artifacts and speckles. The appearance, shape and size of nodules vary, which increases the complexity of ultrasound image segmentation. Moreover, the annotation of nodules is very time-consuming and requires the expertise of experienced clinicians, which increases the time cost. Therefore, it is crucial to develop an automatic segmentation method that can not only improve segmentation accuracy but also significantly reduce the workload of medical staff.

[0004] The development of modern image processing technology has promoted the application of computer-aided diagnosis in the field of ultrasound diagnosis of thyroid nodules, reduced the workload of doctors, and improved the objectivity of diagnosis. Traditional medical image segmentation methods mostly use image processing techniques such as threshold processing, edge detection, and morphology. Senthilkumaran et al. proposed a method for local adaptive threshold segmentation of medical images. Szenasi et al. used a distributed region growing method to segment the main image in colon tissue samples. Singh et al. cropped the region of interest from the input color fundus image and then applied a series of steps including adaptive median filtering, histogram equalization, and morphological processing. Finally, the maximum stable extreme region (MSER) ​​method was used to achieve the segmentation of the optic disc and cup. Deep learning can significantly improve diagnostic accuracy and efficiency, so it has been widely used in the field of ultrasound image segmentation. Yang et al. combined the Res-U-net framework with the prior level set method to improve the accuracy of thyroid tumor segmentation, and then trained the classification network by sharing the shallow layer of the segmentation network, and finally achieved the classification of different thyroid tumors. Nie et al. proposed a fully convolutional neural network that combines a multi-scale input layer and an attention-guided module. They used an attention-guided module to filter features, remove noise, and reduce the negative impact of the background. Yadav et al. used a speckle reduction filter when processing thyroid ultrasound images and then used Hybrid-UNet for segmentation. Experimental results show that this method can effectively segment thyroid nodules and cystic components. Chen et al. used a deep convolutional neural network to encode local view features. They designed a unique graph convolution component that dynamically captures the interaction between high-level and low-level features by building a graph between multiple layers. In addition, to enhance feature fusion, they adopted a channel-aware graph attention mechanism to effectively fuse multi-source feature information to improve the accuracy of nodule segmentation.

[0005] However, traditional medical image segmentation methods are susceptible to image noise and intensity variations, resulting in over-segmentation or under-segmentation. Some deep learning segmentation techniques mainly focus on local feature extraction and lack the ability to capture and utilize the contextual information of the entire input image. For example, Sun et al. combined the position attention module and the channel attention module to propose a dual attention module and integrated it into the skip connection of the TransUNet network, which enhanced the model's ability to capture segmentation details. However, the spatial attention mechanism was not incorporated into the skip connection. Cui et al. introduced a dual dilated gated attention module by incorporating the dilated spatial and channel gated attention mechanisms into the skip connection of the UNet network, achieving satisfactory segmentation results, but did not incorporate global context information. Similarly, Luo et al. combined adaptive spatial attention and channel attention to create an adaptive dual-path attention module integrated with UNet, but they also did not utilize global context. Therefore, it is challenging to effectively integrate global context information. In addition, it is also difficult to capture long-range dependencies from the perspective of the entire image, resulting in the neglect of important scattered features in medical images. Summary of the invention

[0006] In view of this, the present application provides a nodule ultrasound image segmentation method based on a dual-path attention mechanism enhanced UNet++ to capture the key information in the feature map from the global, channel and spatial perspectives and improve the segmentation accuracy.

[0007] To achieve the above objectives, the technical solutions adopted in this application are as follows:

[0008] A nodule ultrasound image segmentation method based on dual-path attention mechanism enhanced UNet++, the method is based on a dual-path attention mechanism (DPAM)-UNet++ network model, the DPAM-UNet++ network model includes an encoder, a decoder, a skip connection and a DPAM module integrated into the skip connection; the method includes the following steps:

[0009] Step 1: The encoder extracts features from the ultrasound image to obtain input features F1;

[0010] Step 2, the input feature F1 is passed through the DPAM module to obtain an enhanced feature map;

[0011] Step 3: Concatenate the enhanced feature map with the input feature F1 to obtain an initial fused feature map, and pass the initial fused feature map to the decoder through a skip connection;

[0012] Step 4: The decoder performs feature recovery on the initial fused feature map to obtain a restored feature map, and then performs feature concatenation on the restored feature map and the initial fused feature map to form a final fused feature map.

[0013] Furthermore, the DPAM module includes a global attention module (GAM) and a global context module (GCBlock, GC module), and the enhanced feature map obtained by the input feature F1 in step 2 after passing through the DPAM module specifically includes:

[0014] Step 2.1, input the input feature F1 into the GAM and GC modules respectively;

[0015] Step 2.2, GAM extracts channel and spatial features, and the GC module captures global context information;

[0016] Step 2.3: The outputs of the GC module and GAM are connected with the original input feature F1 to jointly generate an enhanced feature map, which contains high-resolution features.

[0017] Furthermore, the GAM includes an efficient channel attention (ECA) submodule and a spatial attention (SA) submodule; the GAM extracting channel and spatial features in step 2.2 specifically includes:

[0018] The original input feature F1 first passes through the ECA submodule to obtain Mc(F1); then, F1 and Mc(F1) are element-wise multiplied to obtain feature F2, and feature F2 is input to the SA submodule to obtain Ms(F2); finally, Mc(F1) and Ms(F2) are element-wise multiplied to obtain the final output feature F3.

[0019] Furthermore, the feature F2 is input into the SA submodule to obtain Ms(F2) as follows:

[0020] The input feature F2 is first convolved with a 7x7 kernel to reduce the number of channels from C to C / r; then, the feature map output after the convolution passes through a sigmoid activation function to generate the spatial attention map Ms(F2).

[0021] Furthermore, in step 2.2, the GC module captures the global context information specifically including:

[0022] The GC module first uses a 1x1 convolutional layer and a Softmax function to model the context of the input feature F1; then, the context information is multiplied by the input feature F1 in element order, and the result of the multiplication is transformed through another 1x1 convolutional layer, LayerNorm, and ReLU functions respectively; the transformed features are added to the input feature F1 in element order to generate the final output features.

[0023] Furthermore, after step 4, the following steps are further included:

[0024] Step 5: Construct a comprehensive loss function to improve the edge segmentation accuracy of the DPAM-UNet++ model.

[0025] Furthermore, the comprehensive loss function is specifically:

[0026] L total =L bce +L dice +L lovasz +L ssim (1)

[0027] L in the equation bce , L dice , L lovasz and L ssim They represent BCELoss (Binary Cross Entropy Loss), DiceLoss, LovaszLoss and SSIMLoss (Structural Similarity IndexMeasure

[0028] Loss, structural similarity index measures loss);

[0029] The calculation formula of BCELoss is as follows:

[0030]

[0031] Among them, g i and i Represent the true label and model prediction respectively, and i represents each pixel in the image;

[0032] DiceLoss measures the similarity between the true label and the model prediction; the calculation formula of DiceLoss is:

[0033] L dice =1-2|X∩Y| / |X+Y| (3)

[0034] Where X and Y represent the number of elements in the true label and predicted image, respectively;

[0035] The calculation formula of LovaszLoss is:

[0036]

[0037] Among them, ΔJ i is the marginal contribution of pixel i to the Jaccard index, g i and i are the true value and the predicted value respectively;

[0038] The calculation formula of SSIMLoss is:

[0039] L ssim＝ (2μ x μ y +c 1 )(2σ xy +c 2 ) / (μ x 2 +μ y 2 +c 1 )(σ x 2 +σ y 2 +c 2 ) (5)

[0040] c1 and c2 are constants, x and y represent the actual image and predicted image respectively, μ, σ and σ xy They represent the mean, variance and covariance of the image respectively.

[0041] Compared with the prior art, the beneficial effects of this application are:

[0042] 1. A dual-path attention module is proposed and integrated into the skip connection of the UNet++ network, thereby improving the overall segmentation performance of the model. This dual-path attention mechanism consists of two paths: the first path uses the global attention mechanism to capture the information of the global context, and the second path combines spatial and channel attention to extract channel and spatial information from the feature map. The collaboration of these two pathways enables the model to effectively focus on key areas, thereby improving segmentation accuracy.

[0043] 2. A comprehensive loss function is designed to improve the segmentation effect. This loss function combines multiple loss components and strengthens the model's optimization of edge details in important feature maps by considering similarities, so as to enhance the model's robustness and adaptability to complex environments.

[0044] 3. The method of this application improves the accuracy and efficiency of nodule segmentation from both global and local perspectives, which helps to reduce misdiagnosis caused by limited doctor experience and reduce the doctor's workload. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1This is a flowchart of a nodule ultrasound image segmentation method based on a dual-path attention mechanism enhanced UNet++ for this application;

[0047] Figure 2 This is the structural diagram of the DPAM-UNet++ network model of this application;

[0048] Figure 3 This is the DPAM module structure diagram for this application;

[0049] Figure 4 (a) is the structure diagram of the global attention module (GAM) of this application; (b) is the structure diagram of the efficient channel attention (ECA) submodule of this application; (c) is the structure diagram of the spatial attention (SA) submodule of this application;

[0050] Figure 5 This is the structure diagram of the global context (GC) module of this application;

[0051] Figure 6 Segmentation renderings for different models;

[0052] Figure 7 It is the ROC (Receiver Operating Characteristic) curve of different segmentation models;

[0053] Figure 8 This is the segmentation result diagram of the attention module ablation experiment;

[0054] Fig. 9 Attention heatmaps of different attention modules. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0056] like Figure 1 and 2 As shown, a nodule ultrasound image segmentation method based on dual-path attention mechanism enhanced UNet++ is described. The method is based on a dual-path attention mechanism (DPAM)-UNet++ network model. The DPAM-UNet++ network model includes an encoder, a decoder, a skip connection, and a skip connection integrated into a specific one (such as Figure 2 The DPAM module in the embodiment of the present invention is shown by a red arrow in FIG. 1 ; the method comprises the following steps:

[0057] Step 1: The encoder extracts features from the ultrasound image to obtain input features F1;

[0058] The encoder uses ResNet34 as the backbone network. The input image is first processed by a 7×7 convolutional layer with a stride of 2, and then normalized and ReLU activated to generate the initial feature map x 0,0 After the maximum pooling process, the feature map passes through the four layers of ResNet34 in turn to generate the downsampled feature map x 1,0 、x 2,0 、x 3,0 and x 4,0 , thus completing the downsampling process.

[0059] Step 2, the input feature F1 is passed through the DPAM module to obtain an enhanced feature map;

[0060] As a further embodiment, Figure 3 As shown, the DPAM module includes a global attention module (GAM) and a global context module (GCBlock, GC module), and the input feature F1 in step 2 is subjected to the DPAM module to obtain an enhanced feature map, which specifically includes:

[0061] Step 2.1, input the input feature F1 into the GAM and GC modules respectively;

[0062] Step 2.2, GAM extracts channel and spatial features, and the GC module captures global context information;

[0063] Step 2.3: The outputs of the GC module and GAM are connected with the original input feature F1 to jointly generate an enhanced feature map, which contains high-resolution features.

[0064] In order to optimize computational efficiency and enhance the adaptability of the model to different applications, the Efficient Channel Attention (ECA) module replaces the Channel Attention (CA) module in the traditional GAM. Figure 4 As shown in (a), the GAM includes an efficient channel attention (ECA) submodule and a spatial attention (SA) submodule; in the step 2.2, the GAM extracts channel and spatial features specifically including: the original input feature F1 first passes through the ECA submodule to obtain Mc(F1); then, F1 and Mc(F1) are element-wise multiplied to obtain feature F2, and feature F2 is input into the SA submodule to obtain Ms(F2); finally, Mc(F1) and Ms(F2) are multiplied element-wise to obtain the final output feature F3.

[0065] Figure 4 The GAM structure shown in (a) enables the GAM module to capture and utilize key image features more effectively, thereby improving the overall image segmentation performance.

[0066] like Figure 4As shown in (b), the focus of the ECA submodule is to improve the channel feature representation. The input feature F1 is subjected to global average pooling (GAP) to generate a channel descriptor. This descriptor is passed through a neural network with a single hidden layer to generate an efficient channel attention map Mc(F1).

[0067] Figure 4 The SA submodule shown in (c) is designed to capture the spatial dependencies in the feature map. The input feature F2 is first convolved with a 7x7 kernel to reduce the number of channels from C to C / r; it is then convolved with a 7x7 kernel to restore the number of channels to C. Then, the feature map output after convolution is passed through a sigmoid activation function to generate the spatial attention map Ms(F2). The activation function normalizes the weights, enhances the stability of the attention mechanism, and thus ensures the effectiveness of the attention module.

[0068] Figure 5 The structure of the GC module is shown. The GC module in step 2.2 captures global context information specifically as follows: the GC module first uses a 1x1 convolution layer and a Softmax function to model the context of the input feature; then, the context information is multiplied by the input feature in element order, and then converted by another 1x1 convolution layer, LayerNorm, and ReLU function respectively; the converted features are added to the input features in element order to generate the final output features. The GC module enhances the representation capability of the feature map by combining the global context, enabling the model to perform well in complex scenarios.

[0069] By combining the output of GAM and GCBlock with the original feature map, the DPAM module creates a new feature map rich in context and attention information. This approach improves segmentation accuracy and enhances the representation ability of feature maps, making them more suitable for various complex segmentation tasks. In summary, the DPAM module is crucial for optimizing feature map representation and improving segmentation performance. The pseudocodes of the GAM and GC attention modules are shown in Algorithms 1 and 2, respectively.

[0070] Algorithm 1 GAM

[0071]

[0072] Algorithm 2 GCBlock

[0073]

[0074]

[0075] Step 3: Concatenate the enhanced feature map with the input feature F1 to obtain an initial fused feature map, and pass the initial fused feature map to the decoder through a skip connection;

[0076] Step 4: The decoder performs feature recovery on the initial fused feature map to obtain a restored feature map, and then performs feature concatenation on the restored feature map and the initial fused feature map to form a final fused feature map.

[0077] After the encoder extracts features, the decoder restores the spatial resolution of the feature map. This is done by upsampling the downsampled feature map using nearest neighbor interpolation and then connecting it to the encoder’s high-resolution feature map via jump connections. The decoder module then processes the merged feature map, applying a series of convolutional layers, normalization, and ReLU activations to produce the decoded output feature map.

[0078] After decoding, the feature map x 0,1 、x 0,2 、x 0,3 and x 0,4 After convolutions of different scales, the corresponding output feature maps (logit1, logit2, logit3, and logit4) are generated. These maps capture multi-scale semantic information, allowing the network to effectively balance local and global features. The network then merges the output feature maps through weighted summation and concatenates them with the initial fused features to generate the final fused feature map logit. Subsequently, the logit is upsampled to the size of the input image using bilinear interpolation to ensure that the spatial resolution of the output feature map matches the spatial resolution of the input image.

[0079] Figure 2 In the figure, the black dotted arrows represent jump connections, which transmit high-resolution features between the encoder and decoder to promote the accurate restoration of spatial information. The red dotted arrows represent the DPAM module, which integrates global attention, channel attention, and spatial attention mechanisms. In this way, DPAM enhances the network's ability to pay attention to global contextual information, achieves more accurate feature extraction, and improves the model's ability to accurately identify nodule regions.

[0080] As a further implementation, the step 4 further includes:

[0081] Step 5: Construct a comprehensive loss function to improve the edge segmentation accuracy of the DPAM-UNet++ model.

[0082] The comprehensive loss function combines the advantages of various loss functions to more comprehensively evaluate the performance of the model. This method enables the model to capture finer details in the image, thereby improving segmentation accuracy.

[0083] As a further implementation, the comprehensive loss function is specifically:

[0084] L total =L bce+L dice +L lovasz +L ssim (1)

[0085] L in the equation bce , L dice , L lovasz and L ssim They represent BCELoss (Binary Cross Entropy Loss), DiceLoss, LovaszLoss and SSIMLoss (Structural Similarity IndexMeasure Loss) respectively;

[0086] The calculation formula of BCELoss is as follows:

[0087]

[0088] Among them, g i and i Represent the true label and model prediction respectively, and i represents each pixel in the image;

[0089] DiceLoss measures the similarity between the true label and the model prediction; the calculation formula of DiceLoss is:

[0090] L dice =1-2|X∩Y| / |X+Y| (3)

[0091] Where X and Y represent the number of elements in the true label and predicted image, respectively;

[0092] LovaszLoss is designed specifically for image segmentation tasks and is particularly effective in dealing with class imbalance. Its main function is to optimize the Jaccard index (Intersection over Union, IoU), which is a key indicator for evaluating image segmentation performance. The calculation formula of LovaszLoss is:

[0093]

[0094] Among them, ΔJ i is the marginal contribution of pixel i to the Jaccard index, g i and i are the true value and the predicted value respectively;

[0095] SSIMLoss can help the segmentation model better preserve the structural information of the image. The calculation formula of SSIMLoss is:

[0096] Lssim =(2μ x μ y +c 1 )(2σ xy +c 2 ) / (μ x 2 +μ y 2 +c 1 )(σ x 2 +σ y 2 +c 2 ) (5)

[0097] c1 and c2 are constants, x and y represent the actual image and predicted image respectively, μ, σ and σ xy They represent the mean, variance and covariance of the image respectively.

[0098] The new composite loss function is composed of multiple loss functions. BCELoss evaluates the pixel-by-pixel difference between the predicted value and the actual value. DiceLoss measures the similarity between the precise label and the predicted image. LovaszLoss can solve the class imbalance problem in image segmentation. SSIMLoss can improve the accuracy of boundary segmentation. Combining these loss functions can improve the segmentation effect from multiple angles such as pixel level, sample similarity, image structure and segmentation boundary. By combining the advantages of each loss function, the segmentation accuracy of the model is improved. Figure 6 As shown in Figure 2, the segmentation results of the model proposed in this application are very close to the true label. In addition, Figure 7 The high AUC value and good ROC curve in further verify the excellent performance of the model.

[0099] Experimental evaluation:

[0100] This study was conducted using the TN3K public dataset, which contains 3,493 ultrasound images from 2,421 patients. According to the public division of the TN3K dataset, the training set contains 2,879 ultrasound images and the test set contains 614 ultrasound images. All data have been ethically reviewed.

[0101] The machine specifications used in this experiment are detailed in Table 1. In addition, the model was trained using PyTorch version 2.0.1 and CUDA 11.2. The learning rate of the model was set to 0.00001 and the batch size was 6. The input size of the network was 512×512, and the model was trained for 100 rounds. The input convolution kernel size was 7×7 with a stride of 2. In addition, (3,3) padding was used to retain important edge information. Padding refers to adding extra pixels to the edge of the input feature map to control the spatial size of the feature map after the convolution operation. Specifically, (3,3) padding means adding 3 pixels of zero padding to the height and width of the input feature map. The activation function used is the ReLU function. The RMSprop optimizer was used with a weight decay coefficient of 1e-8 and a momentum coefficient of 0.9. During the training phase, the Torchvision library was used to implement the data augmentation strategy to enhance the generalization and robustness of the model. Specifically, data augmentation is applied with a probability of 75%, including random horizontal flipping, random vertical flipping, and a random combination of horizontal and vertical flipping.

[0102] Table 1 System environment and specifications

[0103]

[0104]

[0105] This experiment uses indicators such as "Intersection over Union (IoU)", "F1_score", "Accuracy", "Precision", "HD95" and "Recall" to measure the segmentation performance of the model. These evaluation indicators are defined as follows:

[0106] IoU=TP / (FP+FN+TP) (6)

[0107] Accuracy=(TN+TP) / (TN+TP+FN+FP) (7)

[0108] Precision=TP / (TP+FP) (8)

[0109] Recall=TP / (TP+FN) (9)

[0110] F1_score=2TP / (FP+FN+2TP) (10)

[0111] HD95 measures the quality of the segmentation boundary by calculating the 95th percentile of the maximum distance between the predicted value and the ground truth value. The above formula uses TP, FP, TN and FN to represent the number of true positive, false positive, true negative and false negative samples respectively.

[0112] The DPAM-UNet++ network model is compared with a series of advanced models widely used in image segmentation tasks, including UNet, CMUNeXt, TRFE+, DeepLabV3+, TransUNet, SGUNet, and TransAttUNet. As shown in Table 2, except for the accuracy indicator TransUNet has a higher score, the DPAM-UNet++ model performs better than other models in the remaining indicators.

[0113] exist Figure 6 In Figure 2, the segmentation results of different models are provided, and each row describes the same ultrasound image segmented by different models. The segmentation results of the DPAM-UNet++ model are obviously closest to the true label and can more accurately retain the nodule boundary information. Figure 7 The receiver operating characteristic curves (ROC) of various segmentation models on the TN3K dataset are presented, showing that DPAM-UNet++ has an AUC value of 0.9213, which is better than other models.

[0114] Multiple nodules are a common clinical finding, and the probability of malignant transformation of these nodules is similar to or even greater than that of single nodules. The DAPM module introduced in this application effectively utilizes the global context information in the feature map to accurately detect nodules, and has significant improvements in identifying multiple nodules. As shown in Table 2, the recall rate of our model is significantly higher than that of other models, 4.31% higher than the second-ranked TransAttUNet model. This shows that the model of this application has a strong ability in detecting multiple nodules. Figure 8 This is also proved by the ablation experiment in . The excellent performance of the model in this application is attributed to the addition of the DPAM module, which extracts features from different angles and scales, enhances the feature representation of small nodules and multiple nodules, and thus improves the recall rate. Fig. 9 The attention heatmap further illustrates that the model with the DPAM module can accurately identify important nodule regions while ignoring irrelevant factors, thereby improving segmentation accuracy. The red frame in the figure marks irrelevant factors.

[0115] Table 2 Segmentation results of different models

[0116]

[0117] DPAM module ablation experiment

[0118] Table 3 lists the ablation experiment results of the DPAM module. The results show that the F1 score of the DPAM module is improved by 1.74% compared with the baseline. In addition, the F1 score and IoU index of the DPAM module are better than those of other attention modules, significantly improving the segmentation accuracy. The ablation experiment shows the segmentation effect of the attention module. Figure 8As shown in the figure, after the introduction of the DPAM module, the segmentation model can capture and identify nodule areas more accurately, and performs well in dealing with multiple nodules and small nodules. In addition, in order to intuitively demonstrate the effectiveness of the proposed attention module, the Grad-CAM visualization technique is used to generate attention heat maps for different attention modules integrated into the UNet++ baseline, as shown in Figure 2. Fig. 9 As shown, it can be seen that the model with the DPAM module can focus on important nodule areas more effectively.

[0119] Table 3 Ablation experiment of attention module

[0120]

[0121]

[0122] Loss function ablation experiment

[0123] In order to demonstrate the effectiveness and necessity of the proposed loss function, ablation experiments were conducted. As shown in Table 4, the results of the ablation study show that both IoU and F1_score are significantly improved after adding LovaszLoss and SSIMLoss. Specifically, after adding LovaszLoss, IoU increased by 1.48% and F1_score increased by 1.28%. In contrast, after adding SSIMLoss, IoU increased by 1.35% and F1_score increased by 1.06%. More importantly, when LovaszLoss and SSIMLoss are used in combination, the improvements in IoU and F1_score are more obvious, increasing by 1.75% and 1.72%, respectively. These results verify the complementarity of the two loss functions and show that their combination can improve the segmentation performance of the model more comprehensively. In addition, the model trained with the comprehensive loss function achieves the lowest HD95 value compared to the model trained with a single loss function. This shows that the proposed comprehensive loss function can effectively optimize the segmentation of nodule boundaries.

[0124] Table 4 Loss function ablation experiment

[0125] Model BCELoss DiceLoss LovaszLoss SSIMLoss IoU F1_score HD95 DPAM-UNet++ √ 0.7270 0.8125 40.50 DPAM-UNet++ √ 0.7304 0.8135 39.73 DPAM-UNet++ √ √ 0.7276 0.8138 39.58 DPAM-UNet++ √ √ √ 0.7424 0.8266 36.12 DPAM-UNet++ √ √ √ 0.7411 0.8244 33.93 DPAM-UNet++ √ √ √ √ 0.7451 0.8310 35.31

[0126] This application introduces the DPAM attention module to capture the global feature information in the feature map and optimize the segmentation of the nodule edge using an integrated loss function. This method improves the accuracy and efficiency of nodule segmentation from both global and local perspectives, helps reduce misdiagnosis caused by limited doctor experience, reduces doctors' workload, and provides new insights for clinical decision support, promoting the accuracy and intelligence of thyroid nodule diagnosis.

[0127] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A nodule ultrasound image segmentation method based on dual-path attention mechanism enhanced UNet++, characterized in that: The method is based on a DPAM-UNet++ network model, which includes an encoder, a decoder, a jump connection, and a DPAM module integrated into the jump connection; the method includes the following steps: Step 1: The encoder extracts features from the ultrasound image to obtain input features F1; Step 2, the input feature F1 is passed through the DPAM module to obtain an enhanced feature map; Step 3: Concatenate the enhanced feature map with the input feature F1 to obtain an initial fused feature map, and pass the initial fused feature map to the decoder through a skip connection; Step 4: The decoder performs feature recovery on the initial fused feature map to obtain a restored feature map, and then performs feature concatenation on the restored feature map and the initial fused feature map to form a final fused feature map.

2. The nodule ultrasound image segmentation method based on dual-path attention mechanism enhanced UNet++ as claimed in claim 1, characterized in that: The DPAM module includes a GAM and a GC module. The enhanced feature map obtained by the input feature F1 in step 2 after passing through the DPAM module specifically includes: Step 2.1, input the input feature F1 into the GAM and GC modules respectively; Step 2.2, GAM extracts channel and spatial features, and the GC module captures global context information; Step 2.3: The outputs of the GC module and GAM are connected with the original input feature F1 to jointly generate an enhanced feature map, which contains high-resolution features.

3. The nodule ultrasound image segmentation method based on dual-path attention mechanism enhanced UNet++ as claimed in claim 2, characterized in that: The GAM includes an ECA submodule and an SA submodule; the GAM extraction channel and spatial features in step 2.2 specifically include: The original input feature F1 first passes through the ECA submodule to obtain Mc(F1); then, F1 and Mc(F1) are element-wise multiplied to obtain feature F2, and feature F2 is input to the SA submodule to obtain Ms(F2); finally, Mc(F1) and Ms(F2) are element-wise multiplied to obtain the final output feature F3.

4. The nodule ultrasound image segmentation method based on dual-path attention mechanism enhanced UNet++ as claimed in claim 3, characterized in that: Input feature F2 into the SA submodule to obtain Ms(F2): The input feature F2 is first convolved with a 7x7 kernel to reduce the number of channels from C to C / r; then, the feature map output after the convolution passes through a sigmoid activation function to generate the spatial attention map Ms(F2).

5. The nodule ultrasound image segmentation method based on dual-path attention mechanism enhanced UNet++ as claimed in claim 4, characterized in that: The GC module captures the global context information in step 2.2 specifically including: The GC module first uses a 1x1 convolutional layer and a Softmax function to model the context of the input feature F1; then, the context information is multiplied by the input feature F1 in element order, and the result of the multiplication is transformed through another 1x1 convolutional layer, LayerNorm, and ReLU functions respectively; the transformed features are added to the input feature F1 in element order to generate the final output features.

6. A nodule ultrasound image segmentation method based on dual-path attention mechanism enhanced UNet++ as described in any one of claims 1 to 5, characterized in that: After step 4, the following steps are also included: Step 5: Construct a comprehensive loss function to improve the edge segmentation accuracy of the DPAM-UNet++ model.

7. The nodule ultrasound image segmentation method based on dual-path attention mechanism enhanced UNet++ as claimed in claim 6, characterized in that: The comprehensive loss function is specifically: L total =L bce +L dice +L lovasz +L ssim (1) L in the equation bce , L dice , L lovasz and L ssim They represent BCELoss, DiceLoss, LovaszLoss and SSIMLoss respectively; The calculation formula of BCELoss is as follows: Among them, g i and i Represent the true label and model prediction respectively, and i represents each pixel in the image; DiceLoss measures the similarity between the true label and the model prediction; the calculation formula of DiceLoss is: L dice =1-2|X∩Y| / |X+Y| (3) Where X and Y represent the number of elements in the true label and predicted image, respectively; The calculation formula of LovaszLoss is: Among them, ΔJ i is the marginal contribution of pixel i to the Jaccard index, g i and i are the true value and the predicted value respectively; The calculation formula of SSIMLoss is: L ssim =(2m x m y +c1)(2σ xy +c2) / (μ x 2 +m y 2 +c1)(σ x 2 +s y 2 +c2) (5) c1 and c2 are constants, x and y represent the actual image and predicted image respectively, μ, σ and σ xy They represent the mean, variance and covariance of the image respectively.

Images