Unet-based longitudinal carotid plaque segmentation method

Abstract

The application discloses a longitudinal carotid artery plaque segmentation method based on unet, and comprises the following steps: acquiring a longitudinal carotid artery ultrasound image, segmenting the longitudinal carotid artery ultrasound image through a segmentation model based on a unet structure to obtain a carotid artery plaque segmentation result; wherein the segmentation model comprises a multi-scale edge enhancement module for downsampling, an edge focusing attention module for upsampling, and a wavelet frequency domain attention module for jump connection of the multi-scale edge enhancement module and the corresponding edge focusing attention module; wherein the multi-scale edge enhancement module is used for enhancing and extracting edge features of the ultrasound image, the wavelet frequency domain attention module is used for optimizing the features extracted by enhancement, and the edge focusing attention module is used for reconstructing the features optimized by the edge focusing attention module according to the assigned weight, and further performing pixel-by-pixel labeling to segment the carotid artery plaque.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image detection, and particularly relates to a longitudinal carotid artery plaque segmentation method based on unet. BACKGROUND

[0002] Carotid artery plaque is an important inducement of ischemic stroke, and accurate identification and segmentation of the carotid artery plaque is crucial for the prevention and treatment of stroke. At present, the segmentation of the carotid artery plaque mainly relies on the manual operation of doctors, which is time-consuming and subjective, and the segmentation result is easily affected by the experience, state and ultrasound probe contact pressure of the doctor, thereby leading to inconsistent plaque performance in the image. With the development of computer science and automation technology, image processing has been widely applied in the field of medical image segmentation. The method based on deep learning can quickly and accurately segment the target region, thereby providing a faster and more objective auxiliary in the segmentation of carotid artery plaque ultrasound images.

[0003] In recent years, with the rapid development of computer vision and deep learning technology, many researchers have begun to explore how to realize the automatic segmentation of carotid artery plaque. Various methods have been proposed in the prior art to improve the accuracy and efficiency of segmentation. For example, a deep guided network based on U-Net introduces an image filtering module to restore structural information. However, due to its high dependence on the quality of the guided image, when the image has a large amount of noise, the segmentation accuracy will decrease significantly. For example, the Swin-UNet architecture performs excellently in global feature extraction, but the local characteristics of convolution operation limit its ability to capture details and small targets. For example, a multi-modal segmentation network combining ultrasound B-mode images and color Doppler images has achieved remarkable results in multi-modal information fusion, but its generalization ability on single-mode ultrasound images is weak.

[0004] In summary, the existing problems of insufficient segmentation accuracy and weak generalization ability on single-mode ultrasound images exist in the prior art. SUMMARY

[0005] To solve the above technical problems, the longitudinal carotid artery plaque segmentation method based on unet is proposed to solve the problems existing in the prior art.

[0006] To achieve the above purpose, the longitudinal carotid artery plaque segmentation method based on unet is provided, which comprises:

[0007] Obtaining a longitudinal carotid artery ultrasound image, segmenting the longitudinal carotid artery ultrasound image through a segmentation model based on a unet network structure to obtain a carotid artery plaque segmentation result;

[0008] The segmentation model is an EdgeWaveNet network, the segmentation model includes a multi-scale edge enhancement module for down-sampling, an edge focus attention module for up-sampling, and a wavelet frequency domain attention module for jump connection of the multi-scale edge enhancement module and the corresponding edge focus attention module, the multi-scale edge enhancement module is used for enhanced extraction of edge features of the ultrasound image, the wavelet frequency domain attention module is used for feature optimization of the enhanced extracted features, the edge focus attention module is used for feature reconstruction of the feature-optimized features according to an assigned weight, and further pixel-by-pixel labeling is performed to segment the carotid plaque.

[0009] Optionally, the segmentation model further includes an initial convolutional layer connected with the multi-scale edge enhancement module, the initial convolutional layer is used for initial feature extraction of the input ultrasound image to meet the input requirement of the multi-scale edge enhancement module, and the edge focus attention module is further connected with a convolutional layer and a sigmoid function for feature segmentation to obtain a segmentation result.

[0010] Optionally, a plurality of layers of the multi-scale edge enhancement module are sequentially connected, a plurality of layers of the edge focus attention module are sequentially connected, and the last layer of the multi-scale edge enhancement module is connected with the first layer of the edge focus attention module.

[0011] Optionally, in the multi-scale edge enhancement module, the process of enhanced extraction of edge features of the ultrasound image includes:

[0012] The feature map of the ultrasound image is down-sampled through maximum pooling and average pooling, the fused down-sampled structure is feature-extracted through a PSConv, the extracted features are normalized and processed through an activation function to obtain processed features, the processed features are weighted through a spatial attention mechanism to obtain spatial attention features, the spatial attention features are weighted through a channel attention mechanism to obtain channel attention features, the spatial attention features, the channel attention features, and the processed features are fused, the feature map is fused with the fused result, and the enhanced extracted features, that is, the output result of the multi-scale edge enhancement module, are obtained.

[0013] Optionally, in the wavelet frequency domain attention module, the process of feature optimization of the enhanced extracted features includes:

[0014] The enhanced extracted features are wavelet-transformed to obtain high-frequency features and low-frequency features, the high-frequency features are processed through a Manhattan self-attention mechanism, a convolution, normalization, and an activation function to obtain activated high-frequency features, and the activated high-frequency features are interpolated to obtain processed high-frequency features.

[0015] The low-frequency features are processed through a linear embedding layer and a Transformer structure, and the processed low-frequency features and the processed high-frequency features are fused and reconstructed to obtain a reconstructed signal, i.e., a feature after optimization, and the reconstructed signal is transmitted to the edge-focused attention module through a skip connection.

[0016] Optionally, in the edge-focused attention module, the feature reconstruction process of the feature after optimization according to the assigned weight includes:

[0017] The feature after optimization is up-sampled through transposed convolution to obtain a high-resolution feature map, the high-resolution feature map is spliced with the output feature of the previous layer and feature extraction is performed to obtain a local feature, the local feature is extracted through local convolution, the local feature is processed through global convolution to obtain local information and global information, and the local information and the global information are spliced and feature extraction is performed to obtain a feature reconstruction result.

[0018] Optionally, the feature extraction process through local convolution includes: performing convolution, normalization and activation function processing on the local feature to obtain local information.

[0019] Optionally, the weight distribution processing of the local feature through global convolution includes:

[0020] The local feature is processed through convolution, normalization and activation function processing several times, and the activation function processing result is processed through convolution again to obtain first branch information; the local feature is processed through normalization, and the normalized result is processed through weight distribution processing of an SS2DwithSCAM module, and the SS2DwithSCAM module processing result is regularized, the regularized result is fused with the SS2DwithSCAM module processing result to obtain second branch information, and the first branch information and the second branch information are processed through convolution to obtain global information, and the SS2DwithSCAM module is an SS2D module added with a spatial channel attention module.

[0021] Optionally, the processing through the SS2DwithSCAM module includes:

[0022] The normalized result is globally modeled through an SS2D module to obtain a modeling result, the modeling result is processed through a spatial channel attention module, and the spatial channel attention module processing result is further processed through weight distribution and conversion to obtain the SS2DwithSCAM module processing result.

[0023] The process of processing by the spatial channel attention module includes: respectively calculating spatial weights and channel weights of the modeling result through spatial attention mechanism and channel attention mechanism, and element-wise multiplying the modeling result with the spatial weights and the channel weights to obtain a spatial channel attention module processing result.

[0024] Compared with the prior art, the present application has the following advantages and technical effects:

[0025] The present application proposes an ultrasound image segmentation method based on hybrid edge perception attention and multi-scale feature enhancement, which effectively improves the shortcomings of traditional UNet in boundary blur and noise interference by introducing a wavelet frequency domain attention module, a multi-scale edge enhancement module and an edge focusing attention module. The method has achieved significant segmentation performance improvement on the self-built carotid plaque dataset, Breast Ultrasound Images Dataset and DDTI thyroid ultrasound image dataset, especially in complex boundary and small target segmentation tasks.

[0026] Specifically, the wavelet frequency domain attention module significantly enhances the significant edge information in the image through frequency domain feature processing, thereby improving the segmentation accuracy of low-quality images; the multi-scale edge enhancement module effectively captures complex boundary information through up-sampling operation and multi-scale feature fusion, thereby improving the segmentation ability of the model to details; and the edge focusing attention module realizes efficient expression of multi-scale information through down-sampling operation and adaptive integration of wavelet features, thereby further optimizing the segmentation performance. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application, and their

[0028] Figure 1 FIG. 1 is an overall structure diagram of an EdgeWaveNet network according to an embodiment of the present application;

[0029] Figure 2 FIG. 2 is a structure diagram of a WFA module according to an embodiment of the present application;

[0030] Figure 3 FIG. 3 is a structure diagram of an MSEE module according to an embodiment of the present application;

[0031] Figure 4 FIG. 4 is a structure diagram of an EFA module according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0033] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0034] In view of the deficiencies of the conventional ultrasound image segmentation method in processing fuzzy boundary, noise interference and multi-scale feature extraction, the present application proposes a lightweight ultrasound image segmentation network EdgeWaveNet. The network first realizes frequency domain decomposition through a wavelet frequency domain attention module (WFA), effectively extracts high-frequency edge information and low-frequency structure information, and combines the attention mechanism to enhance feature expression; secondly, a multi-scale edge enhancement module (MSEE) is used, which combines the combination structure of MaxPool, AvgPool and PSConv, cooperates with the asymmetric padding strategy, realizes the accurate capture of fuzzy edges, and uses the spatial-channel dual attention mechanism to optimize the feature extraction efficiency; finally, an edge focus attention module (EFA) is introduced, which innovatively combines ConvTranspose with local-global feature fusion, significantly improving the edge detail reconstruction quality. Experimental results show that on the thyroid, breast cancer and carotid plaque three types of ultrasound image data sets, the mIoU index of EdgeWaveNet compared with existing methods is increased by 11.63%, 8.86% and 2.55% respectively. In particular, the network improves the segmentation performance through efficient module design, and shows excellent robustness in complex noise environment and small target detection and other challenging tasks.

[0035] The present application proposes innovative solutions from the aspects of feature extraction, noise suppression and computational efficiency to solve the problems of the above method:

[0036] (1) Design an edge focus attention module (Edge Focus Attention Block, EFA). The module innovatively combines ConvTranspose with local-global feature fusion mechanism, significantly enhances the model's ability to reconstruct edge details through adaptive feature weight distribution. Experiments show that this module significantly improves the segmentation accuracy in complex boundary and small target segmentation tasks, and the mIoU on the three data sets is increased by more than 10% on average.

[0037] (2) Wavelet-Frequency Attention Block (WFA) is proposed. Compared with the traditional Fourier transform, wavelet transform has unique advantages in time-frequency positioning, which can more accurately separate high-frequency edge information (such as tissue boundary) and low-frequency structure information (such as background noise). Combined with channel-space dual attention mechanism, the performance degradation of this module under noisy environment is controlled within 5%, which is significantly better than existing methods.

[0038] (3) Multi-Scale Edge Enhancement Block (MSEE) is constructed. This module combines MaxPool (enhances edge mutation response), AvgPool (retains texture details) and PSConv (asymmetric padding strategy adapts to ultrasound edge diffusion mode), and enhances the extraction effect of edge features through spatial-channel dual attention mechanism. Experiments show that the introduction of this module makes the model's segmentation performance on carotid plaque, breast and thyroid three datasets all have significant improvement, and when dealing with fuzzy boundary and small target, the mIoU is more than 8% higher than the baseline model.

[0039] In view of the above technical contents, the technical scheme of the application is described in detail:

[0040] The technical scheme of the application mainly provides an EdgeWaveNet network based on unet network structure for longitudinal carotid plaque segmentation method. The EdgeWaveNet network is used for identifying longitudinal carotid ultrasound images to obtain carotid plaque by segmentation.

[0041] The EdgeWaveNet network as a segmentation model includes a multi-scale edge enhancement module for downsampling, an edge focusing attention module for upsampling, and a wavelet frequency attention module for jump connection of the multi-scale edge enhancement module and the corresponding edge focusing attention module,

[0042] The overall network architecture of the EdgeWaveNet proposed in the application is as follows: Figure 1As shown, containing initial convolution and edge focus attention module (Edge Focus Attention Block, EFA), wavelet frequency attention module (Wavelet-Frequency Attention Block, WFA) and multi-scale edge enhancement module (Multi-Scale Edge Enhancement Block, MSEE). The network first extracts features from the input image (224x224x3) through the initial convolution and down-sampling module. The MSEE module enhances the extraction effect of the edge features of the ultrasound image by combining MaxPool (enhancing edge mutation response) and AvgPool (retaining texture details) parallel processing, cooperating with the asymmetric padding strategy of PSConv. The WFA module uses the time-frequency positioning advantage of wavelet transform to accurately separate high-frequency edge information and low-frequency structure information, and optimizes the features through channel-space double attention mechanism, and improves the capture ability of multi-scale information. The EFA module innovatively combines ConvTranspose with local-global feature fusion mechanism, enhances the reconstruction ability of the model to edge details through adaptive feature weight distribution, especially suitable for processing complex boundary and small target segmentation tasks. Finally, the spatial resolution is gradually restored through the up-sampling module to generate the carotid plaque segmentation result. The loss function L adopts the combination of Dice loss L Dice and cross-entropy loss L CE : L = 0.5L Dice + 0.5L CE .

[0043] WFA module (Wavelet-Frequency Attention Block)

[0044] The anatomical structure edge information of the ultrasound image is mainly contained in the high-frequency component, and the low-frequency component contains large-scale tissue characteristics and noise. The traditional spatial domain convolution is easy to produce spectral confusion when separating high-frequency edges and low-frequency noise, which limits the segmentation accuracy. To solve this problem, the present application proposes a frequency processing module (WFA), the core idea of which is to process high-frequency and low-frequency components respectively through frequency domain decomposition to improve the segmentation performance.

[0045] Firstly, WFA adopts discrete wavelet transform (DWT) to decompose the input feature map into high-frequency and low-frequency components, preliminarily suppressing the noise in the ultrasound image. Considering the real-time requirement of ultrasound image processing and the importance of edge feature preservation, the present application selects Haar wavelet basis as the basis function of DWT. Haar wavelet basis has the characteristics of high computational efficiency and simple structure, which not only effectively preserves the edge jump features of the ultrasound image, but also is particularly suitable for processing speckle noise in the ultrasound image. In specific implementation, a four-layer decomposition structure is adopted, and the'reflect' mode is used for boundary processing, which realizes effective extraction of multi-scale features while ensuring computational efficiency. The high-frequency component contains edge detail information, while the low-frequency component preserves large-scale structural features.

[0046] For the high-frequency component, WFA adopts the Transformer structure for feature enhancement and combines the Manhattan Self-Attention (MaSA) mechanism for secondary processing. Since MaSA models spatial relationships, it can effectively suppress high-frequency noise interference while preserving key edge details. For the low-frequency component, WFA focuses on preserving the local structural information it contains and only performs preliminary denoising to reduce computational complexity.

[0047] Finally, WFA fuses the processed high-frequency and low-frequency components with the original feature map and transmits the information processed by WFA at each layer to the decoder through a jump connection similar to that in unet. As shown in Figure 2 , this inter-layer information transmission mechanism, together with the structure design of WFA, promotes effective utilization of features and improves segmentation performance, while reducing the interference of noise in the ultrasound image on model performance.

[0048] Compared with traditional spatial domain convolution, the WFA module can effectively avoid spectral confusion, thereby improving segmentation accuracy. Traditional spatial convolution, when processing ultrasound images, cannot effectively separate frequency domain information, resulting in overlap of high-frequency edge features and low-frequency structural noise in the spatial domain. Especially during downsampling, high-frequency information will be mixed into the low-frequency region, producing false signals. This spectral confusion causes the edge region to appear fuzzy, and in the process of information jump connection from downsampling to upsampling, the spectral confusion caused by the auxiliary information provided by downsampling will affect the accurate positioning of the carotid plaque boundary, leading to increased measurement error and reduced segmentation results. The WFA module, through multi-resolution analysis of wavelet transformation, can accurately separate frequency components at different scales, effectively avoiding this confusion problem and improving segmentation accuracy. Experimental results show that the WFA module outperforms existing methods on public datasets, especially when dealing with complex edges and background noise.

[0049] The data processing flow of the above WFA module is described in detail:

[0050] First, the input feature map F in ∈R H×W×C Decomposed by Haar wavelet transform (using'reflect' mode) can be expressed as, obtaining a low-frequency component and three high-frequency components.

[0051] LL, LH, HL, HH = DWT(F in ) (1)

[0052] Where LL is the low-frequency component, LH (horizontal high frequency), HL (vertical high frequency), and HH (diagonal high frequency) are three high-frequency components. The three high-frequency components are combined into a unified high-frequency feature layer:

[0053] F low =LL (2)

[0054] F high =Concat(LH, HL, HH) (3)

[0055] In this way, the input feature map is decomposed into a low-frequency feature F low and a combined high-frequency feature F high , which is ready for subsequent processing.

[0056] The high-frequency feature first passes through the Manhattan self-attention mechanism (Manhattan Self-Attention, MaSA) module, and then passes through the 3x3 convolution operation:

[0057] (I high ,I low )=WaveletDecomp(F low ) (4)

[0058] H processed =Conv(MaSA(I high )) (5)

[0059] Next, the processed high-frequency information passes through the batch normalization BatchNorm and the activation function GELU:

[0060] H norm =BatchNorm(H processed ) (6)

[0061] H activated =GELU(H norm ) (7)

[0062] The activated high-frequency part Hactivated is interpolated Interp:

[0063] H interpolated = Interp(H activated ) (8)

[0064] The low-frequency part is processed by a linear embedding layer Emb1 and a Transformer structure:

[0065] L processed = Transformer(Emb(I low )) (9)

[0066] The fused features are reconstructed using a reconstruction module and passed to the decoder through a skip connection, denoted as:

[0067] I reconstructed = Reconstruction(L processed ) (10)

[0068] Finally, the fused features I fused are reconstructed using a reconstruction module Reconstruction and passed to the decoder through a skip connection:

[0069] I fused = Concat(H interpolated , I reconstructed ) (11)

[0070] Through this process, the high-frequency and low-frequency information of the input image is processed, fused, and reconstructed through a series of operations, resulting in the final output features.

[0071] MSEE module (Multi-Scale Edge Enhancement Block)

[0072] In the down-sampling module of the ultrasound image, the module combines MaxPool, AvgPool, and PSConv (Partial Separable Convolution) as well as spatial attention and channel attention mechanisms, aiming to improve the performance of ultrasound image segmentation. Through multi-scale feature extraction and attention mechanisms, the module can better handle complex boundaries and detailed areas in ultrasound images, improving segmentation accuracy and robustness.

[0073] Firstly, MaxPool enhances the edge feature extraction ability by the max-pooling operation, while AvgPool preserves the texture information of the tissue by the average-pooling operation, captures more detailed features, and reduces information loss. The combination of the two can enhance the extraction of edge features while preserving details. Secondly, PSConv, through the design of asymmetric padding and multi-directional diffusion convolution kernel, its receptive field distribution is highly consistent with the spatial characteristics of the ultrasound target. Specifically, asymmetric padding uses different padding methods for different regions to enhance the extraction of local features; multi-directional diffusion convolution kernel captures the diffusion pattern of fuzzy edges through horizontal and vertical convolution kernel diffusion, which is particularly suitable for processing Gaussian distributed edges in ultrasound images. Compared with standard convolution, PSConv can more efficiently capture the diffusion pattern of fuzzy edges in the bottom feature extraction stage, improving the feature expression ability.

[0074] In addition, the introduction of spatial attention and channel attention mechanisms further optimizes the feature extraction process. Spatial attention enhances the feature expression of important regions by weighting the spatial dimensions of the feature map; channel attention enhances the feature expression of important channels by weighting the channel dimensions of the feature map. The output of spatial attention and channel attention is fused to obtain the final feature map, further improving the model's attention to key regions and enhancing segmentation accuracy. It is worth noting that the introduction of PSConv complements the traditional attention mechanism: the traditional attention mechanism mainly enhances the feature expression of important regions by weighting the spatial and channel dimensions of the feature map, while PSConv more efficiently captures fuzzy edges and local details in ultrasound images through the design of asymmetric padding and multi-directional diffusion convolution kernel. This combination makes the model perform better when dealing with complex backgrounds and local information limitations, further improving segmentation accuracy and robustness.

[0075] In summary, the down-sampling module effectively improves the performance of ultrasound image segmentation by combining MaxPool, AvgPool, PSConv, and spatial and channel attention mechanisms, especially when dealing with local information limitations and complex backgrounds. Through multi-scale feature extraction and attention mechanisms, the module can better handle complex boundaries and detail areas in ultrasound images, improving segmentation accuracy and robustness. The MSEE module structure diagram is shown in Figure 3 .

[0076] The specific data processing process of the module is as follows:

[0077] The input feature map is first MaxPool and AvgPool respectively down-sampling the input feature map x:

[0078] X max = MaxPool(X), Xavg = AvgPool(X) (12)

[0079] wherein X max , X avg ∈ R C×2H×2W , X max , X avg denote the corresponding processed feature maps;

[0080] PSConv extracts features through asymmetric padding and multi-directional diffusion convolution kernels:

[0081] X psconv = PSConv(X max ⊕ X avg , k = 3) (13)

[0082] wherein denotes the feature concatenation operation, and k = 3 is the size of the convolution kernel.

[0083] Batch normalization and activation are performed on the PSConv output X psconv

[0084] X bn = BatchNorm(X psconv ), X relu = ReLU(X bn ) (14)

[0085] X psconv = PSConv(X max ⊕ X avg , k = 3) (15)

[0086] A spatial = Sigmoid(PSConv(Concat(AdaptiveMaxPool(X psconv ), AdaptiveAvgPool(X psconv ))), k = 7) (16)

[0087] wherein AdaptiveMaxPool is an adaptive max-pooling layer, AdaptiveAvgPool is an adaptive average-pooling layer, and A spatial denotes the spatial attention weight.

[0088] The spatial attention weights the spatial dimension of the feature map:

[0089]

[0090] wherein, denotes element-wise multiplication.

[0091] ​Channel attention weights the channel dimension of the feature map:

[0092] A channel =Sigmoid(Concat(Conv(ReLU(Conv(AdaptiveMaxPool(X psconv) ),k=1),k=1),Conv(ReLU((AdaptiveAvgPool(X psconv ),k=1)),k=1))) (17

[0093]

[0094] Among them, A channel X represents the channel attention weight. spatial X represents the spatial attention output feature. channel l represents the channel attention output feature.

[0095] If the number of input and output channels is the same (i.e., In_channels == Out_channels), a residual connection is added, and the outputs of spatial attention and channel attention are combined to obtain the final feature map X. out :

[0096] X o =X shortcut +X channel (19)

[0097] X out =Convs(X o (20)

[0098] Where X shortcut The result of the added residual connection is the original input processed by PSconv and conv.

[0099] This module achieves efficient feature extraction and downsampling by combining MaxPool, AvgPool, PSConv, spatial attention, and channel attention mechanisms.

[0100] EFA module (Edge Focus Attention Block)

[0101] Ultrasound images typically have low resolution and limited detail, making it difficult to directly apply traditional interpolation methods (such as bilinear interpolation) without resulting in information loss or image blurring. Furthermore, the complex tissue structures within ultrasound images necessitate both local and global contextual information for accurate interpretation. To address these issues, methods such as... Figure 4As shown, the module uses ConvTranspose (transposed convolution) instead of traditional interpolation methods during upsampling to reduce information loss. ConvTranspose can generate smoother high-resolution images, effectively addressing the low-resolution problem of ultrasound images.

[0102] At the same time, the module realizes more complete upsampling by fusing local information (ConvPath) and global information (ConvSSM). Specifically, local information captures local features (such as edges, textures, etc.) in the image through convolution operations, while ConvSSM models global context information to help the model understand the complex relationships between different regions in the image, which is particularly important for identifying lesion areas.

[0103] In ConvSSM, the module introduces an improved SS2D with SCAM structure. This structure adds SCAM (Spatial Channel Attention Module) after each convolutional layer based on SS2D. SCAM enhances the model's understanding of global context information in each patch through spatial and channel attention mechanisms, allowing it to more accurately capture the relationship between background and target. In addition, SCAM can effectively suppress noise interference in ultrasound images and improve the model's ability to recognize complex structures, further enhancing the upsampling effect.

[0104] The specific data processing flow of the module is as follows:

[0105] Let the input low-resolution ultrasound image be X ∈ R H×W×C , where H, W, and C represent the height, width, and number of channels of the image, respectively.

[0106] Use ConvTranspose to upsample the input image to generate a high-resolution feature map F up :

[0107] F up = ConvTranspose(X; θ up ), F up ∈ R 2H×2W×C (19)

[0108] where θ up represents the parameters of ConvTranspose, and C is the number of channels after upsampling.

[0109] Extract local features F ConvPath ∈ R 2H×2W×C

[0110] F ConvPath = Conv(F up ; θ conv ) (20)

[0111] where θ conv denotes the parameters of the convolutional layer.

[0112] For the above local features, parallel processing is performed through local convolution ConvPath and global convolution ConvSSM, respectively. In the local convolution ConvPath, processing is performed through a plurality of layers of Conv, BN and ReLu in sequence to obtain local information. In the global convolution ConvSSM, the local features are processed through Conv, BN, ReLu, Conv, BN, ReLu and Conv in sequence to form a first branch feature. The local features are also processed through BN, an SS2DwithSCAM module, and a drop path to determine whether to skip the processing to obtain a second branch. The first branch and the second branch are processed through CONV to obtain global information. The global information and the features processed by the local convolution are concatenated, and the concatenated features are processed through CONV to obtain the output of the module.

[0113] In the ConvSSM, the SS2DwithSCAM module is used to extract global context information F global ∈R 2H×2W×C . The SS2D module globally models the input feature map F up :

[0114] F SS2D =SS2D(F up ; θ SS2D ) (21)

[0115] where F SS2D denotes the feature processed by the SS2D module. In the SS2D module, linear processing, conv and Silu are performed in sequence, and the expansion and convolution of another branch are connected in a residual manner.

[0116] Based on the SS2D, the SCAM (spatial channel attention module) is added to enhance the output of each convolutional layer:

[0117] F SCAM =SCAM(F SS2D ; θ SCAM ) (22)

[0118] where θ SCAM denotes the parameters of the SCAM.

[0119] The calculation process of the SCAM includes spatial attention: calculating the spatial weight W spatial :

[0120] W spatial =Sigmoid(Conv(FSS2D ; θ spatial ))(23),

[0121] Channel attention: compute channel weights W channel :

[0122] W channel = Sigmoid(Conv(F SS2D ; θ channel ))(24)

[0123] Feature enhancement: apply spatial and channel weights to feature maps:

[0124]

[0125] where, denotes element-wise multiplication.

[0126] And through the forward function forword core, merge, normalization, linear processing of SS2D in mamba, finally through the dropout processing of the final SS2D with SCAM module output feature.

[0127] Fuse local information F local and global information F global to generate the final high-resolution feature map F final ∈R 2H×2W×C :

[0128] F final = F local + F global (26)

[0129] Pixel-level labeling is performed on the final EFA module to achieve the final carotid plaque segmentation.

[0130] In addition, in order to enhance the generalization of the model, we also use the public ultrasound image dataset, including the malignant dataset part in BreastUltrasound Images Dataset, and the samples in DDTI:Thyroid UltrasoundImages dataset.

[0131] The relevant experimental settings and results of the above are explained:

[0132] Dataset

[0133] There is no standard dataset published in the field of carotid plaque segmentation. In this study, we collected ultrasound images from the hospital and made a new dataset. This study has been approved by the Ethics Committee of the Affiliated Hospital of Inner Mongolia Medical University. All data collection and use comply with the hospital's data security management system. Patient information is de-identified. To facilitate model training, the collected original ultrasound images are cropped to 224x224 RGB images. The cropped images are annotated using the open-source tool Roboflow.

[0134] Training environment and training parameters

[0135] The experiment is based on Pytorch 1.8 framework in ubuntu system using Python. The initial learning rate is set to 0.0001, the batchsize is set to 8, and the training is 100 epochs. Under this setting, the NVIDIA GeForce RTX3090 is used to complete the accelerated training, and the mixed precision is used to accelerate the training. The dataset is randomly divided into training set, test set and validation set according to the ratio of 8:1:1. The loss function used is 0.5Dice+0.5CrossEntropy

[0136] Performance indicators

[0137] In order to evaluate the performance of the proposed method in the task of ultrasound image segmentation, several commonly used evaluation indicators are adopted, including mean intersection over union (mIoU), Dice coefficient (mDice), recall (Recall) and precision (Precision). These indicators help to comprehensively measure the performance of the model in handling segmentation tasks, especially in complex boundary and small target detection.

[0138] Mean Intersection over Union (mIoU)

[0139] Mean Intersection over Union (mIoU, Mean Intersection over Union) is an indicator to measure the overlap between the model's prediction results and the true labels. For each class, the calculation formula of Intersection over Union (IoU) is as follows:

[0140] IoU = TP / (TP + FP + FN) (27)

[0141] Where TP, FP and FN represent the number of samples that are true positive and predicted positive, the number of samples that are predicted positive but true negative, and the number of samples that are true positive but predicted negative, respectively. The mIoU indicator evaluates the overall segmentation performance of the model by averaging the IoU scores of all classes. For binary segmentation tasks, mIoU is equivalent to IoU.

[0142] Dice Coefficient (mDice)

[0143] The Dice coefficient (mDice, Mean Dice Coefficient) is another commonly used metric to measure the overlap between predicted results and true labels. Its formula is as follows:

[0144] Dice = 2 x TP / 2 x TP + FP + FN (28)

[0145] The Dice coefficient measures the similarity between the two by comparing twice the overlapping area with the sum of the predicted and true results. For multi-class tasks, mDice is the average of the Dice coefficients for each class, used to evaluate the model's performance in the overall segmentation task.

[0146] Recall

[0147] Recall reflects the model's ability to identify positive samples, and its calculation formula is:

[0148] Recall = TP / TP + FN (29)

[0149] Recall measures the proportion of correctly identified positive samples in all positive samples. In the ultrasound image segmentation task, a higher Recall indicates that the model can capture more target areas, especially in cases where the boundary is ambiguous or the target is small.

[0150] Precision

[0151] Precision is used to measure the accuracy of the model's prediction of positive samples, and its calculation formula is:

[0152] Precision = TP / TP + FP (30)

[0153] Precision reflects the proportion of actual positive samples among the samples predicted as positive by the model. A higher Precision indicates that the model's prediction of target areas is more accurate, effectively reducing the occurrence of false positives.

[0154] Contrast experiment and visualization

[0155] To verify the effectiveness of the proposed method in the task of ultrasound image segmentation, we conducted experiments on the self-built carotid plaque dataset, the malignant subset of the Breast Ultrasound Images Dataset, and the DDTI: Thyroid Ultrasound Images dataset, and compared the performance with classic segmentation networks, including UNet, UNet++, UCTransNet, Attention UNet, ResUNet, MutiresUNet, and TransNet, etc.

[0156] Performance on the carotid plaque dataset

[0157] To verify the performance of the proposed method in the task of ultrasound image segmentation, experiments were conducted on the self-built carotid plaque dataset and compared with a variety of classic and latest segmentation networks. The experimental results show that the proposed method has significant advantages in multiple key indicators. Specifically, the proposed method achieves the best performance in mIoU (61.38%), mDice (76.07%), Recall (70.15%), and Precision (83.07%), which is 2.55%, 1.99%, 4.90%, and -2.58% higher than the second best method PDF-UNet (2023), respectively.

[0158] In the comparative method, in Table 1, PDF-UNet and Attunet perform more outstandingly, close to the proposed method in mIoU and Precision respectively. However, the Recall and Precision of PDF-UNet, although higher, are still lower than the proposed method in mIoU and mDice, indicating that there is still room for improvement in the overall segmentation performance. Attunet performs best in Precision (85.04%), but its Recall (61.71%) and mIoU (55.66%) are lower, indicating that it is insufficient in identifying part of the target. Other methods such as Net, Resunet and DCSAU-UNet perform well in some indicators, but there is still a big gap in overall performance compared with the proposed method. For example, the Precision (76.60%) of Net is higher, but the mIoU (47.62%) and Recall (55.73%) are lower, indicating that the segmentation results are not comprehensive enough. Mutiresunet and UltraLight_VMUnet(2024) perform the worst in all indicators, indicating that their model design may not be suitable for the current task. In addition, VMUnet(2024) performs well in Precision (78.75%), but its mIoU (45.09%) and Recall (51.34%) are lower, indicating that its overall segmentation performance needs to be improved. UNetv2 performs poorly in all indicators, especially mIoU (36.94%) and Recall (41.99%), which are significantly lower than other methods, indicating that its model design may not be suitable for the current task or its generalization ability on the dataset is poor.

[0159] In summary, the proposed method performs excellent performance in the task of ultrasound image segmentation, especially in mIoU and Recall, which has a significant advantage, reflecting its comprehensive identification ability to target areas. At the same time, the proposed method also maintains a high level in Precision, indicating the accuracy of the segmentation results. These results show that the proposed method has high practicability and promotional value in the task of ultrasound image segmentation, and Table 1 is the comparative test results on the carotid plaque dataset.

[0160] Table 1

[0161]

[0162] Performance on malignant breast ultrasound dataset

[0163] To further verify the performance of the proposed method in the breast ultrasound image segmentation task, we conducted experiments on the malignant breast ultrasound image dataset and compared it with multiple classic and latest segmentation networks. The experimental results show that the proposed method performs significantly better in multiple key indicators. Specifically, the proposed method achieves the best performance in mIoU (55.15%), mDice (71.09%), Recall (64.55%), and Precision (79.11%), which are 8.86%, 7.80%, 6.28%, and 6.59% higher than the second-best method DCSAU-UNet (2023), respectively.

[0164] Among the comparative methods, DCSAU-UNet and Attunet perform more outstandingly in Table 2, with mIoU and Precision close to the proposed method. However, although DCSAU-UNet has higher Recall (58.27%) and Precision (69.25%), its mIoU (46.29%) and mDice (63.29%) are still lower than the proposed method, indicating that there is still room for improvement in its overall segmentation performance. Attunet performs well in Precision (71.86%), but its Recall (54.99%) and mIoU (45.25%) are lower, indicating that it lacks the ability to identify some targets. Other methods such as UNet, Resunet, and Mutiresunet perform well in some indicators, but there is still a big gap between the overall performance and the proposed method. For example, UNet has a high Recall (60.84%), but its mIoU (39.75%) and Precision (53.43%) are low, indicating that its segmentation results are not comprehensive. Mutiresunet and UltraLight_VMUnet (2024) perform poorly in all indicators, indicating that their model design may not be suitable for the current task.

[0165] In summary, the proposed method performs excellently in the breast ultrasound image segmentation task, especially in mIoU and Recall, reflecting its comprehensive identification ability for target regions. At the same time, the proposed method also maintains a high level of Precision, indicating the accuracy of its segmentation results. These results show that the proposed method has high practicality and promotional value in the breast ultrasound image segmentation task. Table 2 shows the comparison test results on the malignant breast ultrasound dataset

[0166] Table 2

[0167]

[0168]

[0169] Performance on thyroid ultrasound dataset

[0170] To verify the performance of the proposed method's generalization ability on the DDTI dataset, we conducted experiments on this dataset and compared them with a variety of classic and latest segmentation networks. The experimental results show that the proposed method performs significantly better in multiple key indicators. Specifically, the proposed method achieves the best performance in mIoU (49.17%), mDice (65.93%), Recall (60.79%), and Precision (72.01%), which is 11.63%, 11.35%, 15.47%, and 3.41% higher than the second best method PDFUNet, respectively.

[0171] Among the compared methods, PDFUNet and DCSAU-UNet (2023) perform relatively outstandingly in Table 3, with mIoU and mDice close to the proposed method. However, PDFUNet's Recall (45.32%) and Precision (68.60%) are relatively high, but its mIoU (37.54%) and mDice (54.58%) are still lower than the proposed method, indicating that there is still room for improvement in its overall segmentation performance. DCSAU-UNet (2023) performs well in mDice (52.86%) and Recall (45.16%), but its mIoU (35.93%) and Precision (63.74%) are relatively low, indicating that it lacks the ability to identify some targets. Other methods such as UNet, Attunet, and Mutiresunet perform well in some indicators, but there is still a large gap between the overall performance and the proposed method. For example, Attunet's Precision (72.47%) is relatively high, but its Recall (34.06%) and mIoU (30.16%) are relatively low, indicating that its segmentation results are not comprehensive enough. Transnet and UltraLight_VMUnet (2024) perform poorly in all indicators, indicating that their model design may not be suitable for the current task.

[0172] In summary, the proposed method performs excellently on the DDTI dataset, especially in mIoU and Recall, reflecting its comprehensive identification ability for target regions. At the same time, the proposed method also maintains a high level of Precision, indicating the accuracy of its segmentation results. These results show that the proposed method has high practicality and promotional value in the DDTI dataset segmentation task.

[0173] Table 3

[0174]

[0175] Ablation experiments

[0176] To verify the effectiveness of each module, ablation experiments were conducted on the three datasets (Mydata, DDTI, and Malignant) respectively. From Tables 4, 5, and 6, it can be seen that on the Mydata dataset, DDTI dataset, and Malignant dataset, after adding the WFA module to the base model, the mIoU and mDice are improved to 54.91% and 70.89% respectively, which proves that the WFA module significantly enhances the feature extraction capability of the model. After further adding the EFA module, the mIoU and mDice are improved to 60.72% and 75.56% respectively, which proves that the EFA module plays an important role in multi-scale feature fusion. Finally, after adding the MSEE module, the mIoU and mDice are improved to 61.38% and 76.07% respectively, and the Rec. is improved to 70.15%, which indicates that the MSEE module further optimizes the feature weight distribution. Through the ablation experiment, the effectiveness of the EFA, WFA, and MSEE modules is verified, and the step-by-step addition of each module significantly improves the model performance, which reflects the generality and reliability of the proposed method. Table 4 is the ablation experiment on the ultrasonic carotid plaque dataset, Table 5 is the ablation experiment on the malignant breast ultrasound dataset, and Table 6 is the ablation experiment on the thyroid ultrasound dataset.

[0177] Table 4

[0178]

[0179] Table 5

[0180]

[0181] Table 6

[0182]

[0183] The above, only for the preferred specific embodiments of the present application, but the scope of protection of the present application is not limited to this, any skilled in the art of the technical personnel in the technical range disclosed in the present application, can easily think of changes or replacement, should be covered in the scope of protection of the present application. Therefore, the scope of protection of the present application should be limited to the scope of protection of the claims.

Claims

1. A longitudinal carotid artery plaque segmentation method based on unet, characterized in that, The method comprises the following steps: obtaining a longitudinal carotid artery ultrasound image, segmenting the longitudinal carotid artery ultrasound image through a segmentation model based on an unet network structure to obtain a carotid artery plaque segmentation result; the segmentation model is an EdgeWaveNet network, wherein the segmentation model comprises a multi-scale edge enhancement module for downsampling, an edge focusing attention module for upsampling, and a wavelet frequency domain attention module for jump connection of the multi-scale edge enhancement module and the corresponding edge focusing attention module, wherein the multi-scale edge enhancement module is used for enhancing and extracting edge features of the ultrasound image, the wavelet frequency domain attention module is used for optimizing the features of the enhanced and extracted features, and the edge focusing attention module is used for feature reconstruction of the features after optimization according to the assigned weights and further pixel-by-pixel labeling to segment the carotid artery plaque; in the multi-scale edge enhancement module, the process of enhancing and extracting edge features of the ultrasound image comprises: downsampling the feature map of the ultrasound image through maximum pooling and average pooling, extracting features of the fused downsampled structure through PSConv, normalizing and activating the extracted features, obtaining processed features, weighting the processed features through a spatial attention mechanism to obtain spatial attention features, weighting the spatial attention features through a channel attention mechanism to obtain channel attention features, fusing the spatial attention features, the channel attention features and the processed features, and fusing the feature map with the fused results to obtain the enhanced and extracted features, i.e. the output results of the multi-scale edge enhancement module; in the wavelet frequency domain attention module, the process of optimizing the features of the enhanced and extracted features comprises: performing wavelet transform on the enhanced and extracted features to obtain high-frequency features and low-frequency features, processing the high-frequency features through a Manhattan self-attention mechanism, convolution, normalization and an activation function to obtain activated high-frequency features, and performing interpolation operation on the activated high-frequency features to obtain processed high-frequency features; processing the low-frequency features through a linear embedding layer and a Transformer structure, fusing and reconstructing the processed low-frequency features and the processed high-frequency features to obtain a reconstructed signal, i.e. the features after optimization, and transmitting the reconstructed signal to the edge focusing attention module through jump connection; in the edge focusing attention module, the process of reconstructing the features after optimization according to the assigned weights comprises: upsampling the features after optimization through transposed convolution to obtain a high-resolution feature map, splicing and extracting features of the high-resolution feature map to obtain local features, extracting features of the local features through local convolution, assigning weights to the local features through global convolution to obtain local information and global information, splicing the local information and the global information and extracting features to obtain a feature reconstruction result.

2. The method of claim 1, wherein The initial convolutional layer is connected with the multi-scale edge enhancement module in the segmentation model, wherein the initial convolutional layer is used for initial feature extraction of the input ultrasound image to meet the input requirement of the multi-scale edge enhancement module, and the edge focus attention module is further connected with the convolutional layer and the sigmoid function for feature segmentation to obtain a segmentation result.

3. The method of claim 1, wherein, A plurality of layers of the multi-scale edge enhancement module are sequentially connected, and a plurality of layers of the edge focus attention module are sequentially connected, wherein the last layer of the multi-scale edge enhancement module is connected with the first layer of the edge focus attention module.

4. The method of claim 1, wherein, The process of feature extraction by local convolution includes: convolution, normalization and activation function processing of local features to obtain local information.

5. The method of claim 1, wherein, The process of weight distribution processing of the local features by global convolution includes: The local features are processed by convolution, normalization and activation function processing for several times, and the activation function processing result is processed by convolution again to obtain first branch information; the local features are processed by normalization, and the normalized result is processed by weight distribution processing through the SS2DwithSCAM module, and the regularization result is fused with the SS2DwithSCAM module processing result to obtain second branch information; the first branch information and the second branch information are processed by convolution to obtain global information, and the SS2DwithSCAM module is an SS2D module added with a spatial channel attention module.

6. The method of claim 5, wherein, The process of processing by the SS2DwithSCAM module includes: The normalized result is processed by global modeling through the SS2D module to obtain a modeling result, the modeling result is processed by the spatial channel attention module, and the spatial channel attention module processing result is further processed by weight distribution and conversion to obtain the SS2DwithSCAM module processing result; The process of processing by the spatial channel attention module includes: the modeling result is respectively processed by spatial attention mechanism and channel attention mechanism to calculate spatial weight and channel weight, and the modeling result is multiplied by the spatial weight and the channel weight element by element to obtain the spatial channel attention module processing result.

Images