A Hybrid Attention UNet-based Ultrasonic Image Segmentation Method and Apparatus Based on Pseudo-Color Enhancement

Abstract

A method and apparatus for segmenting ultrasound images based on a hybrid attention UNet with pseudo-color enhancement, relating to the field of medical image processing technology, are described. The method includes: constructing a convolutional neural network containing an encoding layer and a decoding layer; preprocessing ultrasound images in the training set; performing pseudo-color conversion on the ultrasound images; inputting the ultrasound images into the encoding layer for downsampling and feature extraction, and completing channel information interaction between features; inputting the feature map into the decoding layer to generate a segmentation map and calculating a loss function; adjusting the weights and biases of the convolutional neural network until the loss function converges to obtain the optimal training model; using the optimal training model to process the target ultrasound image to obtain the target segmentation feature map; performing a sigmoid operation on the image, then calculating the loss function, and finally binarizing the image to obtain the final segmentation map. This invention can effectively solve the problem of difficult segmentation of ultrasound images due to poor imaging quality, artifacts, and low contrast.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a hybrid attention UNet ultrasound image segmentation method and apparatus based on pseudo-color enhancement. Background Technology

[0002] Ultrasound imaging is a non-invasive, real-time, and portable imaging technology widely used in clinical diagnosis and monitoring of various diseases. Ultrasound image segmentation technology can help doctors diagnose and quantitatively assess diseases more accurately. Traditional ultrasound image segmentation methods are usually based on manually designed feature extraction algorithms, which require significant manpower and time and are difficult to adapt to different lesion morphologies. The rise of deep learning technology has made ultrasound image segmentation more efficient and accurate. By using deep learning models such as convolutional neural networks (CNNs), features can be learned directly from the original image and segmented at the pixel level. However, the inherent characteristics of ultrasound images, such as noise, shadows, scattering, low contrast, and blurred edges, make target segmentation challenging.

[0003] Currently, deep learning-based ultrasound image segmentation methods mainly include classic CNN models such as U-Net and SegNet, improved models such as Attention U-Net, Dense U-Net, and ResUNet, and the improved model TransUNet, which combines convolution and Transformer. While these improved methods can achieve higher segmentation performance, they come at the cost of high computational cost and low execution efficiency. Furthermore, due to the limited receptive field of traditional convolutions, these convolutional models often ignore global information about the target, leading to inaccurate segmentation. Feature loss during downsampling also causes bias in segmenting target details. For challenging ultrasound segmentation tasks, these shortcomings severely affect the generalization accuracy of segmentation. Therefore, how to address the insufficient global feature extraction capability of convolutions, fully balance efficiency and complexity, and handle ultrasound image segmentation in complex environments remains a key technical challenge for those skilled in the art. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a hybrid attention UNet ultrasound image segmentation method based on pseudo-color enhancement, which combines the classic pseudo-color enhancement algorithm with deep learning methods to solve the problem of difficult segmentation of ultrasound images due to poor imaging quality, artifacts, and low contrast.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a hybrid attention UNet ultrasound image segmentation method based on pseudo-color enhancement, comprising the following steps:

[0006] (1) Construct a convolutional neural network containing an encoding layer and a decoding layer;

[0007] (2) Preprocess the ultrasound images in the training set;

[0008] (3) Perform pseudo-color conversion on ultrasound images according to the pseudo-color enhancement algorithm;

[0009] (4) Input the ultrasound image after pseudo-color conversion into the coding layer for downsampling and feature extraction, and realize the channel information interaction between features;

[0010] (5) Input the downsampled feature map and the attention-weighted feature map, which have been processed by long-range dependency capture, feature representation and normalization, into the decoding layer at the same time to generate the segmentation map and calculate the loss function;

[0011] (6) Adjust the weights and biases of the convolutional neural network until the loss function converges to obtain the optimal training model;

[0012] (7) Call the optimal training model to process the preprocessed target ultrasound image to obtain the target segmentation feature map;

[0013] (8) Perform a Sigmoid operation on the target segmentation feature map, then calculate the loss function, and then binarize the image to obtain the final segmentation map.

[0014] Preferably, in step (3), the pseudo-color conversion method is as follows:

[0015]

[0016] I=g(x,y)………………………………(1.2)

[0017] H=2πg(x,y) / 255……………………(1.3)

[0018]

[0019] Where f(x, y) represents the gray value of a pixel, 0≤f(x, y)≤255; H, S and I represent hue, saturation and intensity in the HIS color space, respectively; where different saturations from 0 to 255 are selected by changing k from 0 to 2.

[0020] Preferably, in step (4), during the interaction of channel information between features, channel attention is learned in the following way:

[0021] ω=σ(C1D k (y))……………………(1.5)

[0022] Among them, the aggregated features y∈R without dimensionality reduction C C1D represents one-dimensional convolution, σ represents the sigmoid function, and k represents the number of adjacent channels of y.

[0023] Preferably, in step (5), the loss function is calculated using a combination of the cross-entropy loss function and the Dice loss function, wherein the cross-entropy loss function is:

[0024]

[0025] Where y represents the true label of the sample, p represents the predicted label, and N is the total number of pixels in the image;

[0026] The Dice loss function is:

[0027]

[0028] Where X represents the label value, Y represents the prediction result, and pi and yi represent the value of the i-th pixel of the expected result and label, respectively; the smoothing parameter s is 1;

[0029] Among them, weighted Dice loss and BCE loss are used for deep supervision loss and final segmentation:

[0030]

[0031] Where λ represents the trade-off parameter between the two losses, λ = 0.3; L final The loss is the final partition.

[0032] Preferably, in step (5), after the feature map is input into the decoding layer, the feature maps of the decoding layer and the corresponding encoding layer are fused.

[0033] Preferably, in step (5), the step of fusing the feature maps of the decoding layer and the corresponding coding layer is as follows:

[0034] First, convolutional kernels of different sizes are used to obtain feature information at multiple scales. Then, this information is combined with channel attention to obtain richer global information interaction. For a given input feature map, a 7×7 convolution is first used to obtain features at a larger scale. Then, a 3×3 convolution is used for fine feature extraction. Finally, the generated feature map is passed through a 3×3 convolutional layer containing a sigmoid activation function to generate a segmentation result.

[0035] Preferably, in step (6), the weights and biases of the convolutional neural network are adjusted according to the backpropagation algorithm.

[0036] Preferably, in step (6), when calculating the loss function, the stochastic gradient descent algorithm is used to solve for the minimum value of the cross-entropy loss function.

[0037] Preferably, in step (8), the image is binarized using a threshold size of 0.5.

[0038] Another object of the present invention is to provide a hybrid attention UNet ultrasound image segmentation device based on pseudo-color enhancement, comprising:

[0039] The initial model building unit is used to construct a convolutional neural network containing encoding and decoding layers;

[0040] The preprocessing unit is used to preprocess the ultrasound images in the training set;

[0041] The pseudo-color conversion unit is used to perform pseudo-color conversion on ultrasound images according to the pseudo-color enhancement algorithm.

[0042] The coding unit is used to input the ultrasound image after pseudo-color conversion into the coding layer for downsampling and feature extraction, and to complete the channel information interaction between features;

[0043] The decoding unit is used to simultaneously input the downsampled feature map, which has undergone long-range dependency capture, feature representation and normalization, and the attention-weighted feature map into the decoding layer to generate a segmentation map and calculate the loss function;

[0044] The convergence unit is used to adjust the weights and biases of the convolutional neural network until the loss function converges, thus obtaining the optimal training model.

[0045] The target processing unit is used to call the optimal training model to process the preprocessed target ultrasound image and obtain the target segmentation feature map;

[0046] The final segmentation map generation unit performs a Sigmoid operation on the target segmentation feature map, calculates the loss function, and then binarizes the image to obtain the final segmentation map.

[0047] The ultrasound image segmentation device may also include a fusion unit for fusing feature maps of the decoding layer and the corresponding coding layer.

[0048] This invention presents a hybrid attention UNet ultrasound image segmentation method based on pseudo-color enhancement. By combining classic pseudo-color enhancement algorithms with deep learning methods, it effectively addresses the segmentation difficulties caused by poor imaging quality, artifacts, and low contrast in ultrasound images. Specifically, this invention enhances channel information interaction without increasing the number of additional parameters, obtaining more refined features. Furthermore, it can fuse feature maps from the decoding and corresponding coding layers to achieve pixel-level prediction results and retain more useful information during decoding information recovery, thus enhancing segmentation performance. After training a convolutional neural network, an optimal training model is obtained. The image to be segmented is then preprocessed, and the optimal training model is used for prediction. The probability output matrix is ​​obtained through a sigmoid function and then transformed into the final segmentation map. This invention fully extracts global and local information from the image and captures the interaction between channel and spatial information, improving the accuracy of ultrasound image segmentation. Attached Figure Description

[0049] Figure 1 The flowchart shows the hybrid attention UNet ultrasound image segmentation method based on pseudo-color enhancement.

[0050] Figure 2 This is a diagram of the structure of a convolutional neural network;

[0051] Figure 3 A structural diagram of the channel and space attention module;

[0052] Figure 4 For high-efficiency channel attention module structure diagram;

[0053] Figure 5 This is a pseudo-color enhancement result image;

[0054] Figure 6 A structural diagram of the multi-scale channel attention module;

[0055] Figure 7 This is a segmentation result image of a sample in the ultrasound dataset. Detailed Implementation

[0056] To help those skilled in the art better understand the improvements of this invention compared to the prior art, the invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0057] This embodiment mainly relates to a hybrid attention UNet ultrasound image segmentation method based on pseudo-color enhancement, hereinafter referred to as "this method", and its implementation process is as follows: Figure 1 As shown.

[0058] The ultrasound image dataset used in this embodiment consists of four publicly available datasets and one self-collected dataset, divided into three parts: a training set, a validation set, and a test set. When applying this method in practice, the images to be segmented should be determined based on the actual acquisition requirements. In this embodiment, the dataset is randomly divided into training, validation, and test sets in a 7:1:2 ratio. The segmentation objects include two categories: lesion regions and background. The lesion segmentation results for some ultrasound images are shown below. Figure 7 As shown, the results of segmenting ultrasound images using this method and other ultrasound image segmentation methods are illustrated. The structure of the convolutional neural network constructed in this embodiment is as follows: Figure 2 As shown.

[0059] (1) First, the ultrasound images and corresponding labels in the training set are preprocessed and data augmented. Then, they are input into the encoder of the constructed ultrasound segmentation network (a convolutional neural network containing an encoder layer and a decoder layer; the encoder layer contains an encoder, and the decoder layer contains a decoder) for image downsampling and abstract feature extraction. The preprocessing and data augmentation here mainly include vertical flipping, horizontal flipping, and image rotation. Finally, the resolution of all images is adjusted to 224×224. Subsequently, the images are converted to pseudo-color according to the traditional pseudo-color enhancement algorithm. For grayscale images displayed with 8 or more bits of grayscale values, the grayscale value of one pixel is represented by f(x, y), where 0 ≤ f(x, y) ≤ 255. The pseudo-color conversion method is as follows:

[0060]

[0061] I=g(x,y)………………………….(1.2)

[0062] H=2πg(x,y) / 255……………………(1.3)

[0063]

[0064] Where H, S, and I represent hue, saturation, and intensity in the HIS color space, respectively. The coefficients in equation (1.3) ensure that H remains within the range of 0 to 2π as f(x, y) changes. Furthermore, different saturations from 0 to 255 can be selected by changing k from 0 to 2. The effect of pseudo-color enhancement is shown in the figure below. Figure 5 As shown.

[0065] (2) The ultrasound image after pseudo-color conversion is input into the encoder of this method, and the interaction of channel information between features is improved through an efficient channel attention module. The channel attention module uses one-dimensional convolution to construct a local cross-channel interaction method that does not require dimensionality reduction and can be completed efficiently. Its structure is as follows: Figure 4As shown, an efficient channel attention module is linked after each residual block to improve the interaction of channel information between features. The efficient channel attention module is a method that utilizes one-dimensional convolution to construct a local cross-channel interaction that does not require dimensionality reduction. Therefore, given aggregated features y∈R without dimensionality reduction... C Channel attention can be learned in the following ways:

[0066] ω=σ(C1D k (y))………………(1.5)

[0067] Where C1D represents one-dimensional convolution, σ represents the sigmoid function, and k represents the number of adjacent channels of y.

[0068] The downsampled feature map after encoder inputs into the bottleneck structure, where the channel and spatial attention module captures long-range dependencies and effectively utilizes multi-channel space for feature representation and normalization. The structure diagram of the channel and spatial attention module is shown below. Figure 3 As shown, a 1×1 convolution is performed before the feature map with the largest receptive field is input into the CSAM block to increase non-linearity. After obtaining the attention-weighted feature map, it is simultaneously input into the decoder branch and passed through the multi-scale channel attention module, which preserves more useful information in the decoding information recovery and enhances the segmentation effect. The multi-scale channel attention module is as follows: Figure 6 As shown, this module fuses feature maps from the decoding and corresponding encoding layers to obtain pixel-level prediction results. First, it uses convolutional kernels of different sizes to obtain feature information at multiple scales, then combines it with channel attention to obtain richer global information interaction. For a given input feature map, a 7×7 convolution is first used to obtain larger-scale features, then a 3×3 convolution is used for fine feature extraction. Finally, the generated feature map is input into the channel attention module, where a 3×3 convolutional layer with a sigmoid activation function generates the segmentation result. The decoder is used to generate the segmentation map and calculate the loss function; here, the sigmoid function is used to predict the pixel category. The loss function is calculated using a combination of cross-entropy and Dice loss functions, where the cross-entropy loss is calculated as follows:

[0069]

[0070] Where yi represents the true label of the sample, pi represents the predicted label, and N is the total number of pixels in the image.

[0071] Similarly, the Dice loss can be written as:

[0072]

[0073] Where X represents the label value, Y represents the predicted result, and pi and yi represent the values ​​of the i-th pixel of the expected result and label, respectively. The smoothing parameter s prevents division by zero and is set to 1.

[0074] To address the class imbalance problem, this embodiment uses weighted Dice loss and BCE loss for deep supervision loss and final segmentation:

[0075]

[0076] Where λ represents the trade-off parameter between the two losses, and in this embodiment, λ is set to 0.3. final This refers to the loss from the final segmentation. Then, the weights and biases of the convolutional neural network are adjusted using the backpropagation algorithm until the loss function converges (error is minimized), resulting in the optimal training model. Specifically, the stochastic gradient descent algorithm is used to find the minimum value of the cross-entropy loss function, thus obtaining the optimized training model.

[0077] (3) To utilize the optimal training model obtained from the training, this embodiment performs image preprocessing on the images and labels in the test set. No data augmentation is performed here; the images are resized to 224×224 to fit the model input, and then pseudo-color conversion is performed. After being fed into the model, the weight parameters of the optimal training model are loaded for forward inference. The corresponding target segmentation feature maps are obtained, and after performing a Sigmoid operation, the loss function is calculated. Finally, the images are binarized using a threshold of 0.5 to obtain the final segmentation map.

[0078] The specific implementation steps are as follows:

[0079] First, the image is preprocessed and resized to 224×224 pixels to fit the required size of the model.

[0080] Then, the optimal model obtained from training is loaded and tested on the preprocessed test image, using two separate decoders to obtain the feature map of the target.

[0081] The sigmoid operation yields a segmentation probability map, and a weighted loss function is used to calculate the loss, resulting in the final probability output matrix. Testing shows that this embodiment achieves a Dice accuracy of over 80% for ultrasound image segmentation, demonstrating high segmentation accuracy.

[0082] This invention first designs a hybrid attention UNet ultrasound image segmentation method based on pseudo-color enhancement, combining classic pseudo-color enhancement algorithms with deep learning methods to effectively solve the segmentation difficulties caused by poor imaging quality, artifacts, and low contrast in ultrasound images. During the encoding stage, an efficient channel attention module and channel and spatial attention modules are combined to enhance channel information interaction and obtain more refined features without increasing the number of additional parameters. This method also proposes a multi-scale channel attention module to retain more useful information and enhance segmentation results during decoding information recovery. After training the network to obtain the optimal training model, the images to be tested are preprocessed, and the optimal training model is used to predict them. The probability output matrix obtained by the sigmoid function is then transformed into the final segmentation map, fully extracting the global and local information of the image and capturing the channel and spatial information interaction, thus improving the accuracy of ultrasound image segmentation.

[0083] The above embodiments are preferred implementations of the present invention. In addition, the present invention can be implemented in other ways. Any obvious substitutions without departing from the concept of the present technical solution are within the protection scope of the present invention.

[0084] Finally, it should be emphasized that, in order to make it easier for those skilled in the art to understand the improvements of the present invention compared with the prior art, some descriptions of the present invention have been simplified, and for clarity, some other elements have been omitted from this application. Those skilled in the art should realize that these omitted elements can also constitute the content of the present invention.

Claims

1. A hybrid attention UNet ultrasound image segmentation method based on pseudo-color enhancement, characterized in that, Includes the following steps: (1) Construct a convolutional neural network containing an encoding layer and a decoding layer; (2) Preprocess the ultrasound images in the training set; (3) Perform pseudo-color conversion on the ultrasound image according to the pseudo-color enhancement algorithm; In step (3), the pseudo-color conversion method is as follows: (1.1) (1.2) (1.3) (1.4) Where f(x, y) represents the gray value of a pixel, 0 ≤ f(x, y) ≤ 255; H, S, and I represent hue, saturation, and intensity in the HIS color space, respectively; and different saturations from 0 to 255 are selected by changing k from 0 to 2. (4) Input the ultrasound image after pseudo-color conversion into the coding layer for downsampling and feature extraction, and realize the channel information interaction between features; In step (4), during the interaction of channel information between features, channel attention is learned in the following way: (1.5) Among them, the aggregated features y∈R without dimensionality reduction C C1D represents one-dimensional convolution, σ represents the sigmoid function, and k represents the number of adjacent channels of y. (5) Input the downsampled feature map after the encoder into the bottleneck structure. The channel and spatial attention module captures the long-range dependency relationship and uses the multi-channel space to perform feature representation and normalization to obtain the feature map. The feature map obtained and the attention-weighted feature map are simultaneously input into the decoding layer to generate the segmentation map and calculate the loss function. In step (5), after the feature map is input into the decoding layer, the feature maps of the decoding layer and the corresponding encoding layer are fused. In step (5), the steps for fusing the feature maps of the decoding layer and the corresponding coding layer are as follows: For a given input feature map, a 7×7 convolution is first used to obtain features at a larger scale, and then a 3×3 convolution is used for fine feature extraction. The generated feature map is then input into the channel attention module, and finally a 3×3 convolutional layer containing a sigmoid activation function generates the segmentation result. (6) Adjust the weights and biases of the convolutional neural network until the loss function converges to obtain the optimal training model; (7) The optimal training model is called to process the preprocessed target ultrasound image to obtain the target segmentation feature map; (8) Perform a Sigmoid operation on the target segmentation feature map, then calculate the loss function, and then binarize the image to obtain the final segmentation map.

2. The ultrasound image segmentation method based on pseudo-color enhancement using hybrid attention UNet according to claim 1, characterized in that, In step (5), the loss function is calculated using a combination of the cross-entropy loss function and the Dice loss function, where the cross-entropy loss function is: ； Where y represents the true label of the sample, p represents the predicted label, and N is the total number of pixels in the image; The Dice loss function is: ； Where X represents the label value, Y represents the prediction result, and pi and yi represent the value of the i-th pixel of the expected result and label, respectively; the smoothing parameter s is 1; The deep supervision loss and final segmentation are performed using weighted Dice loss and BCE loss: ； Where λ represents the trade-off parameter between the two losses, λ=0.3; L final The loss is the final segmentation cost.

3. The ultrasound image segmentation method based on pseudo-color enhancement using hybrid attention UNet according to claim 1, characterized in that, In step (6), the weights and biases of the convolutional neural network are adjusted according to the backpropagation algorithm.

4. The ultrasound image segmentation method based on pseudo-color enhancement using hybrid attention UNet according to claim 1, characterized in that, In step (6), when calculating the loss function, the stochastic gradient descent algorithm is used to find the minimum value of the cross-entropy loss function.

5. The hybrid attention UNet ultrasound image segmentation method based on pseudo-color enhancement according to claim 1, characterized in that, In step (8), the image is binarized using a threshold of 0.

5.

6. A hybrid attention UNet ultrasound image segmentation device based on pseudo-color enhancement, characterized in that, include: The initial model building unit is used to construct a convolutional neural network containing encoding and decoding layers; The preprocessing unit is used to preprocess the ultrasound images in the training set; The pseudo-color conversion unit is used to perform pseudo-color conversion on ultrasound images according to the pseudo-color enhancement algorithm. The pseudo-color conversion method is as follows: (1.1) (1.2) (1.3) (1.4) Where f(x, y) represents the gray value of a pixel, 0 ≤ f(x, y) ≤ 255; H, S, and I represent hue, saturation, and intensity in the HIS color space, respectively; and different saturations from 0 to 255 are selected by changing k from 0 to 2. The coding unit is used to input the ultrasound image after pseudo-color conversion into the coding layer for downsampling and feature extraction, and to complete the channel information interaction between features; During the interaction of channel information between features, channel attention is learned in the following ways: (1.5) Among them, the aggregated features y∈R without dimensionality reduction C C1D represents one-dimensional convolution, σ represents the sigmoid function, and k represents the number of adjacent channels of y. The decoding unit is used to input the downsampled feature map after the encoder into the bottleneck structure. The feature map obtained by the channel and spatial attention module capturing long-range dependencies and using multi-channel space for feature representation and normalization, along with the attention-weighted feature map, is simultaneously input into the decoding layer to generate a segmentation map and calculate the loss function. After the feature map is input into the decoding layer, the feature maps of the decoding layer and the corresponding encoding layer are fused. The steps for fusing the feature maps from the decoding layer and the corresponding coding layer are as follows: For a given input feature map, a 7×7 convolution is first used to obtain features at a larger scale, and then a 3×3 convolution is used for fine feature extraction. The generated feature map is then input into the channel attention module, and finally a 3×3 convolutional layer containing a sigmoid activation function generates the segmentation result. The convergence unit is used to adjust the weights and biases of the convolutional neural network until the loss function converges, thus obtaining the optimal training model. The target processing unit is used to call the optimal training model to process the preprocessed target ultrasound image and obtain the target segmentation feature map; The final segmentation map generation unit performs a Sigmoid operation on the target segmentation feature map, calculates the loss function, and then binarizes the image to obtain the final segmentation map.

Images