An ultrasonic image segmentation method, system, device and medium based on a U-shaped codec structure

Abstract

The application discloses an ultrasonic image segmentation method, system, device and medium based on a U-shaped codec structure, comprising: acquiring an ultrasonic image; inputting the ultrasonic image into an ultrasonic image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasonic image segmentation model is constructed based on a U-shaped codec network and an attention mechanism, the network structure of the ultrasonic image segmentation model is provided with five layers from top to bottom, each layer is a symmetric structure of an encoder part and a decoder part, and an attention module is introduced into the skip link part of each layer. The technical scheme disclosed by the application greatly increases the ultrasonic medical image segmentation precision while the computing power is not increased much, and thus can provide a more powerful basis for clinical diagnosis.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, in particular to an ultrasound image segmentation method, system, device and medium based on a U-shaped codec structure. BACKGROUND

[0002] Ultrasound image is a kind of medical image using sound waves to image in real time, which has the characteristics of convenience, real-time, non-destructive and so on. Ultrasound image can well evaluate the dynamic changes of organs and tissues, and is convenient for doctors to diagnose patients, and is widely used all over the world. However, ultrasound image also has many shortcomings, such as:

[0003] Low contrast and noise problem: ultrasound image is often disturbed by noise, and the image quality is low and the contrast is low, which makes it difficult to extract the target region from the ultrasound image.

[0004] Due to the influence of the resolution and noise of the ultrasound image, the boundaries of many tissues are not clear and it is difficult to accurately locate, and at the same time due to the propagation characteristics of ultrasound and the sound reflection rate of different tissues, there may be large differences in brightness and texture in the image.

[0005] In addition, since ultrasound image is real-time imaging, a lighter, faster and more efficient segmentation network is needed to assist doctors in diagnosis. The existing network requires too much computing power and too long training time, which is not conducive to the segmentation of ultrasound image. SUMMARY

[0006] The purpose of the present application is to provide an ultrasound image segmentation method, system, device and medium based on a U-shaped codec structure to solve the problems existing in the prior art.

[0007] To achieve the above purpose, the present application provides an ultrasound image segmentation method based on a U-shaped codec structure, comprising:

[0008] obtaining an ultrasound image;

[0009] inputting the ultrasound image into an ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-shaped codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is provided with 5 layers from top to bottom, each layer is a symmetrical codec structure, a channel attention module is introduced into the first layer, the second layer, the third layer and the fourth layer, and the second layer, the third layer, the fourth layer and the fifth layer each output an output mask, the output mask is used to calculate edge loss.

[0010] Optionally, the training process of the ultrasound image segmentation model comprises:

[0011] Acquiring training data, wherein the training data includes ultrasound training images and corresponding segmentation results;

[0012] An initial ultrasound image segmentation model is constructed, the training data is input into the initial ultrasound image segmentation model for image segmentation, and training is performed with the goal of minimizing the loss between the initial training result after image segmentation and the segmentation result corresponding to the ultrasound training image to obtain a trained ultrasound image segmentation model.

[0013] Optionally, the processing of the ultrasound image segmentation model specifically includes:

[0014] Inputting the ultrasound image into the second layer, the third layer, and the fourth layer of the encoder part of the ultrasound image segmentation model for downsampling operations to obtain feature maps t1, t2, and t3;

[0015] The feature map t3 is sequentially input into two layers of Tok MLP layers to extract features across height and width, and the fourth layer feature map t4 and the fifth layer feature map t5 are obtained. At the same time, the t5 feature map generates a prediction mask and calculates the edge loss;

[0016] The feature map t5 is input into the third layer of the decoder for upsampling to obtain the first upsampled feature map. The first upsampled feature map is added to the feature map t4, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the first upsampled feature map to obtain feature map t6. At the same time, the t6 feature map generates a prediction mask and calculates the edge loss;

[0017] The feature map t6 is input into the third layer of the decoder for upsampling to obtain the second upsampled feature map. The second upsampled feature map is added to the feature map t3, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the second upsampled feature map to obtain feature map t7. At the same time, the t7 feature map generates a prediction mask and calculates the edge loss;

[0018] The feature map t7 is input into the second layer of the decoder for upsampling to obtain the third upsampled feature map. The third upsampled feature map is added to the feature map t2, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations, and then added to the third upsampled feature map to obtain feature map t8. At the same time, the t8 feature map generates a prediction mask and calculates the edge loss;

[0019] The feature map t8 is input into the first layer of the decoder for upsampling to obtain the fourth upsampled feature map. The fourth upsampled feature map is added to the feature map t1, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations, and then added to the fourth upsampled feature map to obtain feature map t9;

[0020] A 1*1 convolution operation is performed on the feature map t9 to obtain the segmentation result.

[0021] Optionally, the specific process of performing the downsampling operation specifically includes:

[0022] The ultrasound image is subjected to convolution, normalization, pooling and activation processing to obtain a feature map.

[0023] Optionally, the number of channels and the resolution of the feature map t1, the feature map t2, and the feature map t3 are different.

[0024] An ultrasound image segmentation system based on a U-type codec structure, comprising:

[0025] A data acquisition module, used for acquiring ultrasound images;

[0026] An image segmentation module is used to input the ultrasound image into an ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-type codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is provided with 5 layers from top to bottom, each layer is a symmetrical codec structure, the first layer, the second layer, the third layer and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss.

[0027] An electronic device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform an ultrasound image segmentation method based on a U-type codec structure.

[0028] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the ultrasonic image segmentation method based on a U-type codec structure.

[0029] The technical effects of the present invention are:

[0030] The present invention uses a codec network to segment medical ultrasound images, which takes advantage of the advantages of CNN being good at capturing local features and the attention mechanism being good at capturing global features to perform feature map processing and generate more accurate segmentation results. Since ultrasound images have the characteristics of real-time imaging, the corresponding segmentation network is required to be lightweight, fast and effective. The present invention adds three channel attention modules to the basic CNN network, which greatly increases the accuracy of ultrasound medical image segmentation while not increasing the computing power much, thereby providing a more powerful basis for clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0033] Figure 1 Schematic diagram of the original ultrasound image segmentation network in an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of a network with an attention mechanism added in an embodiment of the present invention;

[0035] Figure 3 Schematic diagram of the structure of the CSEA block in an embodiment of the present invention;

[0036] Figure 4 FIG. 4 is a flowchart of an ultrasonic image segmentation implementation in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as limiting the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0038] It should be understood that the terms described herein are intended only to describe particular embodiments and are not intended to limit the present invention. In addition, for numerical ranges herein, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Each smaller range between any intermediate value within a stated value or stated range and any other stated value or intermediate value within the stated range is also encompassed by the present invention. The upper and lower limits of these smaller ranges may be independently included or excluded within the scope.

[0039] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present invention without departing from the scope or spirit of the invention. Other embodiments will be apparent to those skilled in the art from the present invention. The present description and examples are intended to be illustrative only.

[0040] The words “include,” “including,” “have,” “contain,” etc. used in this article are open-ended terms, meaning including but not limited to.

[0041] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] Example 1

[0043] like Figure 1 - Figure 4 As shown, this embodiment provides an ultrasound image segmentation method based on a U-type codec structure, including:

[0044] Acquire ultrasound images;

[0045] The ultrasound image is input into the ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein, the ultrasound image segmentation model is constructed based on a U-type codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is provided with 5 layers from top to bottom, each layer is a symmetrical codec structure, the first layer, the second layer, the third layer and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss.

[0046] This embodiment provides an ultrasound image segmentation method and system based on a U-type codec structure, which relates to the field of computer vision. We introduce the skip connection part of the original network into the channel attention module, correct the target features from the perspective of channel attention, and generate a corresponding prediction mask at each upsampling stage, calculate the edge loss, and help the model learn important features more quickly. Specifically, it includes the following steps: input the collected ultrasound image; the ultrasound image is converted into a feature map after convolution, normalization, pooling and activation processing; then downsampling twice; subsequently, the feature map is passed to the Tok MLP layer for processing and then upsampling; the fifth layer feature map is upsampled once and added to the feature map of the downsampled part; then it passes through a SEA module and is added to the upsampled feature map; then the fourth, third and second layer feature maps of the upsampled part are upsampled and added to the feature maps of the third, second and first layers of the downsampled part; then it passes through a SEA module; finally, a 1*1 convolution is used to generate the final category prediction for each pixel or spatial position.

[0047] The specific implementation plan of this embodiment is as follows:

[0048] Input the acquired ultrasound image;

[0049] The ultrasound image is transformed into feature map t1 after convolution, normalization, pooling and activation;

[0050] Then, the feature map is processed twice in the same way to generate feature maps t2 and t3 respectively, and the detailed features are captured by increasing the number of channels and reducing the resolution.

[0051] The number of channels of feature maps t1, t2, and t3 are 16, 32, and 128 respectively;

[0052] The feature map is then fed into the Tok MLP (Tokenized Multilayer Perceptron), where the input feature map is divided into multiple small blocks and embedded into a vector space, shifted in height and width to induce window locality. The features are then transferred through a deep convolutional layer and max pooling is performed to further reduce the resolution of the feature map to generate feature map t4.

[0053] After that, the upsampling is passed into the same TokMLP for feature map processing to generate feature map t5;

[0054] The up-sampled feature map is added to the previously down-sampled feature map t4 and then processed by the SEA (Squeeze-and-Excitation Attention) module. By performing "squeeze" and "excitation" operations on the input feature map, the network can learn the importance of each channel and perform weighted adjustments on it. The new feature map is then added to the up-sampled feature map to generate a new feature map.

[0055] The generated feature map is then upsampled and added to the feature map t3 generated by the previous downsampling, and then passed through a SEA module and added to the upsampled feature map to generate a feature map;

[0056] The feature map is then upsampled and added to the feature map t2 generated by the previous downsampling, and then passed through a SEA module and added to the upsampled feature map to generate a feature map;

[0057] The feature map generated by upsampling again is added to the previous feature map t1 and passed through the SEA module to generate a feature map again, which is then added to the upsampled feature map to generate the final feature map;

[0058] Finally, a 1*1 convolution is performed to generate the final category prediction for each pixel or spatial position.

[0059] This embodiment uses a codec network to segment medical ultrasound images, and uses the advantages of CNN's ability to capture local features and the attention mechanism's ability to capture global features to process feature maps. At the same time, a deep supervised edge loss function is introduced to assist segmentation and generate more accurate segmentation results. Due to the real-time imaging characteristics of ultrasound images, the corresponding segmentation network is required to be lightweight, fast, and effective. This embodiment adds three channel attention modules to the basic CNN network, which greatly increases the accuracy of ultrasound medical image segmentation while not increasing the computing power much, thereby providing a more powerful basis for clinical diagnosis.

[0060] An ultrasound image segmentation system based on a U-type codec structure, comprising:

[0061] A data acquisition module, used for acquiring ultrasound images;

[0062] An image segmentation module is used to input the ultrasound image into an ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-type codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is provided with 5 layers from top to bottom, each layer is a symmetrical codec structure, the first layer, the second layer, the third layer and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss.

[0063] An electronic device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform an ultrasound image segmentation method based on a U-type codec structure.

[0064] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the ultrasonic image segmentation method based on a U-type codec structure.

[0065] The above description is merely a preferred embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An ultrasound image segmentation method based on a U-type codec structure, characterized in that: include: Acquire ultrasound images; Inputting the ultrasound image into an ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-shaped codec network and an attention mechanism, the network structure of the ultrasound image segmentation model is provided with 5 layers from top to bottom, each layer is a symmetrical codec structure, the first layer, the second layer, the third layer, and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer, and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss; The processing process of the ultrasound image segmentation model specifically includes: Inputting the ultrasound image into the second layer, the third layer, and the fourth layer of the encoder part of the ultrasound image segmentation model for downsampling operations to obtain feature maps t1, t2, and t3; The feature map t3 is sequentially input into two layers of Tok MLP layers to extract features across height and width, and the fourth layer feature map t4 and the fifth layer feature map t5 are obtained. At the same time, the t5 feature map generates a prediction mask and calculates the edge loss; The feature map t5 is input into the third layer of the decoder for upsampling to obtain the first upsampled feature map. The first upsampled feature map is added to the feature map t4, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the first upsampled feature map to obtain feature map t6. At the same time, the t6 feature map generates a prediction mask and calculates the edge loss; The feature map t6 is input into the third layer of the decoder for upsampling to obtain the second upsampled feature map. The second upsampled feature map is added to the feature map t3, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the second upsampled feature map to obtain feature map t7. At the same time, the t7 feature map generates a prediction mask and calculates the edge loss; The feature map t7 is input into the second layer of the decoder for upsampling to obtain the third upsampled feature map. The third upsampled feature map is added to the feature map t2, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations, and then added to the third upsampled feature map to obtain feature map t8. At the same time, the t8 feature map generates a prediction mask and calculates the edge loss; The feature map t8 is input into the first layer of the decoder for upsampling to obtain the fourth upsampled feature map. The fourth upsampled feature map is added to the feature map t1, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations, and then added to the fourth upsampled feature map to obtain feature map t9; A 1*1 convolution operation is performed on the feature map t9 to obtain the segmentation result.

2. The ultrasonic image segmentation method based on the U-type codec structure according to claim 1, characterized in that: The training process of the ultrasound image segmentation model specifically includes: Acquiring training data, wherein the training data includes ultrasound training images and corresponding segmentation results; An initial ultrasound image segmentation model is constructed, the training data is input into the initial ultrasound image segmentation model for image segmentation, and training is performed with the goal of minimizing the loss between the initial training result after image segmentation and the segmentation result corresponding to the ultrasound training image to obtain a trained ultrasound image segmentation model.

3. The ultrasonic image segmentation method based on the U-type codec structure according to claim 1, characterized in that: The specific process of performing the downsampling operation specifically includes: The ultrasound image is subjected to convolution, normalization, pooling and activation processing to obtain a feature map.

4. The ultrasonic image segmentation method based on the U-type codec structure according to claim 1, characterized in that: The number of channels and resolution of the feature map t1, the feature map t2 and the feature map t3 are different.

5. An ultrasound image segmentation system based on a U-type codec structure, characterized in that: include: A data acquisition module, used for acquiring ultrasound images; An image segmentation module is configured to input the ultrasound image into an ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-shaped codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is provided with 5 layers from top to bottom, each layer is a symmetrical codec structure, the first layer, the second layer, the third layer, and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer, and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss; The processing process of the ultrasound image segmentation model specifically includes: Inputting the ultrasound image into the second layer, the third layer, and the fourth layer of the encoder part of the ultrasound image segmentation model for downsampling operations to obtain feature maps t1, t2, and t3; The feature map t3 is sequentially input into two layers of Tok MLP layers to extract features across height and width, and the fourth layer feature map t4 and the fifth layer feature map t5 are obtained. At the same time, the t5 feature map generates a prediction mask and calculates the edge loss; The feature map t5 is input into the third layer of the decoder for upsampling to obtain the first upsampled feature map. The first upsampled feature map is added to the feature map t4, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the first upsampled feature map to obtain feature map t6. At the same time, the t6 feature map generates a prediction mask and calculates the edge loss; The feature map t6 is input into the third layer of the decoder for upsampling to obtain the second upsampled feature map. The second upsampled feature map is added to the feature map t3, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the second upsampled feature map to obtain feature map t7. At the same time, the t7 feature map generates a prediction mask and calculates the edge loss; The feature map t7 is input into the second layer of the decoder for upsampling to obtain the third upsampled feature map. The third upsampled feature map is added to the feature map t2, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations, and then added to the third upsampled feature map to obtain feature map t8. At the same time, the t8 feature map generates a prediction mask and calculates the edge loss; The feature map t8 is input into the first layer of the decoder for upsampling to obtain the fourth upsampled feature map. The fourth upsampled feature map is added to the feature map t1, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations, and then added to the fourth upsampled feature map to obtain feature map t9; A 1*1 convolution operation is performed on the feature map t9 to obtain the segmentation result.

6. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform an ultrasound image segmentation method based on a U-type codec structure according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The device stores a computer program, which, when executed by a processor, implements an ultrasound image segmentation method based on a U-type codec structure according to any one of claims 1 to 4.

Images