Ultrasonic image segmentation method, system and equipment based on U-shaped codec structure and medium

Abstract

The invention discloses an ultrasonic image segmentation method, system and device based on a U-shaped codec structure and a medium. The method comprises the following steps: acquiring an ultrasonic image; inputting the ultrasonic image into an ultrasonic image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasonic image segmentation model is constructed based on a U-shaped codec network and an attention mechanism, the network structure of the ultrasonic image segmentation model is provided with five layers from top to bottom, each layer is provided with an encoder part and a decoder part which are of a symmetrical structure, and an attention module is introduced into a jump link part of each layer. According to the technical scheme, the ultrasonic medical image segmentation precision is greatly improved while the computing power is not greatly increased, so that a more powerful basis can be provided for clinical diagnosis.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to an ultrasonic image segmentation method, system, device and medium based on a U-type codec structure. Background Art

[0002] Ultrasound images are a type of medical image that uses sound waves to form real-time images. They are convenient, real-time, and non-destructive. Ultrasound images can well evaluate the dynamic changes of organs and tissues, making it easier for doctors to diagnose patients. They are widely used around the world. However, ultrasound images also have many disadvantages, such as:

[0003] Low contrast and noise problem: Ultrasound images are often interfered by noise, resulting in low image quality and low contrast, which makes it difficult to extract the target area from the ultrasound image.

[0004] Due to the resolution and noise of ultrasound images, the boundaries of many tissues are unclear and difficult to locate accurately. At the same time, due to the propagation characteristics of ultrasound and the sound wave reflectivity of different tissues, there may be large differences in brightness and texture in the image.

[0005] In addition, since ultrasound images are real-time images, a lighter, faster, and more efficient segmentation network is needed to assist doctors in diagnosis. The existing network requires too much computing power and takes too long to train, which is not conducive to the segmentation of ultrasound images. Summary of the invention

[0006] The purpose of the present invention is to provide an ultrasonic image segmentation method, system, device and medium based on a U-type codec structure to solve the problems existing in the above-mentioned prior art.

[0007] To achieve the above object, the present invention provides an ultrasonic image segmentation method based on a U-type codec structure, comprising:

[0008] Acquiring ultrasound images;

[0009] The ultrasound image is input into the ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-type codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is arranged with 5 layers from top to bottom, and each layer is a symmetrical codec structure, and the first layer, the second layer, the third layer and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss.

[0010] Optionally, the training process of the ultrasound image segmentation model specifically includes:

[0011] Acquiring training data, wherein the training data includes an ultrasound training image and a corresponding segmentation result;

[0012] An initial ultrasound image segmentation model is constructed, the training data is input into the initial ultrasound image segmentation model for image segmentation, and training is performed with the goal of minimizing the loss between the initial training result after image segmentation and the segmentation result corresponding to the ultrasound training image, so as to obtain a trained ultrasound image segmentation model.

[0013] Optionally, the processing process of the ultrasound image segmentation model specifically includes:

[0014] Inputting the ultrasonic image into the second layer, the third layer and the fourth layer of the encoder part of the ultrasonic image segmentation model for downsampling operation to obtain feature maps t1, t2 and t3;

[0015] The feature map t3 is sequentially input into two layers of Tok MLP layers to extract features across height and width, and the fourth layer feature map t4 and the fifth layer feature map t5 are obtained. At the same time, the t5 feature map generates a prediction mask and calculates the edge loss;

[0016] The feature map t5 is input into the third layer of the decoder part for upsampling operation to obtain the first upsampled feature map. The first upsampled feature map is added to the feature map t4, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the first upsampled feature map to obtain feature map t6. At the same time, the t6 feature map generates a prediction mask and calculates the edge loss;

[0017] The feature map t6 is input into the third layer of the decoder part for upsampling operation to obtain the second upsampled feature map. The second upsampled feature map is added to the feature map t3, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the second upsampled feature map to obtain feature map t7. At the same time, the t7 feature map generates a prediction mask and calculates the edge loss.

[0018] The feature map t7 is input into the second layer of the decoder part for upsampling operation to obtain the third upsampled feature map, the third upsampled feature map is added to the feature map t2, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the third upsampled feature map to obtain feature map t8. At the same time, the t8 feature map generates a prediction mask and calculates the edge loss;

[0019] The feature map t8 is input into the first layer of the decoder part for upsampling operation to obtain the fourth upsampling feature map, the fourth upsampling feature map is added to the feature map t1, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations, and then added to the fourth upsampling feature map to obtain the feature map t9;

[0020] A 1*1 convolution operation is performed on the feature map t9 to obtain the segmentation result.

[0021] Optionally, the specific process of performing the downsampling operation specifically includes:

[0022] The ultrasound image is subjected to convolution, normalization, pooling and activation processing to obtain a feature map.

[0023] Optionally, the number of channels and the resolution of the feature map t1, the feature map t2 and the feature map t3 are different.

[0024] An ultrasonic image segmentation system based on a U-type codec structure, comprising:

[0025] A data acquisition module, used for acquiring ultrasound images;

[0026] An image segmentation module is used to input the ultrasound image into an ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-type codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is provided with 5 layers from top to bottom, and each layer is a symmetrical codec structure, and the first layer, the second layer, the third layer and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss.

[0027] An electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform an ultrasound image segmentation method based on a U-type codec structure.

[0028] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for segmenting an ultrasonic image based on a U-type codec structure is implemented.

[0029] The technical effects of the present invention are:

[0030] The present invention uses a codec network to perform medical ultrasound image segmentation, which takes advantage of the advantages of CNN being good at capturing local features and the attention mechanism being good at capturing global features to perform feature map processing and generate more accurate segmentation results. Since ultrasound images have the characteristics of real-time imaging, the corresponding segmentation network is required to be lightweight, fast and effective. The present invention adds three channel attention modules to the basic CNN network, which greatly increases the accuracy of ultrasound medical image segmentation while not increasing the computing power much, thereby providing a more powerful basis for clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0033] Figure 1 is a schematic diagram of an original ultrasound image segmentation network in an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of a network in which an attention mechanism is added in an embodiment of the present invention;

[0035] Figure 3 is a schematic diagram of the structure of a CSEA block in an embodiment of the present invention;

[0036] Figure 4 The figure is a flowchart of ultrasound image segmentation implementation in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as limiting the present invention, but should be understood as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0038] It should be understood that the terms described in the present invention are only for describing special embodiments and are not intended to limit the present invention. In addition, for the numerical range in the present invention, it should be understood that each intermediate value between the upper and lower limits of the scope is also specifically disclosed. Each smaller range between the intermediate value in any stated value or stated range and any other stated value or intermediate value in the described range is also included in the present invention. The upper and lower limits of these smaller ranges can be independently included or excluded in the scope.

[0039] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present invention description without departing from the scope or spirit of the present invention. Other embodiments derived from the present invention description will be apparent to those skilled in the art. The present application description and examples are exemplary only.

[0040] The words “include,” “including,” “have,” “contain,” etc. used in this article are open-ended terms, meaning including but not limited to.

[0041] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] Embodiment 1

[0043] like Figure 1 - Figure 4 As shown, this embodiment provides an ultrasound image segmentation method based on a U-type codec structure, including:

[0044] Acquiring ultrasound images;

[0045] The ultrasound image is input into the ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-type codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is arranged with 5 layers from top to bottom, and each layer is a symmetrical codec structure, and the first layer, the second layer, the third layer and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss.

[0046] This embodiment provides an ultrasound image segmentation method and system based on a U-type codec structure, which relates to the field of computer vision. We introduce the jump connection part of the original network into the channel attention module, correct the target features from the perspective of channel attention, and generate corresponding prediction masks at each stage of upsampling, calculate edge loss, and help the model learn important features faster. Specifically, it includes the following steps: input the collected ultrasound image; the ultrasound image is converted into a feature map after convolution, normalization, pooling and activation processing; then downsampling twice; subsequently, the feature map is passed to the Tok MLP layer for processing and then upsampled; the fifth layer feature map is upsampled once and added to the feature map of the downsampled part; then it passes through a SEA module and then adds it to the upsampled feature map; then the fourth, third and second layer feature maps of the upsampled part are upsampled and added to the feature maps of the third, second and first layers of the downsampled part; then through a SEA module; finally, a 1*1 convolution is used to generate the final category prediction for each pixel or spatial position.

[0047] The specific implementation scheme of this embodiment is as follows:

[0048] Input the acquired ultrasound image;

[0049] The ultrasound image is transformed into feature map t1 after convolution, normalization, pooling and activation;

[0050] Then the feature map is processed twice in the same way to generate feature maps t2 and t3 respectively, capturing detailed features by increasing the number of channels and reducing the resolution;

[0051] The number of channels of feature maps t1, t2, and t3 are 16, 32, and 128, respectively;

[0052] The feature map is then fed into Tok MLP (Tokenized Multilayer Perceptron), where the input feature map is divided into multiple small blocks and embedded into a vector space, shifted in height and width to induce window locality, and then the features are transmitted through deep convolutional layers and max pooling is performed to further reduce the resolution of the feature map to generate feature map t4;

[0053] After that, the upsampling is passed into the same TokMLP for feature map processing to generate feature map t5;

[0054] The feature map generated by upsampling is added to the feature map t4 generated by downsampling before, and then processed by the SEA (Squeeze-and-ExcitationAttention) module. By performing "squeeze" and "excitation" operations on the input feature map, the network can learn the importance of each channel and perform weighted adjustments on it, and then add it to the upsampled feature map to generate a new feature map;

[0055] Then the generated feature map is upsampled and added to the feature map t3 generated by the previous downsampling, and then added to the upsampled feature map after passing through a SEA module to generate a feature map;

[0056] After that, the feature map is upsampled and added to the feature map t2 generated by the previous downsampling, and then added to the upsampled feature map after passing through a SEA module to generate a feature map;

[0057] The feature map generated by upsampling again is added to the previous feature map t1 and passed through the SEA module to generate a feature map again, which is then added to the upsampled feature map to generate the final feature map;

[0058] Finally, a 1*1 convolution is used to generate the final category prediction for each pixel or spatial position.

[0059] This embodiment uses a codec network to segment medical ultrasound images, and uses the advantages of CNN being good at capturing local features and attention mechanism being good at capturing global features to process feature maps. At the same time, a deep supervised edge loss function is introduced to assist segmentation and generate more accurate segmentation results. Due to the real-time imaging characteristics of ultrasound images, the corresponding segmentation network is required to be lightweight, fast, and effective. This embodiment adds three channel attention modules to the basic CNN network, which greatly increases the accuracy of ultrasound medical image segmentation while not increasing the computing power much, thereby providing a more powerful basis for clinical diagnosis.

[0060] An ultrasonic image segmentation system based on a U-type codec structure, comprising:

[0061] A data acquisition module, used for acquiring ultrasound images;

[0062] An image segmentation module is used to input the ultrasound image into an ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-type codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is provided with 5 layers from top to bottom, and each layer is a symmetrical codec structure, and the first layer, the second layer, the third layer and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss.

[0063] An electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform an ultrasound image segmentation method based on a U-type codec structure.

[0064] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for segmenting an ultrasonic image based on a U-type codec structure is implemented.

[0065] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An ultrasonic image segmentation method based on a U-type codec structure, characterized in that: include: Acquiring ultrasound images; The ultrasound image is input into the ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-type codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is arranged with 5 layers from top to bottom, and each layer is a symmetrical codec structure, and the first layer, the second layer, the third layer and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss.

2. The ultrasonic image segmentation method based on the U-type codec structure according to claim 1, characterized in that: The training process of the ultrasound image segmentation model specifically includes: Acquiring training data, wherein the training data includes an ultrasound training image and a corresponding segmentation result; An initial ultrasound image segmentation model is constructed, the training data is input into the initial ultrasound image segmentation model for image segmentation, and training is performed with the goal of minimizing the loss between the initial training result after image segmentation and the segmentation result corresponding to the ultrasound training image, so as to obtain a trained ultrasound image segmentation model.

3. The ultrasonic image segmentation method based on the U-type codec structure according to claim 1, characterized in that: The processing process of the ultrasound image segmentation model specifically includes: Inputting the ultrasonic image into the second layer, the third layer and the fourth layer of the encoder part of the ultrasonic image segmentation model for downsampling operation to obtain feature maps t1, t2 and t3; The feature map t3 is sequentially input into two layers of Tok MLP layers to extract features across height and width, and the fourth layer feature map t4 and the fifth layer feature map t5 are obtained. At the same time, the t5 feature map generates a prediction mask and calculates the edge loss; The feature map t5 is input into the third layer of the decoder part for upsampling operation to obtain the first upsampled feature map. The first upsampled feature map is added to the feature map t4, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the first upsampled feature map to obtain feature map t6. At the same time, the t6 feature map generates a prediction mask and calculates the edge loss; The feature map t6 is input into the third layer of the decoder part for upsampling operation to obtain the second upsampled feature map. The second upsampled feature map is added to the feature map t3, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the second upsampled feature map to obtain feature map t7. At the same time, the t7 feature map generates a prediction mask and calculates the edge loss. The feature map t7 is input into the second layer of the decoder part for upsampling operation to obtain the third upsampled feature map, the third upsampled feature map is added to the feature map t2, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations and then added to the third upsampled feature map to obtain feature map t8. At the same time, the t8 feature map generates a prediction mask and calculates the edge loss; The feature map t8 is input into the first layer of the decoder part for upsampling operation to obtain the fourth upsampling feature map, the fourth upsampling feature map is added to the feature map t1, and a 3*3 convolution operation is performed on the added image. After the convolution operation is completed, it is input into the attention module for compression and excitation operations, and then added to the fourth upsampling feature map to obtain the feature map t9; A 1*1 convolution operation is performed on the feature map t9 to obtain the segmentation result.

4. The ultrasonic image segmentation method based on the U-type codec structure according to claim 3, characterized in that: The specific process of performing the downsampling operation specifically includes: The ultrasound image is subjected to convolution, normalization, pooling and activation processing to obtain a feature map.

5. The ultrasonic image segmentation method based on U-type codec structure according to claim 3, characterized in that: The number of channels and the resolution of the feature map t1, the feature map t2 and the feature map t3 are all different.

6. An ultrasound image segmentation system based on a U-type codec structure, characterized in that: include: A data acquisition module, used for acquiring ultrasound images; An image segmentation module is used to input the ultrasound image into an ultrasound image segmentation model for image segmentation to obtain a segmentation result; wherein the ultrasound image segmentation model is constructed based on a U-type codec network and an attention mechanism, and the network structure of the ultrasound image segmentation model is provided with 5 layers from top to bottom, and each layer is a symmetrical codec structure, and the first layer, the second layer, the third layer and the fourth layer all introduce a channel attention module, and the second layer, the third layer, the fourth layer and the fifth layer all output an output mask, and the output mask is used to calculate the edge loss.

7. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform an ultrasound image segmentation method based on a U-type codec structure according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The device stores a computer program, which, when executed by a processor, implements an ultrasound image segmentation method based on a U-type codec structure as described in any one of claims 1 to 5.

Images