A 2.5D medical CT image segmentation method and device based on an improved UNet model

Abstract

The present invention provides a 2.5D medical CT image segmentation method and device based on an improved UNet model. The method comprises the following steps: obtaining an initial CT image, extracting the initial CT image based on preset extraction parameters to obtain a to-be-segmented image; inputting the to-be-segmented image into an encoder module of a trained UNet model, wherein an information correction module is provided between every two adjacent downsampling transformation modules in the encoder module, and the information correction module is sequentially provided with a channel attention layer and a spatial attention layer; inputting the encoded feature map into a decoder module of the trained UNet model, wherein the decoder module includes multiple upsampling transformation modules and multiple multi-stage multi-scale modules, wherein a multi-stage multi-scale module is provided between every two adjacent upsampling transformation modules in the decoder module, and the multi-stage multi-scale module includes pooling layers and convolution layers of multiple scales, and outputting the segmentation feature map through the decoder module.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a 2.5D medical CT image segmentation method and device based on an improved UNet model. Background Art

[0002] Medical image segmentation plays a very important role in clinical diagnosis and treatment. Currently, image-based disease diagnosis and assessment rely on human experts to process and interpret images. Due to the low efficiency of human experts in post-processing data, such as manual segmentation, and the presence of subjective factors, it is not enough to fully utilize precious imaging data. Therefore, computer-aided diagnosis (CAD) technology based on medical imaging has developed rapidly, greatly promoting research progress in the medical field at home and abroad. In the development of CAD, segmentation is an indispensable and important research content and one of the most popular tasks in medical image processing. It plays a huge role in disease diagnosis and treatment, retrospective studies, and prospective evaluations. Through image segmentation, some anatomical parameters can be accurately calculated. For example, based on cardiac CT, the size changes of the patient's ventricles during systole and diastole can be calculated. Combined with other functional parameters, some clinical diagnostic conclusions can be given.

[0003] Traditional non-deep learning medical image segmentation methods include image threshold-based segmentation, region-growing-based image segmentation, edge-based image segmentation, graph-theory-based image segmentation, and energy functional-based image segmentation. However, deep learning-based convolutional neural network (CNN) image segmentation methods are already the most commonly used technology in CAD. In 2015, the introduction of fully convolutional neural networks (FCNs) expanded image-level classification to pixel-level classification, achieving semantic segmentation. This model replaces the fully connected layers of the image classification model with convolutional layers and utilizes a skip structure to reuse multi-scale information, bringing semantic segmentation research to a new level.

[0004] There are existing 2.5D segmentation methods based on deep learning convolutional neural networks. The 2.5D segmentation method adopts a "many-to-many" segmentation mechanism, that is, sampling C consecutive slices to form a sample and sending it to the segmentation network, and the output is still the segmentation result of the corresponding C slices. However, the differences between different individuals or individuals lead to great differences in organs, such as the abdominal organs of adults and children. Within the same individual, the morphological differences of organs when the positions of CT slices differ greatly are very drastic, especially for dynamically changing organs such as the heart and lungs. Regardless of whether they are the same individual or different individuals, organs or tissues are extremely similar on adjacent CT slices. The existing technology for 2.5D segmentation tasks has high redundancy between adjacent slices and low correlation between non-adjacent slices or slices of individuals with large differences, which increases the difficulty of the model to identify the slice to which the feature belongs, which is not conducive to 2.5D segmentation. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a 2.5D medical CT image segmentation method based on an improved UNet model to eliminate or improve one or more defects in the prior art.

[0006] One aspect of the present invention provides a 2.5D medical CT image segmentation method based on an improved UNet model, the method comprising the following steps:

[0007] Acquire an initial CT image, extract the initial CT image based on preset extraction parameters to obtain an image to be segmented, wherein the image to be segmented includes multiple slice images in the initial CT image;

[0008] The image to be segmented is input into the encoder module of the trained UNet model. The encoder module includes multiple downsampling transformation modules and multiple information correction modules. An information correction module is provided between every two adjacent downsampling transformation modules in the encoder module. The downsampling module performs dimension increase on the input image through a convolutional layer. The information correction module is sequentially provided with a channel attention layer and a spatial attention layer, and outputs an encoded feature map through the encoder module.

[0009] The encoded feature map is input into the decoder module of the trained UNet model. The decoder module includes multiple upsampling transformation modules and multiple multi-stage multi-scale modules. A multi-stage multi-scale module is arranged between every two adjacent upsampling transformation modules in the decoder module. The multi-stage multi-scale module includes pooling layers and convolution layers of multiple scales. The segmentation feature map is output through the decoder module.

[0010] Using the above scheme, the information correction module of this scheme uses the channel attention module to enhance the feature expression of the key slice, weaken the features of the auxiliary slice, and reduce the redundancy between adjacent slices in the sample; the information correction module uses the spatial attention module to make the encoder more sensitive to the region of interest, thereby achieving the purpose of information correction; for the multi-stage multi-scale module, when skipping the output of the information correction module in the encoder and the upsampled feature map in the corresponding decoder, multiple scale pooling layers are added to achieve the purpose of better fusing features of different scales and improving the correlation between non-adjacent slices or individual slices with large differences.

[0011] In some embodiments of the present invention, the downsampling transformation module of the encoder module includes a first downsampling transformation module, a second downsampling transformation module, a third downsampling transformation module and a fourth downsampling transformation module, and the information correction module of the encoder module includes a first information correction module, a second information correction module and a third information correction module; the encoder module includes a first downsampling transformation module, a first information correction module, a second downsampling transformation module, a second information correction module, a third downsampling transformation module, a third information correction module and a fourth downsampling transformation module connected in sequence;

[0012] The upsampling transformation module of the decoder module includes a first upsampling transformation module, a second upsampling transformation module, a third upsampling transformation module and a fourth upsampling transformation module; the multi-stage multi-scale module of the decoder module includes a first multi-stage multi-scale module, a second multi-stage multi-scale module and a third multi-stage multi-scale module; the decoder module includes a first upsampling transformation module, a first multi-stage multi-scale module, a second upsampling transformation module, a second multi-stage multi-scale module, a third upsampling transformation module, a third multi-stage multi-scale module and a fourth upsampling transformation module connected in sequence.

[0013] In some embodiments of the present invention, in the information correction module, the channel attention layer includes a maximum pooling layer, an average pooling layer, a fully connected layer, and an activation function layer arranged in sequence, and the initial feature map input to the maximum pooling layer passes through the average pooling layer and the fully connected layer, and the activation function layer outputs the first feature map to the spatial attention layer;

[0014] In the information correction module, the spatial attention layer includes an average pooling layer, a maximum pooling layer, a convolution layer and an activation function layer arranged in sequence. The first feature map is input into the average pooling layer, passes through the maximum pooling layer and the convolution layer, and the activation function layer outputs the spatial attention feature map. Based on the spatial attention feature map, the first feature map is weighted using the broadcast mechanism to obtain the second feature map. The second feature map is residually connected with the initial feature map to output the third feature map.

[0015] In some embodiments of the present invention, the multi-stage multi-scale module includes pooling layers and convolution layers of multiple scales. The original feature map input into the multi-stage multi-scale module is subjected to pooling layers of different scales respectively, and passes through the convolution layer corresponding to each pooling layer to obtain multiple sub-feature maps. After the multiple sub-feature maps are interpolated to the size of the original feature map through linear interpolation, the multiple sub-feature maps are spliced ​​with the original feature map, and the spliced ​​feature map is then passed through the two-dimensional convolution layer and batch normalization layer of the multi-stage multi-scale module and output as a fused feature map.

[0016] In some embodiments of the present invention, the first downsampling transformation module includes a maximum pooling layer and two convolution units connected in sequence, each convolution unit includes a convolution layer with a convolution kernel of 3*3, a batch normalization layer and a RELU3 activation function layer;

[0017] The second down-sampling transformation module, the third down-sampling transformation module and the fourth down-sampling transformation module all have the same structure as the first down-sampling transformation module;

[0018] The first upsampling transformation module includes an upsampling layer and two convolution units connected in sequence, each convolution unit includes a convolution layer with a convolution kernel of 3*3, a batch normalization layer and a RELU3 activation function layer;

[0019] The second up-sampling conversion module, the third up-sampling conversion module and the fourth up-sampling conversion module all have the same structure as the first up-sampling conversion module.

[0020] In some embodiments of the present invention, the encoder module further includes an input processing module, the input processing module including two sequentially connected convolution units, each convolution unit including a convolution layer with a convolution kernel of 3*3, a batch normalization layer, and a RELU3 activation function layer, the processing module is connected to the image to be segmented by the first convolution unit, and the second convolution unit outputs the processed image to be segmented to the first downsampling transformation module;

[0021] The decoder module also includes an output processing module, which includes a convolution layer with a convolution kernel of 1*1, and the output processing module outputs a segmentation feature map.

[0022] In some embodiments of the present invention, the extraction parameters include the total number of extraction channels and the number of intervals, and the step of extracting the initial CT image based on the preset extraction parameters is as follows:

[0023] First, extract the slice images of the preset initial channel in the initial CT image, and continue to extract the slice images of the channel separated by the interval number from the slice image of the initial channel based on the interval number, and judge whether the total number of slice images currently extracted is equal to the total number of extracted channels. If so, stop extracting and splice the extracted slice images into the image to be segmented; if not, continue to extract the slice images of the channel separated by the interval number from the last extracted slice image based on the interval number until the total number of extracted slice images is equal to the total number of extracted channels.

[0024] In some embodiments of the present invention, the segmentation feature map is a two-channel feature map, including a foreground feature map and a background feature map, and after the segmentation feature map is output by the decoder module, the following steps are further included:

[0025] Comparing the pixel values ​​of the pixel points at the same position in the foreground feature map and the background feature map, if the pixel value of the pixel point in the foreground feature map is larger, marking the pixel value of the pixel point at the corresponding position in the final feature map as the first value; if the pixel value of the pixel point in the background feature map is larger, marking the pixel value of the pixel point at the corresponding position in the final feature map as the second value;

[0026] Compare each pixel in the foreground feature map and the background feature map respectively, complete the pixel value marking of all pixels in the final feature map, and render the pixel points with the first value in the final feature map as the first color, and render the pixel points with the second value in the final feature map as the second color, completing the color rendering of the final feature map.

[0027] In some embodiments of the present invention, the method further comprises model training. In the model training step, sample images corresponding to slice images at interlayer positions in a plurality of slice images of an initial CT image in a preset training data set are obtained, the sample images comprising a foreground sample image and a background sample image, the sample images being pre-marked with target areas and non-target areas, the pixel values ​​of the pixels in the target area in the foreground sample image being a first value, and the pixel values ​​of the pixels in the non-target area being a second value; the pixel values ​​of the pixels in the target area in the background sample image being a second value, and the pixel values ​​of the pixels in the non-target area being a first value;

[0028] The loss function of the foreground feature map and the foreground sample image, and the loss function of the background feature map and the background sample image are calculated according to the following formulas:

[0029]

[0030]

[0031] Where i represents the i-th pixel in the foreground feature map, background feature map, foreground sample image or background sample image, N represents the total number of pixels in the foreground feature map, background feature map, foreground sample image or background sample image, and y i Represents the pixel value of the i-th pixel in the foreground sample image or the background sample image, The pixel value of the i-th pixel in the foreground feature map or the background feature map, DiceLoss represents the DiceLoss function value, and BCELoss represents the cross entropy loss function value;

[0032] Based on the calculation formula of the DiceLoss function value, the DiceLoss function values ​​of the foreground feature map and the foreground sample image, as well as the DiceLoss function values ​​of the background feature map and the background sample image are calculated respectively; based on the calculation formula of the cross entropy loss function value, the cross entropy loss function value of the foreground feature map and the foreground sample image, as well as the cross entropy loss function value of the background feature map and the background sample image are calculated;

[0033] The total loss function is calculated based on the DiceLoss function value of the foreground feature map and the foreground sample image, the DiceLoss function value of the background feature map and the background sample image, the cross entropy loss function value of the foreground feature map and the foreground sample image, and the cross entropy loss function value of the background feature map and the background sample image:

[0034] Total loss function = DiceLoss1 + BCELoss1 + DiceLoss2 + BCELoss2;

[0035] Where DiceLoss1 represents the DiceLoss function value of the foreground feature map and the foreground sample image, DiceLoss2 represents the DiceLoss function value of the background feature map and the background sample image, BCELoss1 represents the cross entropy loss function value of the foreground feature map and the foreground sample image, and BCELoss2 represents the cross entropy loss function value of the background feature map and the background sample image;

[0036] Update the parameters in the UNet model based on the loss function.

[0037] The present invention also provides a 2.5D medical CT image segmentation device based on an improved UNet model, which includes a computer device, the computer device including a processor and a memory, the memory storing computer instructions, and the processor being used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps implemented by the method described above.

[0038] Additional advantages, objects, and features of the present invention will be described in part in the following description and will become apparent to those skilled in the art after studying the following or may be learned by practice of the present invention. The objects and other advantages of the present invention may be particularly pointed out and attained in the description and drawings.

[0039] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0041] Figure 1 Schematic diagram of an embodiment of a 2.5D medical CT image segmentation method based on an improved UNet model of the present invention;

[0042] Figure 2 Schematic diagram of another embodiment of the 2.5D medical CT image segmentation method based on the improved UNet model of the present invention;

[0043] Figure 3 Schematic diagram of the structure of the UNet model of the present invention;

[0044] Figure 4 Schematic diagram of UNet model training;

[0045] Figure 5 This is a structural diagram of the information correction module;

[0046] Figure 6 Schematic diagram of the structure of the multi-stage and multi-scale module;

[0047] Figure 7 A schematic diagram for extracting the image to be segmented;

[0048] Figure 8 is a schematic diagram of the initial CT image;

[0049] Figure 9 This is a schematic diagram of the final feature map after color rendering;

[0050] Figure 10 Schematic diagram of constructing multiple final feature maps into a 3D image. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0052] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0053] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0054] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0055] Introduction to existing technology:

[0056] Medical imaging, a procedure performed by radiology or nuclear medicine departments, is a highly technical field of medicine. This type of examination includes computed tomography (CT), X-rays, ultrasound, and magnetic resonance imaging (MRI). These examinations utilize radiation to identify changes within the body. With the continuous advancement of imaging technology, medical imaging has evolved from a supplementary examination tool to a crucial clinical diagnostic and differential diagnosis tool in modern medicine. In most hospitals, medical imaging equipment accounts for over one-third of the hospital's fixed assets, and medical imaging information accounts for approximately 70% of all medical information. CT utilizes the varying penetration capabilities of X-rays in different tissues and organs of the human body, using detectors to detect the radiation that has passed through different tissues to achieve imaging. Due to its high efficiency, wide applicability, and widespread adoption, it has become one of the most commonly used imaging techniques in radiology departments. In recent years, PET-CT has integrated positron emission tomography (PET) and CT. PET provides information on lesion location or organ metabolism, while CT provides accurate anatomical localization of lesions. Because the two scans are performed simultaneously, a single scan can provide more comprehensive information. In short, CT imaging has been widely used in clinical practice due to its important diagnostic value.

[0057] There are two main approaches to applying deep learning to CT medical image segmentation: (1) Using the common 2D segmentation method, the specific operation is to slice the 3D data of the CT, and then treat a single axial layer as a single image and feed it into the corresponding network for learning. Most common U-net-based segmentation methods are based on this operation. (2) Using 3D CNN models such as V-net to segment 3D data. When applying 3D CNN to CT medical image segmentation, the entire volume data is often cut into small 3D patches for training and testing.

[0058] Both approaches have their own advantages and disadvantages. A 2D network uses a complete slice as input, requiring only a single axis to slide, resulting in less computation and faster inference. However, this approach lacks information about the relationships between slices. While the patch-based 3DCNN approach possesses powerful 3D context perception, it suffers from a limited receptive field, which can easily lead to information confusion in the model. The lack of pre-trained models makes the training process unstable and prone to overfitting. Consequently, a new research direction, namely 2.5D segmentation, has emerged to simulate the segmentation of 3D data by resampling it into 2D multi-channel data based on 2D segmentation models.

[0059] Existing 2.5D segmentation methods often use a "many-to-many" segmentation mechanism, whereby a continuous set of C slices is sampled to form a single sample and fed into the segmentation network. The output is still the segmentation result of the corresponding C slices. Under this mechanism, when C is large, network performance degrades, and the network cannot clearly distinguish which slice a feature belongs to, resulting in information confusion.

[0060] This scheme uses a many-to-one segmentation mechanism to solve the problem of network performance degradation when C is large, where the network cannot clearly distinguish the slice to which the feature belongs and there is information confusion.

[0061] To solve the above problems, Figure 1 As shown, the present invention proposes a 2.5D medical CT image segmentation method based on an improved UNet model, the method comprising the following steps:

[0062] like Figure 8 As shown, in step S100, an initial CT image is acquired, and the initial CT image is extracted based on preset extraction parameters to obtain an image to be segmented, wherein the image to be segmented includes multiple slice images in the initial CT image;

[0063] In some embodiments of the present invention, in the step of acquiring the initial CT image, if the initial CT image is in DICOM format, the DICOM format is converted into NIFIT format.

[0064] In some embodiments of the present invention, the step of obtaining an initial CT image further comprises:

[0065] Receive a selection window, where the selection window includes a minimum HU value and a maximum HU value, obtain the HU value of each pixel in each slice image in the initial CT image, and modify the HU value of the pixel to the minimum HU value if the HU value of the pixel is less than the minimum HU value; if the HU value of the pixel is greater than the maximum HU value, modify the HU value of the pixel to the maximum HU value.

[0066] In some embodiments of the present invention, the selection window is obtained based on a training data set when training the UNet model, and the step of calculating the selection window includes:

[0067] Obtain the HU value of each pixel in the target area of ​​all sample images in the training data set, and sort the HU values ​​to obtain a HU value sequence;

[0068] Based on the preset window range, a window is intercepted in the HU value sequence. After each window is intercepted, it is determined whether the number of pixels corresponding to the HU value in the window is greater than a preset threshold. If so, the window is used as the selection window.

[0069] In some embodiments of the present invention, the step of acquiring the initial CT image further includes normalizing each slice image in the initial CT image to the same scale.

[0070] In a specific implementation process, before the image to be segmented is input into the encoder module, the size of each slice image of the image to be segmented is adjusted to (256, 256) by resizing.

[0071] Step S200: Input the image to be segmented into the encoder module of the trained UNet model. The encoder module includes multiple downsampling transformation modules and multiple information correction modules. An information correction module (ICA) is provided between every two adjacent downsampling transformation modules in the encoder module. The downsampling module performs dimension increase on the input image through a convolutional layer. The information correction module is sequentially provided with a channel attention layer and a spatial attention layer, and outputs a coded feature map through the encoder module.

[0072] Using the above scheme, the information correction module uses the channel attention module to enhance the feature expression of the slice image in the middle layer position among the multiple slice images of the image to be segmented, weaken the features of the auxiliary slices on both sides, and reduce the redundancy between adjacent slices in the sample; in order to improve the feature extraction ability of the encoder, the spatial attention module is cascaded to make the encoder more sensitive to the region of interest, thereby achieving the purpose of information correction.

[0073] Step S300: Input the encoded feature map into the decoder module of the trained UNet model. The decoder module includes multiple upsampling transformation modules and multiple multi-stage multi-scale modules (MSPs). A multi-stage multi-scale module is set between every two adjacent upsampling transformation modules in the decoder module. The multi-stage multi-scale module includes pooling layers and convolution layers of multiple scales, and outputs the segmentation feature map through the decoder module.

[0074] Adopting the above scheme, in the information correction module of this scheme, the channel attention module is used to enhance the feature expression of the slice image in the middle interlayer position among the multiple slice images of the image to be segmented, and the features of the auxiliary slices on both sides of the multiple slice images are weakened, thereby reducing the redundancy between adjacent slices in the sample; in the information correction module, the spatial attention module is used to make the encoder more sensitive to the region of interest, thereby achieving the purpose of information correction; for the multi-stage multi-scale module, when the output of the information correction module in the encoder and the upsampled feature map in the corresponding decoder are skipped (skip connection), pooling layers of multiple scales are added to achieve the purpose of better fusing features of different scales and improving the correlation between non-adjacent slices or individual slices with large differences.

[0075] like Figure 3 As shown, in some embodiments of the present invention, the downsampling transformation module of the encoder module includes a first downsampling transformation module, a second downsampling transformation module, a third downsampling transformation module and a fourth downsampling transformation module, and the information correction module of the encoder module includes a first information correction module, a second information correction module and a third information correction module; the encoder module includes the first downsampling transformation module, the first information correction module, the second downsampling transformation module, the second information correction module, the third downsampling transformation module, the third information correction module and the fourth downsampling transformation module connected in sequence;

[0076] The upsampling transformation module of the decoder module includes a first upsampling transformation module, a second upsampling transformation module, a third upsampling transformation module and a fourth upsampling transformation module; the multi-stage multi-scale module of the decoder module includes a first multi-stage multi-scale module, a second multi-stage multi-scale module and a third multi-stage multi-scale module; the decoder module includes a first upsampling transformation module, a first multi-stage multi-scale module, a second upsampling transformation module, a second multi-stage multi-scale module, a third upsampling transformation module, a third multi-stage multi-scale module and a fourth upsampling transformation module connected in sequence.

[0077] like Figure 5As shown, in some embodiments of the present invention, in the information correction module, the channel attention layer includes a maximum pooling layer, an average pooling layer, a fully connected layer, and an activation function layer arranged in sequence, and the initial feature map input to the maximum pooling layer passes through the average pooling layer and the fully connected layer, and the activation function layer outputs the first feature map to the spatial attention layer;

[0078] In some embodiments of the present invention, the activation function layer in the information correction module is a Sigmoid function layer.

[0079] In the information correction module, the spatial attention layer includes an average pooling layer, a maximum pooling layer, a convolution layer and an activation function layer arranged in sequence. The first feature map is input into the average pooling layer, passes through the maximum pooling layer and the convolution layer, and the activation function layer outputs the spatial attention feature map. Based on the spatial attention feature map, the first feature map is weighted using the broadcast mechanism to obtain the second feature map. The second feature map is residually connected with the initial feature map to output the third feature map.

[0080] In some embodiments of the present invention, the convolution layer in the information correction module is a convolution layer with a convolution kernel of 3*3.

[0081] like Figure 6 As shown, in some embodiments of the present invention, the multi-stage multi-scale module includes pooling layers and convolution layers of multiple scales. The original feature map input into the multi-stage multi-scale module is subjected to pooling layers of different scales respectively, and passes through the convolution layer corresponding to each pooling layer to obtain multiple sub-feature maps. After the multiple sub-feature maps are interpolated to the size of the original feature map by linear interpolation, the multiple sub-feature maps are spliced ​​with the original feature map, and the spliced ​​feature map is then passed through the two-dimensional convolution layer and batch normalization layer of the multi-stage multi-scale module and output as a fused feature map.

[0082] In some embodiments of the present invention, the pooling layers of multiple scales in the multi-stage multi-scale module can be 1*1 pooling layers, 2*2 pooling layers, 3*3 pooling layers, 4*4 pooling layers, 5*5 pooling layers or 6*6 pooling layers, etc., and after passing through each pooling layer, a convolution layer is passed, and the convolution layer can be a 3*3 convolution layer to obtain multiple sub-feature maps.

[0083] In some embodiments of the present invention, the first downsampling transformation module includes a maximum pooling layer and two convolution units connected in sequence, each convolution unit includes a convolution layer with a convolution kernel of 3*3, a batch normalization layer and a RELU3 activation function layer;

[0084] The second down-sampling transformation module, the third down-sampling transformation module and the fourth down-sampling transformation module all have the same structure as the first down-sampling transformation module.

[0085] In some embodiments of the present invention, the first upsampling transformation module includes an upsampling layer and two convolution units connected in sequence, each convolution unit includes a convolution layer with a convolution kernel of 3*3, a batch normalization layer and a RELU3 activation function layer;

[0086] The second up-sampling conversion module, the third up-sampling conversion module and the fourth up-sampling conversion module all have the same structure as the first up-sampling conversion module.

[0087] In some embodiments of the present invention, the encoder module further includes an input processing module, the input processing module including two sequentially connected convolution units, each convolution unit including a convolution layer with a convolution kernel of 3*3, a batch normalization layer, and a RELU3 activation function layer, the processing module is connected to the image to be segmented by the first convolution unit, and the second convolution unit outputs the processed image to be segmented to the first downsampling transformation module;

[0088] The decoder module also includes an output processing module, which includes a convolution layer with a convolution kernel of 1*1, and the output processing module outputs a segmentation feature map.

[0089] In some embodiments of the present invention, the image to be segmented includes three slice images, each slice image size is 256*256, that is, the image to be segmented is a 3-channel, 256*256 image, the input processing module upgrades the feature map to 64 channels, the first downsampling transformation module upgrades the image to 128 channels, the second downsampling transformation module upgrades the image to 256 channels, the third downsampling transformation module upgrades the image to 512 channels, and the fourth downsampling transformation module maintains the number of 512 channels unchanged; the first upsampling transformation module reduces the image to 256 channels, the second upsampling transformation module reduces the image to 128 channels, the third upsampling transformation module reduces the image to 64 channels, and the fourth upsampling transformation module maintains the number of 64 channels unchanged; the output processing module reduces the feature map to 2 channels.

[0090] In the specific implementation process, the input image to be segmented is (3, 256, 256). After the first, second, third, and fourth downsampling transformation modules, the encoder part is x1 (128, 128, 128), x2 (256, 64, 64), x3 (512, 32, 32), and x4 (512, 16, 16). After the first, second, third, and fourth upsampling transformation modules, the encoder part is x5 (256, 32, 32), x6 (128, 64, 64), x7 (64, 128, 128), and x8 (64, 256, 256).

[0091] Preferably, the output 2-channel feature map is subjected to argmax processing.

[0092] In some embodiments of the present invention, the extraction parameters include the total number of extraction channels and the number of intervals, and the step of extracting the initial CT image based on the preset extraction parameters is as follows:

[0093] First, extract the slice images of the preset initial channel in the initial CT image, and continue to extract the slice images of the channel separated by the interval number from the slice image of the initial channel based on the interval number, and judge whether the total number of slice images currently extracted is equal to the total number of extracted channels. If so, stop extracting and splice the extracted slice images into the image to be segmented; if not, continue to extract the slice images of the channel separated by the interval number from the last extracted slice image based on the interval number until the total number of extracted slice images is equal to the total number of extracted channels.

[0094] like Figure 7 As shown, in the specific implementation process, the total number of extracted channels is set to C, the interval number is set to t, if the preset initial channel is the first channel, the interval number is t=1, and the total number of extracted channels is C=3, then the slice images extracted are the 1st, 3rd, and 5th channels.

[0095] In some embodiments of the present invention, the segmentation feature map is a two-channel feature map, including a foreground feature map and a background feature map, and after the segmentation feature map is output by the decoder module, the following steps are further included:

[0096] like Figure 2 As shown, step S400, comparing the pixel values ​​of the pixel points at the same position in the foreground feature map and the background feature map, if the pixel value of the pixel point in the foreground feature map is larger, marking the pixel value of the pixel point at the corresponding position in the final feature map as a first value, if the pixel value of the pixel point in the background feature map is larger, marking the pixel value of the pixel point at the corresponding position in the final feature map as a second value;

[0097] like Figure 2 、 9 As shown, step S500 compares each pixel point in the foreground feature map and the background feature map respectively, completes the pixel value marking of all pixel points in the final feature map, and renders the pixel points with the first value in the final feature map as the first color, and renders the pixel points with the second value in the final feature map as the second color, thereby completing the color rendering of the final feature map.

[0098] In some embodiments of the present invention, the first value may be 1, the second value may be 0, the first color may be white, and the second color may be black.

[0099] like Figure 4As shown, in some embodiments of the present invention, the method further comprises model training. In the model training step, sample images corresponding to slice images at interlayer positions in a plurality of slice images in an initial CT image in a preset training data set are obtained, the sample images comprising a foreground sample image and a background sample image, the sample images being pre-marked with target areas and non-target areas, the pixel values ​​of the pixels in the target area in the foreground sample image being a first value, and the pixel values ​​of the pixels in the non-target area being a second value; the pixel values ​​of the pixels in the target area in the background sample image being a second value, and the pixel values ​​of the pixels in the non-target area being a first value;

[0100] During the specific implementation process, if the initial CT image extracts slice images of the 1st, 3rd, and 5th channels, then in the step of obtaining sample images corresponding to slice images at the intermediate interlayer position of multiple slice images in the initial CT image in the preset training data set, the sample image corresponding to the slice image of the 3rd channel is obtained.

[0101] The foreground sample images and the background sample images are both pre-labeled. The foreground sample images may be labeled in such a way that the pixel values ​​of the pixels in the labeled target area are labeled as 1, and the pixel values ​​of the pixels in the unlabeled area are labeled as 0; the background sample images may be labeled in such a way that the pixel values ​​of the pixels in the labeled target area are labeled as 0, and the pixel values ​​of the pixels in the unlabeled area are labeled as 1.

[0102] In a specific implementation process, if this solution is used for myocardial examination, the target area pre-marked in the sample image is the myocardial area.

[0103] The loss function of the foreground feature map and the foreground sample image, and the loss function of the background feature map and the background sample image are calculated according to the following formulas:

[0104]

[0105]

[0106] Where i represents the i-th pixel in the foreground feature map, background feature map, foreground sample image or background sample image, N represents the total number of pixels in the foreground feature map, background feature map, foreground sample image or background sample image, and y i Represents the pixel value of the i-th pixel in the foreground sample image or the background sample image, The pixel value of the i-th pixel in the foreground feature map or the background feature map, DiceLoss represents the DiceLoss function value, and BCELoss represents the cross entropy loss function value;

[0107] Based on the calculation formula of the DiceLoss function value, the DiceLoss function values ​​of the foreground feature map and the foreground sample image, as well as the DiceLoss function values ​​of the background feature map and the background sample image are calculated respectively; based on the calculation formula of the cross entropy loss function value, the cross entropy loss function value of the foreground feature map and the foreground sample image, as well as the cross entropy loss function value of the background feature map and the background sample image are calculated;

[0108] In the specific implementation process, when calculating the DiceLoss function value of the foreground feature map and the foreground sample image, y i Represents the pixel value of the i-th pixel in the foreground sample image, The pixel value of the i-th pixel in the foreground feature map; when calculating the DiceLoss function value of the background feature map and the background sample image, y i Represents the pixel value of the i-th pixel in the background sample image, The pixel value of the i-th pixel in the background feature map; when calculating the cross entropy loss function value of the foreground feature map and the foreground sample image, y i Represents the pixel value of the i-th pixel in the foreground sample image, The pixel value of the i-th pixel in the foreground feature map; when calculating the cross entropy loss function value of the background feature map and the background sample image, y i Represents the pixel value of the i-th pixel in the background sample image, The pixel value of the i-th pixel in the background feature map.

[0109] The total loss function is calculated based on the DiceLoss function value of the foreground feature map and the foreground sample image, the DiceLoss function value of the background feature map and the background sample image, the cross entropy loss function value of the foreground feature map and the foreground sample image, and the cross entropy loss function value of the background feature map and the background sample image:

[0110] Total loss function = DiceLoss1 + BCELoss1 + DiceLoss2 + BCELoss2;

[0111] Where DiceLoss1 represents the DiceLoss function value of the foreground feature map and the foreground sample image, DiceLoss2 represents the DiceLoss function value of the background feature map and the background sample image, BCELoss1 represents the cross entropy loss function value of the foreground feature map and the foreground sample image, and BCELoss2 represents the cross entropy loss function value of the background feature map and the background sample image;

[0112] Update the parameters in the UNet model based on the loss function.

[0113] In some embodiments of the present invention, the total number of extraction channels is preferably an odd number. Since the sample images corresponding to the slice images at the middle interlayer position in the multiple slice images in the initial CT image in the preset training data set are obtained during the training process, the middle slice in the middle of the scheme is the key slice, and the rest are auxiliary slices. The final output of the network model is the segmentation result corresponding to the key slice. When the total number of extraction channels is preferably an odd number, the scheme performs a many-to-one expression, thereby improving the accuracy of the output image.

[0114] With the above solution, since the "many-to-one" mechanism is adopted, the loss function is calculated based on the segmentation label and prediction results of the key slice.

[0115] like Figure 10 As shown, in the actual processing process, the initial channels can be set to different channels, and multiple final feature maps can be obtained based on the same initial CT image. The multiple final feature maps are constructed into a three-dimensional image using ITK-Snap software, which is convenient for doctors to use.

[0116] The beneficial effects of the present invention include:

[0117] 1. Compared to existing 2.5D segmentation techniques, this invention employs a "many-to-one" mechanism to mitigate the performance degradation caused by a large number of consecutive slices, C. By modifying the construction of data training samples and the model structure, this reduces intra-individual redundancy, increases the correlation between different individuals, and improves segmentation accuracy. Furthermore, compared to 2D, it effectively improves the coherence and accuracy of consecutive slices during CT data segmentation. Compared to 3D, the improved UNet model is smaller, the training process is more stable, and it is less prone to overfitting.

[0118] The present invention also provides a 2.5D medical CT image segmentation device based on an improved UNet model, which includes a computer device, the computer device including a processor and a memory, the memory storing computer instructions, and the processor being used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps implemented by the method described above.

[0119] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps implemented by the aforementioned 2.5D medical CT image segmentation method based on the improved UNet model. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the art.

[0120] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0121] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0122] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0123] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A 2.5D medical CT image segmentation method based on an improved UNet model, characterized in that: The steps of the method include: Acquire an initial CT image, extract the initial CT image based on preset extraction parameters to obtain an image to be segmented, wherein the image to be segmented includes multiple slice images in the initial CT image; The image to be segmented is input into the encoder module of the trained UNet model. The encoder module includes multiple downsampling transformation modules and multiple information correction modules. An information correction module is provided between every two adjacent downsampling transformation modules in the encoder module. The downsampling module performs dimension increase on the input image through a convolutional layer. The information correction module is sequentially provided with a channel attention layer and a spatial attention layer, and outputs an encoded feature map through the encoder module. The encoded feature map is input into the decoder module of the trained UNet model. The decoder module includes multiple upsampling transformation modules and multiple multi-stage multi-scale modules. A multi-stage multi-scale module is arranged between every two adjacent upsampling transformation modules in the decoder module. The multi-stage multi-scale module includes pooling layers and convolution layers of multiple scales. The segmentation feature map is output through the decoder module.

2. The 2.5D medical CT image segmentation method based on the improved UNet model according to claim 1, characterized in that: The downsampling transformation module of the encoder module includes a first downsampling transformation module, a second downsampling transformation module, a third downsampling transformation module and a fourth downsampling transformation module, and the information correction module of the encoder module includes a first information correction module, a second information correction module and a third information correction module; the encoder module includes the first downsampling transformation module, the first information correction module, the second downsampling transformation module, the second information correction module, the third downsampling transformation module, the third information correction module and the fourth downsampling transformation module connected in sequence; The upsampling transformation module of the decoder module includes a first upsampling transformation module, a second upsampling transformation module, a third upsampling transformation module and a fourth upsampling transformation module; the multi-stage multi-scale module of the decoder module includes a first multi-stage multi-scale module, a second multi-stage multi-scale module and a third multi-stage multi-scale module; the decoder module includes a first upsampling transformation module, a first multi-stage multi-scale module, a second upsampling transformation module, a second multi-stage multi-scale module, a third upsampling transformation module, a third multi-stage multi-scale module and a fourth upsampling transformation module connected in sequence.

3. The 2.5D medical CT image segmentation method based on the improved UNet model according to claim 1, characterized in that: In the information correction module, the channel attention layer includes a maximum pooling layer, an average pooling layer, a fully connected layer, and an activation function layer arranged in sequence. The initial feature map input to the maximum pooling layer passes through the average pooling layer and the fully connected layer, and the activation function layer outputs the first feature map to the spatial attention layer; In the information correction module, the spatial attention layer includes an average pooling layer, a maximum pooling layer, a convolution layer and an activation function layer arranged in sequence. The first feature map is input into the average pooling layer, passes through the maximum pooling layer and the convolution layer, and the activation function layer outputs the spatial attention feature map. Based on the spatial attention feature map, the first feature map is weighted using the broadcast mechanism to obtain the second feature map. The second feature map is residually connected with the initial feature map to output the third feature map.

4. The 2.5D medical CT image segmentation method based on the improved UNet model according to claim 1, characterized in that: The multi-stage multi-scale module includes pooling layers and convolution layers of multiple scales. The original feature map input into the multi-stage multi-scale module is subjected to pooling layers of different scales respectively, and passes through the convolution layer corresponding to each pooling layer to obtain multiple sub-feature maps. After the multiple sub-feature maps are interpolated to the size of the original feature map through linear interpolation, the multiple sub-feature maps are spliced ​​with the original feature map. The spliced ​​feature map is then passed through the two-dimensional convolution layer and batch normalization layer of the multi-stage multi-scale module and output as a fused feature map.

5. The 2.5D medical CT image segmentation method based on the improved UNet model according to claim 2, characterized in that: The first downsampling transformation module includes a maximum pooling layer and two convolution units connected in sequence, each convolution unit includes a convolution layer with a convolution kernel of 3*3, a batch normalization layer and a RELU3 activation function layer; the second downsampling transformation module, the third downsampling transformation module and the fourth downsampling transformation module have the same structure as the first downsampling transformation module; The first upsampling transformation module includes an upsampling layer and two convolution units connected in sequence; the second upsampling transformation module, the third upsampling transformation module and the fourth upsampling transformation module all have the same structure as the first upsampling transformation module.

6. The 2.5D medical CT image segmentation method based on the improved UNet model according to claim 1, characterized in that: The encoder module also includes an input processing module, which includes two convolution units connected in sequence, each of which includes a convolution layer with a convolution kernel of 3*3, a batch normalization layer, and a RELU3 activation function layer. The processing module is connected to the image to be segmented by the first convolution unit, and the second convolution unit outputs the processed image to be segmented to the first downsampling transformation module; The decoder module also includes an output processing module, which includes a convolution layer with a convolution kernel of 1*1, and the output processing module outputs a segmentation feature map.

7. The 2.5D medical CT image segmentation method based on the improved UNet model according to claim 1, characterized in that: The extraction parameters include the total number of extraction channels and the number of intervals. The steps of extracting the initial CT image based on the preset extraction parameters are as follows: First, extract the slice images of the preset initial channel in the initial CT image, and continue to extract the slice images of the channel separated by the interval number from the slice image of the initial channel based on the interval number, and judge whether the total number of slice images currently extracted is equal to the total number of extracted channels. If so, stop extracting and splice the extracted slice images into the image to be segmented; if not, continue to extract the slice images of the channel separated by the interval number from the last extracted slice image based on the interval number until the total number of extracted slice images is equal to the total number of extracted channels.

8. The 2.5D medical CT image segmentation method based on the improved UNet model according to any one of claims 1 to 7, characterized in that: The segmentation feature map is a two-channel feature map, including a foreground feature map and a background feature map. After the segmentation feature map is output by the decoder module, the method further includes the following steps: Comparing the pixel values ​​of the pixel points at the same position in the foreground feature map and the background feature map, if the pixel value of the pixel point in the foreground feature map is larger, marking the pixel value of the pixel point at the corresponding position in the final feature map as the first value; if the pixel value of the pixel point in the background feature map is larger, marking the pixel value of the pixel point at the corresponding position in the final feature map as the second value; Compare each pixel in the foreground feature map and the background feature map respectively, complete the pixel value marking of all pixels in the final feature map, and render the pixel points with the first value in the final feature map as the first color, and render the pixel points with the second value in the final feature map as the second color, completing the color rendering of the final feature map.

9. The 2.5D medical CT image segmentation method based on the improved UNet model according to claim 1, characterized in that: The method further includes model training. In the model training step, sample images corresponding to slice images at interlayer positions in a plurality of slice images in an initial CT image in a preset training data set are obtained, the sample images including a foreground sample image and a background sample image, the sample images being pre-marked with a target area and a non-target area, the pixel values ​​of the pixels in the target area in the foreground sample image being a first value, and the pixel values ​​of the pixels in the non-target area being a second value; the pixel values ​​of the pixels in the target area in the background sample image being a second value, and the pixel values ​​of the pixels in the non-target area being a first value; The loss function of the foreground feature map and the foreground sample image, and the loss function of the background feature map and the background sample image are calculated according to the following formulas: Where i represents the i-th pixel in the foreground feature map, background feature map, foreground sample image or background sample image, N represents the total number of pixels in the foreground feature map, background feature map, foreground sample image or background sample image, and y i Represents the pixel value of the i-th pixel in the foreground sample image or the background sample image, The pixel value of the i-th pixel in the foreground feature map or the background feature map, DiceLoss represents the DiceLoss function value, and BCELoss represents the cross entropy loss function value; Based on the above DiceLoss function value calculation formula, the DiceLoss function values ​​of the foreground feature map and the foreground sample image and the DiceLoss function values ​​of the background feature map and the background sample image are calculated respectively; The cross entropy loss function value between the foreground feature map and the foreground sample image and the cross entropy loss function value between the background feature map and the background sample image are calculated based on the calculation formula of the cross entropy loss function value; The total loss function is calculated based on the DiceLoss function value of the foreground feature map and the foreground sample image, the DiceLoss function value of the background feature map and the background sample image, the cross entropy loss function value of the foreground feature map and the foreground sample image, and the cross entropy loss function value of the background feature map and the background sample image: Total loss function = DiceLoss1 + BCELoss1 + DiceLoss2 + BCELoss2; Where DiceLoss1 represents the DiceLoss function value of the foreground feature map and the foreground sample image, DiceLoss2 represents the DiceLoss function value of the background feature map and the background sample image, BCELoss1 represents the cross entropy loss function value of the foreground feature map and the foreground sample image, and BCELoss2 represents the cross entropy loss function value of the background feature map and the background sample image; Update the parameters in the UNet model based on the loss function.

10. A 2.5D medical CT image segmentation device based on an improved UNet model, characterized in that: The apparatus includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus implements the steps implemented by the method according to any one of claims 1 to 9.

Images