Method for brain hematoma image segmentation based on image enhancement and weakly supervised learning

Abstract

The present application relates to a kind of cerebral hematoma image segmentation method based on image enhancement and weak supervision learning, belong to digital image processing field.The method includes: acquisition classification map data and segmentation map data, the cerebral hematoma pseudo-label in classification map and segmentation map is extracted by threshold processing method, while generating pseudo-normal map, the residual enhancement map of bleeding area is obtained by original drawing minus the pseudo-normal map;Brain hematoma image segmentation model is constructed, and residual enhancement map is used for training;The brain hematoma image segmentation model structure is mainly TransUnet as trunk network, there is classifier module and activation map extractor module after its encoder, there is segmentation module after its decoder;The hematoma region of cerebral hemorrhage picture is segmented by the brain hematoma image segmentation model trained.The present application uses a large amount of classification data and a small amount of segmentation data set, so that model can learn a large number of classification features, can enhance the robustness of model segmentation.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing and relates to the processing of brain hematoma images, specifically to a brain hematoma image segmentation method based on image enhancement and weakly supervised learning. Background Technology

[0002] Cerebral hematoma segmentation is a crucial task in medical imaging, primarily applied to head image analysis, such as CT (computed tomography) or MRI (magnetic resonance imaging), to detect and locate hematomas in a patient's brain. A cerebral hematoma is a pool of blood within the cranium, which can be caused by trauma, ruptured blood vessels, or other pathological factors. In clinical practice, rapid and accurate segmentation of cerebral hematoma regions is essential for developing effective treatment plans and improving patient survival rates. The application of medical image segmentation technology in this field aims to automatically extract the boundaries of cerebral hematomas from complex image data. The challenge of this task lies in the fact that medical images are typically highly variable and also affected by noise, uneven contrast, and other technical factors. Therefore, developing efficient algorithms and models to accurately segment cerebral hematoma regions is crucial for the field of medical imaging.

[0003] With the rapid development of deep learning technology, especially the application of convolutional neural networks (CNNs), significant progress has been made in hematoma segmentation. These deep learning models are able to learn complex feature representations, thereby improving the resolution of different brain structures and lesions. In addition to basic image segmentation, some models combine 3D and time-series information to gain a more comprehensive understanding of the dynamic changes in hematomas. In practical applications, hematoma segmentation technology helps doctors develop treatment plans more quickly, improves surgical accuracy, and facilitates earlier intervention, thus improving patient prognosis. However, hematoma segmentation technology still faces challenges, such as difficulties in data annotation and the generalization performance of models, requiring continuous research and improvement. The development of hematoma segmentation technology will play an increasingly important role in medical imaging diagnosis and treatment, providing patients with more precise medical services.

[0004] Existing methods for segmenting brain hematoma images can be divided into two categories: fully supervised segmentation methods and semi-supervised segmentation methods. However, due to the limited training set, it is difficult to achieve good robustness during model learning. Furthermore, because the hematoma in the training set is irregular, the model struggles to converge to the expected results. Therefore, fully supervised segmentation methods perform worse than semi-supervised segmentation methods on controlled datasets. Since classification datasets are relatively easy to obtain and far more numerous than segmentation datasets, semi-supervised segmentation methods outperform fully supervised segmentation methods. Therefore, constraining the network using CAM activation maps generated by a classification network and segmentation labels allows for a more comprehensive and accurate focus area. Simultaneously, the network can observe more features through a large amount of classification data, which helps improve the model's generalization ability and thus segment brain hematoma images more efficiently. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a brain hematoma segmentation method based on image enhancement and weakly supervised learning. By combining weakly supervised methods and image enhancement-based methods, and simultaneously achieving a mutually reinforcing effect through multi-task collaborative learning of classification and segmentation, the robustness of the brain hematoma segmentation model is enhanced.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A brain hematoma image segmentation method based on image enhancement and weakly supervised learning, the method comprising:

[0008] S1. Collect classification map data and segmentation map data, extract cerebral hematoma pseudo-labels from the classification map and segmentation map using a threshold processing method, and generate pseudo-normal maps. Subtract the pseudo-normal maps from the original maps to obtain the residual enhancement map of the hemorrhage area; wherein, the number of classification maps is much greater than the number of segmentation maps.

[0009] S2. Construct a brain hematoma image segmentation model and train it using residual enhancement maps. The brain hematoma image segmentation model is structured with TransUnet as the backbone network, with a classifier module and an activation map extractor module added after its encoder, and a segmentation module added after its decoder.

[0010] S3. The hematoma region of the brain hemorrhage image is segmented using a trained brain hematoma image segmentation model.

[0011] Furthermore, in step S1, the process of generating the residual enhancement map of the bleeding area is as follows: First, pseudo-labels are generated using medical prior knowledge through threshold processing; then, a generator and a discriminator are built, a pseudo-normal map is generated through the generator, and the pseudo-normal map is used by the discriminator to generate a pseudo-segmentation map; finally, the original map and the pseudo-normal map are subtracted to obtain the residual enhancement map.

[0012] Furthermore, in step S2, the training process of the brain hematoma image segmentation model includes:

[0013] S21. Input the residual enhancement map into the brain hematoma image segmentation model; wherein the segmentation map is respectively entered into the decoder, activation map extractor module and segmentation module, the decoder obtains the segmentation label, the activation map extractor module obtains the activation map, and the segmentation module obtains the segmentation result; at the same time, the classification map and the segmentation map are simultaneously entered into the classifier module to generate the classification result;

[0014] S22. Downsample the segmentation labels to obtain CAM labels, and extract the classification labels from the classification map at the same time;

[0015] S23. Calculate the loss between the classification label and the classification result, the loss between the CAM label and the CAM activation map, and the loss between the segmentation label and the segmentation result, respectively.

[0016] The loss between the classification label and the classification result is calculated as shown in the following formula:

[0017]

[0018] F cls =C(E(I) cls +I seg ))

[0019] In the formula, N represents the batch size, and F... cls L represents the classification result. cls Indicates category label, I cls and I seg E(·) represents the residual augmentation maps generated from the classification map and the segmentation map, respectively. E(·) represents the encoder and C(·) represents the classifier module.

[0020] The loss between CAM labels and CAM activation maps is calculated as follows:

[0021]

[0022] L CAM =D′(L seg )

[0023] F CAM =G(E(I) seg ))

[0024] In the formula, L CAM Indicates CAM label, F CAM L represents the CAM activation map. seg The segmentation label is represented by D′(·), the decoder is represented by D′(·), and the activation map extractor module is represented by G(·).

[0025] The loss between the segmentation label and the segmentation result is calculated as follows:

[0026]

[0027] Fseg=S(D′(E(Iseg)))

[0028] In the formula, F seg S represents the segmentation result, and S(·) represents the segmentation module.

[0029] The beneficial effects of this invention are as follows: This invention combines the ideas of image enhancement and weakly supervised learning. By using a large amount of classification data and a small amount of segmentation dataset in the same network, it collaboratively considers both segmentation and classification tasks, allowing classification and segmentation to mutually promote and improve each other. This integrates the advantages of both tasks, increasing the number of features the model can learn. Simultaneously, segmentation training is performed in the decoder, and because the model learns a large number of classification features, the robustness of the model's segmentation is enhanced. This invention can be used for hematoma classification and can also use classification features for model learning to improve the accuracy of hematoma segmentation.

[0030] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0032] Figure 1 This is a schematic diagram of an image enhancement model;

[0033] Figure 2 This is a schematic diagram of the CAM activation map generation process;

[0034] Figure 3 This is a schematic diagram of the residual enhancement plot generation process;

[0035] Figure 4 A schematic diagram of the cerebral hematoma segmentation model;

[0036] Figure 5 This is a schematic diagram of the training process for a brain hematoma segmentation model. Detailed Implementation

[0037] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0038] like Figure 1 The diagram shown illustrates the structure of the image enhancement model in this invention. The process of using this model for residual enhancement is as follows: Figure 3 As shown, the details are as follows:

[0039] Step 1: Select the CT-ICH hemorrhage dataset as the training set for brain hematoma segmentation. At the same time, the brain hematoma segmentation test also uses the CT-ICH hemorrhage dataset.

[0040] The dataset is preprocessed as follows:

[0041] Step 101: Import the CT-ICH hemorrhage dataset in nii format.

[0042] Step 102: Load the CT-ICH hemorrhage dataset using the nib library and perform window transformation, with a window range of level 40 and width 120.

[0043] Step 103: Slice the CT-ICH hemorrhage dataset along the Z-axis, with a slice range of [28, 40].

[0044] Step 104: Convert each slice of the CT-ICH hemorrhage dataset to three channels and normalize the size to 512×512.

[0045] Step 2: The encoder uses ResNet18, the decoder uses deconvolution for upsampling, and the classifier uses VGG16. At the same time, pseudo-labels are generated using prior knowledge of medical images.

[0046] Step 3: Encoder E uses a six-layer ResNet module. Each ResNet module contains one skip connection and two consecutive convolutions, with the stride of the last convolution being 2. Using the labeled point image as input, the final output feature map is 128×7×7, which is then processed by global average pooling to obtain a 128-dimensional feature vector.

[0047] Step 4: The decoder is implemented through deconvolution, similar to Unet. Each deconvolution halves the number of channels and doubles the size, finally outputting a 3×512×512 pseudo-normal image.

[0048] Step 5: In the last layer, use VGG16 as the classification trainer, take the pseudo-normal image as input, and divide it into three categories: background, bone window, and brain tissue. Finally, output a 3×512×512 category segmentation map.

[0049] Step 6: Discriminator D uses the standard UNet as its backbone. Discriminator D has four layers, each consisting of ReLU, convolutional, and pooling layers. Finally, it is downsampled to 256×16×16 and then fed into the decoder of discriminator D. The decoder consists of deconvolutional layers. The 2×512×512 segmentation result image output by the decoder is used as the loss along with the segmentation label.

[0050] Step 7: Finally, subtract the pseudo-normal image from the original image to obtain the residual enhancement image of the bleeding area.

[0051] Please see Figure 4 and Figure 5 The training process of the brain hematoma image segmentation model proposed in this invention is as follows:

[0052] Step 1: For each training image I l First, the residual enhancement map I is generated through the image enhancement module. eh I eh The pseudo-normal image is obtained by subtracting the original image from the pseudo-normal image. The process can be expressed as the following formula:

[0053] F fake =G(I l )

[0054] I res =I l -F fake

[0055] I eh =I eh +I res

[0056] In the formula, G(·) represents the activation map extractor, and F fake This indicates a pseudo-normal image.

[0057] Step 2: Put the residual enhancement map into the TransUnet backbone network of the brain hematoma segmentation model.

[0058] For the residual enhancement map I generated from the segmentation map seg The segmentation label L is obtained by sending it to the decoder. seg ;

[0059] The activation image F is obtained by sending it into the activation image extractor module. CAM ,like Figure 2 As shown:

[0060] FCAM =G(E(I) seg ))

[0061] Where E(·) represents the encoder;

[0062] The segmentation module is used to obtain the segmentation result F. seg :

[0063] Fseg=S(D′(E(Iseg)))

[0064] Where S(·) represents the segmentation module and D′(·) represents the decoder.

[0065] Step 3: Simultaneously feed the classification image and segmentation image into the classifier module to generate the classification result F. cls :

[0066] F cls =C(E(I) cls +I seg ))

[0067] In the formula, C(·) represents the classifier module, and I cls This represents the residual enhancement map generated from the classification map; furthermore, for the segmentation label L seg CAM labels are obtained by downsampling:

[0068] L CAM =D′(L seg )

[0069] Step 4: via L CAM With F CAM Establish the Dice loss, which is expressed as:

[0070]

[0071] Step 5: Feed the classification result into the cross-entropy loss function to obtain the final result, where the cross-entropy loss is expressed as:

[0072]

[0073] In the formula, N represents the batch size, and L... cls Indicates category label.

[0074] Step 6: Finally, calculate the loss between the segmentation results and the segmentation labels. The loss function is also the Dice loss, expressed as:

[0075]

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for segmenting brain hematoma images based on image enhancement and weakly supervised learning, characterized in that: The method includes the following steps: S1. Collect classification map data and segmentation map data, extract cerebral hematoma pseudo-labels from the classification map and segmentation map using the threshold processing method, and generate pseudo-normal map at the same time. Subtract the pseudo-normal map from the original map to obtain the residual enhancement map of the hemorrhage area. S2. Construct a brain hematoma image segmentation model and train it using residual enhancement maps. The brain hematoma image segmentation model is structured with TransUnet as the backbone network, with a classifier module and an activation map extractor module added after its encoder, and a segmentation module added after its decoder. The training process of the brain hematoma image segmentation model includes: S21. Input the residual enhancement map into the brain hematoma image segmentation model; wherein the segmentation map is respectively entered into the decoder, activation map extractor module and segmentation module, the decoder obtains the segmentation label, the activation map extractor module obtains the activation map, and the segmentation module obtains the segmentation result; at the same time, the classification map and the segmentation map are simultaneously entered into the classifier module to generate the classification result; S22. Downsample the segmentation labels to obtain CAM labels, and extract the classification labels from the classification map; S23. Calculate the loss between the classification label and the classification result, the loss between the CAM label and the CAM activation map, and the loss between the segmentation label and the segmentation result, respectively. The loss between the calculated classification label and the classification result is expressed as follows: In the formula, N Indicates the batch size. Indicates the classification result. Indicates category labels, and These represent the residual enhancement maps generated from the classification map and the segmentation map, respectively. Indicates encoder, This represents the classifier module; The loss between the calculated CAM label and the CAM activation map is expressed as: In the formula, Indicates CAM label, This represents the CAM activation map. Indicates a separator label. Indicates decoder, Indicates activation of the graph extractor module; The loss between the calculated segmentation label and the segmentation result is expressed as: In the formula, Indicates a separator label. Indicates a segmentation module; S3. The hematoma region of the brain hemorrhage image is segmented using a trained brain hematoma image segmentation model.

2. The brain hematoma image segmentation method according to claim 1, characterized in that: In step S1, the process of generating the residual enhancement map of the bleeding area is as follows: First, pseudo-labels are generated using medical prior knowledge through threshold processing; then, a generator and a discriminator are built, a pseudo-normal map is generated through the generator, and the pseudo-normal map is used by the discriminator to generate a pseudo-segmentation map; finally, the original map and the pseudo-normal map are subtracted to obtain the residual enhancement map.

Images