Semi-Supervised Automatic Segmentation Method for Hippocampal Subregions Based on Ultra-High-Field 7T Magnetic Resonance Image Synthesis

Abstract

The present invention belongs to the field of medical image processing, and particularly relates to a semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis. The method includes obtaining 3T magnetic resonance images and 7T magnetic resonance images and performing preprocessing; processing the 3T magnetic resonance images by using a trained image generation model to generate 7T magnetic resonance images; processing the 7T magnetic resonance images by using the first 3D U-Net network cascaded in a trained image segmentation model to extract multi-scale rough segmentation results of the 7T magnetic resonance images; and processing the multi-scale rough segmentation results of the 7T magnetic resonance images by using the second 3D U-Net network cascaded in a trained image segmentation model to extract fine segmentation results of the 7T magnetic resonance images. The present invention can improve the accuracy of the segmentation network and automatically segment 7T magnetic resonance images.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and particularly relates to a semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis. Background Art

[0002] The hippocampus is located deep in the medial temporal lobe of the brain and plays an important role in human memory and spatial navigation. It is one of the most studied neuronal systems in the brain tissue. Clinically, a large number of studies have shown that structural and functional abnormalities of the hippocampus occur in a variety of neuropsychiatric diseases. The hippocampus plays an important role in episodic memory and long-term memory encoding and retrieval.

[0003] However, the hippocampus is not a homogeneous structure. According to the cytoarchitectonic differences, it can be divided into different hippocampal subfields, including the cornu ammonis (hippocampal body, CA), dentate gyrus (DG), and subiculum. Among them, the pyramidal cell layer is the main component of the cornu ammonis, and it can be further subdivided into 4 subregions as shown in Figure 1 Figure 1, including CA1, CA2, CA3, and CA4.

[0004] More and more neuroscience research findings show that the hippocampal subregions may generate more valuable information than the entire hippocampus. Based on Magnetic Resonance Imaging (MRI), accurately and reliably segmenting the hippocampal subregions and analyzing them can help understand the subtle physiological and pathological changes of neuropsychiatric diseases involving the hippocampus, and lay an important foundation for the clinical diagnosis and prediction of diseases. Currently, most manual or automatic segmentation methods for hippocampal subregions are based on 7T MRI images to obtain better segmentation results. However, 7T MRI scanners are extremely expensive, scarce in number, and have extremely high imaging costs, making it difficult to popularize and use them in ordinary medical institutions. Therefore, it is difficult to obtain a large amount of 7T MRI image data for the training and testing of 7T MRI image segmentation models. Summary of the Invention

[0005] Aiming at the problems that the effect of 3T magnetic resonance images in the current hippocampal sub-region segmentation task is inferior to that of 7T magnetic resonance images, and the 7T magnetic resonance image data has high cost and small data volume, a semi-supervised automatic segmentation method for hippocampal sub-regions based on the synthesis of ultra-high field 7T magnetic resonance images is proposed. First, based on the powerful cross-modal image generation effect of the Cycle GAN model, a hippocampal generation model for 7T magnetic resonance images of the brain is established to realize the synthesis of ultra-high field 7T magnetic resonance images from conventional 3T magnetic resonance images, and use the 7T magnetic resonance images to guide the segmentation of the hippocampal sub-regions with high precision in conventional 3T magnetic resonance images; on this basis, based on the 3D U-Net model, a hippocampal sub-region segmentation network is established, and the generated 7T magnetic resonance images are used for the segmentation of the hippocampal sub-regions; finally, aiming at the key problem of small data volume of 7T magnetic resonance images, a semi-supervised learning method is designed to utilize the information of unlabeled data for the training of the entire network model; finally, the effectiveness and practicability of the network model are verified.

[0006] (1) Construct a new image generation model for 7T magnetic resonance images based on conventional 3T magnetic resonance images, combined with the downstream end-to-end segmentation model, which provides an effective way for high-precision segmentation of hippocampal sub-regions on conventional 3T magnetic resonance images.

[0007] (2) Construct a new image segmentation model for the automatic segmentation of the hippocampal sub-region, which fully utilizes the anatomical prior of the hippocampal sub-region and the idea of multi-scale feature fusion to improve the accuracy and stability of the automatic segmentation of the hippocampal sub-region.

[0008] (3) Construct a new semi-supervised learning framework for the automatic segmentation of the hippocampal sub-region, introduce a large number of unlabeled data for model training, make up for the key problem of extremely small data volume of 7T magnetic resonance images, and effectively improve the segmentation accuracy and generalization ability of the hippocampal sub-region.

[0009] The present invention provides the following technical solutions to solve the above technical problems and achieve the above technical objectives. A semi-supervised automatic segmentation method for hippocampal sub-regions based on the synthesis of ultra-high field 7T magnetic resonance images, the method includes:

[0010] Obtain 3T magnetic resonance images and 7T magnetic resonance images, and preprocess the 3T magnetic resonance images and 7T magnetic resonance images;

[0011] Process the 3T magnetic resonance images with the trained image generation model to generate 7T magnetic resonance images;

[0012] Process the 7T magnetic resonance images with the first 3D U-Net network cascaded by the trained image segmentation model to extract the multi-scale rough segmentation results of the 7T magnetic resonance images;

[0013] The multi-scale rough segmentation result of the 7T magnetic resonance image is processed by the second 3D U-Net network cascaded with the trained image segmentation model to extract the fine segmentation result of the 7T magnetic resonance image.

[0014] Advantages of the present invention:

[0015] 1. The present invention adopts an image generation model based on a generative adversarial network, which can convert a conventional 3T MRI image into a 7T MRI image based on the strong modal transfer ability of the above-mentioned adversarial generative network. In order to keep the generated image highly semantically consistent with the original image, a semantic consistency loss is introduced between the generated image and the original image to improve the accuracy of the generated image; an attention mechanism is introduced into the generator and the discriminator to make the image generation process pay more attention to the hippocampal region.

[0016] 2. The present invention adopts a multi-scale nested cascaded image segmentation model. The first 3D U-Net network is used for rough segmentation of the image, and the second 3D U-Net network is used for fine segmentation of the image; the combination of rough and fine segmentation is used to improve the globality of image segmentation; at the same time, a weighted cross-entropy loss and a multi-scale supervision loss are introduced to improve the accuracy of image segmentation, and the two 3D U-Net networks are cascaded layer by layer to ensure that the gradient does not disappear during the training process, improving the accuracy of the segmentation network.

[0017] 3. The present invention adopts a semi-supervised learning method to train the image segmentation model. Based on the consistency hypothesis principle of semi-supervised learning, the input real 7T magnetic resonance image is subjected to two different image enhancement operations, and then input into the image segmentation model respectively. The output obtained by the segmentation model is subjected to consistency regularization, and multi-scale depth consistency regularization is introduced to perform deep supervision on its consistency learning process. Then a corrected pseudo-supervision method is introduced to reduce the noise of the pseudo-labels to improve the segmentation accuracy of the model. Description of the Drawings

[0018] Figure 1 is a schematic diagram of the hippocampal subregion;

[0019] Figure 2 is a flowchart of the semi-supervised automatic segmentation method of the embodiment of the present invention;

[0020] Figure 3 is a structural diagram of the image generation model of the embodiment of the present invention;

[0021] Figure 4 is a structural diagram of the image generation model of the preferred embodiment of the present invention;

[0022] Figure 5 is a structural diagram of the image segmentation model of the embodiment of the present invention;

[0023] Figure 6 It is the structural diagram of the image segmentation model of the preferred embodiment of the present invention. Detailed implementation manners

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0025] Figure 2 It is the flowchart of the semi-supervised automatic segmentation method of the embodiment of the present invention. As Figure 2 shown, the method includes:

[0026] 101. Obtain 3T magnetic resonance images and 7T magnetic resonance images, and preprocess the 3T magnetic resonance images and 7T magnetic resonance images;

[0027] In the embodiment of the present invention, considering that 3T magnetic resonance images are easy to obtain, the present invention generates a segmentation model for 7T magnetic resonance images through 3T magnetic resonance images, so as to automatically segment 7T magnetic resonance images.

[0028] Among them, the 3T magnetic resonance images and 7T magnetic resonance images include magnetic resonance images of four modalities: 3T T1-weighted images, 3T T2-weighted images, 7T T1-weighted images, and 7T T2-weighted images.

[0029] Among them, the preprocessing includes first manually labeling the 7T T2-weighted magnetic resonance images. Then, denoising and bias field correction were performed on the MRI data of four modalities. Finally, these images were registered to the same space for segmentation. Specifically, first, two experienced experts were invited to manually label the hippocampal sub-region of the 7T T2-weighted magnetic resonance images. Then, rigid registration was used to align the 3T T2 and 7T T2 MRI with the corresponding 3T T1 and 7T T1 MRI. Next, the 7T T1 MRI was linearly registered to the 3T T1 MRI, and the 3T T1 MR was linearly registered to an unbiased template using ANTS deformable registration. Two cubic ROIs containing the left and right hippocampi were extracted from the unbiased template. For each side, the binary segmentation was mapped to the template space by merging the anatomical labels on that side into a single label, and a rectangular box covering all the segmentations in the template space was extracted to obtain the cubic ROI. Then, the left and right template ROIs were upsampled to a resolution of 0.4×0.4×0.4 mm3 to match the resolution of the 7T T2 MRI. Finally, using the deformation matrices obtained during the registration process alone or in combination, the MRI data and manual labels of all modalities were mapped to the corresponding hemispheric template ROIs, and subsequent segmentation was performed in these ROIs. Finally, 7T magnetic resonance images paired with 3T magnetic resonance images with manually segmented labels and the corresponding 3T magnetic resonance images were obtained for subsequent network generation and segmentation training.

[0030] 102. Use the trained image generation model to generate 7T magnetic resonance images from the 3T magnetic resonance images;

[0031] In the embodiment of the present invention, the image generation model adopts a generative adversarial network. The traditional generative adversarial network has a pair of generators and a pair of discriminators for adversarial generation.

[0032] In some embodiments of the present invention, the image generation model adopts a Cycle GAN network, such as Figure 2As shown, based on the Cycle GAN network, 3T MRI and 7T MRI are regarded as two different modalities of images. The input 3T magnetic resonance image is used to generate a fake 7T magnetic resonance image through the first generator, and then the generated fake 7T magnetic resonance image is used to calculate the generative adversarial loss through the first discriminator to determine whether the fake 7T magnetic resonance image can be regarded as a real 7T magnetic resonance image. At the same time, the cycle loss is calculated between the fake 7T magnetic resonance image and the real 7T magnetic resonance image. The fake 7T magnetic resonance image determined to be true is sent to the second generator to be generated into a fake 3T magnetic resonance image. The generated fake 3T magnetic resonance image is passed through the second discriminator to calculate the generative adversarial loss to determine whether the fake 3T magnetic resonance image can be regarded as a real 3T magnetic resonance image. At the same time, the cycle loss is calculated between the fake 3T magnetic resonance image and the real 3T magnetic resonance image; by optimizing the generative adversarial loss and the cycle loss, the image generation model is trained.

[0033] In a preferred embodiment of the present invention, a semantic consistency loss is introduced between the Fake 7T MRI and the real 3T MRI to maintain a high semantic consistency between the two modalities of images, so that the segmentation of the generated image and the original image remains the same category.

[0034] Among them, Figure 3 the first generator is denoted as G1, and the second generator is denoted as G2; the first discriminator is denoted as D1, and the second discriminator is denoted as D2; therefore, the adversarial loss L GAN and the cycle loss L Cycle are respectively defined as:

[0035] L GAN (G1, D1, 3T, 7T) = E Real7T～7T [logD1(Real7T)] + E Real3T～3T [log(1 - D1(Fake7T))]

[0036] L GAN (G2, D2, 7T, 3T) = E Real3T～3T [logD2(Real3T)] + E Real7T～7T [log(1 - D2(Fake3T))]

[0037] L Cycle (G1, G2, 3T) = E Real3T～3T [||G2(G1(Real3T)) - Real3T||1]

[0038] L Cycle (G2, G1, 7T) = E Real7T～7T[||G2(G1(Real7T)) - Real7T||1]

[0039] Among them, Fake7T represents a false 7T magnetic resonance image; Real7T represents; Real3T represents a real 3T magnetic resonance image; Fake3T represents a false 3T magnetic resonance image.

[0040] Since this image generation network is mainly used to guide the downstream hippocampal sub-region segmentation task on conventional 3T MRI images, the images should maintain a high degree of semantic consistency before and after synthesis. Therefore, the present invention pre-trains a model F for hippocampal sub-region segmentation of conventional 3T MRI s , fixes its weights, and represents the predicted output of F s as P s (X). Then the semantic consistency loss is defined as:

[0041] L SC = L seg (P s (Fake7T), Fake7T) + L seg (P s (Real3T), Real3T)

[0042] L seg (X, Y) = L ce (X, Y) + L Dice (X, Y)

[0043] Among them, L ce represents the cross-entropy loss, and L Dice represents the Dice loss; Fake7T represents a false 7T magnetic resonance image; Real3T represents a real 3T magnetic resonance image.

[0044] The total loss of the image generation model is:

[0045] L total1 = λ1·L GAN (G1, D1, 3T, 7T) + λ2·L GAN (G2, D2, 7T, 3T)

[0046] + λ3·L Cycle (G1, G2, 3T) + λ4·L Cycle (G2, G1, 7T) + λ5·L SC

[0047] By iteratively training the above total loss L total1 , a trained image generation model can be obtained. This trained image generation model can be used for the generation process of 3T magnetic resonance images to obtain more 7T magnetic resonance images.

[0048] In a preferred embodiment of the present invention, in order to better learn the correlation between global features and make the synthesized image pay more attention to the detailed texture of the hippocampal region, a self-attention mechanism (3D Self-attention modules) is introduced into the G and D modules of the generation network. It can capture long-term dependencies by calculating the feature weighted sum between a certain voxel and all other voxels, thereby making more effective use of long-range feature information.

[0049] Specifically, in order to make the generation process pay more attention to the hippocampus and its sub-region areas, 3D self-attention modules are introduced into the generator and the discriminator, that is, a first 3D self-attention module is further included between the first generator and the first discriminator, and a second 3D self-attention module is further included between the second generator and the second discriminator; this way can improve the quality of the generated image.

[0050] As Figure 4 shown, Figure 4 shows the structural diagram of the image generation model with 3D self-attention modules introduced; in this structure, first, the feature map of the hidden layer is converted into and two feature spaces for calculating attention.

[0051] S j,i = f(x j ) T g(x i )

[0052] where β j,i is the attention of voxel j' to voxel i, and S j,i is the correlation between vector j and vector i, that is, the attention value. Then the third feature space is combined together to calculate the attention feature map The attention feature of each voxel j is as follows:

[0053] h(x i ) = W h x, v(x i ) = W v x

[0054] where v(x i ) represents the value of β j,i multiplied by the third feature space h(x). The final output of the attention module is expressed as:

[0055] y j = αO j+x j

[0056] Among them, α represents a learnable scalar initialized to 0. is a weight matrix learned through 1×1×1 convolution. To improve the memory efficiency, in specific implementation, it can be set where C is the number of channels of the input feature map.

[0057] 103. Use the first 3D U-Net network in the cascaded trained image segmentation model to process the 7T magnetic resonance image, and extract the multi-scale rough segmentation result of the 7T magnetic resonance image;

[0058] 104. Use the second 3D U-Net network in the cascaded trained image segmentation model to process the multi-scale rough segmentation result of the 7T magnetic resonance image, and extract the fine segmentation result of the 7T magnetic resonance image.

[0059] In the embodiment of the present invention, based on the good performance of the 3D U-Net network in 3D medical image segmentation, the design of the hippocampal sub-region segmentation network is carried out based on the 3D U-Net network. The overall network framework is as Figure 5 shown. This network is composed of two identical cascaded 3D U-Net sub-networks, divided into the first 3D U-Net network and the second 3D U-Net network. Each encoding layer and decoding layer of each U-Net network only contains 3×3×3 convolution to reduce the training parameters; batch normalization and ReLU activation are performed after each 3×3×3 convolution; except for the bottleneck layer, the encoding part includes a max-pooling operation after ReLU, and the decoding part performs a transposed convolution before feature concatenation. To alleviate the problem of gradient disappearance and accelerate the network convergence, the network performs deep supervision on different side output layers, and designs an AWCloss function to calculate the loss of multi-scale deep supervision. Then design a fusion module to fuse the multi-scale features of the side output of the first sub-network, and then use the fused features as the input of the second U-Net network for fine segmentation. To prevent the network from being too complex and difficult to converge, the decoding layer of the first U-Net network is cascaded with the encoding layer of the corresponding second sub-network.

[0060] In an embodiment of the present invention, the training process of the image segmentation model includes inputting the 7T magnetic resonance image into the first 3D U-Net network, and using the encoding layer structure of the first 3D U-Net network to extract hierarchical 7T magnetic resonance image features; using the decoding layer structure of the first 3D U-Net network to extract multi-scale rough segmentation results of the hierarchical 7T magnetic resonance image; inputting the multi-scale rough segmentation results of the 7T magnetic resonance image into the second 3D U-Net network, and using the encoding layer structure of the second 3D U-Net network to extract hierarchical 7T magnetic resonance image features; using the decoding layer structure of the second 3D U-Net network to extract multi-scale fine segmentation results of the hierarchical 7T magnetic resonance image; calculating the weighted cross-entropy loss between the true 7T magnetic resonance image label and the fine segmentation result predicted by the last encoding layer of the second 3D U-Net network; calculating the multi-scale supervision loss using the true 7T magnetic resonance image label and the multi-scale fine segmentation results predicted by the remaining encoding layers of the second 3D U-Net network; and training the image segmentation model by optimizing the weighted cross-entropy loss and the multi-scale supervision loss.

[0061] Among them, in order to make full use of the prior knowledge of the hippocampal subregion to further improve the accuracy of its segmentation, the present invention designs an AWC loss function. Cross-entropy loss is one of the most common pixel-level losses in the segmentation task, which is used to calculate the entropy based on the predicted probability and the true label of each pixel. In order to alleviate the class imbalance problem during the training process, class weighting is performed on the cross-entropy loss. Higher weights are given to the labels that need to be concerned and are more difficult to segment finely. The higher the weight, the greater the loss, and the better the model learns this label.

[0062] The formula for the weighted cross-entropy (WCE) loss is as follows:

[0063] L WCE =-w c ∑Y gt logY pred

[0064] Among them, Y gt represents the true label, Y pred represents the predicted segmentation image, and w c is the weight of different labels, which is set according to the segmentation difficulty of each subregion.

[0065] In addition, we extract the probability map of each label from the training images to obtain the position information of different labels. Each value in the probability map represents the probability that the label appears at that point. We perform the following Gaussian transformation on the obtained probability map to obtain the depth atlas prior (DAP) of each label:

[0066]

[0067] Where PA is the probability atlas, σ1 and σ2 are hyperparameters, c ∈ C is the label, and the AWCloss is defined as follows:

[0068]

[0069] Where h, w, and d are the height, width, and depth of the input image, respectively.

[0070] The image segmentation network based on the traditional 3D U-Net only involves the supervision of the top-layer output of the expansion path and ignores the hidden-layer features. Due to the vanishing gradient problem, the final loss cannot be effectively backpropagated to the shallower layers, which makes the 3D medical image segmentation task with only a small amount of labeled data more difficult. The deep supervision mechanism has a strong "regularization" effect on smaller training data and relatively shallow networks, which will significantly affect the final classification accuracy and feature learning. Specifically, first, the outputs of the middle three layers in the decoding path are upsampled to the size of the top layer, and the number of output feature maps is set to the number of labels. Then, the multiple side outputs at different scales are segmented through the softmax layer. By calculating the segmentation losses on the output layers at different scales, the hidden layers can be directly supervised for more effective gradient backpropagation, thereby improving the learning efficiency of the network. The loss function L of its multi-scale supervision mds is calculated as follows:

[0071]

[0072] Where is the segmentation loss of the i-th side output layer, that is, the i-th decoding layer, and N is the total number of side output layers.

[0073] Then the total loss of the image segmentation model is:

[0074] L total2 = L mds + L AWC

[0075] By iterating the above total loss L total2 , the training of the image segmentation model can be obtained, and the trained image segmentation model can be used for the segmentation processing of 7T magnetic resonance images.

[0076] Considering the difficulty of obtaining and annotating 7T magnetic resonance images, in order to make up for the problem of the small amount of labeled training data, a corrected pseudo-supervision loss and a weighted cross-entropy loss are introduced based on the semi-supervised learning method. The structure of the image segmentation network after introducing the above two losses is as Figure 6 shown. For 7T magnetic resonance images with real labels, first perform two different image enhancements on them to obtain the first enhanced 7T magnetic resonance image after enhancement and the second 7T magnetic resonance image The enhancement methods include random intensity offset, random intensity scaling, adding Gaussian noise, etc. Then these three types of images are respectively fed into the image segmentation network to obtain the predicted segmentation maps Pred, and Calculate the loss by comparing Pred with its true label, while and are respectively subjected to corrected pseudo - supervision with Pred. Then the unlabeled data is enhanced twice by different image enhancement methods to obtain the enhanced images and Then the enhanced images are respectively fed into the same segmentation network to obtain their respective predicted segmentation outputs and Then, based on the consistency hypothesis principle of semi - supervised segmentation, and are subjected to consistency supervision to add the information learned by the network from the unlabeled data to the labeled data.

[0077] Therefore, the loss also includes performing different data enhancements on the 7T magnetic resonance image with the true label to generate the first 7T magnetic resonance image and the second 7T magnetic resonance image; respectively inputting the 7T magnetic resonance image with the true label, the first 7T magnetic resonance image, and the second 7T magnetic resonance image into the image segmentation network; using each encoding layer of the second 3D U - Net network to predict the fine segmentation results of the 7T magnetic resonance image with the true label, the first 7T magnetic resonance image, and the second 7T magnetic resonance image; calculating the multi - scale consistency loss through the fine segmentation results of the first 7T magnetic resonance image and the second 7T magnetic resonance image output by the corresponding encoding layers; calculating the corrected pseudo - supervision loss through the 7T magnetic resonance image with the true label respectively with the fine segmentation results of the first 7T magnetic resonance image and the second 7T magnetic resonance image; jointly training the image segmentation model with the multi - scale consistency loss calculation, the corrected pseudo - supervision loss, the weighted cross - entropy loss, and the multi - scale supervision loss.

[0078] Specifically, for the consistency regularization paradigm of semi - supervised learning, currently it is all based on the final output after training the network with perturbed image inputs, and the consistency between the features in the hidden layer of the training network is not considered. As a result, it is possible that the training network fits in a direction deviating from the target task, leading to a reduction in the task effect. The present invention and The multi - scale side output layers of are respectively subjected to corresponding consistency supervision, so as to achieve the purpose of effectively supervising the consistency of the features in the hidden layer of the network. The consistency supervision loss function is defined as:

[0079]

[0080] Among them, represents the consistency loss of the i-th decoding layer of the second 3D U-Net network, that is, the consistency loss at the i-th scale; represents the fine segmentation result output by the first 7T magnetic resonance image at the i-th decoding layer; represents the fine segmentation result output by the second 7T magnetic resonance image at the i-th decoding layer; dis represents the similarity distance function.

[0081] In some embodiments, the similarity distance function adopts the Jaccard distance function, the Euclidean distance function, the cosine similarity distance function, or the KL divergence distance function.

[0082] In some embodiments, the similarity distance function adopts the cosine distance function, which is expressed as:

[0083]

[0084] In order to better supervise the segmentation prediction of labeled data, and the consistency assumption also applies to labeled data, a corrected pseudo-supervision method is designed. By supervising the loss function between the network prediction output after the same image enhancement of the labeled data and the original input image, more effective supervision of the segmentation prediction of labeled data is achieved. Its supervised loss function is defined as follows:

[0085]

[0086]

[0087]

[0088] Among them, L urp (Pred a , pred) represents the corrected pseudo-supervision loss of the 7T magnetic resonance image with the true label and the fine segmentation results of the first 7T magnetic resonance image and the second 7T magnetic resonance image respectively; represents the corrected pseudo-supervision loss of the 7T magnetic resonance image with the true label and the fine segmentation result of the first 7T magnetic resonance image; represents the corrected pseudo-supervision loss of the 7T magnetic resonance image with the true label and the fine segmentation result of the second 7T magnetic resonance image; represents the uncertainty estimation of the 7T magnetic resonance image with the true label and the fine segmentation result of the first 7T magnetic resonance image; represents the uncertainty estimation of the 7T magnetic resonance image with the true label and the fine segmentation result of the second 7T magnetic resonance image; Denotes the cross-entropy loss between the 7T magnetic resonance image with the true label and the fine segmentation result of the first 7T magnetic resonance image; Denotes the cross-entropy loss between the 7T magnetic resonance image with the true label and the fine segmentation result of the second 7T magnetic resonance image; Denotes the predicted value of the fine segmentation result of the first 7T magnetic resonance image, Denotes the predicted value of the fine segmentation result of the second 7T magnetic resonance image, and pred denotes the predicted value of the fine segmentation result of the 7T magnetic resonance image with the true label.

[0089] In some embodiments, the uncertainty estimation includes:

[0090]

[0091] Wherein, Denotes the uncertainty estimation, which is calculated by the K-L divergence. The uncertainty estimation introduces adaptive voxel weighting into the pseudo-supervised loss, where more confident voxels have higher weights, while less confident voxels have lower weights. Such correction reduces the influence of label noise and improves the robustness of the segmentation.

[0092] In some preferred embodiments, the present invention further improves the uncertainty estimation, expressed as:

[0093]

[0094] Wherein, μ i Is a control parameter. To optimize the supervision effect, during the training implementation process, not only the single unlabeled prediction map Or Is used for uncertainty estimation, but the joint uncertainty estimation of the two is performed. The regulation deviation of its joint estimation is regulated by the control parameter μ i Such a joint uncertainty estimation method can ensure that the network's correction pseudo-supervised process is more global, thereby improving the overall semi-supervised learning performance of the network and ultimately achieving the goal of effectively improving the segmentation accuracy.

[0095] In some specific embodiments of the present invention, the present invention can introduce the HCP dataset during the semi-supervised learning training process for unlabeled supervised data for consistency supervision in semi-supervised training. The HCP project has mapped the neural pathways of brain function and behavior through high-quality neuroimaging data of more than 1,100 healthy young people, solving a key problem in the challenge of mapping the human brain map. HCP uses improved data acquisition, analysis, and sharing methods to provide data and discoveries to the scientific community, greatly enhancing the understanding of the human brain structure, function, connectivity, and their relationship with behavior. Among the various datasets it has made public, the 7T MRI whole-brain data of 184 healthy subjects is applicable to the semi-supervised training paradigm of this project. Testing on the above data shows that the present invention has good segmentation effects.

[0096] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and this program can be stored in a computer-readable storage medium. The storage medium can include: ROM, RAM, magnetic disk, or optical disc, etc.

[0097] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis, characterized in that, The method includes: Obtain a 3T magnetic resonance image and a 7T magnetic resonance image, and preprocess the 3T magnetic resonance image and the 7T magnetic resonance image; Process the 3T magnetic resonance image using a trained image generation model to generate a 7T magnetic resonance image; Process the 7T magnetic resonance image using the first 3D U-Net network in the cascaded trained image segmentation model to extract the multi-scale rough segmentation result of the 7T magnetic resonance image; Process the multi-scale rough segmentation result of the 7T magnetic resonance image using the second 3D U-Net network in the cascaded trained image segmentation model to extract the fine segmentation result of the 7T magnetic resonance image; The training process of the image segmentation model includes inputting the 7T magnetic resonance image into the first 3D U-Net network, and using the encoding layer structure of the first 3D U-Net network to extract the hierarchical 7T magnetic resonance image features; using the decoding layer structure of the first 3D U-Net network to extract the multi-scale rough segmentation result of the hierarchical 7T magnetic resonance image; inputting the multi-scale rough segmentation result of the 7T magnetic resonance image into the second 3D U-Net network, and using the encoding layer structure of the second 3D U-Net network to extract the hierarchical 7T magnetic resonance image features; using the decoding layer structure of the second 3D U-Net network to extract the multi-scale fine segmentation result of the hierarchical 7T magnetic resonance image; calculating the weighted cross-entropy loss between the real 7T magnetic resonance image label and the fine segmentation result predicted by the last encoding layer of the second 3D U-Net network; calculating the multi-scale supervision loss using the real 7T magnetic resonance image label and the multi-scale fine segmentation results predicted by the remaining encoding layers of the second 3D U-Net network; training the image segmentation model by optimizing the weighted cross-entropy loss and the multi-scale supervision loss; The loss also includes performing different data augmentations on the 7T magnetic resonance image with real labels to generate a first 7T magnetic resonance image and a second 7T magnetic resonance image; inputting the 7T magnetic resonance image with real labels, the first 7T magnetic resonance image, and the second 7T magnetic resonance image into the image segmentation network respectively; using each encoding layer of the second 3D U-Net network to predict the fine segmentation results of the 7T magnetic resonance image with real labels, the first 7T magnetic resonance image, and the second 7T magnetic resonance image; calculating the multi-scale consistency loss through the fine segmentation results of the first 7T magnetic resonance image and the second 7T magnetic resonance image output by the corresponding encoding layer; calculating the corrected pseudo-supervision loss through the 7T magnetic resonance image with real labels and the fine segmentation results of the first 7T magnetic resonance image and the second 7T magnetic resonance image respectively; jointly training the image segmentation model with the multi-scale consistency loss, the corrected pseudo-supervision loss, the weighted cross-entropy loss, and the multi-scale supervision loss.

2. The semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis according to claim 1, wherein, The training process of the image generation model includes generating a fake 7T magnetic resonance image from the input 3T magnetic resonance image through a first generator, and then calculating the generative adversarial loss of the generated fake 7T magnetic resonance image through a first discriminator to determine whether the fake 7T magnetic resonance image can be regarded as a real 7T magnetic resonance image. At the same time, a cycle loss is calculated between the fake 7T magnetic resonance image and the real 7T magnetic resonance image. The fake 7T magnetic resonance image determined to be true is sent to a second generator to be generated into a fake 3T magnetic resonance image. The generated fake 3T magnetic resonance image is passed through a second discriminator to calculate the generative adversarial loss to determine whether the fake 3T magnetic resonance image can be regarded as a real 3T magnetic resonance image. At the same time, a cycle loss is calculated between the fake 3T magnetic resonance image and the real 3T magnetic resonance image; the image generation model is trained by optimizing the generative adversarial loss and the cycle loss.

3. A semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis according to claim 2, characterized in that, The loss also includes calculating a semantic consistency loss between the input 3T magnetic resonance image and the fake 7T magnetic resonance image, and jointly training the image generation model with the semantic consistency loss, the generative adversarial loss, and the cycle loss.

4. A semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis according to claim 2, characterized in that, A first 3D self-attention module is further included between the first generator and the first discriminator, and a second 3D self-attention module is further included between the second generator and the second discriminator.

5. A semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis according to claim 1, characterized in that The multi-scale consistency loss includes: Among them, represents the consistency loss of the i-th decoding layer of the second 3D U-Net network, that is, the consistency loss at the i-th scale, represents the fine segmentation result output by the first 7T magnetic resonance image at the i-th decoding layer; represents the fine segmentation result output by the second 7T magnetic resonance image at the i-th decoding layer; dis represents the similarity distance function.

6. A semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis according to claim 5, characterized in that, The similarity distance function adopts the Jaccard distance function, the Euclidean distance function, the cosine similarity distance function, or the KL divergence distance function.

7. A semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis according to claim 1, characterized in that, The calibration pseudo-supervised loss includes: Among them, L urp (Pred a , pred) represents the corrected pseudo - supervised loss of the 7T magnetic resonance image with the true label respectively with the fine - segmentation results of the first 7T magnetic resonance image and the second 7T magnetic resonance image; represents the corrected pseudo - supervised loss of the 7T magnetic resonance image with the true label and the fine - segmentation result of the first 7T magnetic resonance image; represents the corrected pseudo - supervised loss of the 7T magnetic resonance image with the true label and the fine - segmentation result of the second 7T magnetic resonance image; represents the uncertainty estimation of the 7T magnetic resonance image with the true label and the fine - segmentation result of the first 7T magnetic resonance image; represents the uncertainty estimation of the 7T magnetic resonance image with the true label and the fine - segmentation result of the second 7T magnetic resonance image; represents the cross - entropy loss of the 7T magnetic resonance image with the true label and the fine - segmentation result of the first 7T magnetic resonance image; represents the cross - entropy loss of the 7T magnetic resonance image with the true label and the fine - segmentation result of the second 7T magnetic resonance image; represents the predicted value of the fine - segmentation result of the first 7T magnetic resonance image, represents the predicted value of the fine - segmentation result of the second 7T magnetic resonance image, and pred represents the predicted value of the fine - segmentation result of the 7T magnetic resonance image with the true label.

8. A semi-supervised automatic segmentation method for hippocampal subregions based on ultra-high field 7T magnetic resonance image synthesis according to claim 7, characterized in that The uncertainty estimation includes: Where j = 1 or j = 2, j = 1 represents the first 7T magnetic resonance image, and j = 2 represents the second 7T magnetic resonance image.

Images