Semi-supervised segmentation method and system for fetal brain tissue images
By using a diffused U-Net encoder and decoder structure, combined with difficulty-aware loss and a guided consistency module, the problems of data annotation difficulties and class imbalance in fetal brain tissue segmentation are solved, achieving high-precision semi-supervised segmentation.
Patent Information
- Application Number
- CN202411235803.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-09-04
AI Technical Summary
Existing technologies for fetal brain tissue segmentation suffer from problems such as difficulty in data labeling, class imbalance, and inability to fully utilize information from unlabeled samples, resulting in insufficient segmentation accuracy.
By employing a diffusion U-Net encoder and decoder structure, combined with difficulty-aware loss and guided consistency module, and performing semi-supervised learning by adding noise to labeled images and utilizing information from unlabeled images, segmentation accuracy is improved.
With a small number of labeled samples, high-accuracy fetal brain tissue segmentation was achieved, solving the class imbalance problem and improving segmentation precision.
Smart Images

Figure CN119338839B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a semi-supervised segmentation method and system for fetal brain tissue images. Background Technology
[0002] Currently, the diagnosis of fetal brain development abnormalities mainly relies on manual judgment. However, manual judgment is highly dependent on the expert's skill level, lacks quantitative analysis capabilities, is labor-intensive, and different experts may make different judgments on the same data, or even the same expert may make different judgments at different times. In recent years, with the development of computer vision technology, the latest research has begun to use artificial intelligence methods to automatically segment different tissues. This can not only accelerate the expert's diagnostic process and allow for quantitative analysis of brain volume through segmentation results, but also eliminate the subjective factors between different experts.
[0003] Meanwhile, as a medical image segmentation problem, fetal brain tissue segmentation faces numerous challenges related to medical imaging: First, medical images involve the privacy of medical case data, resulting in a very limited amount of data. Second, the annotation of medical images typically requires professionals with medical background knowledge, leading to high costs and a large workload.
[0004] Huang et al. designed a lightweight fetal brain tissue segmentation model that integrates a CNN-based encoder-decoder architecture with a transformer module called the Context Transformer Block (CoT). The CoT block integrates context information mining and self-attention learning into a unified architecture. Furthermore, a hybrid dilated convolution module is introduced in the last layer of the decoder.
[0005] However, Huang et al.'s existing technology has at least the following drawbacks:
[0006] First, most of them are concentrated in the field of fully supervised learning, which cannot effectively solve the problem of data labeling difficulties.
[0007] Secondly, no weights are assigned to any category of learning, treating difficult and easy categories equally.
[0008] Third, the information transfer effect of labeled samples on unlabeled samples was not fully utilized. Summary of the Invention
[0009] In view of this, it is necessary to provide a semi-supervised segmentation method and system for fetal brain tissue images.
[0010] This invention provides a semi-supervised segmentation method for fetal brain tissue images. The method is applied to a diffusion model, which includes an electrically connected diffusion U-Net encoder, a diffusion U-Net decoder, and a U-Net decoder. The method includes the following steps: S1, acquiring fetal brain tissue images; the fetal brain tissue images include labeled images and unlabeled images; S2, adding noise to the labels of the labeled images to generate a noisy image, then inputting the noisy image, the time of generating the noisy image, and the labeled image into the diffusion U-Net encoder and diffusion U-Net decoder to obtain the segmentation result output by the supervised diffusion model, and finally calculating the loss function L with the labels of the labeled images. deno S3, For the diffusion U-Net encoder, the intermediate information is decoded using the U-Net decoder to obtain the decoded segmentation result, and then the loss function L is calculated by comparing it with the labels of the labeled image. diff S4 inputs the unlabeled image into the diffusion U-Net encoder to extract features, uses the diffusion U-Net decoder and the U-Net decoder to obtain the segmentation results of the unlabeled image, and then calculates the loss function L. u S5, using the bootstrapping consistency module for the U-Net decoder, the loss function L is calculated. u_consis S6, based on the obtained loss function L deno L diff L u L u_consis The final loss function is calculated, and the image is encoded using a diffusion U-Net encoder and then decoded using a U-Net decoder to obtain the final segmentation result of the fetal brain tissue image.
[0011] Preferably, the labeled image includes: a labeled image X L and its label Y L X is a preprocessed three-dimensional fetal brain MRI image, with a size of CxDxHxW, where C represents the number of channels, D represents the depth, H represents the height, and W represents the width; the unlabeled image includes: unlabeled image X. U .
[0012] Preferably, step S2 includes:
[0013] Perception is performed using a diffusion U-Net encoder and a diffusion U-Net decoder, given a labeled image X. L ∈R D×W×H and label Y L ∈R D×W×H First, convert the tag to one-hot encoding format. Then, t steps of noise are added consecutively to obtain noise labels. This is the forward diffusion process:
[0014]
[0015] Then, label the noise. With labeled image X L The connection serves as the input to the diffusion U-Net encoder;
[0016] Next, Using time t as input to the diffusion U-Net encoder, multi-scale features embedded at time steps are generated. Where i represents the scale, and F represents the size of the basic feature. The U-Net decoder is further used to predict clear labels.
[0017] Preferably, step S3 includes:
[0018] The U-Net decoder deconvolves the features of the last layer output by the diffusion U-Net encoder in step S1 to increase the feature map size and reduce the number of channels. Then, it performs a convolution operation after concatenating each layer with the intermediate output of the diffusion U-Net encoder, and finally outputs the segmentation result.
[0019] Preferably, step S3 further includes:
[0020] Define τ as the cumulative iteration count, and λ as... k,e The Dice score for class k in the e-th iteration. and Let d represent the categories of those who did not learn successfully and those who learned successfully after the τth iteration, respectively. k,e Defined as the difficulty of the k-th class after the e-th iteration, then: classes with faster learning speeds have smaller d values. k,e The corresponding weights in the loss function are smaller, thus slowing down the learning speed;
[0021]
[0022] definition The inverse Dice weights are inversely proportional to the Dice score; the lower the Dice score, the larger the weight, forcing the diffusion model to focus more on learning the difficult categories. The overall difficulty-perceived weight for the k-th category is defined as...
[0023]
[0024] The difficulty-aware loss function is defined as:
[0025]
[0026] Preferably, step S4 includes:
[0027] For the same unlabeled image X U The unlabeled image X is obtained using the diffusion U-Net decoder and the U-Net decoder, respectively. U Based on the two segmentation results, the loss function L is calculated. u .
[0028] Preferably, step S5 includes:
[0029] For labeled and unlabeled images, the features F obtained from the diffusion U-Net encoder l and F u After passing through the U-Net decoder, the features of the intermediate layers are obtained respectively. Combined into Feat_l and The data are combined to form Feat_u. Feat_l is processed by the bootstrapping consensus module to obtain Feat_Maps_l and attention query updates_Qs_l. Feat_u is processed by the bootstrapping consensus module to obtain Feat_Maps_consis. Then, Feat_Maps_consis and updates_Qs_l are input into the bootstrapping consensus module to obtain Feat_Maps_u. Finally, the difference between Feat_Maps_l and Feat_Maps_u is calculated to obtain the loss function L. u_consis .
[0030] Preferably, step S5 further includes:
[0031] When there is no input Giuded_Q, the consistency module is initialized to 0. After feature projection and feature normalization, the Feature is fed into the classification decoder along with the Giuded_Q. Then, for each scale of features, the module outputs the fused multi-head attention features and the updated Giuded_Q. Finally, the results of each scale are collected to obtain Feature_map and update_Q.
[0032] Preferably, step S5 further includes:
[0033] The classification decoder takes the normalized Feature and Guided_Q as input, and then applies linear mapping to each. The linearly mapped Feature is treated as V and K in the attention mechanism, and the mapped Guided_Q is treated as Q. Q and K are multiplied to obtain a new Feature, and Feature and V are multiplied to obtain a new Guided_Q. Then, these are processed by normalization, MLP, and dropout, and then added to the original variables to obtain the fused multi-head attention features and the updated Guided_Q.
[0034] Preferably, step S6 includes:
[0035] The final loss function L is determined by L deno L diff L u L u_consis composition:
[0036] L = L deno +L diff +μ(L u +L u_consis ).
[0037] This invention addresses the problem of a large number of unlabeled samples and a small number of labeled samples in medical image data, further improving the accuracy of fetal brain tissue segmentation. This invention introduces a diffusion model into semi-supervised segmentation of fetal brain tissue images. Compared to ordinary fully supervised algorithms, this invention can achieve similar results with less labeled data. The network adopts a single encoder and dual decoder structure, and introduces a difficulty-aware loss when calculating the loss. This difficulty-aware loss allows the diffusion model to focus more on learning difficult categories, thus solving the class imbalance problem encountered in medical image segmentation. Simultaneously, this invention introduces a guided consistency module, which can learn information from labeled samples and, through an attention mechanism, uses this learned information to guide the learning of unlabeled samples, improving the model's learning performance. Attached Figure Description
[0038] Figure 1 A schematic diagram illustrating the overall framework of the technical solution provided in the embodiments of the present invention;
[0039] Figure 2 This is a flowchart of the semi-supervised segmentation method for fetal brain tissue images according to the present invention;
[0040] Figure 3 The loss function L provided for embodiments of the present invention u_consis A diagram illustrating the specific calculation method;
[0041] Figure 4 This is a schematic diagram illustrating the specific process of the boot consistency module provided in an embodiment of the present invention. Detailed Implementation
[0042] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0043] The overall framework diagram of the technical solution of the present invention is as follows: Figure 1 As shown.
[0044] The diffusion model comprises a diffusion U-Net encoder, a diffusion U-Net decoder, and a U-Net decoder. The diffusion U-Net encoder, the diffusion U-Net decoder, and the U-Net decoder are electrically connected.
[0045] During the training phase, a diffusion model is used to capture invariant features, and a guided consistency module is used to guide the learning of more accurate features.
[0046] The input image includes: a labeled image X L and its label Y L Unlabeled image X U Furthermore:
[0047] X is the preprocessed three-dimensional fetal brain MRI image, with a size of (CxDxHxW), where C represents the number of channels, D represents the depth, H represents the height, and W represents the width. In this embodiment, since the image is a three-dimensional black and white image, C is 1.
[0048] See Figure 2 The diagram shown is a flowchart of a preferred embodiment of the semi-supervised segmentation method for fetal brain tissue images of the present invention.
[0049] Step S1: Obtain fetal brain tissue images. The fetal brain tissue images include labeled images and unlabeled images.
[0050] Step S2: Noise is added to the labels of the labeled image to generate a noisy image. Then, the noisy image, the time of generating the noisy image, and the labeled image are input into the diffusion U-Net encoder and diffusion U-Net decoder to obtain the segmentation result output by the supervised diffusion model. Finally, the loss function L is calculated by comparing it with the labels of the labeled image. deno Specifically:
[0051] In this embodiment of the invention, the labeled image X L The label Y L Noise is added to generate a noisy image. Then, the noisy image, the time t when the noisy image was generated, and the labeled image X are compared. L Input the diffusion U-Net encoder and diffusion U-Net decoder to obtain the segmentation result output by the supervised diffusion model. Then, based on the above segmentation result and the label Y... L Calculate the loss function L deno That is to say:
[0052] Use L deno The difference between the segmentation result generated by the diffusion U-Net decoder and the original label is calculated to learn image information and enhance the accuracy of the diffusion model.
[0053] Furthermore:
[0054] Perception is performed using a diffusion model, comprising a diffusion U-Net encoder and a diffusion U-Net decoder, given a labeled image X. L ∈R D×W×H and label Y L ∈R D×W×H First, convert the tag to one-hot encoding format. Then, t steps of noise are added consecutively to obtain noise labels. This is the forward diffusion process:
[0055]
[0056] Then, label the noise. With labeled image X L The connection serves as the input to the diffusion U-Net encoder. It is worth noting that the high-level features differ between the different data streams.
[0057] For the noise reduction process, firstly, Using time t as input to the diffusion U-Net encoder, multi-scale features embedded at time steps are generated. Where i represents the scale, and F represents the size of the basic feature. The U-Net decoder is further used to predict clear labels.
[0058] Step S3: For the diffusion U-Net encoder, the intermediate information is decoded using the U-Net decoder to obtain the decoded segmentation result, and then the loss function L is calculated by comparing it with the labels of the labeled images. diff Specifically:
[0059] In this embodiment of the invention, for the diffusion U-Net encoder, the intermediate information is decoded using the U-Net decoder to obtain the decoded segmentation result, and then the decoded segmentation result is compared with the label Y. L Calculate the loss function L diff That is to say:
[0060] L diff The difficulty-aware loss function is L. diff The difficulty category is calculated, and different learning weights are assigned to different categories so that the diffusion model can learn more information about the difficulty category.
[0061] Furthermore:
[0062] The U-Net decoder deconvolves the features of the last layer output by the diffusion U-Net encoder in step S1 to increase the feature map size and reduce the number of channels. Then, it performs a convolution operation after concatenating each layer with the intermediate output of the diffusion U-Net encoder, and finally outputs the segmentation result.
[0063] To alleviate the class imbalance problem encountered in medical image segmentation, this embodiment adopts a difficulty-aware weighting strategy. This strategy forces the diffusion model to focus on the most difficult class, that is, the class with the slowest learning speed and poorest performance. The difficulty is learned through the probability map p generated by the U-Net decoder.
[0064] Specifically, in this embodiment, τ is defined as the number of cumulative iterations, empirically set to 50; λ k,e The Dice score for class k in the e-th iteration; and These represent the categories of those who did not learn successfully and those who learned successfully after the τth iteration, respectively; d k,e Let be the difficulty of the k-th category after the e-th iteration. Then: categories that learn faster have smaller d values. k,e The corresponding weights in the loss function are smaller to slow down the learning speed.
[0065]
[0066] In this embodiment, the definition is... The inverse Dice weights are inversely proportional to the Dice score; the lower the Dice score, the larger the weight, forcing the diffusion model to focus more on learning difficult categories. The overall difficulty-perceived weight for the k-th category is defined as... Based on experience, α is set to 0.2 to mitigate the impact of outliers.
[0067]
[0068] The difficulty-aware loss function is defined as follows:
[0069]
[0070] Step S4: Input the unlabeled image into the diffusion U-Net encoder to extract features, then use the diffusion U-Net decoder and U-Net decoder to obtain the segmentation results of the unlabeled image, and then calculate the loss function L. u Specifically:
[0071] In this embodiment of the invention, the unlabeled image X U The input is fed into the diffusion U-Net encoder to extract features, and then the diffusion U-Net decoder and the U-Net decoder are used to obtain the unlabeled image X. U The two segmentation results are used to calculate the loss function L. u That is to say:
[0072] Calculate the consistency L between the diffusion U-Net decoder and the U-Net decoder for the same unlabeled image. u .
[0073] Step S5: Using the bootstrap consistency module, calculate the loss function L for the U-Net decoder. u_consis Specifically:
[0074] In this embodiment of the invention, a guided consistency module is used to calculate the consistency L across multiple scales between the unlabeled image segmentation result guided by the labeled image and the unlabeled image segmentation result without guidance. u_consis .
[0075] Furthermore:
[0076] The guiding consistency module works by learning the attention query updates_Qs_l for labeled images, and then comparing the consistency of the multi-scale features output without the attention query with the multi-scale features output with the attention query, thereby achieving the effect of guiding consistency.
[0077] like Figure 3 As shown, the features F obtained from the diffusion U-Net encoder for labeled and unlabeled images are... l and F u After passing through the U-Net decoder, the features of the intermediate layers are obtained respectively. Combined into Feat_l and The data are combined to form Feat_u. Feat_l is processed by the bootstrapping consensus module to obtain Feat_Maps_l and attention query updates_Qs_l. Feat_u is processed by the bootstrapping consensus module to obtain Feat_Maps_consis. Then, Feat_Maps_consis and updates_Qs_l are input into the bootstrapping consensus module to obtain Feat_Maps_u. Finally, the difference between Feat_Maps_l and Feat_Maps_u is calculated to obtain the loss function L. u_consis .
[0078] The specific process of bootstrapping the consistency module is as follows: Figure 4 As shown, when there is no input Giuded_Q, it is directly initialized to 0. After feature projection and feature normalization, the Feature is fed into the classification decoder along with the Giuded_Q. Then, for the features at each scale, the fused multi-head attention features and the updated Giuded_Q are output. Finally, the results at each scale are collected to obtain Feature_map and update_Q.
[0079] The classification decoder process is as follows: Input the normalized Feature and Guided_Q, and then use linear mapping respectively. Treat the linearly mapped Feature as V and K in the attention mechanism, and the mapped Guided_Q as Q. Multiply Q and K to get a new Feature, and multiply Feature and V to get a new Guided_Q. Then pass through normalization, MLP, and dropout respectively, and add them to the original variables to obtain the fused multi-head attention features and the updated Guided_Q.
[0080] Step S6: Calculate the final loss function, encode using a diffusion U-Net encoder, and then decode using a U-Net decoder to obtain the final segmentation result of the fetal brain tissue image.
[0081] Specifically:
[0082] The embodiments of the present invention use the following formula to calculate the final loss function L, the final loss function L being derived from L... deno L diff L u composition:
[0083] L = L deno +L diff +μ(L u +L u_consis )
[0084] Based on the calculated final loss function, the segmentation is performed using a diffusion U-Net encoder and then decoded using a U-Net decoder to obtain the final segmentation result.
[0085] This invention can achieve high-accuracy image segmentation even with only a small number of labeled samples.
[0086] Although the present invention has been described with reference to the present preferred embodiments, those skilled in the art should understand that the above preferred embodiments are only used to illustrate the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A semi-supervised segmentation method for fetal brain tissue images, characterized in that, The method is applied to a diffusion model, which includes a diffusion U-Net encoder, a diffusion U-Net decoder, and a U-Net decoder that are electrically connected to each other. The method includes the following steps: S1, acquire fetal brain tissue images; the fetal brain tissue images include: labeled images and unlabeled images; S2, noise is added to the labels of the labeled image to generate a noisy image. Then, the noisy image, the time of generating the noisy image, and the labeled image are input into the diffusion U-Net encoder and diffusion U-Net decoder to obtain the segmentation result output by the supervised diffusion model. Finally, the loss function is calculated by combining the noisy image with the labels of the labeled image. ; S3, for the diffusion U-Net encoder, the U-Net decoder is used to decode the intermediate information to obtain the decoded segmentation result, and then the loss function is calculated by comparing it with the labels of the labeled images. ; S4 inputs the unlabeled image into the diffusion U-Net encoder to extract features, uses the diffusion U-Net decoder and the U-Net decoder to obtain the segmentation results of the unlabeled image, and then calculates the loss function. ; S5, using the bootstrap consistency module for the U-Net decoder, calculates the loss function. ; S6, based on the obtained loss function , , , The final loss function is calculated, and the image is encoded using a diffusion U-Net encoder and then decoded using a U-Net decoder to obtain the final segmentation result of the fetal brain tissue image. Step S3 includes: definition The cumulative number of iterations, The Dice score for class k in the e-th iteration. and They represent the first After the second iteration, there were no categories of successful learning or successful learning. Defined as the difficulty of the k-th class after the e-th iteration, then: classes with faster learning speeds have lower difficulty levels. The corresponding weights in the loss function are smaller, thus slowing down the learning speed; , definition The inverse Dice weights are inversely proportional to the Dice score; the lower the Dice score, the larger the weight, forcing the diffusion model to focus more on learning the difficult categories. The overall difficulty-perceived weight for the k-th category is defined as... : The difficulty-aware loss function is defined as: 。 2. The method as described in claim 1, characterized in that, The labeled images include: labeled images and its labels X is a preprocessed three-dimensional fetal brain MRI image, with a size of CxDxHxW, where C represents the number of channels, D represents the depth, H represents the height, and W represents the width; the unlabeled image includes: unlabeled images .
3. The method as described in claim 2, characterized in that, Step S2 includes: Sensing is performed using a diffusion U-Net encoder and diffusion U-Net decoder, given a labeled image. and tags First, convert the tag to one-hot encoding format. Then add t steps of noise continuously. Obtain noise labels This is the forward diffusion process: , Then, label the noise. With labeled images The connection serves as the input to the diffusion U-Net encoder; Next, Using time t as input to the diffusion U-Net encoder, multi-scale features embedded at time steps are generated. ,in, This indicates which scale, and F represents the size of the basic feature. The U-Net decoder is further used to predict clear labels. .
4. The method as described in claim 3, characterized in that, Step S3 includes: The U-Net decoder deconvolves the features of the last layer output by the diffusion U-Net encoder in step S1 to increase the feature map size and reduce the number of channels. Then, it performs a convolution operation after concatenating each layer with the intermediate output of the diffusion U-Net encoder, and finally outputs the segmentation result.
5. The method as described in claim 4, characterized in that, Step S4 includes: For the same unlabeled image Unlabeled images were obtained using the diffusion U-Net decoder and the U-Net decoder, respectively. Based on the two segmentation results, the loss function is calculated. .
6. The method as described in claim 5, characterized in that, Step S5 includes: Features obtained from the diffusion U-Net encoder for labeled and unlabeled images and After passing through the U-Net decoder, the features of the intermediate layers are obtained respectively. Combined into Feat_l and The data are combined to form Feat_u. Feat_l is processed by the bootstrapping consensus module to obtain Feat_Maps_l and attention query updates_Qs_l. Feat_u is processed by the bootstrapping consensus module to obtain Feat_Maps_consis. Then, Feat_Maps_consis and updates_Qs_l are input into the bootstrapping consensus module to obtain Feat_Maps_u. Finally, the difference between Feat_Maps_l and Feat_Maps_u is calculated to obtain the loss function. .
7. The method as described in claim 6, characterized in that, Step S5 further includes: When there is no input Giuded_Q, the consistency module is initialized to 0. After feature projection and feature normalization, the Feature is fed into the classification decoder along with the Giuded_Q. Then, for each scale of features, the module outputs the fused multi-head attention features and the updated Giuded_Q. Finally, the results of each scale are collected to obtain Feature_map and update_Q.
8. The method as described in claim 7, characterized in that, Step S5 further includes: The classification decoder takes the normalized Feature and Guided_Q as input, and then applies linear mapping to each. The linearly mapped Feature is treated as V and K in the attention mechanism, and the mapped Guided_Q is treated as Q. Q and K are multiplied to obtain a new Feature, and Feature and V are multiplied to obtain a new Guided_Q. Then, these are processed by normalization, MLP, and dropout, and then added to the original variables to obtain the fused multi-head attention features and the updated Guided_Q.
9. The method as described in claim 8, characterized in that, Step S6 includes: The final loss function Depend on , , , composition: + + 。
Citation Information
Patent Citations
Semi-supervised medical image segmentation method based on boundary perception
CN116030044A
Semi-supervised segmentation method based on potential diffusion model and multi-level context cross consistency
CN116958554A
