A Semi-Supervised Segmentation Method and System for Medical Images with Dynamic and Static Adaptive Boundary Awareness

Abstract

This invention discloses a semi-supervised medical image segmentation method and system with dynamic and static adaptive boundary awareness. The method first acquires medical image data and constructs training samples, including labeled and unlabeled medical image data. Next, a student-teacher semi-supervised medical image segmentation model is constructed and trained using both labeled and unlabeled medical image data. Finally, the parameters of the student-teacher semi-supervised medical image segmentation model are optimized based on the total loss function, and the segmentation result of the medical image to be segmented is output using the trained model. This invention helps improve the reliability of supervision information for unlabeled medical images, alleviates the problem of unstable student segmentation branch training in scenarios with few labels, and reduces the impact of pseudo-label noise at the boundary of unlabeled samples on the training process.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and deep learning technology, and in particular to a semi-supervised medical image segmentation method based on dynamic adaptation and boundary-aware supervision. Background Technology

[0002] Medical image segmentation is a crucial technical component in computer-aided diagnosis, lesion identification, organ structure analysis, preoperative planning, and quantitative medical assessment. In recent years, deep learning-based medical image segmentation methods have achieved promising results. However, these methods typically rely on large amounts of pixel-level labeled data for training. Pixel-level annotation of medical images often requires personnel with specialized medical knowledge, a time-consuming and costly process, and is susceptible to subjective differences among annotators. Therefore, in practical applications, only a small number of labeled medical images and a large number of unlabeled medical images are often obtained.

[0003] Semi-supervised medical image segmentation methods, by jointly utilizing a small number of labeled medical images and a large number of unlabeled medical images, can alleviate the problem of insufficient medical image annotation to some extent. Existing semi-supervised segmentation methods typically employ consistency learning, pseudo-label learning, multi-branch collaborative training, or teacher-student models to mine supervised information from unlabeled medical images. However, in scenarios with few annotations, there may be significant differences in appearance distribution between labeled and unlabeled medical images, such as differences in imaging equipment, scanning protocols, tissue morphology, lighting conditions, texture features, and lesion morphology. When the model directly transfers supervised information learned from labeled data to unlabeled data, it is prone to unstable feature transfer and unreliable pseudo-supervised signals.

[0004] On the other hand, target regions in medical images often suffer from problems such as blurred boundaries, low contrast, irregular shapes, and difficulty in distinguishing fine structures. Conventional region-level segmentation losses mainly focus on the overall overlap between the predicted and labeled regions, and are insufficiently constrained by issues such as boundary offset, edge breaks, and local contour discontinuities. In semi-supervised learning, pseudo-labels or consistency supervision from unlabeled samples are more prone to noise in boundary regions. If these uncertain boundary regions are not explicitly constrained, erroneous supervision may be continuously reinforced during training, leading to unstable boundary localization.

[0005] In recent years, segmentation base models have provided strong structural prior information, offering a new technical foundation for medical image segmentation. However, medical images differ significantly from natural images in terms of imaging mechanisms, texture distribution, structural morphology, and boundary features. Using only fixed base model features or static adaptation methods is insufficient to adequately accommodate the appearance differences between various medical image inputs. Furthermore, existing semi-supervised medical image segmentation methods based on base models typically focus more on region-level knowledge distillation or prediction consistency, neglecting the reliability of boundary region supervision and the suppression of boundary pseudo-label noise.

[0006] Therefore, there is an urgent need for a semi-supervised medical image segmentation method that can, under conditions of limited labeled data, utilize the structural prior information of the basic segmentation model and dynamically adapt it to the input medical image, while providing more reliable boundary-aware supervision for the boundary regions of the medical image, thereby improving the accuracy of medical image segmentation results and the stability of boundary localization. Summary of the Invention

[0007] The purpose of this invention is to provide a semi-supervised medical image segmentation method and system based on dynamic adaptation and boundary-aware supervision, in order to solve the problems of unstable feature transfer caused by the difference in appearance distribution between labeled and unlabeled data in existing semi-supervised medical image segmentation methods in scenarios with few labels, unreliable supervision signals in boundary regions, and easy accumulation of false label noise at the boundaries.

[0008] To achieve the above objectives, the present invention provides a semi-supervised medical image segmentation method based on dynamic adaptation and boundary-aware supervision, comprising the following steps:

[0009] S1. Acquire medical image data and construct training samples, wherein the medical image data includes labeled medical image data and unlabeled medical image data.

[0010] S2. Construct a student-teacher semi-supervised medical image segmentation model, and train the student-teacher semi-supervised medical image segmentation model using the labeled medical image data and unlabeled medical image data. The student-teacher semi-supervised medical image segmentation model includes a student segmentation branch and a teacher branch based on the segmentation base model.

[0011] S21. In the image encoder of the teacher branch, a dynamic-static domain adaptive adapter is introduced. The intermediate features of the input medical image are dynamically-statically adapted by the dynamic-static domain adaptive adapter to obtain adapted teacher features. Based on the adapted teacher features, a teacher segmentation prediction is generated for knowledge distillation of the student segmentation branch.

[0012] S22. The student segmentation branch is trained based on a boundary-aware supervision mechanism, which includes boundary band weighted supervision, stable edge supervision, and a boundary gate discriminator. The boundary band weighted supervision is used to enhance the supervision weight of the boundary region, the stable edge supervision is used to constrain the edge prediction results, and the boundary gate discriminator is used to guide the student segmentation branch to focus on the boundary-sensitive region and fuse the output segmentation result map.

[0013] S3. Optimize the parameters of the student-teacher semi-supervised medical image segmentation model based on the total loss function, and use the trained medical image segmentation model to output the segmentation result of the medical image to be segmented.

[0014] Further, step S1 includes: acquiring labeled medical image data and unlabeled medical image data; performing image enhancement processing on the labeled medical image data and unlabeled medical image data to obtain enhanced medical image data; and constructing training samples for training the student-teacher semi-supervised medical image segmentation model based on the labeled medical image data, unlabeled medical image data, and enhanced medical image data.

[0015] Furthermore, the image enhancement process includes style perturbation enhancement, which is used to alter the style statistics of the medical image to improve the robustness of the student segmentation branch to appearance differences between labeled and unlabeled medical image data.

[0016] Furthermore, the student segmentation branch includes a first student segmentation sub-branch, a second student segmentation sub-branch, and a fusion branch; the first student segmentation sub-branch is used to predict the input medical image and generate a first student prediction result; the second student segmentation sub-branch is used to predict the input medical image and generate a second student prediction result; the fusion branch is used to generate a fused prediction result based on the first student prediction result and the second student prediction result, and finally obtains the segmentation result map.

[0017] Furthermore, the teacher branch includes an image encoder and a decoder of a pre-trained segmentation base model. The dynamic-static domain adaptive adapter is set in the image encoder to perform residual adaptation on the intermediate features extracted by the image encoder and input the adapted features into the subsequent encoding layer or decoder to generate the teacher segmentation prediction.

[0018] Furthermore, the dynamic-static domain adaptive adapter includes a meta-adjustment unit, a convolutional kernel mixing unit, and a feature refinement unit. The meta-adjustment unit is used to generate dynamic combination coefficients corresponding to multiple dynamic convolutional templates based on the input intermediate features. The convolutional kernel mixing unit includes a static convolutional branch and a dynamic convolutional branch. The dynamic convolutional branch is used to perform weighted combination of the convolution results of multiple dynamic convolutional templates based on the dynamic combination coefficients. The static convolutional branch is used to extract static structural features from the input intermediate features based on a shared convolutional kernel. The feature refinement unit is used to perform input feature weighted fusion and channel weighted refinement on the features fused by the static convolutional branch and the dynamic convolutional branch to obtain adapted output features.

[0019] Furthermore, the meta-regulation unit performs global context aggregation on the input intermediate features to obtain a context representation, and sequentially inputs the context representation into the first fully connected layer, the nonlinear activation layer, the second fully connected layer, and the normalization layer to generate dynamic combination coefficients corresponding to the multiple dynamic convolution templates. As an optional implementation, the nonlinear activation layer is a GELU activation layer, the normalization layer is a Softmax layer, and the sum of the dynamic combination coefficients corresponding to the multiple dynamic convolution templates is 1.

[0020] Furthermore, the plurality of dynamic convolution templates include a two-dimensional convolution template, a pointwise convolution template, a vertical convolution template, and a horizontal convolution template.

[0021] Furthermore, the convolution kernel hybrid unit fuses the static structural features output by the static convolution branch with the dynamic compensation features output by the dynamic convolution branch to obtain dynamic-static hybrid features; wherein, the dynamic combination coefficient is used to perform weighted combination of the convolution results of the multiple dynamic convolution templates.

[0022] Furthermore, the feature refinement unit performs weighted fusion of the input intermediate features and the dynamic-static hybrid features output by the convolutional kernel mixing unit to obtain fused features; the fused features are then subjected to global max pooling and global average pooling to obtain channel description information, and channel weights are generated based on the channel description information; the fused features are then weighted according to the channel weights to obtain channel-weighted features; and the channel-weighted features are then mapped using a multilayer perceptron to obtain adapted output features.

[0023] Furthermore, the teacher branch generates teacher segmentation predictions based on the adaptation features output by the dynamic-static domain adaptive adapter, and constrains the first student prediction results and the second student prediction results through knowledge distillation loss, so that the student segmentation branch learns the structural prior information provided by the teacher branch.

[0024] Further, the boundary band weighted supervision includes: converting the pixel-level annotations corresponding to the labeled medical image data into one-hot encoded annotations; generating a semantic edge map based on the differences in category annotations between adjacent pixels in the one-hot encoded annotations; performing dilation or max pooling on the semantic edge map to obtain the boundary band region. A pixel-level weight map is generated based on the boundary band region, and this pixel-level weight map is used as the pixel-by-pixel loss weight to weight the pixel-by-pixel segmentation error between the predicted probability map output by the student segmentation branch and the pixel-level ground truth annotations corresponding to the labeled medical image data, resulting in the boundary band weighted segmentation loss.

[0025] Furthermore, in the pixel-level weight map, the pixel weights located in the boundary band region are greater than the pixel weights located in the non-boundary band region, and the boundary band weighted segmentation loss includes weighted cross-entropy loss and Dice loss.

[0026] Further, the stable edge supervision includes: performing edge prediction on the features of the first student segmentation sub-branch, the second student segmentation sub-branch, and the fusion branch respectively through the edge prediction module to obtain a first edge prediction map, a second edge prediction map, and a fused edge prediction map; for labeled medical image data, generating a true edge map based on its pixel-level annotations, and constraining the first edge prediction map, the second edge prediction map, and the fused edge prediction map based on the true edge map; for a mixed sample obtained by mixing labeled medical image data and unlabeled medical image data, generating a pseudo edge map based on the fusion prediction result, and filtering reliable edge regions in the pseudo edge map according to confidence, and using the reliable edge regions to constrain the first edge prediction map and the second edge prediction map.

[0027] Furthermore, the edge loss includes binary cross-entropy loss and Dice loss, and the contribution of the edge loss to model training is adjusted during training by gradually increasing weights with training iterations.

[0028] Furthermore, the boundary gating discriminator generates a boundary gating map based on the prediction discrepancy, prediction uncertainty, and edge response between the first student prediction result and the second student prediction result, and modulates the pixel-level discrepancy map and uncertainty feature map based on the boundary gating map to obtain a fused prediction result, i.e., a segmentation result map.

[0029] Furthermore, the boundary gating discriminator includes a difference calculation unit, an uncertainty calculation unit, an edge embedding unit, and a gating fusion unit. The difference calculation unit is used to generate a pixel-level divergence map by first binarizing the first student prediction result and the second student prediction result, and then performing a pixel-by-pixel XOR operation. The uncertainty calculation unit is used to generate an uncertainty feature map based on the pixel-level entropy map of the first student prediction result and the second student prediction result. The edge embedding unit is used to generate an edge embedding feature based on the edge response of the first student prediction result and the second student prediction result. The gating fusion unit is used to generate a boundary gating map based on the edge embedding feature, and modulate the pixel-level divergence map and the uncertainty feature map using the boundary gating map. The two modulated result maps are then concatenated with the first student prediction result and the second student prediction result, and a fused prediction result, i.e., a segmentation result map, is generated through a mapping function.

[0030] Furthermore, the total loss function includes supervised segmentation loss, knowledge distillation loss, entropy minimization loss, mutual consistency loss, and edge supervision loss; the supervised segmentation loss is used to constrain the segmentation prediction results corresponding to labeled medical image data; the knowledge distillation loss is used to constrain the student segmentation branch to learn the teacher segmentation prediction generated by the teacher branch; the entropy minimization loss is used to improve the confidence of the prediction results for unlabeled medical image data; the mutual consistency loss is used to constrain the prediction consistency between the first student segmentation sub-branch and the second student segmentation sub-branch; and the edge supervision loss is used to constrain the edge prediction results.

[0031] Further, step S3 includes: performing end-to-end training of the student-teacher semi-supervised medical image segmentation model based on the total loss function; updating the parameters of the student segmentation branch and the dynamic-static domain adaptive adapter during the training process; and after the model training is completed, inputting the medical image to be segmented into the trained medical image segmentation model and outputting the target region segmentation result of the medical image to be segmented.

[0032] In another aspect, the present invention also provides a semi-supervised medical image segmentation system with dynamic and static adaptive boundary awareness, comprising the following modules:

[0033] The medical image data module is used to acquire medical image data and construct training samples, wherein the medical image data includes labeled medical image data and unlabeled medical image data.

[0034] The image segmentation module is used to build a student-teacher semi-supervised medical image segmentation model, which is trained using labeled and unlabeled medical image data.

[0035] The training and testing module optimizes the parameters of the student-teacher semi-supervised medical image segmentation model based on the total loss function, and outputs the segmentation results of the medical image to be segmented using the trained medical image segmentation model.

[0036] Compared with the prior art, the present invention has at least the following beneficial effects:

[0037] First, this invention introduces a dynamic-static domain adaptive adapter in the teacher branch based on a pre-trained segmentation base model. The static convolutional branch preserves the stable structural prior of the pre-trained segmentation base model, while the dynamic convolutional branch generates input-related dynamic compensation features based on the input medical image features. This enables the teacher branch to adapt to the appearance differences between different medical image samples while maintaining the structural prior.

[0038] Second, this invention uses a student-teacher semi-supervised learning framework, which uses the teacher predictions after dynamic-static adaptation as knowledge distillation constraints to guide student segmentation branch training. This helps to improve the reliability of supervision information for unlabeled medical images and alleviate the problem of unstable student segmentation branch training in scenarios with few labels.

[0039] Third, this invention improves the supervision weight of the boundary region by using boundary band weighted supervision, so that the model pays more attention to the edge region of organs, tissues or lesions during training, which can improve the problem of insufficient boundary offset constraint of conventional region-level loss.

[0040] Fourth, this invention constrains the edge prediction of student segmentation branches through stable edge supervision, and retains reliable pseudo-edge information from the mixed sample prediction results through confidence screening, which helps to reduce the impact of pseudo-label noise at the boundary of unlabeled samples on the training process.

[0041] Fifth, this invention uses a boundary gating discriminator to fuse prediction divergence, uncertainty information, and edge response information between two student segmentation sub-branches to generate a boundary gating map and guide the fusion prediction to focus on boundary-sensitive areas, thereby improving the consistency and stability of boundary area prediction. Attached Figure Description

[0042] Figure 1 A schematic diagram of the semi-supervised medical image segmentation method based on dynamic adaptation and boundary-aware supervision provided in an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the overall framework of the student-teacher semi-supervised medical image segmentation model provided in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the dynamic-static domain adaptive adapter structure provided in an embodiment of the present invention;

[0045] Figure 4 This is a comparison chart of the visualization segmentation results on the ACDC dataset. Detailed Implementation

[0046] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this invention.

[0047] It should be noted that, unless otherwise specified, the embodiments and technical features described in this invention can be combined with each other. The structures, proportions, connections, and module shapes shown in the accompanying drawings are for illustrative purposes only and do not constitute a limitation on the scope of protection of this invention.

[0048] Example 1

[0049] like Figures 1 to 3 As shown, this embodiment provides a semi-supervised medical image segmentation method based on dynamic adaptation and boundary-aware supervision. This method is designed for scenarios where a small number of labeled medical images and a large number of unlabeled medical images are used in the training process. By introducing a dynamic-static domain adaptive adapter in the teacher branch and a boundary-aware supervision mechanism in the student segmentation branch, the model can improve the reliability of supervision over boundary regions while utilizing the prior structure of the pre-trained segmentation model.

[0050] Step S1: Acquire medical image data and construct training samples

[0051] Medical image data is acquired, which may be cardiac images, endoscopic images, or other medical images; this invention does not limit the scope of the data. In this embodiment, the ACDC dataset or the colonoscopy polyp segmentation dataset may be used as training data; no limitation is imposed.

[0052] Before model training, style perturbation enhancement can be performed on labeled and unlabeled medical images to give the enhanced medical images different appearance statistical features, thereby improving the student segmentation branches' ability to adapt to changes in the appearance of medical images.

[0053] As an alternative implementation, hybrid training samples can be constructed based on labeled and unlabeled medical images. These hybrid training samples can be constructed using an uncertainty-guided hybrid strategy. The hybrid training samples enhance the adaptability of the student segmentation branch to different image styles and target morphologies, and provide hybrid sample edge constraints for subsequent stable edge supervision.

[0054] Step S2: Construct a student-teacher semi-supervised medical image segmentation model

[0055] A student-teacher semi-supervised medical image segmentation model is constructed, which includes a student segmentation branch and a teacher branch based on a pre-trained segmentation base model.

[0056] like Figure 2 As shown, the student segmentation branch includes a first student segmentation sub-branch, a second student segmentation sub-branch, and a merging branch. The first student segmentation sub-branch is denoted as... The second student's sub-branch is denoted as The first student segmentation sub-branch predicts the input medical image and generates the first student prediction result. The second student segmentation sub-branch predicts the input medical image and generates the second student prediction result. .

[0057] The first student segmentation sub-branch and the second student segmentation sub-branch employ different medical image segmentation network structures to obtain complementary prediction results.

[0058] The fusion branch is used to predict the results of the first student. Second Student Prediction Results Generate fusion prediction results The fusion prediction result can be used as the comprehensive prediction result during the training process, or it can be output as the final medical image segmentation result during the model inference stage.

[0059] The teacher branch is constructed based on a pre-trained segmentation base model. As an optional implementation, the pre-trained segmentation base model is a segmentation base model with an image encoder and decoder. The teacher branch is used to extract structural prior features from the input medical image and generate teacher segmentation predictions. Teacher segmentation prediction This is used to constrain the first and second student splitting branches through knowledge distillation, enabling the student splitting branches to learn the structural prior information provided by the teacher branch.

[0060] Step S3: Generate teacher predictions and perform knowledge distillation based on the dynamic-static domain adaptive adapter.

[0061] like Figure 2 and Figure 3 As shown, a dynamic-static domain adaptive adapter is introduced into the image encoder of the teacher branch. This dynamic-static domain adaptive adapter can be configured to correspond to multiple feature transformation layers in the image encoder, and is used to perform residual adaptation on the intermediate input features of each feature transformation layer.

[0062] For any feature transformation layer in the image encoder, the intermediate input features of that feature transformation layer are represented as follows: Dynamic-static domain adaptive adapter according to Generate residual adaptation features The residual adaptation features are then used to adjust the feature extraction process of the teacher branch image encoder, thereby achieving dynamic-static adaptation of the teacher branch.

[0063] like Figure 3 As shown, the dynamic-static domain adaptive adapter includes a meta-adjustment unit, a convolutional kernel mixing unit, and a feature refinement unit.

[0064] The meta-regulation unit is used to generate dynamic combination coefficients based on the input intermediate features. Specifically, the meta-regulation unit adjusts the input intermediate features... Global context aggregation is performed to obtain a context representation. As an optional implementation, global average pooling can be used to aggregate the intermediate input features to obtain channel-level context information.

[0065] The context representation is sequentially input into the first fully connected layer, the nonlinear activation layer, the second fully connected layer, and the normalization layer to generate dynamic combination coefficients corresponding to multiple dynamic convolution templates. The nonlinear activation layer can be a GELU activation layer or other nonlinear activation function layers; the normalization layer can be a Softmax layer.

[0066] As an optional implementation method, the dynamic combination coefficients are expressed as:

[0067]

[0068] in, Indicates global average pooling. This represents the first fully connected layer. Represents a non-linear activation function. This indicates the second fully connected layer. This represents the normalization function.

[0069] The dynamic combination coefficient can be expressed as:

[0070]

[0071] in, These correspond to combined weights of different dynamic convolution templates, and satisfy the following:

[0072]

[0073] Through the aforementioned meta-adjustment unit, input-related dynamic control signals can be generated based on the intermediate features of the current input medical image, enabling subsequent dynamic convolution branches to perform adaptive feature compensation for different input samples.

[0074] The convolutional kernel hybrid unit includes a static convolutional branch and a dynamic convolutional branch. The static convolutional branch uses a shared convolutional kernel to convolve the intermediate input features, preserving the stable structural priors in the pre-trained segmentation base model. The dynamic convolutional branch, based on the dynamic combination coefficients generated by the meta-adjustment unit, weights and combines the convolution results of multiple dynamic convolutional templates to generate input-related dynamic compensation features.

[0075] As an optional implementation, the output of the static convolution branch is represented as:

[0076]

[0077] in, This indicates a shared static convolution kernel.

[0078] The output of the dynamic convolution branch is represented as:

[0079]

[0080] in, Indicates the first Each dynamic convolution template corresponds to a convolution operation.

[0081] As an optional implementation, the plurality of dynamic convolution templates include a two-dimensional convolution template, a pointwise convolution template, a vertical convolution template, and a horizontal convolution template. For example, the dynamic convolution template may include... Convolution template, Convolution template, Convolution template and Convolution templates. Different convolution templates are used to capture structural information in medical images at different orientations, scales, or receptive fields.

[0082] Output the static convolution branch and dynamic convolution branch output By fusing the features, we obtain a dynamic-static hybrid feature:

[0083]

[0084] Through the aforementioned convolutional kernel hybrid unit, the static convolutional branch is used to preserve the stable structural features of the basic segmentation model, while the dynamic convolutional branch is used to generate adaptive compensation features based on the current input medical image. The two work together to enable the teacher branch to maintain both structural priors and input adaptability.

[0085] The feature refinement unit is used to perform input feature weighted fusion and channel weighted refinement on the features fused from the static and dynamic convolution branches to obtain adapted output features. Specifically, it processes the input intermediate features... Dynamic-static hybrid features of the output of the convolutional kernel hybrid unit Weighted fusion is performed to obtain the fusion features:

[0086]

[0087] in, These are the weighting coefficients.

[0088] Fusion features Global max pooling and global average pooling are performed separately to generate channel description information, and channel weights are generated based on the channel description information. As an optional implementation, the channel weights are represented as follows:

[0089]

[0090] in, Indicates global max pooling. This represents a multilayer perceptron. express function.

[0091] Based on the channel weights, the fused features are weighted by channels to obtain refined features:

[0092]

[0093] in, This indicates element-wise multiplication.

[0094] Furthermore, regarding the refined features Perform lightweight mapping to obtain adapted output features. The adapted output features, as residual adaptation features, are residually fused with the input features of the corresponding layer of the image encoder and then input into the subsequent feature transformation layer.

[0095] After processing by the dynamic-static domain adaptive adapter, the image encoder of the teacher branch generates adapted teacher features. These adapted teacher features are then input into the decoder of the teacher branch to generate teacher segmentation predictions. .

[0096] Based on the teacher segmentation prediction Prediction results for the first student Second Student Prediction Results Knowledge distillation constraints are applied. As an optional implementation, the knowledge distillation loss is expressed as:

[0097]

[0098] in, express divergence, This indicates the distillation temperature parameter.

[0099] Through knowledge distillation constraints, student segmentation branches can learn the structural prior information provided by teacher branches, enabling student segmentation branches to obtain more stable supervision signals in scenarios with few annotations.

[0100] Step S4: Train students to segment branches based on a boundary-aware supervision mechanism

[0101] like Figure 2 As shown, a boundary-aware supervision mechanism is introduced into the student segmentation branch. This mechanism includes boundary-band weighted supervision, stable edge supervision, and a boundary-gated discriminator. These three components train the student segmentation branch from three aspects: boundary region weighting, edge prediction constraints, and boundary-sensitive fusion, respectively.

[0102] 1. Boundary band weighted supervision

[0103] For labeled medical images, label them at the pixel level. Convert to one-hot encoding label. Let the one-hot encoding label be... Where C represents the number of categories, and H and W represent the image height and width, respectively.

[0104] A semantic edge map is generated based on the differences in category labeling between adjacent pixels. As an optional implementation, for pixel position p, the semantic edge map... It can be represented as:

[0105]

[0106] in, This represents the one-hot encoding label corresponding to class c. and These represent the offsets of adjacent pixels in the horizontal and vertical directions, respectively. ( (,0,1) indicates that the value is restricted to the range of 0 to 1.

[0107] The semantic edge map is subjected to dilation or max pooling to obtain the boundary band region:

[0108]

[0109] in, This indicates max pooling or an equivalent inflation process. Used to control the width of the boundary zone area.

[0110] Generate pixel-level weight maps based on the boundary zone regions:

[0111]

[0112] in, This is the boundary weight adjustment coefficient. When a pixel is located within the boundary band region, its weight is greater than that of pixels outside the boundary band region.

[0113] The boundary band weighted segmentation loss is calculated based on the pixel-level weight map. As an optional implementation, the boundary band weighted cross-entropy loss is expressed as:

[0114]

[0115] in, This represents pixel-level cross-entropy loss. This represents the predicted result of the student's branch split.

[0116] Furthermore, by combining the Dice loss with the boundary-band weighted cross-entropy loss, we obtain the boundary-band weighted supervision loss:

[0117]

[0118] The boundary-band weighted supervision loss can be applied to the first student's prediction result, the second student's prediction result, and the fused prediction result, respectively, to enhance the student's ability to learn the boundary region through segmentation branches.

[0119] 2. Stable edge supervision

[0120] To further constrain the edge prediction results of the student segmentation branches, this embodiment includes an edge prediction module. This module performs edge prediction on the features of the first student segmentation sub-branch, the second student segmentation sub-branch, and the fusion branch, respectively, to obtain corresponding edge prediction maps.

[0121] For labeled medical images, a ground truth edge map is generated based on the pixel-level annotations. This ground truth edge map can be obtained using the semantic edge map generation method described above. The ground truth edge map is then used as the edge supervision target to constrain the edge prediction maps corresponding to the first student segmentation sub-branch, the second student segmentation sub-branch, and the fusion branch.

[0122] As an optional implementation, the edge loss of a single edge prediction graph includes binary cross-entropy loss and Dice loss:

[0123]

[0124] in, Represents the edge prediction map. Indicates the target of peripheral supervision. This represents the binary cross-entropy loss.

[0125] For labeled medical images, the supervised edge loss can be expressed as:

[0126]

[0127] in, , , These represent the edge prediction maps of the labeled medical image corresponding to the first student segmentation sub-branch, the second student segmentation sub-branch, and the fusion branch, respectively. Represents the true edge map.

[0128] For a hybrid sample obtained by mixing labeled and unlabeled medical images, a pseudo-edge map is generated based on the fusion prediction results, and reliable edge regions in the pseudo-edge map are selected based on confidence levels. As an optional implementation, let the fusion prediction probability map of the hybrid sample be... Then the pseudo-edge supervision target can be represented as:

[0129]

[0130] Among them, E This indicates the edge generation operation. [ ] indicates an indicator function. This represents the confidence threshold.

[0131] Based on the edge prediction map corresponding to the mixed samples constrained by the filtered pseudo-edge map, the edge loss of the mixed samples is obtained:

[0132]

[0133] To avoid the unstable pseudo-edge information having an excessive impact on the model in the early stages of training, the influence of edge loss on model training can be adjusted by gradually increasing the weights during training iterations. The final edge loss can be expressed as:

[0134]

[0135] in, This represents the weight function that changes with training iterations.

[0136] By using stable edge supervision, this invention can utilize both real edge maps and reliable pseudo-edge maps to constrain the edge prediction results of student segmentation branches, thereby reducing the impact of pseudo-label noise on unlabeled sample boundaries on model training.

[0137] 3. Boundary Gating Discriminator

[0138] The boundary gating discriminator is used to generate a boundary gating map based on the prediction difference, prediction uncertainty and edge response between the first student prediction result and the second student prediction result, and to use the boundary gating map to guide the fusion branch to focus on the boundary sensitive area.

[0139] The boundary gating discriminator includes a difference calculation unit, an uncertainty calculation unit, an edge embedding unit, and a gating fusion unit.

[0140] The difference calculation unit is used to generate a pixel-level divergence map based on the binarization results of the first student's prediction and the second student's prediction. As an optional implementation, the pixel-level divergence map is represented as follows:

[0141]

[0142] in, This indicates a binarization operation. This represents the XOR operation. A pixel-level divergence map is used to represent regions of inconsistency in the pixel-level prediction results between two student splitting sub-branches.

[0143] The uncertainty calculation unit is used to generate an uncertainty feature map based on the pixel-level entropy maps of the first student's prediction and the second student's prediction. For any prediction result, its pixel-level entropy can be expressed as:

[0144]

[0145] in, Indicates pixel position Belongs to the The predicted probability of the class. Entropy diagrams are obtained based on the prediction results of the first student and the second student, respectively. and And both are used as inputs for uncertainty information.

[0146] The edge embedding unit is used to generate edge embedding features based on the edge responses of the first student's prediction and the second student's prediction. Specifically, edge generation operations are performed on the first student's prediction and the second student's prediction respectively to obtain the edge responses. and The edge responses are then spliced ​​together and input into the edge embedding unit.

[0147] The gating fusion unit generates a boundary gating map based on edge embedding features. As an optional implementation, the boundary gating map is represented as follows:

[0148]

[0149] in, Represents edge embedding mapping, Let G represent the gating mapping function, and let G represent the boundary gating graph with values ​​ranging from 0 to 1.

[0150] Consistent region features are constructed based on the pixel-level divergence map, and low-uncertainty region features are constructed based on the pixel-level entropy map. The consistent region features and low-uncertainty region features are then modulated using a boundary gating map. As an optional implementation, the modulation process is expressed as follows:

[0151]

[0152]

[0153] in, This represents the feature obtained by encoding consistent region information. This represents the features obtained by encoding information from low-uncertainty regions. and This represents the gate strength adjustment coefficient.

[0154] The features modulated by boundary gating are fused with the first student's prediction and the second student's prediction to generate a fused prediction result, i.e., a segmentation result image. :

[0155]

[0156] in, ( ) represents the fusion mapping function.

[0157] By using a boundary gating discriminator, this invention can jointly model the divergence region, uncertain region, and edge response region between two student segmentation sub-branches, making the fusion prediction results more focused on boundary-sensitive regions, thereby improving the stability of medical image boundary segmentation.

[0158] Step S5: Optimize model parameters and output medical image segmentation results

[0159] Based on the aforementioned student-teacher semi-supervised medical image segmentation model, dynamic-static domain adaptive adapter, and boundary-aware supervision mechanism, a total loss function is constructed and the model parameters are jointly optimized.

[0160] As an optional implementation, the total loss function includes supervised segmentation loss, knowledge distillation loss, entropy minimization loss, mutual consistency loss, and edge supervision loss:

[0161]

[0162] in, Indicates the loss of supervision and division. This represents the loss from knowledge distillation. This represents the loss by minimizing entropy. Indicates mutual consistency loss, This indicates edge supervision loss.

[0163] The supervised segmentation loss is used to constrain the segmentation prediction results corresponding to the labeled medical image data. As an optional implementation, the supervised segmentation loss can be applied separately to the first student prediction result, the second student prediction result, and the fused prediction result:

[0164]

[0165] in, This indicates pixel-level annotations corresponding to labeled medical images.

[0166] The entropy minimization loss is used to improve the confidence of prediction results for unlabeled medical images. As an optional implementation, the entropy minimization loss can be expressed as:

[0167]

[0168] in, The entropy function represents the probability graph of the prediction.

[0169] The mutual consistency loss is used to constrain the prediction consistency between the first student split sub-branch and the second student split sub-branch. As an optional implementation, the mutual consistency loss can be expressed as:

[0170]

[0171] in, This indicates that the gradient operation is stopped.

[0172] Based on the aforementioned total loss function, the student-teacher semi-supervised medical image segmentation model is trained end-to-end using gradient backpropagation. During training, the parameters of the first student segmentation sub-branch, the second student segmentation sub-branch, the fusion branch, the edge prediction module, the boundary gating discriminator, and the dynamic-static domain adaptive adapter can be updated. As an optional implementation, the main parameters of the pre-trained segmentation base model can be kept frozen or partially updated to preserve its structural priors and reduce training costs.

[0173] During the inference phase, the medical image to be segmented is input into the trained medical image segmentation model. The student segmentation branch generates a first student prediction result and a second student prediction result, and the fusion branch generates a fused prediction result. The fused prediction result is output as the target region segmentation result of the medical image to be segmented.

[0174] Through the above steps, the method provided in this embodiment can train a medical image segmentation model under conditions of a small number of labeled medical images and a large number of unlabeled medical images. It uses a dynamic-static domain adaptive adapter to alleviate the problem of unstable feature transfer caused by differences in the appearance of medical images, and uses a boundary-aware supervision mechanism to improve the reliability of boundary region supervision, thereby improving the accuracy of medical image segmentation results and the stability of boundary localization.

[0175] This embodiment also provides a semi-supervised medical image segmentation system with dynamic and static adaptive boundary awareness, used to implement the aforementioned semi-supervised medical image segmentation method, including a medical image data module, an image segmentation module, and a training and testing module:

[0176] The medical image data module is used to acquire medical image data and construct training samples. The medical image data includes labeled medical image data and unlabeled medical image data.

[0177] The image segmentation module is used to construct a student-teacher semi-supervised medical image segmentation model, which is trained using labeled and unlabeled medical image data.

[0178] The training and testing module optimizes the parameters of the student-teacher semi-supervised medical image segmentation model based on the total loss function, and outputs the segmentation results of the medical image to be segmented using the trained medical image segmentation model.

[0179] To verify the effectiveness of the semi-supervised medical image segmentation method based on dynamic adaptation and boundary-aware supervision described in this embodiment, the method was validated. As an exemplary validation method, experiments were conducted using the ACDC segmentation dataset and the colonoscopy polyp segmentation image dataset to evaluate the applicability of the method in different medical image types and target region segmentation tasks. The ACDC segmentation data was used to evaluate the segmentation performance of the method on organ structures, and the colonoscopy polyp segmentation image data was used to evaluate the segmentation performance on lesion regions. The aforementioned datasets and annotation ratios are only used to illustrate the technical effects of the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0180] This embodiment uses the following evaluation metrics: (1) Jaccard index, which calculates the union and intersection (IoU) of the prediction and the ground reality; (2) Dice coefficient, a set similarity measure function; (3) 95th percentile Hausdorff distance (95HD), which measures the maximum distance between the predicted boundary and the ground reality; and (4) Average surface distance (ASD), which calculates the average distance between corresponding points on the predicted surface and the ground reality. The Dice coefficient and IoU are used to evaluate the degree of overlap between the predicted segmented region and the actual labeled region. Higher Dice coefficients and IoU indicate more accurate segmentation. HD95 and ASD are used to evaluate the distance error between the predicted boundary and the actual boundary. Lower HD95 and ASD indicate more stable boundary localization. For the colonoscopy polyp segmentation dataset, Dice coefficient, IoU, and HD95 are used as evaluation metrics.

[0181] To verify the overall segmentation performance of the method in this embodiment under conditions of limited labeled data, the method in this embodiment was compared with several semi-supervised medical image segmentation methods on the ACDC dataset with 10% labeled samples. These comparison methods included the Mean Teacher Method (MT), Mutual Correction Framework (MCF), Multi-Attention Cross-Distillation Method (CDMA), Correlation-Aware Mutual Learning Method (CAML), and Knowledge-Guided Learning and Analysis (SAM) Method (KnowSAM). The results are shown in Table 1.

[0182] Table 1. Quantitative evaluation of the method of the present invention under the condition that 10% of the samples in the ACDC dataset are labeled.

[0183]

[0184] As shown in Table 1, under the condition that 10% of the samples in the ACDC dataset are labeled, the method in this embodiment achieves superior results in all four metrics: Dice coefficient, IoU, HD95, and ASD. Compared with the KnowSAM method based on knowledge distillation of the segmentation baseline model, the method in this embodiment improves the Dice coefficient from 89.29% to 90.23%, the IoU from 81.22% to 82.65%, reduces HD95 from 1.29 to 1.19, and reduces ASD from 0.32 to 0.28. These results indicate that the method in this embodiment not only improves the overlap between the predicted segmented region and the actual labeled region but also reduces the boundary distance error, thereby improving the segmentation accuracy and boundary localization stability of medical images under conditions of limited labeled data. Figure 4As can be seen, compared with existing semi-supervised medical image segmentation methods, the method in this embodiment can obtain segmentation regions that are closer to the actual annotations, even when the boundaries of the heart structure are weak, the local contrast is low, or the target contour is discontinuous. Especially in the target edge region, the segmentation results of this embodiment have better contour continuity and structural integrity, and can reduce local missed segmentation, boundary offset, and structural breakage.

[0185] To further verify the generalization ability of the method in this embodiment in different medical image types and different target region segmentation tasks, the method in this embodiment was applied to multiple colonoscopy polyp segmentation datasets and compared with various semi-supervised medical image segmentation methods. The results are shown in Table 2.

[0186] Table 2. Quantitative evaluation of the method of the present invention in a colonoscopy polyp segmentation dataset with 10% labeled samples.

[0187]

[0188] As shown in Table 2, on multiple colonoscopy polyp segmentation datasets, the method of this embodiment improved the Dice coefficient from 78.40% to 80.76% and the IoU from 71.93% to 74.49% on the CVC-ClinicDB dataset, while decreasing the HD95 from 4.11 to 3.86; on the ETIS dataset, the Dice coefficient improved from 49.18% to 56.10%, the IoU from 42.55% to 48.93%, and the HD95 from 4.47 to 4.26; and on the Kvasir dataset, the Dice coefficient improved from 85.82% to 87.24% and the IoU from 79.01% to 80.58%. These results demonstrate that the method of this embodiment is not only applicable to organ structure segmentation in cardiac magnetic resonance imaging but also to lesion region segmentation in colonoscopy images, exhibiting good applicability across medical image types.

Claims

1. A semi-supervised segmentation method for medical images with dynamic and static adaptive boundary awareness, characterized in that, Includes the following steps: S1. Acquire medical image data and construct training samples, wherein the medical image data includes labeled medical image data and unlabeled medical image data; S2. Construct a student-teacher semi-supervised medical image segmentation model and train it using labeled and unlabeled medical image data. S3. Optimize the parameters of the student-teacher semi-supervised medical image segmentation model based on the total loss function, and use the trained medical image segmentation model to output the segmentation result of the medical image to be segmented.

2. The semi-supervised segmentation method for medical images with dynamic and static adaptive boundary awareness as described in claim 1, characterized in that, Step S1 includes: acquiring labeled medical image data and unlabeled medical image data; performing image enhancement processing on the labeled medical image data and unlabeled medical image data to obtain enhanced medical image data; and constructing training samples for training the student-teacher semi-supervised medical image segmentation model based on the labeled medical image data, unlabeled medical image data, and enhanced medical image data. The image enhancement process includes style perturbation enhancement.

3. The semi-supervised segmentation method for medical images with dynamic and static adaptive boundary awareness as described in claim 2, characterized in that, The student-teacher semi-supervised medical image segmentation model includes a student segmentation branch and a teacher branch based on the segmentation base model, which are implemented as follows: S21. In the image encoder of the teacher branch, a dynamic-static domain adaptive adapter is introduced. The intermediate features of the input medical image are dynamically-statically adapted by the dynamic-static domain adaptive adapter to obtain adapted teacher features. Based on the adapted teacher features, a teacher segmentation prediction is generated for knowledge distillation of the student segmentation branch. S22. Train student segmentation branches based on boundary-aware supervision mechanism. The boundary-aware supervision mechanism includes boundary band weighted supervision, stable edge supervision and boundary gating discriminator. Boundary band weighted supervision is used to enhance the supervision weight of the boundary region, stable edge supervision is used to constrain the edge prediction results, and boundary gating discriminator is used to guide student segmentation branches to focus on boundary-sensitive regions and fuse the output segmentation result map.

4. The semi-supervised segmentation method for medical images with dynamic and static adaptive boundary awareness according to claim 3, characterized in that, The student segmentation branch includes a first student segmentation sub-branch, a second student segmentation sub-branch, and a fusion branch; the first student segmentation sub-branch is used to predict the input medical image and generate a first student prediction result; the second student segmentation sub-branch is used to predict the input medical image and generate a second student prediction result; the fusion branch is used to generate a fused prediction result based on the first student prediction result and the second student prediction result. The teacher branch includes an image encoder and a decoder of a pre-trained segmentation base model. A dynamic-static domain adaptive adapter is set in the image encoder to perform residual adaptation on the intermediate features extracted by the image encoder, and input the adapted features into the subsequent encoding layer or decoder to generate the teacher segmentation prediction.

5. The semi-supervised segmentation method for medical images with dynamic and static adaptive boundary awareness according to claim 4, characterized in that, The dynamic-static domain adaptive adapter includes a meta-adjustment unit, a kernel mixing unit, and a feature refinement unit. The meta-adjustment unit is used to generate dynamic combination coefficients corresponding to multiple dynamic convolution templates based on the input intermediate features. The kernel mixing unit includes a static convolution branch and a dynamic convolution branch. The dynamic convolution branch performs a weighted combination of the convolution results of multiple dynamic convolution templates based on the dynamic combination coefficients. The static convolution branch extracts static structural features from the input intermediate features based on the shared convolution kernel. The feature refinement unit is used to perform input feature weighted fusion and channel weighted refinement on the features fused from the static convolutional branch and the dynamic convolutional branch to obtain adapted output features. The meta-regulation unit performs global context aggregation on the input intermediate features to obtain context representations, and then sequentially inputs the context representations into the first fully connected layer, the nonlinear activation layer, the second fully connected layer, and the normalization layer to generate dynamic combination coefficients corresponding to multiple dynamic convolution templates, which are used to perform weighted combination of the convolution results of multiple dynamic convolution templates. The nonlinear activation layer is a GELU activation layer, the normalization layer is a Softmax layer, and the sum of the dynamic combination coefficients corresponding to multiple dynamic convolution templates is 1. The multiple dynamic convolution templates include two-dimensional convolution templates, pointwise convolution templates, vertical convolution templates, and horizontal convolution templates; The convolutional kernel hybrid unit fuses the static structural features output by the static convolutional branch with the dynamic compensation features output by the dynamic convolutional branch to obtain dynamic-static hybrid features. The feature refinement unit performs weighted fusion of the input intermediate features and the dynamic-static hybrid features output by the convolutional kernel mixing unit to obtain fused features; Global max pooling and global average pooling are performed on the fused features to obtain channel description information, and channel weights are generated based on the channel description information. The fused features are then weighted according to the channel weights to obtain channel-weighted features. The channel-weighted features are then mapped using a multilayer perceptron to obtain the adapted output features.

6. The semi-supervised segmentation method for medical images with dynamic and static adaptive boundary awareness according to claim 5, characterized in that, The teacher branch generates teacher segmentation predictions based on the adaptation features output by the dynamic-static domain adaptive adapter, and constrains the prediction results of the first student and the second student through knowledge distillation loss, so that the student segmentation branch learns the structural prior information provided by the teacher branch.

7. The semi-supervised segmentation method for medical images with dynamic and static adaptive boundary awareness according to claim 5, characterized in that, The boundary band weighted supervision includes: converting pixel-level annotations corresponding to labeled medical image data into one-hot encoded annotations; generating a semantic edge map based on the differences in category annotations between adjacent pixels in the one-hot encoded annotations; performing dilation or max pooling on the semantic edge map to obtain the boundary band region; generating a pixel-level weight map based on the boundary band region; and using the pixel-level weight map as the pixel-by-pixel loss weight to weight the pixel-by-pixel segmentation error between the student segmentation branch's prediction result for labeled medical image data and the corresponding pixel-level annotation, thereby obtaining the boundary band weighted segmentation loss and enhancing the student segmentation branch's learning of the boundary region. In the pixel-level weight map, the pixel weights located in the boundary band region are greater than the pixel weights located in the non-boundary band region. The boundary band weighted segmentation loss includes weighted cross-entropy loss and Dice loss.

8. The semi-supervised segmentation method for medical images with dynamic and static adaptive boundary awareness according to claim 7, characterized in that, The stable edge supervision includes: performing edge prediction on the features of the first student segmentation sub-branch, the second student segmentation sub-branch, and the fusion branch through the edge prediction module to obtain a first edge prediction map, a second edge prediction map, and a fused edge prediction map; for labeled medical image data, generating a true edge map based on pixel-level annotations, and constraining the first edge prediction map, the second edge prediction map, and the fused edge prediction map based on the true edge map; for mixed samples obtained by mixing labeled and unlabeled medical image data, generating a pseudo edge map based on the fusion prediction result, and filtering reliable edge regions in the pseudo edge map based on confidence, and using the reliable edge regions to constrain the first edge prediction map and the second edge prediction map; The stable edge supervision loss includes binary cross-entropy loss and Dice loss, and the contribution of the edge loss to model training is adjusted by gradually increasing the weights during training iterations.

9. The semi-supervised segmentation method for medical images with dynamic and static adaptive boundary awareness according to claim 8, characterized in that, The boundary gating discriminator generates a boundary gating map based on the prediction discrepancy, prediction uncertainty, and edge response between the first student prediction result and the second student prediction result. It then performs gating modulation on the pixel-level discrepancy map and the uncertainty feature map based on the boundary gating map and fuses them to obtain a fused prediction result. The boundary gating discriminator includes a difference calculation unit, an uncertainty calculation unit, an edge embedding unit, and a gating fusion unit. The difference calculation unit is used to generate a pixel-level divergence map by first binarizing the first student prediction result and the second student prediction result, and then performing a pixel-by-pixel XOR operation. The uncertainty calculation unit is used to generate an uncertainty feature map based on the pixel-level entropy map of the first student prediction result and the second student prediction result. The edge embedding unit is used to generate edge embedding features based on the edge responses of the first student prediction result and the second student prediction result. The gated fusion unit generates a boundary gate map based on edge embedding features through a mapping function, and modulates a pixel-level divergence map and an uncertainty feature map using the boundary gate map. The two modulated result maps are then concatenated with the first student prediction result and the second student prediction result, and a fusion prediction result, i.e., a segmentation result map, is generated through a mapping function. The total loss function includes supervised segmentation loss, knowledge distillation loss, entropy minimization loss, mutual consistency loss, and edge supervision loss. The supervised segmentation loss is used to constrain the segmentation prediction results corresponding to labeled medical image data. The knowledge distillation loss is used to constrain the student segmentation branch to learn the teacher segmentation prediction generated by the teacher branch. The entropy minimization loss is used to improve the confidence of the prediction results for unlabeled medical image data. The mutual consistency loss is used to constrain the prediction consistency between the first student segmentation sub-branch and the second student segmentation sub-branch. The edge supervision loss is used to constrain the edge prediction results.

10. A semi-supervised medical image segmentation system with dynamic and static adaptive boundary awareness, used to implement the semi-supervised medical image segmentation method according to any one of claims 1 to 9, characterized in that, Includes the following modules: A medical image data module is used to acquire medical image data and construct training samples, wherein the medical image data includes labeled medical image data and unlabeled medical image data; The image segmentation module is used to build a student-teacher semi-supervised medical image segmentation model, which is trained using labeled and unlabeled medical image data. The training and testing module optimizes the parameters of the student-teacher semi-supervised medical image segmentation model based on the total loss function, and outputs the segmentation results of the medical image to be segmented using the trained medical image segmentation model.

Images