A medical image segmentation method and device based on causal intervention and text guidance

Abstract

The application discloses a medical image segmentation method based on causal intervention and text guidance, which comprises the following steps: acquiring multi-source medical images, dividing them into a labeled training set and an unlabeled target domain test set, and pre-training a first model for the training set; and when testing, self-adapting, comprising: decomposing visual features into domain-invariant causal features and domain-specific false features through a mask network; extracting text features of anatomical structures, injecting the text features into the causal features and the false features respectively through dynamic convolution to obtain corresponding prediction results, realizing semantic conditioning of text guidance; generating pseudo labels based on uncertainty and feature prototype consistency, calculating a forward segmentation loss between the prediction results and the pseudo labels, constructing an adversarial loss based on the prediction results of the false features, and performing adversarial optimization on the mask network and the false feature prediction, and then performing statistical alignment optimization through batch normalization, and completing self-adaptation during testing. The application has high segmentation accuracy and robustness under the condition of dynamic domain shift.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and specifically to a medical image segmentation method and device based on causal intervention and text guidance. Background Technology

[0002] Deep learning has demonstrated excellent performance in the field of image semantic segmentation. However, in actual clinical deployment, in addition to the distribution shift caused by differences in imaging equipment, acquisition protocols and patient groups between different medical centers, medical image segmentation also generally faces problems such as blurred lesion boundaries, small target scale and large morphological changes, imbalance between foreground and background categories, difficulty in obtaining labeled data, and interference from image noise and artifacts, which seriously affect the generalization ability and segmentation accuracy of the model.

[0003] Temporal Test Adaptation (TTA) methods can adjust the model using unlabeled target data without requiring a large amount of training data in the target domain beforehand. Building on this, Continuous Temporal Test Adaptation (CTTA) further expands the adaptation scenario, requiring not only adaptation to a single new domain but also continuous model performance stability under constantly changing data distributions. Existing CTTA methods can be broadly categorized into two types: parameter fine-tuning and lightweight adaptation based on frozen source models. Parameter fine-tuning methods lack true labels during long-term inference, heavily relying on unreliable self-supervised signals, easily leading to severe error accumulation and catastrophic forgetting of source domain knowledge, resulting in poor stability. While frozen source model-based methods reduce the risk of forgetting to some extent, in medical image segmentation scenarios, they often focus on aligning low-level style features, lacking in-depth modeling of high-level semantic information for the segmentation task. This makes it difficult to ensure the consistency of lesion structure, organ contours, and task-related semantic information during the adaptation process, leading to problems such as missed segments, misclassifications, or inaccurate boundary localization.

[0004] To address the aforementioned issues, existing research has attempted to combine style transfer with cue learning to improve the performance of CTTA. For example, invention patent CN122290119A discloses a test-temporal adaptive medical image segmentation method and device based on cue learning and style transfer. This method uses a hybrid style transfer module that combines Fourier transform and batch normalization statistics to generate images with source domain style while preserving the semantic structure of the target domain, thus aiding domain alignment. Simultaneously, a semantically aware cue learning module provides image-level learnable cues to guide the model to adapt to the new domain without modifying the backbone network. Cue learning is conducted under the guidance of reliable pseudo-labels and batch normalization statistical alignment.

[0005] However, this approach primarily operates at the image appearance level and shallow network features. While it can alleviate low-level distribution differences, it struggles to explicitly model the cross-domain invariance of high-level anatomical semantics such as organs, lesions, and boundaries. From a causal inference perspective, images from different medical domains exhibit significant differences in low-level imaging features such as texture, noise patterns, and contrast. These domain-specific appearance styles constitute spurious factors that interfere with model decisions, making the model highly susceptible to being misled by false style features. Furthermore, in the absence of true labels for the target domain, relying solely on the alignment and cue injection of single-modal visual features makes it difficult to provide an absolutely stable semantic reference benchmark across domains, and cannot guarantee the consistency of task-related semantics such as lesions or organ structures during continuous adaptation.

[0006] To address the issue of misleading segmentation, some studies have improved segmentation robustness from the perspective of causal intervention. For example, patent CN119693642A discloses a medical image segmentation method based on causal intervention, which reduces the interference of confounding factors on segmentation decisions by separating visual features into causal features and confounding features. However, this approach is designed for static training scenarios and does not address adaptive scenarios in the testing domain. Therefore, it still cannot solve problems such as online model adaptation, error accumulation, and catastrophic forgetting when the target domain distribution is constantly changing and there are no labels, and it cannot guarantee the stability of segmentation decisions under cross-domain conditions. In addition, some studies have introduced text-guided segmentation. For example, patent CN122618227A uses the CLIP text encoder to extract textual description features of medical image categories, fuses the text features with the visual context to obtain class-conditional biases, and superimposes them onto the visual prediction results to achieve text-guided medical image segmentation. Similarly, this approach does not address the direct application of text guidance to visual features or prediction results in test-time adaptive scenarios. It is not decoupled from causal features, cannot distinguish between causal features and domain-specific spurious features, and is difficult to avoid interference from domain-specific styles on text semantic guidance in test-time adaptive scenarios. Summary of the Invention

[0007] In scenarios involving continuous domain offset across medical centers and devices, how can we effectively reduce the interference of domain-specific appearance information on segmentation decisions, maintain the consistency of task-related semantics during continuous adaptation, improve the reliability of pseudo-label supervision signals, and avoid error accumulation and catastrophic forgetting, thereby enhancing the cross-domain generalization ability and segmentation accuracy of medical image segmentation models, without the presence of true labels in the target domain? This invention provides a medical image segmentation method and device based on causal intervention and text guidance. The feature decoupling mechanism based on causal intervention explicitly identifies and eliminates domain-specific false confusion factors, prompting the model to focus on domain-invariant structured causal representations. Independent dynamic convolutions are performed on the decomposed causal and false features to obtain causal feature prediction results and false feature prediction results, allowing cross-domain stable text semantic priors to act on the two feature branches separately rather than simply being superimposed on visual predictions. Simultaneously, a cross-domain absolutely stable multimodal clinical text prior, unaffected by the underlying appearance of the image, is introduced as a conditional constraint, driving pixel-level predictions to align with the true category semantics. This provides an independent objective reference for continuous adaptation, ensuring that anatomical semantic information is retained in the causal features while domain-specific information is pushed into the false features. Thus, during the adaptive process in tests without target domain labels and with continuously changing distributions, a synergistic closed loop of causal decoupling, semantic anchoring, false label optimization, and adversarial purification is formed, improving segmentation stability and cross-domain generalization ability.

[0008] To achieve the above-mentioned objectives, an embodiment provides a medical image segmentation method based on causal intervention and text guidance, comprising the following steps: Acquire multiple medical images and divide them into a labeled training set and an unlabeled target domain test dataset; The first model is pre-trained using a labeled training set to obtain the pre-trained first model. The first model is tested and adapted using an unlabeled target domain test dataset. This adaptation includes: decomposing the visual features extracted by the first model into domain-invariant causal features and domain-specific spurious features using a masking network to achieve dynamic causal intervention; extracting text features from a pre-trained text encoder based on preset anatomical structures, and injecting these text features into the domain-invariant causal features and domain-specific spurious features respectively through dynamic convolution to obtain causal feature prediction results and spurious feature prediction results, thus achieving semantic conditionalization of text guidance; generating refined spurious labels based on prediction uncertainty and feature prototype consistency; calculating a positive segmentation loss based on the causal feature prediction results and refined spurious labels; constructing an adversarial loss based on the spurious feature prediction results; and performing adversarial optimization on the masking network and spurious feature prediction, where the spurious feature prediction branch minimizes the segmentation loss, and the masking network maximizes the segmentation loss, thereby prompting the masking network to reduce the information content related to the segmentation task in the spurious features; and optimizing the masking network and dynamic convolutional network through batch normalized statistical alignment loss to complete the test-time adaptation. The target medical image is segmented based on the first model adapted during testing, and the segmentation results are output.

[0009] In one embodiment, the medical images are derived from one or more medical images from different medical devices, different medical institutions, different imaging protocols, or different patient groups.

[0010] In one embodiment, the backbone network of the first model is a convolutional neural network, a Transformer network, or a hybrid architecture combining both. During the adaptive process of the first model during testing, the backbone network weights remain frozen, and only the lightweight module parameters related to the dynamic causal intervention and the semantic conditionalization of the text guidance are updated.

[0011] In one embodiment, the decomposition of the visual features extracted by the first model through a masking network into domain-invariant causal features and domain-specific spurious features includes: Extract the intermediate visual features from the penultimate layer of the first model as the initial visual representation; The initial visual representation is processed using a lightweight masking network to generate causal soft masks and spurious soft masks. The initial visual representation is multiplied element-wise with the causal soft mask and the spurious soft mask respectively to obtain domain-invariant causal features and domain-specific spurious features.

[0012] In one embodiment, the domain-invariant causal feature is anatomical structural information that remains stable under at least one of different medical centers, different medical devices, and different imaging protocols; Domain-specific spurious features are at least one domain-specific appearance information caused by domain offset, such as texture, noise, lighting changes, or contrast.

[0013] In one embodiment, the text features of the preset anatomical structure refer to the text features corresponding to the standardized text description constructed for the anatomical target to be segmented.

[0014] In one embodiment, text features are injected into domain-invariant causal features and domain-specific spurious features respectively through dynamic convolution, including: Text features are mapped to dynamic convolution kernel weights and biases using a lightweight projection network. The domain-invariant causal features and domain-specific spurious features are upsampled to the original image resolution, respectively. Dynamic convolution operations are then performed on the upsampled domain-invariant causal features and domain-specific spurious features using the convolution kernel weights and biases to obtain causal feature prediction results and spurious feature prediction results.

[0015] Furthermore, the pre-trained text encoder is a frozen BiomedCLIP text encoder.

[0016] In one embodiment, the process of generating refined pseudo-labels based on prediction uncertainty and feature prototype consistency includes: The first pre-trained model is used to predict pixel-level probability maps, and initial pseudo-labels are generated by setting a probability threshold. Monte Carlo Dropout is used to calculate prediction entropy and construct an uncertainty mask; Extract the deep features from the penultimate layer of the first model, and calculate the foreground feature prototype and background feature prototype respectively; Calculate the feature distance from the deep features of each pixel to the foreground feature prototype and the background feature prototype; Based on the uncertainty mask and the feature distance, a reliability mask is constructed, and reliable pixels are selected to generate refined pseudo-labels; wherein the reliability mask is represented as: ,in, For pixels Predicted entropy, For uncertainty threshold, and These are the Euclidean distances to the foreground feature prototype and the background feature prototype, respectively. These are the initial pseudo-tags.

[0017] Furthermore, the foreground feature prototype and background feature prototype are calculated in the following ways: , , in, Foreground feature prototype, As a prototype of background features, For pixels The deep feature vector, Pixel-level prediction probability, For initial pseudo-tags, For indicator functions, and These are foreground and background indicator variables, respectively.

[0018] Furthermore, the total loss established by optimizing the mask network and the dynamic convolutional network using batch normalized statistical alignment loss is expressed as: , in, For pixel-level segmentation loss, To combat the losses, To batch normalized statistical alignment loss, and These are the corresponding weighting coefficients.

[0019] Furthermore, Pixel-level segmentation loss for predicting branches of causal features; To combat the losses, and These are the causal feature prediction branch and the spurious feature prediction branch, respectively. and These are causal characteristics and false characteristics, respectively. For text features, To refine pseudo-labels, This is the pixel-level cross-entropy loss function.

[0020] In one embodiment, the batch normalized statistical alignment loss is calculated by: calculating the alignment statistics via momentum update. , ; The batch normalized statistical alignment loss is: ,in, , and , The source domain and the target domain are respectively in the th... The mean and standard deviation of the batch normalized statistics for each layer. The momentum coefficient, This represents the total number of batch normalization layers in the mask network.

[0021] The present invention also provides a medical image segmentation device based on causal intervention and text guidance, including a memory and a processor. The memory is used to store a computer program, and the processor is used to implement the medical image segmentation method based on causal intervention and text guidance when the computer program is executed.

[0022] The present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, performs the aforementioned medical image segmentation method based on causal intervention and text guidance.

[0023] Compared with the prior art, the beneficial effects of the present invention include at least the following: (1) Unlike existing solutions that rely on style transfer and shallow cue alignment to mitigate domain shift, this invention uses a masking network to explicitly decompose features into domain-invariant causal features and domain-specific spurious features at the intermediate visual feature level. Combined with adversarial training, this prevents spurious feature branches from carrying correct segmentation semantics, thereby actively removing domain-specific interference factors. This allows segmentation decisions to focus on stable anatomical structures, rather than merely mitigating low-level appearance distribution differences.

[0024] (2) Unlike existing solutions that fuse text features with visual context into a scalar bias and superimpose it onto the prediction result, this invention maps text features to the kernel weights and biases of dynamic convolutions through a projection network. Independent dynamic convolutions are performed on causal features and spurious features respectively to obtain causal feature prediction results and spurious feature prediction results. The causal branch receives positive segmentation supervision from refined pseudo-labels, while the spurious branch is subjected to reverse constraints in conjunction with adversarial loss. Thus, text semantics not only provide cross-domain stable semantic anchors for causal features, but also avoids dilution by domain-specific noise and participates in the purification of spurious features.

[0025] (3) Based on prediction uncertainty and feature prototype consistency, the present invention performs dual screening of initial pseudo-labels to generate refined pseudo-labels, and forms a closed loop of "screening, supervision, purification and re-screening" with adversarial training: high-quality pseudo-labels provide reliable supervision for causal branch segmentation loss and adversarial training. After adversarial training purifies causal features, it improves the inter-class discrimination of foreground or background feature prototypes, which in turn improves the accuracy of pseudo-label screening, thereby improving the quality of self-supervised signals without target domain labels.

[0026] (4) This invention forms a closed-loop synergy between dynamic causal intervention, text-guided semantic conditionation, dual pseudo-label screening and adversarial purification in adaptive testing. Causal decoupling provides pure input for text guidance, text guidance provides semantic anchors for causal features, refined pseudo-labels provide supervision for adversarial training, and adversarial training reversely purifies causal features and improves the reliability of pseudo-label screening. This synergistically improves the adaptive stability and cross-domain segmentation accuracy of medical image segmentation in continuous domain offset scenarios across medical centers and devices. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0028] Figure 1 This is a flowchart illustrating the medical image segmentation method based on causal intervention and text guidance provided by the present invention.

[0029] Figure 2 A schematic diagram illustrating the principle of the medical image segmentation method based on causal intervention and text guidance provided in the embodiments.

[0030] Figure 3 This is a visualization of causal and spurious features provided in the embodiments. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the accompanying drawings and... The embodiments further illustrate the present invention in detail. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of the invention.

[0032] This method constructs a CTTA framework that combines dynamic causal intervention with text-guided semantic conditionalization. 2 TTA is used to address the challenges posed by distribution shifts in medical image segmentation. Without modifying the backbone network weights, C... 2 TTA achieves efficient and stable adaptation through causal feature decomposition, cross-domain stable text semantic prior guidance, and adversarial pseudo-label optimization, while effectively avoiding error accumulation and catastrophic forgetting. Specifically, the embodiment provides a medical image segmentation method based on causal intervention and text guidance, such as... Figure 1 As shown, it includes the following steps: S1: Acquire multiple medical images and divide them into a labeled training set and an unlabeled target domain test dataset.

[0033] In this embodiment, multiple medical image data are acquired. These medical image data may originate from different medical devices, medical institutions, imaging protocols, or patient groups, including but not limited to fundus images and endoscopic images. For other medical images with domain shifts caused by different imaging devices, acquisition protocols, or medical centers, this method can also be applied to medical image segmentation tasks through appropriate input preprocessing.

[0034] Based on differences in data source, acquisition conditions, or domain attributes, the multiple medical image datasets are divided into a first training set and an unlabeled target domain test dataset. The first training set consists of source domain data, while the unlabeled target domain test dataset consists of target domain data. The first training set comprises manually labeled medical image data used for source domain supervised training; the unlabeled target domain test dataset consists of target domain medical image data acquired during the testing phase, used for temporal adaptation during testing. Because the two datasets differ in imaging equipment, resolution, lighting conditions, color distribution, noise levels, and tissue morphology, a domain shift occurs, providing a data foundation for subsequent model training and adaptation.

[0035] In the optic disc / optic cup (OD / OC) segmentation task of this embodiment, the data comes from five different medical centers (such as RIM-ONE-r3, REFUGE, etc.), and an 800×800 region of interest is extracted and scaled to 512×512. In the polyp segmentation task, the data comes from four public datasets (such as BKAI-IGH-NeoPolyp, CVC-ClinicDB, etc.), and the images are scaled to 352×352 pixels.

[0036] S2. Pre-train the first model using the labeled training set to obtain the pre-trained first model.

[0037] In this embodiment, a first model is pre-trained using a first training set, enabling the first model to learn lesion features, tissue boundary information, and semantic segmentation knowledge from source domain medical images. The first model can be a medical image segmentation network, whose backbone network can employ a convolutional neural network, a Transformer network, or a hybrid architecture combining both. In this embodiment, the OD / OC segmentation task uses a pre-trained ResUNet-34 network, and the polyp segmentation task uses a PraNet network. Specifically, after the medical image is input into the network, the encoder extracts multi-scale features, which are then progressively upsampled by the decoder and combined with skip connections to fuse details and semantic information. Finally, the output layer yields pixel-level segmentation results. Supervised training on the first training set enables the first model to acquire stable source domain representation capabilities. After pre-training, the backbone network weights of the first model are frozen, and the target domain medical images from the unlabeled target domain test dataset are further used to test the lightweight auxiliary module connected to the first model. Compared to the adaptation approach of directly updating all parameters of the backbone network, this embodiment uses a frozen segmentation model that has been fully pre-trained in the source domain as a stable foundation. During the testing phase, only lightweight auxiliary modules are updated, which helps to retain source domain segmentation knowledge while reducing the risk of error accumulation and catastrophic forgetting.

[0038] Furthermore, the unlabeled target domain test dataset does not participate in the source domain supervised pre-training of the first model, but is used as a sequence of target domain medical images during the testing phase. The current target domain medical image is obtained from the unlabeled target domain test dataset according to the order in which the target domain medical images enter the testing phase. and the current target domain medical image The first model, with its backbone network parameters frozen, is used to obtain pixel-level prediction probability maps. Intermediate visual feature maps used for causal feature decomposition And deep features used for category prototype calculation.

[0039] Specifically, the intermediate visual feature map For the adaptation process during testing, a lightweight masking network generates complementary soft masks and decomposes them into causal feature maps and spurious feature maps. These causal feature maps and spurious feature maps are used for dynamic causal intervention, semantically conditionalizing them with the text features of the anatomical structure to obtain pixel-level prediction results. The pixel-level prediction probability map... The deep features are used for pseudo-label optimization. Reliable pixels are selected based on prediction uncertainty and class prototype distance to generate refined pseudo-labels. Causal branch loss and spurious branch adversarial loss are constructed based on the refined pseudo-labels, and batch normalized statistical alignment loss is constructed based on the batch normalized statistics generated by the current target domain medical image. While keeping the weights of the first model backbone network and the text encoder parameters frozen, only the causal intervention module and the lightweight auxiliary module related to pixel-level prediction are updated. After completing the test-time adaptation of the current target domain medical image, the corresponding medical image segmentation result is obtained using the adapted first model, and subsequent target domain medical images in the unlabeled target domain test dataset are processed according to the test order. The above test-time adaptation process does not rely on manually labeled target domain information.

[0040] S3. Adapting the first model during testing using an unlabeled target domain test dataset. This adaptation includes: decomposing the visual features extracted by the first model into domain-invariant causal features and domain-specific spurious features using a masking network to achieve dynamic causal intervention; extracting text features of a preset anatomical structure using a pre-trained text encoder, and injecting the text features into the domain-invariant causal features and domain-specific spurious features respectively through dynamic convolution to obtain causal feature prediction results and spurious feature prediction results, thus achieving semantic conditionalization of text guidance; filtering the initial spurious labels based on prediction uncertainty and feature prototype consistency to generate refined spurious labels; calculating the positive segmentation loss based on the causal feature prediction results and refined spurious labels; constructing an adversarial loss based on the spurious feature prediction results; and performing adversarial optimization on the masking network and spurious feature prediction, wherein the spurious feature prediction branch minimizes the segmentation loss, and the masking network maximizes the segmentation loss, so as to prompt the masking network to reduce the information content related to the segmentation task in the spurious features; and optimizing the masking network and dynamic convolutional network through batch normalized statistical alignment loss to complete the test-time adaptation.

[0041] In the embodiments, such as Figure 2 As shown, from a causal perspective, domain offset can be considered as a confounding factor mainly composed of texture, noise, and contrast. This module freezes the intermediate visual feature maps output by the backbone network. As the initial visual representation, a lightweight masking network consisting of several convolutional layers is introduced for dynamic causal intervention. This network uses intermediate visual feature maps... As input, output two channel numbers and Same complementary soft mask and To ensure the completeness of the eigenvalue decomposition, it is mandatory to satisfy... Consistency constraints.

[0042] Specifically, through element-wise masking, the causal feature map and spurious feature map obtained are as follows: , ,in, and These are causal soft masks and spurious soft masks, respectively. This represents element-wise multiplication. This explicit factorization mechanism allows... It can focus on stable anatomical structures, and This captures domain-specific noise such as background texture, which will be penalized by subsequent adversarial targets. To generate accurate predictions, thereby ensuring Accurately capture causal characteristics.

[0043] Next, semantic conditionalization of text guidance is performed. Figure 2 Specifically, due to the high consistency in anatomical structures across different medical image domains, this step utilizes text priors to guide segmentation. A frozen BiomedCLIP is used as the text encoder to construct standardized text descriptions for the anatomical targets to be segmented, extracting dimensions... Text features This frozen representation exhibits strong generalization ability and significantly reduces computational cost during adaptation. Text embeddings are mapped to structured convolutional parameters, including kernel weights, through a lightweight, learnable projection network. and bias . Causal feature map and fake feature maps The samples are upsampled to full resolution via interpolation and convolution, respectively. Then, dynamic convolution operations are performed using the kernel weights and biases generated from the text features to obtain causal feature prediction results and spurious feature prediction results. , in, For input visual features, This represents a convolution operation. This cross-domain invariant textual semantic prior provides stable guidance for pixel-level prediction without modifying the backbone network weights, effectively blocking the confusion paths brought about by domain-specific styles.

[0044] In this embodiment, a frozen source model is used to predict a probability map, an initial pseudo-label is generated by a threshold, and an uncertainty mask is constructed by calculating entropy using Monte Carlo Dropout; deep features are extracted, foreground and background feature prototypes are calculated, and a final reliability mask is constructed by combining feature distance; an adversarial loss is constructed, the causal module is optimized by minimax objective, and batch normalization is combined with alignment loss to stabilize feature normalization.

[0045] Specifically, for unlabeled target domain images, a frozen source model is used to predict a probability map, and initial pseudo-labels are generated by applying a threshold. Simultaneously, Monte Carlo Dropout is used to calculate the prediction entropy and construct an uncertainty mask. Deep features are extracted from the penultimate layer, and the foreground feature prototype and background feature prototype are calculated as follows: , , , , in, Foreground feature prototype, As a prototype of background features, For pixels The deep feature vector, Pixel-level prediction probability, For initial pseudo-tags, For indicator functions, and These are foreground and background indicator variables, respectively.

[0046] Specifically, calculate the Euclidean distance from each pixel feature to the two prototypes. and Based on the estimated prototype and feature distance, construct the final reliability mask: .

[0047] Finally, the pseudo-label supervision loss is defined as masked cross-entropy. This mask combines prediction uncertainty with feature space consistency, ensuring that the pseudo-label is preserved only when the pixel feature is closer to the prototype of its assigned class, thus effectively filtering out erroneous predictions due to domain offset.

[0048] In this embodiment, the adversarial training and batch normalized statistical alignment optimization causal module includes: Pixel-level segmentation loss is calculated based on the segmentation results consistent with causal features and refined pseudo-labels. Based on false features, prediction results are generated, and an adversarial loss is constructed. Minimax optimization is performed on the lightweight masking network and spurious feature prediction to reduce the information content of spurious features that is relevant to the segmentation task. The batch normalized statistical alignment loss between the source and target domains is calculated by updating momentum, and the constraint feature normalization process is adapted to the target domain distribution.

[0049] Specifically, the total loss established based on adversarial training and batch normalized statistical alignment to optimize the causal module is expressed as: , in, For pixel-level segmentation loss, To combat the losses, To batch normalized statistical alignment loss, and These are the corresponding weighting coefficients.

[0050] in, Pixel-level segmentation loss for predicting branches of causal features; To combat the losses, and These are causal characteristics and false characteristics, respectively. For text features, To refine pseudo-labels, This is a pixel-level cross-entropy loss function. Adversarial loss is used to decouple causal features from spurious features. Causal feature prediction branch. Based on causal characteristics Generating and refining pseudo tags Consistent segmentation results; spurious feature prediction branch Used to detect false features Does it still contain information related to the segmentation category? By implementing minimax optimization on the false feature prediction branch and the masking network, the masking network is prompted to reduce the information related to the segmentation task in false features, while retaining the anatomical semantic information used for accurate segmentation in the causal features, thereby improving the purification degree of causal features. The batch normalized statistical alignment loss... Batch normalized statistics used to constrain the source and target domains.

[0051] Specifically, to mitigate distribution shift, a batch normalization statistical alignment loss is proposed (Batch Normalization Statistical Alignment Loss). Figure 2 The alignment statistic calculated via momentum update is: , ; The batch normalized statistical alignment loss is: ,in, , and , The source domain and the target domain are respectively in the th... The mean and standard deviation of the batch normalized statistics for each layer. The momentum coefficient, This represents the total number of batch normalization layers in the mask network. This loss function ensures that the feature normalization process continuously adapts to the dynamically changing target domain distribution by minimizing the difference between the alignment statistic and the current target domain statistic.

[0052] S4. Segment the target medical image based on the first model adapted during testing and output the segmentation results.

[0053] When applied, any medical image is used as input, and the adapted model is used to perform medical image segmentation to obtain the segmentation result.

[0054] Based on the same technical concept, this embodiment also provides a medical image segmentation device based on causal intervention and text guidance, including a memory and a processor. The memory stores a computer program, and the processor, when executing the computer program, implements the aforementioned medical image segmentation method based on causal intervention and text guidance. Those skilled in the art will understand that all or part of the steps in the above method embodiments can be implemented by hardware related to program instructions. These program instructions can be stored in one or more computer-readable storage media and, when called and executed by a processor, implement the corresponding steps in the above embodiments of the medical image segmentation method. In other words, the above method embodiments can be implemented entirely in software, in hardware, or in a combination of software and hardware.

[0055] The computer-readable storage medium can be any tangible medium capable of storing program code, including but not limited to static random access memory (SRAM), solid-state hard disk, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic storage, optical storage, magnetic disk, optical disk, and any combination of the above media.

[0056] To better illustrate the effectiveness of the method provided by this invention, combined with experimental results and visualization analysis, this invention can still obtain segmentation results that are close to those of manual annotation without relying on manual annotation of the target domain and only using the target domain data in the testing phase for adaptive processing. It shows good boundary characterization ability, structural integrity and segmentation stability in polyp segmentation and optic disc / cup segmentation tasks.

[0057] To further verify the effectiveness of the dynamic causal intervention mechanism employed in this invention in decoupling domain-invariant causal features from domain-specific spurious features, Figure 3 The visualization results of feature representations of test images in different target domains are shown in the visual disc and visual cup segmentation task. Specifically, the first model was pre-trained using domain A as the source domain, and adaptively adapted when tested using domains B, C, D, and E as target domains respectively. Figure 3 In the diagram, A→B, A→C, A→D, and A→E represent adaptive scenarios when the model is tested from the source domain A to the corresponding target domain.

[0058] Figure 3In the image, each row corresponds to the test results under different target domains, and each column, from left to right, consists of the test image, causal features of the optic disc, false features of the optic disc, causal features of the optic cup, false features of the optic cup, predicted segmentation results, and ground truth annotation results.

[0059] Depend on Figure 3 It can be seen that, under different target domain conditions, causal features can respond relatively stably to anatomical structures related to the segmentation task, such as the optic disc and optic cup, and maintain their position and contour information well. In contrast, spurious features mainly respond to background texture, brightness, contrast, and other appearance information related to changes in the imaging domain. These results indicate that the present invention, through dynamic causal intervention, can, to some extent, separate domain-invariant anatomical information related to the segmentation task from domain-specific appearance information, thereby reducing the model's dependence on domain-specific spurious features. Furthermore, Figure 3 The predicted segmentation results and the corresponding real annotation results show good consistency in the regional location and contour structure of the optic disc and optic cup. This indicates that after dynamic causal intervention and adaptive testing, the model can still maintain relatively stable segmentation performance under different target domain conditions, thereby improving the adaptability of the medical image segmentation model to domain shift and its cross-domain generalization ability.

[0060] In the experimental setup of this embodiment, the learning rate for the OD / OC segmentation task was set to 0.05, the learning rate for the polyp segmentation task was set to 0.01, and the hyperparameters were... and The values ​​were set to 0.05 and 1000 respectively, and the momentum preheating coefficient was set to 5. All experiments were implemented on an NVIDIA RTX 4090 GPU using the PyTorch framework.

[0061] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A medical image segmentation method based on causal intervention and text guidance, characterized in that, Includes the following steps: Acquire multiple medical images and divide them into a labeled training set and an unlabeled target domain test dataset; The first model is pre-trained using a labeled training set to obtain the pre-trained first model. The first model is tested and adapted using an unlabeled target domain test dataset. This adaptation includes: decomposing the visual features extracted by the first model into domain-invariant causal features and domain-specific spurious features using a masking network to achieve dynamic causal intervention; extracting text features from a pre-trained text encoder based on preset anatomical structures, and injecting these text features into the domain-invariant causal features and domain-specific spurious features respectively through dynamic convolution to obtain causal feature prediction results and spurious feature prediction results, thus achieving semantic conditionalization of text guidance; generating refined spurious labels based on prediction uncertainty and feature prototype consistency; calculating a positive segmentation loss based on the causal feature prediction results and refined spurious labels; constructing an adversarial loss based on the spurious feature prediction results; and performing adversarial optimization on the masking network and spurious feature prediction, where the spurious feature prediction branch minimizes the segmentation loss, and the masking network maximizes the segmentation loss, thereby prompting the masking network to reduce the information content related to the segmentation task in the spurious features; and optimizing the masking network and dynamic convolutional network through batch normalized statistical alignment loss to complete the test-time adaptation. The target medical image is segmented based on the first model adapted during testing, and the segmentation results are output.

2. The medical image segmentation method based on causal intervention and text guidance according to claim 1, characterized in that, The backbone network of the first model is a convolutional neural network, a Transformer network, or a hybrid architecture combining both. During the adaptive process of the first model during testing, the weights of the backbone network remain frozen, and only the parameters related to dynamic causal intervention and semantic conditionalization of text guidance are updated.

3. The medical image segmentation method based on causal intervention and text guidance according to claim 1, characterized in that, The process of decomposing the visual features extracted by the first model into domain-invariant causal features and domain-specific spurious features using a masking network includes: Extract the intermediate visual features from the penultimate layer of the first model as the initial visual representation; The initial visual representation is processed using a lightweight masking network to generate causal soft masks and spurious soft masks. The initial visual representation is multiplied element-wise with the causal soft mask and the spurious soft mask respectively to obtain domain-invariant causal features and domain-specific spurious features.

4. The medical image segmentation method based on causal intervention and text guidance according to claim 3, characterized in that, Domain-invariant causal characteristics are anatomical structural information that remains stable under at least one condition, such as different medical centers, different medical devices, and different imaging protocols. Domain-specific spurious features are at least one domain-specific appearance information caused by domain offset, such as texture, noise, lighting changes, or contrast.

5. The medical image segmentation method based on causal intervention and text guidance according to claim 1, characterized in that, By dynamically convolutional, text features are injected into domain-invariant causal features and domain-specific spurious features, including: Text features are mapped to dynamic convolution kernel weights and biases using a lightweight projection network. Domain-invariant causal features and domain-specific spurious features are upsampled to the original image resolution, respectively. By using the convolution kernel weights and biases, dynamic convolution operations are performed on the upsampled domain-invariant causal features and domain-specific spurious features, respectively, to obtain causal feature prediction results and spurious feature prediction results.

6. The medical image segmentation method based on causal intervention and text guidance according to claim 1, characterized in that, Based on prediction uncertainty and feature prototype consistency, refined pseudo-labels are generated by filtering the initial pseudo-labels, including: The first pre-trained model is used to predict pixel-level probability maps, and initial pseudo-labels are generated by setting a probability threshold. Monte Carlo Dropout is used to calculate prediction entropy and construct an uncertainty mask; Extract the deep features from the penultimate layer of the first model, and calculate the foreground feature prototype and background feature prototype respectively; Calculate the feature distance from the deep features of each pixel to the foreground feature prototype and the background feature prototype; Based on the uncertainty mask and the feature distance, a reliability mask is constructed, and reliable pixels are selected to generate refined pseudo-labels; wherein the reliability mask is represented as: ,in, For pixels Predicted entropy, For uncertainty threshold, and These are the Euclidean distances to the foreground feature prototype and the background feature prototype, respectively. These are the initial pseudo-tags.

7. The medical image segmentation method based on causal intervention and text guidance according to claim 6, characterized in that, The foreground feature prototype and background feature prototype are calculated in the following ways: ， ， in, Foreground feature prototype, As a prototype of background features, For pixels The deep feature vector, Pixel-level prediction probability, For initial pseudo-tags, For indicator functions, and These are foreground and background indicator variables, respectively.

8. The medical image segmentation method based on causal intervention and text guidance according to claim 1, characterized in that, The total loss established by optimizing the mask network and dynamic convolutional network using batch normalized statistical alignment loss is expressed as: ， in, For pixel-level segmentation loss, To combat the losses, To batch normalized statistical alignment loss, and These are the corresponding weighting coefficients.

9. A medical image segmentation device based on causal intervention and text guidance, comprising a memory and a processor, wherein the memory is used to store a computer program, characterized in that, The processor is used to implement the medical image segmentation method based on causal intervention and text guidance as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it performs the medical image segmentation method based on causal intervention and text guidance as described in any one of claims 1 to 8.

Images