A cardiac medical image segmentation method and device based on decoupling representation and knowledge fusion and a computer storage medium

Abstract

The application discloses a cardiac medical image segmentation method and device based on decoupling representation and knowledge fusion, which comprises the following steps: inputting cardiac image data into a basic model and an expert model respectively, and extracting multi-scale features; performing structure-residual decomposition on the multi-scale features to obtain structure components and residual components; adopting a center kernel alignment constraint to align the same components across models, and applying an orthogonal decoupling constraint to the structure components and the residual components within the same model; inputting the aligned components into a pixel decoder to obtain corresponding segmentation responses; generating adaptive gating weights based on category condition information, re-fusing two types of segmentation responses, and predicting a final cardiac medical image segmentation result. Through the technical path of decoupling, alignment and fusion, the application effectively absorbs the task-specific knowledge of the expert model while retaining the general ability of the basic model, and significantly improves the segmentation accuracy and generalization robustness of out-of-distribution cardiac medical images.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and artificial intelligence technology, specifically relating to a cardiac medical image segmentation method, device, and computer storage medium based on decoupled representation and knowledge fusion. Background Technology

[0002] Cardiac medical image segmentation is crucial for quantitative assessment of cardiac structure, measurement of lesion extent, clinical diagnostic assistance, and treatment planning. Currently, the generalization ability of expert models based on architectures such as U-Net and nnUNet is often limited by the source of training data, and performance tends to degrade when faced with cross-center, cross-modal, cross-device, and varying lesion distributions. On the other hand, while multimodal medical models possess strong general representation capabilities and a wide task coverage, their ability to discern lesion details and boundaries remains insufficient in out-of-distribution cardiac scenarios.

[0003] One existing direct approach is to fine-tune the base model to adapt it to specific cardiac tasks. For example, the invention patent with publication number CN120765931A discloses a medical image super-resolution segmentation method based on multi-scale resolution autoencoders and task-aware feature decoupling. While this direct fine-tuning approach may improve the performance of the target domain, it is prone to catastrophic forgetting, which reduces the original general anatomical knowledge and cross-domain generalization ability of the base model.

[0004] Furthermore, traditional feature fusion methods, such as the invention patent disclosed in CN119557840A which describes a feature fusion method based on multimodal medical data, typically involve directly splicing, weighting, or aligning features from different sources. However, they fail to explicitly distinguish between domain-invariant and domain-related information within the features, easily injecting domain bias information from expert models into the base model, leading to negative transfer and bias leakage. In the highly heterogeneous scenario of cardiac medical images, features from different sources often simultaneously contain relatively stable anatomical topological information and domain-sensitive texture, noise, artifact, and lesion difference information. Without explicit decoupling and component-level control mechanisms, robust out-of-distribution transfer is difficult to achieve. Summary of the Invention

[0005] In view of the problems existing in the above-mentioned background technology, such as insufficient generalization ability of expert models, easy catastrophic forgetting caused by direct fine-tuning of basic models, and bias leakage and negative transfer caused by the failure of traditional feature fusion methods to distinguish between domain-invariant information and domain-related information, the present invention provides a cardiac medical image segmentation method and device based on decoupled representation and knowledge fusion.

[0006] To achieve the above-mentioned objectives, an embodiment provides a cardiac medical image segmentation method based on decoupled representation and knowledge fusion, characterized by comprising the following steps: S1. Cardiac medical image data are input into the basic model and expert model respectively, and basic features and expert features are extracted; S2. Decompose the basic features and expert features to obtain structural components and residual components. The structural components are used to characterize the anatomical structural information that is stable across domains, and the residual components are used to characterize the fine-grained information that is related to the specific domain. S3. Employ center kernel alignment constraints to align structural components across models and residual components across models; and apply orthogonal decoupling constraints to structural components and residual components within the same model. S4. The aligned structural components and residual components are respectively processed by the pixel decoder to obtain the corresponding segmentation responses; based on the category condition information, the corresponding segmentation responses are adaptively refused to obtain the cardiac medical image segmentation results.

[0007] Preferably, the basic features are obtained by inputting cardiac medical image data into the basic model encoding, and are used to retain basic medical multimodal knowledge; The expert features are obtained by inputting cardiac medical image data into an expert model for encoding, and are used to preserve lesion details and boundary discrimination.

[0008] More preferably, the base model is BiomedParse, and the expert model is nnUNet.

[0009] Preferably, the decoupling and decomposition of multi-scale features to obtain structural components and residual components includes: using a dual-branch parallel channel projection structure, decomposing each scale feature in the corresponding multi-scale features through convolutional layers to generate structural components and residual components for that scale.

[0010] More preferably, a reconstruction loss constraint is introduced during decoupling decomposition to constrain the sum of the decomposed structural components and residual components to approximate the multi-scale features before decomposition. The calculation formula is as follows: , In the formula, To reconstruct the loss constraints; For the first Input features at various scales; For the result component, For residual components, Denotes the square of the Frobenius norm. Representation layer normalization, It is a numerically stable term.

[0011] By introducing reconstruction constraint loss, we can avoid losing the original feature information during the structure-residual decomposition process and ensure that the two types of components can recover the multi-scale features before decomposition as much as possible after summation.

[0012] Preferably, aligning structural components across models and aligning residual components across models includes: The center kernel alignment similarity loss is used to measure the representation consistency between the structural components on the base model side and the structural components on the expert model side, as well as the representation consistency between the residual components on the base model side and the residual components on the expert model side, and knowledge alignment is performed with the goal of minimizing the representation consistency. The knowledge alignment loss is expressed as: In the formula, For center kernel alignment similarity loss, and These represent the basic model at the 1st... Aligned structural components and residual components at each scale and These represent the expert model at the 1st and 2nd stages, respectively. Aligned structural components and residual components at each scale.

[0013] Through the aforementioned component-level knowledge alignment, the structural components of the base model can more fully inherit the expert model's ability to geometrically characterize complex cardiac boundaries, while the residual components of the base model can more fully absorb the expert model's sensitivity to lesions and abnormal regions, thereby reducing the negative migration risk brought about by the overall injection of domain bias information.

[0014] Preferably, the applied orthogonal decoupling constraint is expressed as: : , In the formula, This represents the average of the absolute values ​​of each element in the cross-correlation matrix of the two sets of features; and These represent the basic model at the 1st... Aligned structural components and residual components at each scale and These represent the expert model at the 1st and 2nd stages, respectively. Align the structural and residual components at each scale. By minimizing the orthogonal decoupling constraint, the linear correlation between structural and residual components within the same model can be reduced, allowing the structural components to more centrally retain domain-stable anatomical information, and the residual components to more centrally represent fine-grained information (lesion details, texture differences, and domain-related supplementary information) related to specific domains.

[0015] Preferably, the adaptive re-fusion of the corresponding segmentation responses based on category condition information includes: Category conditional query vectors are generated based on category embeddings; the category embeddings are frozen category semantic embeddings, and the category conditional query vectors are obtained by adding the frozen category semantic embeddings to the learnable query offsets; Using the multi-scale encoded features of the expert model as keys and values, the category condition query vector is modulated through a multi-head attention mechanism to obtain the category condition descriptor; The category condition descriptor is input into a multilayer perceptron to generate adaptive gating weights; The residual component segmentation response of the structural component segmentation response domain is re-fused based on adaptive gating weights. The re-fusion is expressed as follows: ,in, For category condition descriptors, and The first Structural component segmentation response and residual component segmentation response at each scale The adaptive gating weights are defined as follows.

[0016] By introducing a class-conditional adaptive mechanism, different classes and spatial locations can dynamically determine whether to rely more on the structural component or the residual component. When the class condition indicates that the current region has high anatomical prior reliability, the structural component response is enhanced; when the class condition indicates that the current region has more significant lesion differences or out-of-distribution uncertainty, the contribution of the residual component response is increased. Class conditional information is generated from class embeddings, textual cue features, or class priors.

[0017] Preferably, during the training phase, all parameters of the base model and the expert model are frozen, and only the parameters involved in feature decomposition, cross-model alignment, and adaptive re-fusion are trained; During training, the total loss function is expressed as follows: ,in, To divide the loss, To reconstruct the loss constraints, For orthogonal decoupling constraints, For knowledge alignment loss, To reconstruct the loss constraint weights, For orthogonal decoupling constraint weights, For knowledge alignment loss weights, This represents the total number of scales.

[0018] On the other hand, the embodiment also provides a cardiac medical image segmentation device based on decoupled representation and knowledge fusion, used to implement the cardiac medical image segmentation method based on decoupled representation and knowledge fusion, including: The feature extraction module is used to input the preprocessed cardiac medical image data into the basic model and the expert model respectively, and extract basic features and expert features. The feature decoupling module is used to decompose basic features and expert features to obtain structural components and residual components. The structural components are used to characterize cross-domain stable anatomical structure information, and the residual components are used to characterize fine-grained information related to specific domains. The knowledge alignment module is used to align structural components across models and residual components across models using central kernel alignment constraints; and to apply orthogonal decoupling constraints to structural components and residual components within the same model. The adaptive re-fusion module is used to obtain the corresponding segmentation responses from the aligned structural components and residual components through the pixel decoder; and to perform adaptive re-fusion on the corresponding segmentation responses based on category condition information. The output module is used to input adaptive re-fusion features into the predictor to obtain the final cardiac medical image segmentation result.

[0019] On the other hand, the embodiment also provides a computer-readable storage medium on which a program is stored, which, when executed by a processor, performs the cardiac medical image segmentation method based on decoupled representation and knowledge fusion.

[0020] Compared with the prior art, the beneficial effects of the present invention include at least the following: (1) The technical approach of structure-residual decoupling and component-level knowledge alignment proposed in this invention effectively avoids the problem of injecting domain bias information from the expert model into the base model as a whole in traditional feature fusion methods by explicitly separating the structural components and residual components of features and performing cross-model alignment only on components of the same type, thereby significantly reducing the risk of negative transfer. At the same time, by introducing orthogonal constraints inside the model, it is ensured that the structural components focus on preserving cross-domain stable anatomical topological information, and the residual components focus on capturing domain-related lesion and texture difference information, further enhancing the robustness of the model in heterogeneous medical image scenarios.

[0021] (2) The category-condition-guided adaptive re-fusion mechanism proposed in this invention enables the model to dynamically adjust its dependence on structural and residual information based on different cardiac structures and spatial locations. In regions with reliable anatomical priors, the model preferentially relies on structural components to maintain anatomical consistency; in regions with high uncertainty outside the lesion or distribution, the model automatically enhances the contribution of residual components to preserve fine-grained discriminative cues. This flexible fusion strategy enables this invention to more intelligently respond to different types of cardiac structures and lesion morphologies in the face of complex and ever-changing real-world clinical scenarios.

[0022] (3) This invention achieves the complementarity and fusion of the advantages of both by simultaneously utilizing the general representation capability of the medical multimodal basic model and the high-precision discrimination capability of the task expert segmentation model. Experimental results show that in cross-center and cross-modal out-of-distribution cardiac image segmentation tasks, the average Dice coefficient of this invention is improved by about 6% in the CT modality and by more than 7% in the MRI modality compared with the current best nnUNet, which fully demonstrates that this invention can effectively improve the segmentation accuracy of complex cardiac structures and lesion regions while ensuring generalization ability.

[0023] (4) This invention abandons the traditional full-model fine-tuning strategy. By freezing all parameters of the base model and training only a very small number of lightweight adaptation modules, it fundamentally avoids the loss of the base model's original general knowledge when adapting to new tasks. Experimental data shows that after directly performing full fine-tuning on the base model, its segmentation performance in out-of-distribution scenarios drops significantly to a Dice coefficient of 0.645. In contrast, this invention maintains a segmentation accuracy as high as 0.871 while introducing expert knowledge, fully verifying the effectiveness of this invention in protecting the general capabilities of the base model during knowledge transfer. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a cardiac medical image segmentation method based on decoupled representation and knowledge fusion provided in the embodiment.

[0026] Figure 2 This is a logical process diagram of the cardiac medical image segmentation method based on decoupled representation and knowledge fusion provided in the embodiment.

[0027] Figure 3 This is a schematic diagram of the t-SNE scatter plot distribution of the structural components and residual components obtained from the decomposition of the embodiment.

[0028] Figure 4 This is a schematic diagram of the cardiac medical image segmentation device based on decoupled representation and knowledge fusion provided in the embodiment.

[0029] Figure 5 This is a comparison of the 3D visualization results of different methods provided in the embodiments for the cardiac multi-structure segmentation task. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0031] In related technologies, many disease segmentation and diagnosis methods based on deep learning, transfer learning, and basic model adaptation have been proposed. For example, expert model frameworks such as U-Net and nnUNet are used to achieve high-precision segmentation of specific cardiac structures. Alternatively, medical multimodal basic models are used, leveraging their strong general representation capabilities and cross-task transfer capabilities to uniformly model different medical image tasks. In addition, some methods adapt general models to specific cardiac segmentation tasks through feature distillation, feature fusion, and efficient parameter fine-tuning to improve the accuracy of target region recognition.

[0032] However, most existing technologies focus on directly learning, fine-tuning, or fusing overall features, lacking an effective technical solution for out-of-distribution cardiac scenarios that simultaneously considers generalization ability and task-specific discrimination ability. Specifically, existing methods typically do not explicitly distinguish between domain-invariant anatomical structural information and domain-related lesion difference information within features, easily injecting domain bias information from expert models into the base model during transfer or fusion, leading to bias leakage, negative transfer, and catastrophic forgetting problems. Furthermore, there is a lack of a complete technical paradigm that aligns component-level knowledge based on decoupling and achieves adaptive re-fusion by combining category semantic conditions.

[0033] In view of this, the inventive concept of this invention is as follows: to solve the problems of insufficient generalization ability of expert models, catastrophic forgetting caused by direct fine-tuning of the basic model, and bias leakage and negative transfer caused by direct fusion of features from different sources in the prior art. By adopting a decoupling-alignment-fusion technical approach, the generalization ability of the generalized basic model and the segmentation accuracy of the expert model are transferred in a structured way, so as to avoid negative transfer while retaining their respective advantages, thereby achieving robust segmentation for out-of-distribution cardiac medical images.

[0034] Based on the above inventive concept, such as Figure 1 As shown in the embodiment, a cardiac medical image segmentation method based on decoupled representation and knowledge fusion is provided, including the following steps: S1. Cardiac medical image data are input into the basic model and expert model respectively, and basic features and expert features are extracted.

[0035] The core objective of step S1 is to acquire cardiac medical image data to be segmented and perform preprocessing operations to reduce input differences between different centers, devices, and protocols. The preprocessed images are then input into the base model and expert model respectively to obtain multi-scale feature representations at different levels. These multi-scale features can come from different stages or resolution layers of the encoder, and are used to simultaneously support structural information modeling, residual information modeling, and component-level knowledge alignment in subsequent processing.

[0036] First, cardiac medical image data from one or more medical centers is acquired. This image data can be computed tomography (CT) images, magnetic resonance imaging (MRI), cardiovascular magnetic resonance imaging (CMR), or other modalities of medical image data. The preprocessing process sequentially resamples the voxel spacing, performs non-foreground region cropping, intensity normalization, and input size alignment to reduce differences in spatial resolution, grayscale distribution, and effective anatomical region range between different data sources, facilitating data processing in the downstream model. Specifically, resampling unifies the spatial resolution across different data sources, non-zero foreground region cropping removes large areas of invalid background, intensity normalization reduces grayscale distribution differences caused by different imaging modalities and equipment conditions, and input size alignment ensures consistency for subsequent training and inference.

[0037] In this embodiment, the medical image dataset was collected from the 2025 International Conference on Medical Imaging Challenge (MICCAI CARE2025). This challenge aims to alleviate the problems of modal heterogeneity and domain heterogeneity in real-world medical scenarios, and includes a multimodal, cross-center, multi-sequence dataset containing CT, MRI, and CMR data. It mainly includes whole-heart segmentation tasks in the CT and MRI modalities, and myocardial scar segmentation and recognition tasks in the CMR modal.

[0038] In this embodiment, the performance of patients in the CT modality under an out-of-distribution (OOD) configuration is primarily considered. That is, cross-center data partitioning is used, with CT data from center A for training and validation (20 case samples in total), and CT data from center B for testing (20 case samples in total). The training, validation, and testing process of nnUNet is strictly consistent with the method in this patent to prevent data leakage. The segmentation task aims for seven classifications, including: left ventricle (LV), right ventricle (RV), left atrium (LA), right atrium (RA), myocardium (Myo), ascending artery (AO), and pulmonary artery (PA).

[0039] In this embodiment, the preprocessing stage adopts an nnUNet-style preprocessing workflow. The base model is BiomedParse, and the expert model is nnUNet. BiomedParse maintains its original weights frozen, while nnUNet is trained and internally validated on the A-center CT data. Subsequent feature extraction uses the BiomedParse backbone (base model encoder). The preprocessed image is mapped to the basic latent variable. And perform feature extraction to obtain basic features. The nnUNet encoder part (expert model encoder) Preprocessed images of the same size and type are mapped to expert latent variables. And feature extraction is performed to obtain expert features. Finally, the feature map of the basic model is obtained. Feature maps of expert models ( Figure 2 Each of these contains multi-scale feature maps of the relevant models.

[0040] S2. Decompose the basic features and expert features to obtain structural components and residual components. The structural components are used to characterize the anatomical structural information that is stable across domains, and the residual components are used to characterize the fine-grained information that is related to the specific domain.

[0041] In this embodiment, the multi-scale feature representation is subjected to structure-residual decomposition to obtain structural components and residual components. The structural components are used to preserve domain-invariant anatomical topological information, while the residual components are used to preserve domain-related fine-grained lesion information and differential information.

[0042] The decomposition employs a dual-branch parallel channel projection structure targeting the same input feature, generating structural components and residual components respectively. The sum of these two components is then constrained by a reconstruction loss to approximate the original input feature, thus maintaining representation integrity. Specifically, the feature decomposition stage involves feature maps from both the base model and expert model sides. and Two 3D convolutional neural networks (CNNs) of size (1,1,1) are fed into their respective scales. The output channels of the CNNs on the base model side remain unchanged, while the output channels of the CNNs on the expert model side must be consistent with the feature map channels on the base model side. Through modulation of the CNNs during the decoupling phase, the feature maps of the expert model nnUNet and the base model BiomedParse are projected onto the same semantic space. At the same time, the feature maps of BiomedParse and nnUNet are each decoupled into two clusters, namely structural components and residual components, laying the groundwork for subsequent knowledge fusion.

[0043] In a formal way, such as Figure 2As shown, for the basic model in the first... Input features at each scale Through the decoupling module corresponding to the basic model Decomposition yields: ; For expert models in the 1st Input features at each scale Through the decoupling module corresponding to the expert model Decomposition yields: ; in, and These represent the basic model at the 1st... Aligned structural components and residual components at each scale and These represent the expert model at the 1st and 2nd stages, respectively. Align the structural and residual components at each scale. To avoid losing original feature information during structure-residual decomposition and to ensure that the two types of components can recover the input features as much as possible after summation, this invention further introduces a reconstruction loss constraint. At the... At each scale, the reconstruction loss can be expressed as: , In the formula, To reconstruct the loss constraints; For the first Input features at various scales; For the result component, For residual components, Denotes the square of the Frobenius norm. Representation layer normalization, It is a numerically stable term.

[0044] S3. Employ center kernel alignment constraints to align structural components across models and residual components across models; and apply orthogonal decoupling constraints to structural components and residual components within the same model.

[0045] In the knowledge alignment stage, a component-aware alignment strategy is adopted to perform component-level knowledge fusion on similar components in both models. Structural components in BiomedParse are paired with structural components in nnUNet, and residual components in BiomedParse are paired with residual components in nnUNet, with knowledge alignment performed separately. This knowledge alignment is achieved using Channel Center Kernel Alignment (CKA) constraints. Channel CKA measures the similarity structure between two sets of features along the channel dimension, enhancing the consistency of the overall semantic structure of the feature representations on both sides without requiring complete point-by-point numerical consistency. During the knowledge alignment process, orthogonality constraints and reconstruction constraints are introduced simultaneously. The orthogonality constraint reduces the linear correlation between structural and residual components within the same model, allowing them to function separately. The reconstruction constraint ensures that the decoupled structural and residual components, after combination, can still recover the original input features as much as possible, thereby maintaining representation integrity and reducing information loss caused by decoupling. Unlike directly fusing the original overall features, this invention aligns the structural components of the base model with the structural components of the expert model, and aligns the residual components of the base model with the residual components of the expert model. This allows the base model to absorb the stable anatomical priors of the expert model in the structural subspace and the lesion-sensitive information of the expert model in the residual subspace, thereby reducing the negative migration risk brought about by the overall injection of domain bias information.

[0046] Specifically, such as Figure 2 As shown, the knowledge alignment is achieved using a similarity constraint based on center kernel alignment. For the first... For each scale, the knowledge alignment loss is defined as: , in, The center kernel alignment similarity loss is used to measure the consistency of representations between the base model and the expert model on the same components.

[0047] Furthermore, if we denote the two sets of features to be compared as... and Then, the linear center kernel alignment similarity can be expressed as: , in, To prevent constants with a denominator of zero.

[0048] Through the aforementioned component-level knowledge alignment, the structural components of the base model can more fully inherit the expert model's ability to geometrically characterize complex cardiac boundaries, while the residual components of the base model can more fully absorb the expert model's sensitivity to lesions and abnormal regions. To further ensure the functional division of labor between structural and residual components, this invention introduces orthogonal decoupling constraints for the first... The orthogonal decoupling constraints applied at each scale are expressed as follows: : , In the formula, This represents the average of the absolute values ​​of each element in the cross-correlation matrix of the two sets of features. By minimizing this constraint term, the linear correlation between structural components and residual components within the same model can be reduced, allowing structural components to more effectively preserve domain-invariant anatomical topological priors, and allowing residual components to more effectively represent lesion details, texture differences, and domain-related supplementary information.

[0049] S4. The aligned structural components and residual components are respectively processed by the pixel decoder to obtain the corresponding segmentation responses. Based on the category condition information, the corresponding segmentation responses are adaptively refused to predict the final cardiac medical image segmentation result.

[0050] Step S4 mainly includes forward propagation, component refusion, and segmentation result prediction.

[0051] S4-1 forward propagation: used to input the structural components and residual components after component-level knowledge alignment into the pixel decoder to obtain the corresponding component-level segmentation response.

[0052] In this embodiment, after knowledge alignment is completed, the base model BiomedParse will align the structural component feature maps. and residual component characteristic map The components are fed into their respective pixel decoders for forward propagation to obtain the structural component responses. and residual component response The pixel decoder performs feature enhancement, multi-scale feature aggregation, and spatial information interaction on both types of components, resulting in a more stable anatomical representation within the structural component and a more sensitive lesion and abnormal response within the residual component. Since both types of components share a unified pixel decoding system, they can be projected into a consistent spatial representation domain, providing directly modulated component-level features for subsequent component fusion modules.

[0053] After processing by the pixel decoder, two sets of resource libraries can be obtained for subsequent prediction, one of which is the structural component resource library. The other group is the residual component resource library. .in and Used to preserve component feature information at different scales and Used to preserve high-resolution spatial representations related to segmentation mask generation. The two sets of resource libraries mentioned above correspond to stable anatomical information resources and fine-grained difference information resources, respectively. They are not directly used as the final output, but are instead used in the subsequent re-fusion stage.

[0054] S4-2 Component Fusion: This component generates adaptive spatial gating or fusion weights based on category-conditional information, re-fusing the structural component segmentation response and the residual component segmentation response to output the final cardiac medical image segmentation result. Unlike fixed-weight summation or direct concatenation, this invention introduces a category-conditional adaptive mechanism, allowing different categories and spatial locations to dynamically determine whether to rely more on the structural component or the residual component. When the category condition indicates that the current region has high anatomical prior reliability, the structural component response is enhanced; when the category condition indicates that the current region has more significant lesion differences or out-of-distribution uncertainty, the contribution of the residual component response is increased.

[0055] In this embodiment, BiomedParse uses a MedBERT pre-trained text encoder to encode and vectorize the prompts (text embeddings). Therefore, text descriptors based on text embeddings are generated for different types of prompts. These descriptors are obtained by adding the text embeddings of different types of prompts to a learnable residual vector. The text descriptors interact with the deep semantic features of the expert model nnUNet to obtain class-prototype descriptors based on text and image features. The class-prototype descriptors are responsible for generating channel-level gating, adaptively performing convex combinations of structural component resource libraries and residual component resource libraries for each class. Finally, the fused resource library is forward-propagated to the prediction results of BiomedParse, interacting with the text content to produce the final segmentation result.

[0056] Specifically, category condition information is generated from category embeddings, textual cue features, or category priors. Let the frozen category semantic embedding be... The learnable query offset is Then the category condition query vector can be written as: .

[0057] Multi-scale coding using expert models As keys and values, the category condition descriptor most relevant to the current category is obtained through a multi-head attention mechanism. : , in, This represents a multi-head attention operator. This indicates a flattening operation, which is performed after obtaining the category condition descriptor. Then, gating weights are generated using a multilayer perceptron, and adaptive refusion is performed on the structural component response and the residual component response. For the th At each scale, the refusion result can be expressed as: , in, For category condition descriptors, and The first Structural component segmentation response and residual component segmentation response at each scale The adaptive gating weights are defined as follows.

[0058] When the adaptive gating weights are large, the structural component response is enhanced; when the adaptive gating weights are small, the residual component response is enhanced. This approach prioritizes maintaining anatomical consistency in structural regions and preserving domain-related discriminative cues in lesion regions. Finally, features from all scales are re-fused. The text embeddings from the base model are fed into the predictor to obtain the final cardiac medical image segmentation results.

[0059] During training, all parameters of the base model BiomedParse, including its backbone network, pixel decoder, and predictor, were frozen, along with all parameters of the expert model nnUNet. Only the lightweight structure-structure residual decomposition module, alignment module, and class-conditional adaptive refusion module were allowed to be trained, resulting in 9.1 M trainable parameters, approximately 2.14% of the total 425 M model parameters. 3D CT cardiac image data were converted to 2D images by slice, following the CARE Cardiac 2025 benchmark preprocessing workflow. A cross-center data partitioning strategy was adopted, with 20 cases in Center A, totaling 37,135 slices, used for training and internal validation. The AdamW optimizer was used, with an initial learning rate of 1×10⁻⁶. -4 The weight decays to 1×10 -4 The overall objective of the loss function is: ,in, To divide the loss, To reconstruct the loss constraints, For orthogonal decoupling constraints, For knowledge alignment loss, To reconstruct the loss constraint weights, For orthogonal decoupling constraint weights, For knowledge alignment loss weights, This represents the total number of scales.

[0060] Segmentation loss We employ a joint loss method combining standard Dice and cross-entropy; we reconstruct the loss constraint weights. =0.1, orthogonal decoupling constraint weight =0.15, knowledge alignment loss weight =0.25. The model was trained for 100 epochs with a batch size of 1; a single NVIDIA GeForce RTX 3090 GPU (24 GB VRAM) was used, and the total training time reached convergence in approximately 2 hours; no additional acceleration strategies such as mixed precision were used during training. The expert model nnUNet was pre-trained independently for 500 epochs on the same training set using a 2D configuration, and the optimal checkpoint on the validation set was selected as the frozen expert prior.

[0061] The inference phase only runs the frozen base model backbone, lightweight decomposition module, and re-fusion module, without running the expert model. The inference computation is 1.931 TFLOPS / patch (patch size 4×512×512, 7 category hints), which is only about 5% more overhead than the original BiomedParse's 1.837 TFLOPS / patch. A sliding window inference engine consistent with nnUNet is used; Gaussian importance weighting is applied to fused overlapping predictions on 2D slices, with a Gaussian sigma scale set to 0.125; the sliding window tiling step size is set to 0.5. The Gaussian weighted prediction results for all slices are normalized pixel-by-pixel, and then the slice-level segmentation masks are stitched together in the original spatial order and mapped back to the original 3D image coordinate space to obtain the final 3D heart structure segmentation result. A single inference run on an NVIDIA GeForce RTX 3090 takes approximately 45.5 seconds per patient (containing several slices).

[0062] Based on the same technical concept, the embodiment also provides a cardiac medical image segmentation device based on decoupled representation and knowledge fusion, used to implement the aforementioned cardiac medical image segmentation method based on decoupled representation and knowledge fusion, such as... Figure 4 As shown, it includes: The feature extraction module is used to input cardiac medical image data into the basic model and expert model respectively, and extract basic features and expert features. The feature decoupling module is used to decompose basic features and expert features to obtain structural components and residual components. The structural components are used to characterize cross-domain stable anatomical structure information, and the residual components are used to characterize fine-grained information related to specific domains. The knowledge alignment module is used to align structural components across models and residual components across models using central kernel alignment constraints; and to apply orthogonal decoupling constraints to structural components and residual components within the same model. The adaptive re-fusion module is used to obtain the corresponding segmentation responses from the aligned structural components and residual components through the pixel decoder; and to perform adaptive re-fusion on the corresponding segmentation responses based on category condition information. The output module is used to input adaptive refusion features into the predictor to obtain cardiac medical image segmentation results.

[0063] Based on the same technical concept, the embodiment also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, performs the cardiac medical image segmentation method based on decoupled representation and knowledge fusion.

[0064] To demonstrate the effectiveness of the above technical solutions, a comparison of the 3D visualization results of different methods in the cardiac multi-structure segmentation task was conducted.

[0065] like Figure 5 As shown, the columns represent the left ventricle (LV), right ventricle (RV), left atrium (LA), right atrium (RA), myocardium (MYO), aorta (AO), and pulmonary artery (PA) in sequence; the rows represent the visualization segmentation results of ground truth annotation (GT), BiomedParse, full-scale fine-tuned BiomedParse (BiomedParse-FT), nnUNet, and the present invention (OODKA) in sequence.

[0066] In this test case, the un-fine-tuned BiomedParse failed to effectively segment most cardiac structures, with a Dice of 0.00 for LV, RV, LA, MYO, and PA, only reaching 0.26 and 0.74 for RA and AO, respectively. The fully fine-tuned BiomedParse segmentation results were significantly improved, with Dice of 0.95, 0.64, 0.85, 0.73, 0.89, 0.90, and 0.02 for LV, RV, LA, RA, MYO, AO, and PA, respectively, but significant undersegmentation of PA remained. The corresponding Dice for nnUNet were 0.95, 0.64, 0.75, 0.63, 0.83, 0.59, and 0.47, capable of segmenting the main structures, but boundary deviations and some missegmentation were still observed in the RA, AO, and PA regions. In this case, the Dice values ​​of OODKA in this invention are 0.94, 0.91, 0.94, 0.91, 0.93, 0.92, and 0.86, respectively. Although its LV result is slightly lower than that of BiomedParse-FT and nnUNet (0.95), it achieves higher results in this case on RV, LA, RA, MYO, AO, and PA, and the predicted regions are generally closer to the GT in terms of morphology and boundaries, especially in recovering slender structures such as PA more completely.

[0067] Furthermore, as shown in Table 1, the quantitative results of cardiac multi-structure segmentation using different methods under CT modal distribution are presented.

[0068] Table 1

[0069] Table 1 shows the Dice coefficients and average values ​​for each method on LV, RV, LA, RA, Myo, AO, and PA. Out-of-distribution partitioning refers to splitting the dataset across centers, where the training and test data do not contain cases from the same center.

[0070] Since the training and test sets did not include cases from the same center, the experimental setup in Table 1 mainly examines the model's segmentation performance when faced with differences in imaging equipment, scanning protocols, and data distribution between different centers. The untuned BiomedParse had an average Dice of only 0.127, with LV, RV, Myo, and PA all at 0, and LA at only 0.006. It only reached 0.231 and 0.651 on RA and AO, respectively, indicating that its pre-training capability cannot directly adapt to current CT cardiac multi-structure segmentation tasks. After full fine-tuning, the average Dice of BiomedParse-FT increased to 0.645, with LV, RV, LA, RA, Myo, and AO reaching 0.841, 0.629, 0.687, 0.694, 0.789, and 0.824, respectively. However, PA was only 0.051, indicating that its segmentation of complex structures such as the pulmonary artery is still insufficient.

[0071] The average Dice of nnUNet is 0.811, with 0.895 and 0.864 on LV and Myo, respectively, which are the highest values ​​among all methods. Meanwhile, its RV, LA, and RA reach 0.809, 0.887, and 0.818, respectively, showing good overall performance. However, the Dice on AO and PA are 0.786 and 0.620, respectively, significantly lower than its results on cardiac chambers and myocardial structures, indicating that it still faces certain difficulties in segmenting large vessel structures under transcentric conditions.

[0072] OODKA achieved the highest average Dice of 0.871, an improvement of 0.060 over nnUNet. Specifically, OODKA's Dice on RV, LA, RA, AO, and PA were 0.811, 0.920, 0.840, 0.960, and 0.845, respectively, all the highest results in the table; on LV and Myo, it reached 0.884 and 0.841, respectively, which, although slightly lower than nnUNet's 0.895 and 0.864, still remained at a high level. OODKA's main improvement over nnUNet was in AO and PA, with improvements of 0.174 and 0.225, respectively, indicating that it has better segmentation performance for large vascular structures such as the aorta and pulmonary artery.

[0073] Looking at the distribution of results across different structures, OODKA achieved a Dice higher than 0.81 for all categories, with the lowest value being 0.811 for RV, while the lowest value for nnUNet was 0.620 for PA, and the lowest value for BiomedParse-FT was only 0.051. Therefore, OODKA not only achieved a higher average Dice but also showed a more balanced performance across different cardiac structures.

[0074] In addition, Table 2 shows the quantitative results of cardiac multi-structure segmentation using different methods under MRI modality distribution.

[0075] Table 2

[0076] Table 2 shows the Dice coefficients and average values ​​for each method on LV, RV, LA, RA, Myo, AO, and PA in MRI modalities.

[0077] Table 2 shows that, under the MRI modality, the average Dice of BiomedParse and BiomedParse-FT were 0.458 and 0.473, respectively, with limited overall improvement. Specifically, BiomedParse performed better on LA and RA, with Dice of 0.845 and 0.755, respectively, but only 0.008, 0.264, and 0.099 on Myo, AO, and PA. After full fine-tuning, PA increased to 0.411, but structures such as LV and RV showed a decrease.

[0078] The average Dice of nnUNet was 0.599, with relatively balanced results across different structures, reaching 0.596 for Myo and 0.482 for AO. OODKA achieved the highest average Dice of 0.670, with values ​​of 0.849, 0.630, 0.871, 0.795, and 0.668 for LV, RV, LA, RA, and Myo, respectively, all of which were the highest results. The PA Dice was 0.417, slightly higher than nnUNet's 0.411, with only the AO Dice of 0.462 slightly lower than nnUNet. Therefore, OODKA performed best overall in the MRI modality, especially showing a significant advantage in cardiac chambers and myocardial structures.

[0079] To verify the decoupling effect between the structural component p and the residual component s in OODKA, PCA was used to perform dimensionality reduction and visualization of the two types of features. The results are as follows: Figure 3As shown, the structural component p and the residual component s form two cluster regions with relatively clear boundaries and little overlap in the two-dimensional projection space, indicating that the two types of components have good feature separability. This result shows that under the action of orthogonal constraints and complementary constraints, the redundant correlation between the structural component and the residual component is suppressed, and the two types of components can represent different types of information respectively, thus forming a specific functional division.

[0080] In summary, the method provided by this invention has the following effects: By decoupling the features of the base model and the expert model through structure and residuals, the domain-invariant anatomical information and the domain-related difference information are explicitly separated, thereby improving the representation stability during cross-domain transfer.

[0081] By using component-aware knowledge fusion, cross-model alignment is performed only on components of the same type, avoiding the overall injection of domain bias information from expert models into the base model and reducing negative transfer.

[0082] By using orthogonal and complementary constraints, the redundant coupling between structural components and residual components is reduced, allowing the two types of components to form a functional division of labor.

[0083] Adaptive refusion guided by category conditions enables different categories to dynamically choose whether to rely more on structural components or residual components at different spatial locations, thereby improving the segmentation accuracy of complex distribution regions.

[0084] By freezing the backbone network and training only lightweight adaptation modules, the task-specific knowledge of expert models can be absorbed while retaining the general capabilities of the basic model, thereby reducing the risk of catastrophic forgetting and improving deployment efficiency.

[0085] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cardiac medical image segmentation method based on decoupled representation and knowledge fusion, characterized in that, Includes the following steps: S1. Cardiac medical image data are input into the basic model and expert model respectively, and basic features and expert features are extracted; S2. Decompose the basic features and expert features to obtain structural components and residual components. The structural components are used to characterize the anatomical structural information that is stable across domains, and the residual components are used to characterize the fine-grained information that is related to the specific domain. S3. Employ center kernel alignment constraints to align structural components across models and residual components across models; In addition, orthogonal decoupling constraints are applied to the structural components and residual components within the same model; S4. The aligned structural components and residual components are respectively processed by the pixel decoder to obtain the corresponding segmentation responses; Adaptive re-fusion is performed on the corresponding segmentation responses based on category condition information to obtain cardiac medical image segmentation results.

2. The cardiac medical image segmentation method based on decoupled representation and knowledge fusion according to claim 1, characterized in that, The aforementioned basic features are obtained by inputting cardiac medical image data into the basic model encoding, and are used to retain basic medical multimodal knowledge; The expert features are obtained by inputting cardiac medical image data into an expert model for encoding, and are used to identify lesion details and boundaries.

3. The cardiac medical image segmentation method based on decoupled representation and knowledge fusion according to claim 1, characterized in that, The method of decoupling and decomposing multi-scale features to obtain structural components and residual components includes: using a dual-branch parallel channel projection structure, decomposing each scale feature in the corresponding multi-scale features through convolutional layers to generate structural components and residual components for that scale.

4. The cardiac medical image segmentation method based on decoupled representation and knowledge fusion according to claim 3, characterized in that, During decoupling and decomposition, a reconstruction loss constraint is introduced to ensure that the sum of the decomposed structural components and residual components approximates the multi-scale features before decomposition. The calculation formula is as follows: ， In the formula, To reconstruct the loss constraints; For the first Input features at each scale; For the result component, For residual components, Denotes the square of the Frobenius norm. Representation layer normalization, It is a numerically stable term.

5. The cardiac medical image segmentation method based on decoupled representation and knowledge fusion according to claim 1, characterized in that, Cross-model alignment of structural components and cross-model alignment of residual components are performed, including: The center kernel alignment similarity loss is used to measure the representation consistency between the structural components on the base model side and the structural components on the expert model side, as well as the representation consistency between the residual components on the base model side and the residual components on the expert model side, and knowledge alignment is performed with the goal of minimizing the representation consistency. The knowledge alignment loss is expressed as: In the formula, For center kernel alignment similarity loss, and These represent the basic model at the 1st... Aligned structural components and residual components at each scale and These represent the expert model at the 1st and 2nd stages, respectively. Aligned structural components and residual components at each scale.

6. The cardiac medical image segmentation method based on decoupled representation and knowledge fusion according to claim 1, characterized in that, The applied orthogonal decoupling constraint is expressed as : ， In the formula, This represents the average of the absolute values ​​of each element in the cross-correlation matrix of the two sets of features; and These represent the basic model at the 1st... Aligned structural components and residual components at each scale and These represent the expert model at the 1st and 2nd stages, respectively. Aligned structural components and residual components at each scale.

7. The cardiac medical image segmentation method based on decoupled representation and knowledge fusion according to claim 1, characterized in that, The adaptive re-fusion of the corresponding segmentation responses based on category condition information includes: Category conditional query vectors are generated based on category embeddings; the category embeddings are frozen category semantic embeddings, and the category conditional query vectors are obtained by adding the frozen category semantic embeddings to the learnable query offsets; Using the multi-scale encoded features of the expert model as keys and values, the category condition query vector is modulated through a multi-head attention mechanism to obtain the category condition descriptor; The category condition descriptor is input into a multilayer perceptron to generate adaptive gating weights; The residual component segmentation response of the structural component segmentation response domain is re-fused based on adaptive gating weights. The re-fusion is expressed as follows: ,in, For category condition descriptors, and The first Structural component segmentation response and residual component segmentation response at each scale The adaptive gating weights are defined as follows.

8. The cardiac medical image segmentation method based on decoupled representation and knowledge fusion according to claim 1, characterized in that, During the training phase, all parameters of the base model and the expert model are frozen, and training is performed only on the parameters involved in feature decomposition, cross-model alignment, and adaptive re-fusion. During training, the total loss function is expressed as follows: ,in, To divide the loss, To reconstruct the loss constraints, For orthogonal decoupling constraints, For knowledge alignment loss, To reconstruct the loss constraint weights, For orthogonal decoupling constraint weights, For knowledge alignment loss weights, This represents the total number of scales.

9. A cardiac medical image segmentation device based on decoupled representation and knowledge fusion, characterized in that, The method for implementing the cardiac medical image segmentation method based on decoupled representation and knowledge fusion as described in any one of claims 1-8 includes: The feature extraction module is used to input cardiac medical image data into the basic model and expert model respectively, and extract basic features and expert features. The feature decoupling module is used to decompose basic features and expert features to obtain structural components and residual components. The structural components are used to characterize cross-domain stable anatomical structure information, and the residual components are used to characterize fine-grained information related to specific domains. The knowledge alignment module is used to align structural components across models and residual components across models using central kernel alignment constraints; and to apply orthogonal decoupling constraints to structural components and residual components within the same model. The adaptive re-fusion module is used to obtain the corresponding segmentation responses from the aligned structural components and residual components through the pixel decoder; and to perform adaptive re-fusion on the corresponding segmentation responses based on category condition information. The output module is used to input adaptive refusion features into the predictor to obtain cardiac medical image segmentation results.

10. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it performs the cardiac medical image segmentation method based on decoupled representation and knowledge fusion as described in any one of claims 1-8.

Images