Model training method, knowledge distillation-based unsupervised domain adaptive heart image segmentation method and device

Abstract

The invention provides a model training method and an unsupervised domain adaptive heart image segmentation method and device based on knowledge distillation, and the method comprises the steps: obtaining and preprocessing multi-modal heart MRI images of a plurality of sample patients, and obtaining a source domain image and a target domain image; constructing an unsupervised domain adaptive heart image segmentation model based on knowledge distillation, and inputting the source domain image and the target domain image into the unsupervised domain adaptive heart image segmentation model; based on knowledge self-distillation, implicit feature alignment is carried out in an encoder, and meanwhile explicit feature alignment is carried out on the top layer of the decoder based on a Wasserstein distance; in the forward propagation process, feature extraction is performed on a source domain image, multi-scale features are captured, and structural information migration between a source domain and a target domain is realized; in the direction propagation process, the model parameters are adjusted by minimizing the total loss of the model, and the target unsupervised domain adaptive heart image segmentation model is output.

Description

A model training method, an unsupervised domain-adaptive cardiac image segmentation method and apparatus based on knowledge distillation Technical Field

[0001] This application relates to the fields of medical image processing, cardiac image segmentation, and domain adaptation, and in particular to a model training method, an unsupervised domain adaptation cardiac image segmentation method and apparatus based on knowledge distillation. Background Technology

[0002] Currently, the accuracy of myocardial segmentation is highly dependent on the imaging diagnosis and surgical decisions for cardiac diseases in clinical practice. In recent years, late gadolinium-enhanced cardiac magnetic resonance imaging (LGE CMR) has played an irreplaceable role in the assessment of cardiac diseases such as myocardial infarction. Accurate myocardial segmentation is crucial for the identification and quantitative analysis of scar tissue, for disease diagnosis, and for the development of personalized treatment plans. Deep neural networks provide strong technical support for automated segmentation. However, the reliability of LGE CMR segmentation models is highly dependent on the sufficiency and annotation quality of training data. Due to the significant modal differences between LGE CMR and sequences such as T2-weighted (T2) and equilibrium steady-state free precession (bSSFP), models trained directly on one modality are difficult to generalize to other modalities. Furthermore, due to limitations imposed by imaging equipment and technology, sample annotations are generally scarce. Existing segmentation methods struggle to balance high-resolution structural boundaries with cross-modal adaptability, often resulting in segmentation results with high noise, discontinuous boundaries, and weak generalization ability, which affects quantitative disease analysis and diagnostic decisions.

[0003] To alleviate the performance bottleneck caused by insufficient new domain annotation, unsupervised domain adaptation (UDA) technology has emerged, improving the transfer and generalization capabilities of models to some extent. However, current unsupervised segmentation techniques for medical images, especially cross-modal cardiac MRI, still have the following problems:

[0004] First, it is difficult to maintain the consistency of anatomical structure in cross-domain transformation: existing style transfer methods based on generative adversarial networks often destroy the anatomical structure and clinical semantics in the original medical images when realizing cross-domain image transformation, resulting in blurred or erroneous segmentation results that are unacceptable for clinical diagnosis.

[0005] Second, the feature alignment granularity is coarse and lacks deep semantic constraints: The explicit distribution distance measures adopted by most scholars, such as maximum mean square loss and Sliced ​​Wasserstein Discrepancy, although they reduce the distribution difference between the source domain and the target domain to a certain extent, are not enough to accurately express and preserve high-dimensional, fine-grained medical structures, resulting in the loss of segmentation details and boundary information.

[0006] Third, traditional adversarial training processes are unstable and model convergence is difficult: Existing unsupervised domain adaptation methods typically employ an end-to-end joint training model, with the segmentation network and the discriminator network being updated simultaneously. Because the objective functions of the two networks conflict, this can easily lead to training oscillations, gradient vanishing, or mode collapse, making it difficult for the model to converge to the optimal solution. Furthermore, these methods are extremely sensitive to hyperparameters and are difficult to reproduce.

[0007] Fourth, there is a lack of dedicated knowledge distillation mechanisms for medical structures: existing knowledge distillation methods mostly focus on natural image tasks and lack targeted design for unsupervised domain adaptive segmentation of medical images. They fail to fully utilize the fine-grained cardiac structure knowledge obtained by the teacher model during source domain training and transfer it to the target domain, resulting in limited target segmentation accuracy.

[0008] Fifth, the training and inference efficiency of the model is low: some methods have high network complexity and consume a lot of computing resources when achieving cross-domain feature alignment and structure preservation, which is not conducive to the real-time or efficient intelligent diagnosis needs in clinical settings. Summary of the Invention

[0009] In view of the above problems, this application provides an unsupervised domain adaptive cardiac image segmentation method and apparatus based on knowledge distillation. It aims to address the problem that existing segmentation methods have difficulty accurately capturing the boundaries of complex cardiac structures. By fusing cross attention-driven knowledge distillation (CAKD) and Wasserstein distance for multimodal cardiac MRI segmentation, it significantly improves the integrity and generalization ability of structural representation in multimodal unsupervised domain adaptive (UDA) cardiac segmentation tasks, and accurately achieves structural preservation and distribution transfer of cross-modal cardiac MRI images.

[0010] In a first aspect, embodiments of this application provide a model training method, the method comprising:

[0011] Multimodal cardiac MRI images of multiple patients were acquired and preprocessed to obtain standardized source domain and target domain images;

[0012] An unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation is constructed, and standardized source and target domain images are input into the model.

[0013] A phased adversarial training strategy was adopted to adjust the parameters of an unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation;

[0014] An unsupervised adaptive cardiac image segmentation model based on knowledge distillation is developed, using the model after the last parameter adjustment as the trained target.

[0015] Optionally, the unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation uses a U-Net network containing an encoder and a decoder as the backbone network, with the decoder acting as a feature extractor and the decoder acting as a segmenter; and introduces a cross-attention feature knowledge distillation module and a Wasserstein distance distribution knowledge distillation module on the basis of the U-Net backbone network; a discriminator module is also set at the back end of the backbone network.

[0016] Optionally, the cross-attention feature knowledge distillation module is located at the end of the feature extractor. It uses the source domain features extracted by the feature extractor as the teacher and the target domain features as the student. Based on the cross-attention mechanism, it is used to calculate the feature differences between the source domain and target domain feature maps in the latent space to achieve implicit feature alignment.

[0017] The Wasserstein distance distribution knowledge distillation module, located at the segmenter output, uses the source domain image as the teacher and the target domain image as the student. It calculates the Wasserstein distance between the predicted distributions of the source domain and the predicted distributions of the target domain to achieve explicit distribution alignment in the global output space.

[0018] The discriminator module receives the segmentation prediction results from the source and target domains and uses an adversarial learning mechanism to distinguish the domain affiliation of the input samples, thereby constraining the segmenter to generate prediction results with domain invariance.

[0019] Optionally, the step of adjusting the parameters of the knowledge distillation-based unsupervised adaptive cardiac image segmentation model using a phased adversarial training strategy includes:

[0020] Phase 1: Keep the discriminator parameters frozen, use the feature extractor to extract features from the source and target domain images, and perform implicit feature alignment at the encoder end through the cross-attention feature knowledge distillation module. At the decoder end, perform explicit distribution alignment through the Wasserstein distance distribution knowledge distillation model, and combine the reconstruction path to achieve structural information transfer. In this phase, the parameters of the feature extractor and segmenter are updated by minimizing the joint loss and using the backpropagation algorithm.

[0021] Phase 2: Keep the parameters of the feature extractor and segmenter frozen, use the discriminator to perform adversarial discrimination on the prediction results of the source domain and the target domain, calculate the classification loss of the discriminator for the source domain and target domain samples, and update the discriminator parameters by minimizing the classification loss and using the error backpropagation algorithm.

[0022] Optionally, the minimized joint loss is the sum of the source domain segmentation loss, the reconstruction loss between the source and target domains, and the domain distribution difference loss, wherein the domain distribution difference loss is the sum of the cross-attention feature knowledge distillation loss and the Wasserstein distance distribution knowledge distillation loss.

[0023] Optionally, the cross-attention feature knowledge distillation loss for:

[0024]

[0025] in, For cross-attention distillation loss, Distillation loss is a domain-specific feature. To segment the loss term.

[0026] Optionally, the knowledge distillation loss of the Wasserstein distance distribution is:

[0027]

[0028] in, Let be the probability distribution of the source domain. The probability distribution of the target domain; It represents the "transfer amount" from the source domain to the destination domain, and is used to measure the degree of matching between the source and destination domains; Represents the transmission cost function; This represents a regularization parameter to ensure the smoothness of the transmission plan; This represents the joint probability distribution between the source domain distribution and the target domain distribution; and: ,in, Let be the probability mass of the i-th sample in the target domain. Let be the probability mass of the j-th sample in the source domain.

[0029] Secondly, embodiments of this application provide an unsupervised domain-adaptive cardiac image segmentation method based on knowledge distillation, the method comprising:

[0030] Acquire multimodal cardiac MRI images of the target patient;

[0031] Multimodal cardiac MRI images of the target patient are input into a target unsupervised adaptive cardiac image segmentation model to automatically segment key cardiac structures; the target unsupervised adaptive cardiac image segmentation model is obtained by the model training method described above.

[0032] Output a segmentation mask for the multimodal cardiac MRI image of the target patient.

[0033] Thirdly, embodiments of this application provide a model training apparatus, the apparatus comprising:

[0034] The preprocessing module is used to acquire and preprocess multimodal cardiac MRI images from multiple patients to obtain standardized source and target domain images.

[0035] The model building module is used to construct an unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation, and inputs standardized source domain images and target domain images into the unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation.

[0036] The training module is used to adjust the parameters of the knowledge distillation-based unsupervised domain-adaptive cardiac image segmentation model using a phased adversarial training strategy.

[0037] The output module is used to train an unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation, using the model after the last parameter adjustment as the target.

[0038] Fourthly, embodiments of this application provide an unsupervised domain-adaptive cardiac image segmentation device based on knowledge distillation, the device comprising:

[0039] The input module acquires multimodal cardiac MRI images of the target patient;

[0040] The segmentation module is used to input the multimodal cardiac MRI images of the target patient into the target unsupervised adaptive cardiac image segmentation model to perform automated segmentation of key cardiac structures; the target unsupervised adaptive cardiac image segmentation model is obtained by the model training method described above;

[0041] The output module outputs a segmentation mask of the multimodal cardiac MRI image of the target patient.

[0042] Compared with the prior art, the specific beneficial effects of the present invention are as follows:

[0043] First, this invention constructs a network architecture based on UDA-VAE, which utilizes the reconstruction mechanism of the source and target domains in the feature space to effectively alleviate the problem of loss of cardiac anatomical structure that may occur during cross-domain style transfer, and ensures the structural stability of the segmentation results in terms of morphological integrity and boundary continuity.

[0044] Secondly, this invention innovatively implements a dual alignment machine combining implicit and explicit methods. By introducing cross-attention feature distillation at the encoder, fine-grained implicit feature alignment in the latent space is achieved; simultaneously, by introducing Wasserstein distance distillation at the decoder, explicit alignment of the global prediction distribution in the output space is achieved. This inside-out deep alignment strategy significantly improves the model's generalization ability and representational accuracy in the target domain.

[0045] Furthermore, this invention employs a unique phased adversarial training strategy combined with a parameter freezing mechanism. By decoupling the optimization processes of the segmenter and discriminator, and using the joint loss function of the segmentation phase and the discriminative loss to alternately guide model updates, it effectively solves the problems of convergence difficulty and training instability in traditional adversarial training. This reduces computational resource consumption while improving the training efficiency and convergence quality of the model.

[0046] Finally, this invention significantly enhances the model's environmental adaptability and robustness through the above mechanism, enabling it to be widely applied to cardiac medical image segmentation tasks in different centers and imaging modalities. It effectively improves the dependence of existing models on a large amount of labeled data and solves the overfitting problem in small sample scenarios.

[0047] In summary, this invention, through a UDA-VAE architecture combined with dual knowledge distillation and phased adversarial training, systematically solves the technical challenges of structural information loss, insufficient feature alignment, and unstable training in unsupervised cross-modal cardiac image segmentation. It achieves more refined, stable, and efficient automated cardiac segmentation, meeting the high standards required for clinical auxiliary diagnosis and personalized medicine. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 is a structural framework diagram of the CAKD network proposed in this invention.

[0050] Figure 2 is a flowchart of the detailed feature distillation and Cross-Attention mechanism.

[0051] Figure 3 is a visual comparison of the segmentation results of our method and the state-of-the-art method on the MS-CMRSeg 2019 dataset, where LV, RV and Myo are represented by blue, green and yellow, respectively.

[0052] Figure 4 is a visual comparison of the segmentation results of our method and the state-of-the-art methods on the MyoPS 2020 dataset, where LV, RV and Myo are represented by blue, green and yellow, respectively. Detailed Implementation

[0053] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0054] Example 1:

[0055] This invention proposes a multimodal cardiac image segmentation method based on Cross-Attention and Wasserstein Knowledge Distillation (CAWKD). By fusing the Cross-Attention Knowledge Distillation (CAWKD) module with the Wasserstein Distance Distillation (WDD) module, this method significantly improves the structural representation integrity and generalization ability in multimodal unsupervised domain adaptive (UDA) cardiac segmentation tasks. This invention fully utilizes the information connection between source domain images (e.g., bSSFP, T2) and target domain images (e.g., LGE) sequences to achieve more refined segmentation of key cardiac structures such as the myocardium, providing a solid foundation for clinical diagnosis and quantitative analysis of cardiac diseases.

[0056] This invention uses the UDA-VAE framework as the basic method for style transfer. Within the context of UDA, the labeled source domain is represented as... Unlabeled target domains are represented as The main goal of UDA is to utilize and The model is trained and its performance is evaluated on the target domain. The VAE technique maximizes the likelihood function, thereby... and Mapped to a shared low-dimensional latent space In the middle. Its general objective function is expressed as:

[0057] (1)

[0058] The key task is to minimize the KL divergence. As a core component of transfer learning, the Kullback-Leibler divergence primarily measures the difference between the latent spatial distributions of the source and target domains, and can be expressed as:

[0059] (2)

[0060] The KV injection mechanism has been widely applied in several key areas of multimodal image segmentation, particularly in cross-modal feature fusion, image style transfer, and fine-grained object segmentation. The core function of the attention mechanism is to calculate the similarity between the query (Q) and the key (K), generating a weight matrix. This weight matrix is ​​then used to perform weighted aggregation on the values ​​(V), expressed as:

[0061] (3).

[0062] Furthermore, this embodiment provides a model training method, which includes the following steps:

[0063] Step 1: Acquire multimodal cardiac MRI images from multiple sample patients and preprocess them to obtain standardized source domain images and target domain images;

[0064] Specifically, based on publicly available datasets such as MS-CMRSeg 2019 and MyoPS 2020, multimodal cardiac MRI images of multiple patient samples were acquired, including bSSFP, LGE, and T2 sequence cardiac MRI images. A specific type of multimodal cardiac MRI image (e.g., bSSFP, T2) was used as the source domain image, and an LGE sequence cardiac MRI image was used as the target domain image. Both the source and target domain images were then preprocessed. The preprocessing included: resampling the acquired multimodal cardiac MRI images and labels at a uniform resolution / voxel size, removing irrelevant regions, and standardizing the intensity distribution; simultaneously, acquiring the source domain training samples and their corresponding anatomical structure labels, as well as the label information for the target domain validation samples. These labels covered key anatomical regions such as the left ventricle (LV), right ventricle (RV), and myocardium (Myo).

[0065] Step 2: Construct an unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation, and input the standardized source domain image and target domain image into the unsupervised domain-adaptive cardiac image segmentation model;

[0066] The knowledge distillation-based unsupervised domain-adaptive cardiac image segmentation model employs a variational autoencoder-based unsupervised domain-adaptive architecture (UDA-VAE). Specifically, this model uses a U-Net network as the backbone network for feature extraction and image segmentation, comprising an encoder and a decoder.

[0067] In the U-Net architecture, the encoder acts as a feature extractor, and the decoder acts as a segmenter. A parameter-sharing mechanism (i.e., the source and target domain branches share the same set of encoder weights) is employed to extract deep semantic features from multimodal images. Based on the VAE remapping mechanism, the input image is mapped to an aligned latent space. To address the domain offset problem, this invention introduces a cross-attention feature knowledge distillation module and a Wasserstein distance distribution knowledge distillation module on top of the U-Net backbone network to ensure the performance of feature extraction and segmentation.

[0068] Furthermore, in the encoder stage, a Cross-Attention Knowledge Distillation (CAKD) module is embedded to implicitly shorten the intermediate layer features between the source and target domains using the cross-attention mechanism. At the connection between the latent space and the decoder, a Wasserstein Distance Distillation (WDD) module is introduced to explicitly constrain the global distribution distance between the source and target domains in the latent space. Specifically, considering the task requirements and the critical nature of feature transfer, CAKD is introduced in layers 2, 3, and 5 of the U-Net network encoder for implicit feature alignment, while WDD is used for explicit feature alignment at the top layer of the decoder.

[0069] Specifically, referring to Figures 1-2, in order to specifically optimize the latent space distribution difference metric Ldomai and achieve efficient domain alignment, this model introduces two key knowledge distillation modules:

[0070] (1) Cross-attention feature knowledge distillation module

[0071] The cross-attention feature knowledge distillation module, based on cross-attention mechanism and knowledge self-distillation, deeply fuses and aligns the semantic features of source and target domain images in the latent feature space;

[0072] To address the issues of target domain image features being easily assimilated by "source domain style" and resulting in incomplete representation of medical anatomical structures in traditional knowledge distillation, a fine-grained feature alignment loss function (Equation 8) is proposed. In a multimodal cardiac magnetic resonance imaging segmentation task, visual information from different modalities is effectively integrated through a KV injection mechanism, enhancing the model's ability to integrate and understand modality-specific features.

[0073] In practice, a cross-attention mechanism is used to enable the interaction between the semantic features of the source domain and the structural features of the target domain. Specifically, by mapping the image features of the target domain to keys and values ​​and mapping the image features of the source domain to query injection information, efficient cross-modal feature fusion is achieved.

[0074] In this framework, source and target domain data flow through a parameter-shared encoder. Through knowledge distillation, the semantic consistency of the source domain can be used to constrain the feature extraction process of the target domain, thereby ensuring that the target domain image retains accurate anatomical structural information in the latent space. Specifically, source and target domain images are simultaneously input into the parameter-shared U-Net encoder. The extracted source domain features serve as the teacher, and the target domain features serve as the student during training. Queries are extracted from the source domain encoded features, and keys and values ​​are extracted from the target domain encoded features. A cross-attention mechanism is used to interact with the features, thereby calculating the correlation between the source domain query and the target domain key-value pair. This forces the generated attention map to be consistent with the self-attention map of the target domain, thus causing the target domain feature distribution to converge towards the source domain, achieving feature alignment under unsupervised domain adaptation.

[0075] Align the source domain query with the target domain key / value, calculate their differences, and achieve structural information migration. The specific algorithm is as follows:

[0076] (4)

[0077] in, , , and , , These are the Query, Key, and Value characteristics of the source and target domains, respectively. This is the scaling factor.

[0078] The cross-attention distillation loss generated when the query from the source domain guides the key and value of the target domain to generate a cross-attention representation. for:

[0079] (5)

[0080] in, To utilize the cross-domain attention feature representation generated by the source domain query vector and the target domain key-value pair, This is a self-attention feature representation of the target domain generated using only the query vector and key-value pairs of the target domain itself.

[0081] Cross-attention domain feature distillation relies on a residual mechanism of difference between the source and target domains, focusing on the differences in domain features rather than directly analyzing the details of each domain feature. For different imaging modalities, such as cardiac MRI, this mechanism helps mitigate the ambiguity of cross-attention, thus ensuring a complete understanding of cardiac structural features. Therefore, the domain feature distillation loss produced by domain feature distillation... for:

[0082] (6)

[0083] Meanwhile, to ensure the effectiveness of cross-attention in segmentation tasks, a segmentation loss term is introduced. :

[0084] (7)

[0085] in, The Dice loss function, The segmentation labels are the actual anatomical structures corresponding to the source domain input image. Specifically, it refers to the segmentation loss of attention fusion results under cross-attention feature knowledge distillation.

[0086] Based on equations (5)-(7), the cross-attention feature knowledge distillation loss is obtained. :

[0087] (8)

[0088] As shown in Figure 2, cross-attention distillation, based on cross-attention mechanism (CA) loss and query-key-value (QKV) loss, promotes effective domain feature transfer by capturing representational differences and predicted semantic distribution differences in the latent space. Detailed distillation implicitly involves hidden features in the cross-attention process by measuring the differences between the source and target domain query vectors, ensuring that intrinsic differences in key structures such as the myocardium under special modalities are preserved while the target domain is style-transformed to the source domain. This module effectively alleviates common problems in cardiac structure segmentation such as boundary ambiguity and missed region detection. Compared to simple feature transfer, it significantly improves cross-modal structural reasoning capabilities, exhibits strong robustness to complex morphological changes, and ensures the complete transfer of details in cardiac MRI images.

[0089] (2) Wasserstein distance distribution knowledge distillation module

[0090] The Wasserstein distribution alignment module is used to align the distributions between source and target domain images based on the Wasserstein distance.

[0091] Traditional KL divergence is widely used in UDA-VAE to achieve distribution alignment between the source and target domains. However, while KL divergence has advantages in modeling intra-class variations, it has limitations in capturing inter-class relationships. Specifically, although the Softmax function in the backpropagation process does introduce the probabilistic influence of one class on other classes, this effect is usually weak and cannot fully reflect inter-class dependencies. Given the inherent limitations of KL divergence in accurately capturing cross-modal and cross-class differences when applied to UDA architectures, especially when dealing with complex clinical image data, this invention proposes using Wasserstein distance to optimize the knowledge transfer process from the target domain to the source domain.

[0092] Wasserstein distance measures the minimum cost required to "transfer" one probability distribution to another. Compared to the standard KL divergence, it provides a smoother and more stable optimization path, making it particularly suitable for alignment scenarios involving cross-modal and high distributional offsets. Multimodal medical image information is extracted based on cross-class comparisons and used as the standard for transfer guidance. Labeled source domain images are considered as teachers, and unlabeled target domain images are considered as students. The optimization process can be formally defined as follows:

[0093] (9)

[0094] in, Let be the probability distribution of the source domain segmentation results. The probability distribution of the target domain segmentation results; It represents the "transfer amount" from the source domain to the destination domain, and is used to measure the degree of matching between the source and destination domains; Represents the transmission cost function; This represents a regularization parameter to ensure the smoothness of the transmission plan; This represents the joint probability distribution between the source domain distribution and the target domain distribution (i.e., the elements in the optimal transmission scheme matrix). This represents the knowledge distillation loss from the Wasserstein distance distribution.

[0095] Formula (9) satisfies the following constraints: ,in, Let be the probability mass of the i-th sample in the target domain. Let be the probability mass of the j-th sample in the source domain.

[0096] Transmission cost function It relates to the similarity function IRT between two regions, and its expression is:

[0097] (10)

[0098] in, The parameters that control the sharpness of the IRT function, This is a feature similarity measurement function. For the first in the source domain space domain Feature vectors at each position For the target domain spatial domain, the first Feature vectors at each position;

[0099] Through the aforementioned optimal transmission mechanism (i.e., the mathematical calculation process and alignment strategy constructed by formulas (9) and (10)), the Wasserstein distance can accurately align high-level semantic features such as spatial domain and texture, effectively reducing the domain bias of image distribution under different modalities. Compared with traditional statistical matching methods, this mechanism avoids the serious structural distortion problem caused by simple "statistical matching," preserving the "style" and "structure" features of different modalities while achieving distribution alignment. In addition, the Wasserstein distance provides more stable gradient propagation and optimization convergence, especially showing significant advantages when dealing with high-dimensional feature distribution alignment.

[0100] This method significantly improves the model's generalization segmentation performance on target domain samples with very few labels. It achieves accurate transfer of source domain knowledge to the target domain through an optimized transfer plan, providing a more robust and effective distribution alignment strategy for cross-modal medical image segmentation.

[0101] Step 3: Adjust the parameters of the knowledge distillation-based unsupervised domain adaptive cardiac image segmentation model using a phased adversarial training strategy;

[0102] This application adopts a phased adversarial strategy: during the segmenter training phase, the discriminator parameters are kept frozen, and the segmenter is guided to achieve high accuracy and style consistency through a joint loss function, with the target domain segmentation accuracy, feature space reconstruction quality, and latent space distribution differences as the core indicators; during the discriminator training phase, the segmenter parameters are frozen, and the discriminant accuracy is used as an optimization signal to strictly optimize the discriminator parameters, so that it can improve the discriminability between the two domains while distinguishing the distributions of the two domains, thereby enhancing the overall performance of cross-domain tasks.

[0103] Optionally, step 3 specifically includes:

[0104] Phase 1 (Segmenter Training Phase): Keeping the discriminator parameters frozen, the source and target domain images are input into the network. Forward propagation is performed using the feature extractor and segmenter. Implicit feature alignment is performed at the encoder end through the cross-attention feature knowledge distillation module, and explicit distribution alignment based on Wasserstein distance is performed at the decoder end. Structural information transfer is achieved by combining the reconstruction path. In this phase, the source domain segmentation loss, the reconstruction loss between the source and target domains, and the domain distribution difference loss are calculated. By minimizing the joint loss, the parameters of the feature extractor and segmenter are updated using the backpropagation algorithm.

[0105] Wherein, the joint loss function L vae for:

[0106] L vae =L seg +L rec +L domain

[0107] In the formula, L seg The source domain segmentation loss is the segmentation loss of the backbone network's segmentation prediction result; L rec L is the reconstruction loss between the source and target domains. domain This is the domain distribution difference loss, which is also the cross-attention feature knowledge distillation loss. Knowledge of distillation loss relative to Wasserstein distance distribution sum;

[0108] Phase 2 (Discriminator Optimization): Keep the parameters of the feature extractor and segmenter frozen, use the discriminator to perform adversarial discrimination on the prediction results of the source domain and the target domain, calculate the classification discrimination loss of the discriminator for the source domain and the target domain samples, and update the discriminator parameters by minimizing the classification discrimination loss and using the error backpropagation algorithm to improve the discriminability of inter-domain features, so that it can accurately distinguish the distribution differences between the source domain and the target domain.

[0109] This application achieves high-quality cross-modal cardiac structure segmentation by using the CAWKD framework for joint optimization. The model can effectively adapt to the data distribution characteristics of the target domain, while maintaining the structural knowledge learned from the source domain and making full use of the visual feature information of the target domain.

[0110] Step 4: Use the model after the last parameter adjustment as the target for unsupervised domain adaptation of the cardiac image segmentation model;

[0111] Furthermore, the model after the last parameter adjustment is the model that performs best in segmentation during the verification phase.

[0112] Example 2: This example provides an unsupervised domain-adaptive cardiac image segmentation method based on knowledge distillation. The method includes the following steps:

[0113] Step 1: Acquire multimodal cardiac MRI images of the target patient;

[0114] Step 2: Input the multimodal cardiac MRI images of the target patient into the target unsupervised adaptive cardiac image segmentation model; the target unsupervised adaptive cardiac image segmentation model is obtained by the model training method described above;

[0115] Step 3: Output the segmentation mask of the multimodal cardiac MRI image of the target patient to achieve segmentation of key cardiac structures, including the left ventricle, right ventricle, and myocardium.

[0116] Experimental Case: To verify the effectiveness and feasibility of the proposed method, training and testing were performed on the MS-CMRSeg 2019 and MyoPS 2020 datasets, and the method was compared with current mainstream state-of-the-art methods, demonstrating that the proposed method exhibits better structural integrity and cross-modal robustness.

[0117] (1) Dataset and task settings

[0118] The MS-CMRSeg 2019 dataset contains 45 sets of multi-sequence cardiac magnetic resonance (CMR) images, covering bSSFP, LGE, and T2 modalities, and provides annotation information for the main cardiac structures. During training, 35 sets of bSSFP images with their annotations are used as the source domain, and 35 sets of unannotated LGE images are used as the target domain. Validation is based on 5 sets of annotated target domain images; the ground truth LGE labels are only used to validate model performance. During testing, an additional 40 sets of LGE images are used to evaluate segmentation accuracy and measure the algorithm's generalization ability on new data.

[0119] The MyoPS 2020 dataset: The segmentation task also focuses on key cardiac structures such as the left ventricle (LV), right ventricle (RV), and myocardium (Myo). It adopts a domain-adaptive segmentation setting from bSSFP to LGE to test the algorithm's cross-modal adaptability and accuracy.

[0120] (2) Experimental environment and model setup

[0121] This simulation experiment was conducted on an Ubuntu 18.04.1 system, with an Intel® Core™ i7-9800X CPU, 62GB of RAM, and an NVIDIA RTX3090 GPU. The development framework was PyTorch. The Adam optimizer was used during model training, with specific parameter settings including a batch size of 1, weight decay of 0.0005, and an initial learning rate of 0.0001. Online data augmentation operations such as random scaling and flipping were introduced to mitigate the risk of overfitting.

[0122] (3) Experiment content

[0123] Basic segmentation performance and comparative experiments: On the MS-CMRSeg 2019 and MyoPS 2020 public datasets, a comprehensive comparison was conducted with various publicly available state-of-the-art methods, including MSL, SWD, VarDA, RKD, PKT, GKLKD, RDiKL, PCL, RKDO, etc.

[0124] Visualization Validation: The segmentation prediction of the heart structure and the real labels are presented in an intuitive visualization across multiple scenarios, highlighting the method's ability to handle complex boundaries.

[0125] Ablation experiments: to verify the independent contribution of each key module of the model (such as CA Style and QKV constraints, Distillation mechanism, etc.) to the overall performance, and to verify the performance improvement brought about by combinatorial innovation.

[0126] (4) Evaluation criteria

[0127] In this experimental case, the quantitative metrics for segmentation performance are the Dice coefficient (Dice) and the Hausdorff distance (HD). The Dice coefficient measures the overlap between the predicted segmented region and the ground truth label (the higher the better). The Hausdorff distance measures the distance between the farthest point of the predicted and ground truth structural boundary; the lower the Hausdorff distance, the higher the precision of the boundary segmentation.

[0128]

[0129]

[0130] (5) Experimental results

[0131] Results on the MS-CMRSeg 2019 dataset

[0132] Table 1 compares the performance of this invention with mainstream methods in the bSSFP→LGE conversion task. The CAWKD method achieves an average Dice score of 84.49% and an HD of 5.60 mm for the three structures (LV, RV, and Myo), reaching the current best performance. CAWKD performs excellently for segmentation of all myocardial structures, especially surpassing SWD by approximately 5.7% in average Dice, and the model demonstrates superior predictive stability and generalization. Some comparative methods (RKD, SWD) have slightly better HD values ​​for individual structures (such as RV and Myo), but their overall performance is not significant and fluctuates considerably, while CAWKD exhibits a more balanced and robust overall performance.

[0133] Based on Figure 3, the segmentation visualization results show that the present invention is significantly superior to the comparative algorithm in terms of accurate reconstruction of the heart's anatomical structure, especially in terms of boundary smoothing and small structure recognition.

[0134] Table 1 compares the Dice and HD metrics of our method with those of the state-of-the-art methods on the MS-CMRSeg 2019 dataset (bSSFP → LGE).

[0135]

[0136] Results on the MyoPS 2020 dataset

[0137] Table 2 shows the performance of CAWKD in cross-modal segmentation from bSSFP to LGE, with an average Dice score of 80.00% and an HD of 6.18 mm, ranking first among all methods. CAWKD not only boasts the highest overall Dice score but also demonstrates significant advantages for complex structures (such as Myo), particularly excelling in handling segmentation contour smoothness and edge consistency. Although its HD in LV is slightly lower than MSL, the performance of MSL during testing is extremely unstable and heavily influenced by data. In contrast, CAWKD consistently maintains leading generalization ability across multiple tests, exhibiting no significant performance fluctuations, demonstrating the robustness of this invention.

[0138] Table 2 compares the Dice and HD metrics of our method with state-of-the-art methods on the MyoPS 2020 dataset (bSSFP→LGE).

[0139]

[0140] As shown in Figure 4, the CAWKD method has excellent characterization of complex dividing lines such as RV and myocardium, and effectively inhibits both over- and under-segmentation. It also significantly outperforms in noise reduction and smoothness.

[0141] Ablation test results

[0142] Tables 3 and 4 analyze the independent contributions of different key modules and their combinations, such as CA Style, QKV, and Cross-Attention. For example, using KL divergence distillation alone results in an average Dice of only 74.68%; however, using strategies such as Wasserstein distance, QKV distillation, and CA style distillation increases the average Dice to 78.42%, 76.78%, and 78.22%, respectively. After using the full module combination CAWKDDistillation, the average Dice reaches 80.00%, and the HD decreases to 6.18 mm, significantly outperforming the results of using each component individually, thus confirming the complementary gains of each innovative module.

[0143] Table 3 shows the ablation experimental results on the MS-CMRSeg 2019 dataset regarding the effectiveness of different components of CAWKD.

[0144]

[0145] Table 4 shows the ablation experiment results on the MyoPS 2020 dataset regarding the effectiveness of different components of CAWKD.

[0146]

[0147] Example 3: This example provides a model training device, the device comprising:

[0148] The preprocessing module is used to acquire and preprocess multimodal cardiac MRI images from multiple patients to obtain standardized source and target domain images.

[0149] The model building module is used to construct an unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation, and inputs standardized source domain images and target domain images into the unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation.

[0150] The training module is used to adjust the parameters of the knowledge distillation-based unsupervised domain-adaptive cardiac image segmentation model using a phased adversarial training strategy.

[0151] The output module is used to train an unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation, using the model trained with the last parameter adjustment as the target.

[0152] Example 4: This example provides an unsupervised domain-adaptive cardiac image segmentation device based on knowledge distillation. The device includes:

[0153] The input module acquires multimodal cardiac MRI images of the target patient;

[0154] The segmentation module is used to input the multimodal cardiac MRI images of the target patient into the target unsupervised adaptive cardiac image segmentation model to perform automated segmentation of key cardiac structures; the target unsupervised adaptive cardiac image segmentation model is obtained by the model training method of Example 1;

[0155] The output module outputs a segmentation mask of the multimodal cardiac MRI image of the target patient.

[0156] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0157] Finally, it should be noted that in this text, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0158] The above provides a detailed description of the model training method, the unsupervised domain-adaptive cardiac image segmentation method and apparatus based on knowledge distillation provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A model training method, characterized in that, The method includes: acquiring multimodal cardiac MRI images from multiple sample patients and preprocessing them to obtain standardized source domain images and target domain images; constructing an unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation, and inputting the standardized source domain images and target domain images into the unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation; adjusting the parameters of the unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation using a phased adversarial training strategy; and using the model after the last parameter adjustment as the trained target unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation.

2. The method according to claim 1, characterized in that, The unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation uses a U-Net network containing an encoder and a decoder as its backbone network, with the decoder acting as both a feature extractor and a segmenter. Furthermore, a cross-attention feature knowledge distillation module and a Wasserstein distance distribution knowledge distillation module are introduced on top of the U-Net backbone network. A discriminator module is also included at the back end of the backbone network.

3. The method according to claim 2, characterized in that, The cross-attention feature knowledge distillation module, located at the end of the feature extractor, uses the source domain features extracted by the feature extractor as the teacher and the target domain features as the student. Based on the cross-attention mechanism, it is used to calculate the feature differences between the source and target domain feature maps in the latent space, achieving implicit feature alignment. The Wasserstein distance distribution knowledge distillation module, located at the segmenter output, uses the source domain image as the teacher and the target domain image as the student. By calculating the Wasserstein distance between the source and target domain prediction distributions, it is used to achieve explicit distribution alignment in the global output space. The discriminator module receives the segmentation prediction results from the source and target domains and distinguishes the domain affiliation of input samples through an adversarial learning mechanism, thereby constraining the segmenter to generate prediction results with domain invariance.

4. The method according to claim 3, characterized in that, The proposed phased adversarial training strategy for adjusting the parameters of the knowledge distillation-based unsupervised domain-adaptive cardiac image segmentation model includes: Phase 1: Keeping the discriminator parameters frozen, the feature extractor extracts features from the source and target domain images, and performs implicit feature alignment at the encoder end through a cross-attention feature knowledge distillation module, and explicit distribution alignment at the decoder end through a Wasserstein distance distribution knowledge distillation model, combined with the reconstruction path to achieve structural information transfer. In this phase, the parameters of the feature extractor and segmenter are updated using the backpropagation algorithm by minimizing the joint loss; Phase 2: Keeping the feature extractor and segmenter parameters frozen, the discriminator performs adversarial discrimination on the prediction results of the source and target domains, calculates the discriminator's classification loss for the source and target domain samples, and updates the discriminator parameters using the error backpropagation algorithm by minimizing the classification loss.

5. The method according to claim 4, characterized in that, The minimized joint loss is the sum of the source domain segmentation loss, the reconstruction loss between the source and target domains, and the domain distribution difference loss, wherein the domain distribution difference loss is the sum of the cross-attention feature knowledge distillation loss and the Wasserstein distance distribution knowledge distillation loss.

6. The method according to claim 4, characterized in that, The cross-attention feature knowledge distillation loss for: in, For cross-attention distillation loss, Distillation loss is a domain-specific feature. To segment the loss term.

7. The method according to claim 4, characterized in that, The knowledge distillation loss of the Wasserstein distance distribution is: in, Let be the probability distribution of the source domain. Let be the probability distribution of the target domain; It represents the "transfer amount" from the source domain to the destination domain, and is used to measure the degree of matching between the source and destination domains; Represents the transmission cost function; This represents a regularization parameter to ensure the smoothness of the transmission plan; This represents the joint probability distribution between the source domain distribution and the target domain distribution; and: ,in, Let be the probability mass of the i-th sample in the target domain. Let be the probability mass of the j-th sample in the source domain.

8. An unsupervised domain-adaptive cardiac image segmentation method based on knowledge distillation, characterized in that, Includes the following steps: Acquire multimodal cardiac MRI images of the target patient; input the multimodal cardiac MRI images of the target patient into a target unsupervised adaptive cardiac image segmentation model to perform automated segmentation of key cardiac structures; the target unsupervised adaptive cardiac image segmentation model is obtained by the model training method described in any one of claims 1-7; output the segmentation mask of the multimodal cardiac MRI images of the target patient.

9. A model training device, characterized in that, The device includes: a preprocessing module for acquiring and preprocessing multimodal cardiac MRI images from multiple sample patients to obtain standardized source domain images and target domain images; a model building module for constructing an unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation, and inputting the standardized source domain images and target domain images into the unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation; a training module for adjusting the parameters of the unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation using a phased adversarial training strategy; and an output module for using the model after the last parameter adjustment as the trained target unsupervised domain-adaptive cardiac image segmentation model based on knowledge distillation.

10. An unsupervised domain-adaptive cardiac image segmentation device based on knowledge distillation, characterized in that, The device includes: an input module for acquiring multimodal cardiac MRI images of a target patient; a segmentation module for inputting the multimodal cardiac MRI images of the target patient into a target unsupervised adaptive cardiac image segmentation model for automated segmentation of key cardiac structures; the target unsupervised adaptive cardiac image segmentation model is obtained by the model training method as described in any one of claims 1-7; and an output module for outputting a segmentation mask of the multimodal cardiac MRI images of the target patient.

Images