Task-guided and domain-consistent cross-modal medical image synthesis method

Abstract

The application relates to the technical field of medical image processing and computer-aided diagnosis, and discloses a cross-modal medical image synthesis method based on task guidance and domain feature consistency. The method acquires an unpaired source domain and a target domain dataset, constructs a synthesis model containing a multi-scale generator, a double-view discriminator and a task guidance auxiliary diagnosis discriminator, generates a synthesis image and a reconstructed image through forward propagation, calculates an adversarial generation loss and a cycle consistency loss, simultaneously extracts an encoding layer feature, calculates a domain feature consistency loss by using Euclidean distance to constrain a semantic direction, fuses source domain and synthesis image features to input an auxiliary network, calculates a task guidance interaction loss and guides a generator optimization through gradient back propagation, and alternately updates model parameters based on a total objective function. The application effectively solves the problems of pathological information loss and semantic drift in cross-modal synthesis, and significantly improves the accuracy and practicality of the synthesis image in auxiliary diagnosis.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer-aided diagnosis technology, specifically a cross-modal medical image synthesis method based on task guidance and domain feature consistency. Background Technology

[0002] Early and accurate diagnosis of Alzheimer's disease (AD) and its prodromal stages is crucial for slowing disease progression and developing intervention strategies. In the current field of medical imaging analysis, the fusion of multimodal neuroimaging data has become a mainstream approach for building efficient and intelligent diagnostic frameworks. Structural magnetic resonance imaging (sMRI) provides high-resolution information on brain anatomy, such as hippocampal and cortical atrophy; while fluorodeoxyglucose positron emission tomography (FDG-PET) sensitively reflects neuronal functional status and glucose metabolism levels. These two modalities are highly complementary in information, and their combined use can significantly improve the diagnostic accuracy of neurodegenerative diseases.

[0003] However, obtaining complete and one-to-one corresponding multimodal paired data faces significant challenges in actual clinical applications. On the one hand, PET equipment is expensive and has a relatively low adoption rate, and the examination process involves radiation exposure, resulting in a natural scarcity of its data in clinical accumulation. On the other hand, due to patient attrition, contraindications to examination, or data quality issues, existing medical databases often exhibit modality gaps, meaning that many patients only have MRI data but lack corresponding PET data. This data incompleteness severely restricts the training and generalization of downstream multimodal assisted diagnostic models. To address this challenge, cross-modal medical image synthesis technology has emerged, aiming to utilize readily available modalities (such as MRI) to generate missing modalities (such as PET) to complete the data chain.

[0004] While deep learning-based generative adversarial networks (GANs) have made some progress in cross-modal synthesis, their application in medical imaging remains significantly limited. Traditional cross-modal synthesis methods often heavily rely on large-scale paired data, requiring images of the same subject in both modalities simultaneously, which is highly impractical given the time-consuming and resource-intensive nature of clinical data collection. Although non-paired synthesis methods based on cycle consistency reduce the stringent requirements for data pairing, their reliance on edge distributions for mapping makes them highly susceptible to feature erosion during complex nonlinear transformations, leading to the loss of details in fine anatomical structures of the brain and making it difficult to preserve domain-invariant features between modalities.

[0005] More critically, existing synthesis methods generally suffer from a deficiency in prioritizing visual representation over pathological semantics. Most generative models focus solely on enhancing the visual realism of synthesized images, making them texturally similar to real PET images, while neglecting the preservation of deep pathological semantics. In AD diagnosis, subtle pathological changes, such as specific areas of reduced metabolism, are crucial for disease identification. Generators lacking pathological constraints cannot ensure the accurate preservation of these diagnostically discriminative features during cross-modal transformations. This results in visually realistic images that lack practical value in clinical diagnosis and may even produce misleading artifacts. Furthermore, because the generation task and downstream diagnostic task are often disconnected, the generative model cannot perceive which features are most important for diagnosis and lacks a task-oriented feedback mechanism to optimize generation quality. This also makes it difficult for synthesized images to meet the high standards required for clinical auxiliary diagnosis. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a cross-modal medical image synthesis method based on task guidance and domain feature consistency, which solves the problems of anatomical structure distortion and loss of pathological features with clinical diagnostic value caused by semantic drift in existing unpaired cross-modal medical image synthesis technologies.

[0007] To achieve the above objectives, this invention provides a cross-modal medical image synthesis method based on task guidance and domain feature consistency. This method first acquires a source domain dataset containing unpaired first-modal medical images and a target domain dataset containing unpaired second-modal medical images. Subsequently, a cross-modal medical image synthesis model is constructed, which integrates a first generator, a second generator, a first discriminator, a second discriminator, and a task-guided assisted diagnostic discriminator.

[0008] During the model training phase, a first modality medical image is input into a first generator to generate a first synthesized image via forward propagation. This first synthesized image is then input into a second generator to generate a source domain reconstructed image. Similarly, the second and first generators are used to generate a second synthesized image and a target domain reconstructed image based on the second modality medical image. Building upon this, the present invention introduces domain feature consistency loss and task-guided interaction loss, combining them with adversarial generation loss and cyclic consistency loss to construct a total objective loss function. This total objective loss function is then used to iteratively train the synthetic model.

[0009] Regarding the construction and principle of the domain feature consistency loss, this invention introduces a distance constraint mechanism in the latent feature space of the generator. Specifically, the generator's preset coding layer extracts the original feature vector of the first modality medical image and the reconstructed feature vector of the source domain reconstructed image. Then, by calculating the L2 norm of the difference between these two feature vectors, the Euclidean distance between them in the feature space is obtained. This step uses this Euclidean distance as the optimization factor for minimizing the domain feature consistency loss. This mechanism forces the generator to ensure that the deep semantic features of the image are as close as possible in both numerical and spatial positions during image transformation and reconstruction, preventing semantic information deviation caused by unpaired training, and ensuring that the synthesized image maintains a high degree of consistency with the original image in terms of deep anatomical semantics.

[0010] Regarding the construction and principle of the task-guided interactive loss, this invention introduces a task-guided auxiliary diagnostic discriminator as a supervision module to establish an interaction mechanism between the generation task and the downstream diagnostic task. Specifically, the anatomical feature map of the first modality medical image and the metabolic feature map of the first synthesized image are extracted and fused along the channel dimension. The fused features are input into the task-guided auxiliary diagnostic discriminator to obtain the predicted probability distribution of the disease state. Based on this predicted probability distribution and the real disease label, the classification error is calculated using a multi-class cross-entropy function. The gradient generated by this classification error is passed back to the first generator through a backpropagation algorithm. This mechanism enables the generator to perceive the needs of the downstream diagnostic task during training, guiding it to focus on preserving or restoring pathological features with disease discriminative power (such as metabolic abnormalities in specific brain regions) when synthesizing images, thereby improving the clinical diagnostic value of the synthesized images.

[0011] In terms of network structure design, both the first and second generators of this invention employ a multi-scale encoder structure. This structure includes a first convolutional path and a second convolutional path set in parallel. The first convolutional path uses a larger-sized convolutional kernel to extract global spatial structure features, while the second convolutional path uses a smaller-sized convolutional kernel to extract local texture detail features. The two feature paths are concatenated along the channel dimension and then input into the downsampling module, thus taking into account both the feature extraction of macroscopic anatomical structures and microscopic tissue textures in medical images. Furthermore, the discriminator adopts a dual-view discrimination structure. It evaluates the realism of the entire image through a global discrimination branch and outputs a feature matrix through a local discrimination branch to evaluate the texture realism of local regions of the image, further improving the detail fidelity of the generated image.

[0012] Regarding the specific calculation of the loss function, the adversarial generation loss is calculated using the least squares loss function, which covers the mean square error of the discriminator in distinguishing between real and generated images, thus stabilizing the training process. The cycle consistency loss is calculated using the L1 norm distance formula, which quantifies the pixel-level absolute error between the source domain image and the source domain reconstructed image, and between the target domain image and the target domain reconstructed image, respectively, ensuring the reversibility of image transformation.

[0013] This invention provides a cross-modal medical image synthesis method based on task guidance and domain feature consistency. It has the following beneficial effects: 1. This invention constructs a task-guided interactive mechanism by introducing a task-guided auxiliary diagnostic discriminator. It uses a multi-class cross-entropy function to calculate the classification error and guides the generator optimization through gradient backpropagation. This mechanism forces the generator to retain pathological features with clinical diagnostic value during image synthesis, solving the problem that traditional generative adversarial networks only focus on visual realism, resulting in the loss of key pathological information. This significantly improves the practicality of synthesized images in disease screening and auxiliary diagnosis.

[0014] 2. This invention utilizes Euclidean distance to calculate the directional differences between feature vectors in the generator's encoding layer, constructing a domain feature consistency loss. This technique constrains the semantic directional consistency between the original and reconstructed images from a high-dimensional feature space, effectively suppressing the semantic drift phenomenon commonly seen in unpaired cross-modal conversion, and ensuring the stability and accuracy of the synthesized image in terms of anatomical structure and deep semantics.

[0015] 3. The present invention employs an encoder structure containing parallel multi-scale convolutional paths in the generator, and trains it in conjunction with a dual-view discriminant structure. Through the synergistic effect of large-size convolutional kernels extracting global structure and small-size convolutional kernels extracting local texture, combined with multi-scale supervision of global and local discriminant, the clarity and detail restoration capability of the synthesized image are effectively improved. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the overall structure of a non-paired cross-modal medical image synthesis method based on task guidance and domain feature consistency according to the present invention. Figure 3 This is a schematic diagram of a multi-scale encoder structure for implementing an unpaired cross-modal medical image synthesis method based on task guidance and domain feature consistency according to the present invention. Figure 4 This is a schematic diagram illustrating the bidirectional domain feature consistency constraint mechanism of a non-paired cross-modal medical image synthesis method based on task guidance and domain feature consistency, as implemented in this invention. Figure 5This is a bar chart comparing the generator performance in various implementation scenarios of the present invention. Figure 6 This is a performance comparison diagram for various implementation scenarios of the present invention.

[0017] in, Figure 2 This paper demonstrates the overall workflow of the cross-modal medical image synthesis model proposed in this invention, based on task guidance and domain feature consistency. Input data includes RealMRI (first modality image / real MRI) as source domain data and RealPET (second modality image / real PET) as target domain data. During generation, G (Generator) contains an Encoder and a Decoder, responsible for converting RealMRI into SynPET / FakePET (first synthesized image) or RealPET into SynMRI / FakeMRI (second synthesized image). To ensure cycle consistency, the model further generates RecMRI (source domain reconstructed image) and RecPET (target domain reconstructed image). Model training is constrained by various loss functions, including those designed to fool D (Discriminator). (Adversarial Loss, adversarial generation loss), used to maintain structural consistency. (CycleLoss, cycle consistency loss), and the core of this invention. (ConsistLoss, Domain Feature Consistency Loss). Furthermore, an Auxiliary Discriminator is introduced to classify images for diseases, outputting AD / CN (Disease Label / Classification Result), and calculating... (TaskLoss, task-guided interaction loss) to guide the generator to preserve pathological features.

[0018] Figure 3 The specific network structures of the generator and discriminator in this invention are shown in detail. The generator front end employs a Multi-scale Feature Encoder, which contains two parallel paths: one uses Conv7×7×7 (first convolutional path / 7×7×7 convolution) to extract global features, and the other uses Conv3×3×3 (second convolutional path / 3×3×3 convolution) to extract local details. After the features are processed by InstanceNorm (instance normalization) and ReLU / LeakyReLU (activation layers), they are then processed by Concat. The image is then fused using a combination of stitching and feature fusion. Subsequently, the features are processed through DownSampling, a deep ResNetBlock (residual module), and UpSampling to reconstruct the image. The discriminator employs a dual-view architecture, consisting of a GlobalDiscriminator branch for evaluating the overall image's realism, and a LocalDiscriminator branch for refining the discrimination of local image regions.

[0019] Figure 4 This demonstrates the principle of domain feature consistency. Figure 3 The computational principle of domain feature consistency loss in this invention is explained. This mechanism operates in the Latent Space, utilizing the encoder to extract the MRI-DomainFeature (source domain feature vector) of the input image and the PET-DomainFeature (target domain feature vector) of the reconstructed image. This feature consistency is quantified using the Euclidean distance formula, thereby ensuring the stability of deep anatomical semantics and pathological information during cross-modal transformation. Detailed Implementation

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] See attached document Figure 1 This invention provides a cross-modal medical image synthesis method based on task guidance and domain feature consistency, which may include the following steps: Obtain the source domain dataset and the target domain dataset. The source domain dataset contains multiple first-modality medical images, and the target domain dataset contains multiple second-modality medical images. In this embodiment, the first-modality medical images are magnetic resonance imaging (MRI) data, and the second-modality medical images are positron emission tomography (PET) scan data. The first-modality medical images in the source domain dataset and the second-modality medical images in the target domain dataset are not paired, meaning there is no one-to-one anatomical structure matching relationship between them. Preprocessing is performed on the source domain dataset and the target domain dataset, including removing non-brain tissue regions, intensity normalization, and image slice extraction.

[0022] See attached document Figure 2A cross-modal medical image synthesis model was constructed. This model includes a first generator, a second generator, a first discriminator, a second discriminator, and a task-guided diagnostic discriminator. The first generator establishes a mapping from the source domain to the target domain, converting the first-modal medical image into a synthesized second-modal medical image. The second generator establishes a mapping from the target domain to the source domain, converting the second-modal medical image into a synthesized first-modal medical image.

[0023] Both the first and second generators employ a network structure based on a multi-scale encoder. The multi-scale encoder contains convolutional paths of different sizes set in parallel to capture global structural features and local texture features, respectively. Both the first and second discriminators employ a dual-view discriminant structure, which includes a global discriminant branch for evaluating the realism of the entire image and a local discriminant branch for evaluating the realism of local image segments. A task-guided assisted diagnostic discriminator receives image features and performs a disease classification prediction task.

[0024] A forward propagation computation is performed on the cross-modal medical image synthesis model. A first modality medical image is randomly selected from the source domain dataset as input, and a first synthesized image is generated using a first generator. The first synthesized image is then input into a second generator to generate a source domain reconstructed image. A second modality medical image is randomly selected from the target domain dataset as input, and a second synthesized image is generated using a second generator. The second synthesized image is then input into the first generator to generate a target domain reconstructed image.

[0025] Calculate the domain feature consistency loss. Extract the feature vectors of the first modality medical image from the first generator's preset coding layer, and the feature vectors of the source domain reconstructed image from the corresponding coding layer of the first generator, then calculate the Euclidean distance difference between them. Similarly, extract the feature vectors of the second modality medical image from the second generator's preset coding layer, and the feature vectors of the target domain reconstructed image from the corresponding coding layer of the second generator, then calculate the Euclidean distance difference between them. The domain feature consistency loss is used to constrain the consistency of intra-domain feature representations during forward generation and reverse reconstruction.

[0026] Calculate the task-guided interaction loss. Extract the feature map of the first modality medical image and the feature map of the first synthetic image generated by the first generator. Perform channel concatenation or weighted fusion of the feature maps of the first modality medical image and the first synthetic image to obtain fused features. Input the fused features into the task-guided auxiliary diagnostic discriminator to obtain the predicted probability distribution for a specific disease category. Calculate the classification loss based on the predicted probability distribution and the corresponding real disease label of the image. The task-guided interaction loss is used to guide the first generator to retain pathological features with disease discriminative power by utilizing the gradient feedback of the classification task.

[0027] The adversarial generation loss and cycle consistency loss are calculated. The adversarial generation loss is calculated based on the discriminant outputs of the first discriminator on the second modality medical image and the first synthesized image, and the discriminant outputs of the second discriminator on the first modality medical image and the second synthesized image. It is used to constrain the generated image to conform to the data distribution of the target domain. The cycle consistency loss is calculated based on the pixel-level differences between the first modality medical image and the source domain reconstructed image, and the pixel-level differences between the second modality medical image and the target domain reconstructed image. It is used to constrain the generator to maintain the consistency of anatomical structures.

[0028] Based on adversarial generation loss, cycle consistency loss, domain feature consistency loss, and task-guided interaction loss, a total objective loss function is constructed by weighted summation. The total objective loss function is then used to iteratively train the first generator, second generator, first discriminator, second discriminator, and task-guided auxiliary diagnostic discriminator in the cross-modal medical image synthesis model. Network parameters are updated using backpropagation until the model converges or reaches the preset number of training epochs.

[0029] During the inference phase, a first modality medical image to be processed is acquired and subjected to the same preprocessing as in the training phase. The preprocessed first modality medical image is then input into the trained first generator, which outputs a synthesized second modality medical image. This synthesized second modality medical image is used to assist doctors in clinical diagnosis or as input data for downstream analysis algorithms.

[0030] To train the aforementioned cross-modal medical image synthesis model, this embodiment constructs an unpaired dataset containing source domain data and target domain data. The data acquisition and preprocessing process directly affects the training stability of the model and the quality of the final generated images. The specific implementation steps are as follows: This embodiment selects the Alzheimer's Disease Neuroimaging Project (ADNI) database as the data source. Source domain dataset The target domain dataset consists of T1-weighted magnetic resonance imaging (MRI) data. The dataset consists of 18F-FDG positron emission tomography (PET) data. Based on clinical diagnostic labels, the acquired image data were divided into two categories: Alzheimer's disease (AD) and cognitively normal (CN). To construct the unpaired training set, it was not required that the same subject possess both MRI and PET images; the subject identities in the source domain dataset and the target domain dataset could be independent.

[0031] Raw medical imaging data typically includes non-brain tissue structures such as the skull, scalp, and eyeballs. These structures are not significant for the diagnosis of neurodegenerative diseases and may introduce background noise that interferes with feature extraction by the generator. This embodiment utilizes a brain extraction tool (e.g., FSL-BET) to perform skull dissection on raw MRI and PET 3D volumetric data. By setting specific fractional anisotropy or gradient thresholds, a binarized brain mask is generated. The mask is then multiplied by the original image to remove background and non-brain tissue regions, retaining only the brain parenchyma.

[0032] Due to individual differences in brain anatomy among different subjects, and variations in head posture during scanning, it is necessary to map all images to a unified anatomical space. In this embodiment, all craniotomized MRI and PET images are registered to the standard MNI152 template space. Specifically, affine transformation is used for linear registration to correct the spatial orientation of the images. After registration, all 3D images are resampled to a uniform spatial resolution, such as 182×218×182 pixels, to ensure that the data input to the network has a consistent spatial dimension.

[0033] Off-field correction and intensity normalization: To address the common phenomenon of intensity non-uniformity in MRI images caused by magnetic field inhomogeneity, this embodiment uses the N4 off-field correction algorithm to process and eliminate the interference of low-frequency intensity gradients in the image.

[0034] Subsequently, to eliminate differences in pixel value ranges caused by different scanning devices or parameters, intensity normalization processing was performed on the images. For each 3D image... Calculate the maximum value of its pixel intensity. and minimum value A linear transformation is used to map the pixel intensity of the image to the interval [-1, 1]. The transformation formula is as follows: This normalization method matches the range of the hyperbolic tangent activation function in the generator output layer, which helps to accelerate network convergence.

[0035] Dataset partitioning: The processed data was divided into training, validation, and test sets according to the subject dimension. This ensured that the three-dimensional data of the same subject existed in only one set, avoiding inflated test results due to data leakage. The final training set included the source domain dataset. and target domain data set ,in and These represent the total number of source and target domains, respectively. and There is no need for a correspondence in anatomical structure and time.

[0036] See attached document Figure 3 This embodiment details the specific structural design of each sub-network in the cross-modal medical image synthesis model. The model mainly consists of a generator based on a multi-scale encoder, a dual-view discriminator, and a task-guided auxiliary network.

[0037] The first generator in this embodiment Second generator They have the same network topology. To overcome the problem that single-scale convolution cannot take into account both global anatomical contours and local texture details during feature extraction, the generator adopts a multi-branch parallel encoder design.

[0038] Specifically, the generator's encoding stage includes an input layer, a multi-scale feature extraction module, and a downsampling module. The input layer receives single-channel or multi-channel medical image slices. The multi-scale feature extraction module contains two parallel convolutional paths: the first path uses a large-size convolutional kernel (e.g., a 7×7 kernel) with a large receptive field to capture the global spatial structure and low-frequency semantic information of the image; the second path uses a small-size convolutional kernel (e.g., a 3×3 kernel) to capture high-frequency details such as local edges and textures of the image.

[0039] The feature maps extracted from the two paths are concatenated along the channel dimension, and the fused feature map then enters the downsampling module. The downsampling module consists of strided convolutional layers with a stride of 2, used to progressively reduce the spatial resolution of the feature maps and increase the number of channels. After each convolutional layer, an instance normalization layer and a non-linear activation function layer (e.g., ReLU) are sequentially connected.

[0040] Following the encoding stage, the network accesses a bottleneck layer containing multiple residual blocks. Each residual block contains two 3×3 convolutional layers and skip connections to maintain the stability of feature propagation in deep networks and prevent gradient vanishing.

[0041] The decoding stage consists of an upsampling module and an output layer. The upsampling module uses transposed convolution or bilinear interpolation combined with convolution to gradually restore the spatial resolution of the feature map to the size of the input image. The output layer uses a 7×7 convolutional layer to map the feature map back to the image space and uses the hyperbolic tangent function as the activation function to output a synthesized image with pixel values ​​in the range of [-1, 1] [-1, 1].

[0042] Dual-view discriminator construction: To simultaneously ensure the global structural realism and local texture clarity of the synthesized image, the first discriminator... Second discriminator All are configured as a dual-view architecture, including a global discriminant branch and a local discriminant branch that are independent of each other in terms of parameters.

[0043] Global Discrimination Branch: This branch receives the entire image as input and consists of a series of stacked convolutional layers with a stride of 2. As the network depth increases, the spatial size of the feature maps gradually decreases until they are compressed into one-dimensional vectors or extremely small feature maps. Finally, a scalar score is output through a fully connected layer, representing the probability that the entire image is classified as a real sample. This branch is primarily responsible for constraining the generated medical images to conform to the distribution of the target modality in terms of macroscopic anatomical structure.

[0044] Local discriminant branch: This branch employs a Markov discriminator structure. It has fewer network layers, and the final output is not a single scalar, but rather a... The feature matrix is ​​used. Each element in this matrix corresponds to a local receptive field (patch) of a specific size in the input image. The local discrimination branch distinguishes between true and false for each element in the matrix, which is essentially an independent texture evaluation of different local regions of the image. This design forces the generator to focus on recovering high-frequency texture details, thereby eliminating blur and blocky artifacts in the generated image.

[0045] Task guidance and auxiliary diagnostic discriminator construction: To incorporate semantic constraints for disease diagnosis into the image synthesis process, this embodiment constructs a task-guided assisted diagnostic discriminator. This network does not accept only a single image input, but instead employs a feature-level fusion strategy.

[0046] Feature Extraction and Fusion: The front end of the task-guided assisted diagnostic discriminator contains two independent feature extraction subnetworks, used to extract anatomical features from source domain images (e.g., MRI) and metabolic features from synthetic images (e.g., synthetic PET) output by the generator, respectively. The two sets of extracted feature maps are stacked and fused along the channel dimension to form a joint feature representation containing multimodal information.

[0047] Classification Prediction: The fused features are input into the classification backbone network. This backbone network consists of alternating convolutional layers, max-pooling layers, and batch normalization layers. After multi-layer processing, the feature maps are flattened and mapped to the class space through fully connected layers. The output layer uses the Softmax function to output probability distribution vectors corresponding to different disease categories (e.g., Alzheimer's disease, normal controls). During training, this auxiliary network not only updates its own parameters based on the classification labels but also propagates the backpropagation of the classification gradient to the generator, guiding the generator to synthesize images with correct pathological features.

[0048] This embodiment describes in detail the adversarial generative loss construction method used to constrain the generated images to approximate the distribution of real data during model training.

[0049] To ensure that the generated medical images are visually and statistically indistinguishable from real images, this invention introduces adversarial constraints in both the source and target domains. Considering that the sigmoid cross-entropy loss function used in conventional generative adversarial networks (GANs) is prone to gradient vanishing or training instability, this embodiment employs the loss form of least squares generative adversarial networks (LSGANs). Least squares loss, by penalizing samples far from the decision boundary, provides stronger and more stable gradient signals, thereby guiding the generator to produce higher-quality images.

[0050] For the first modality (MRI, source domain) From the second mode (PET, target domain) to the second mode (PET). The forward mapping process of ) defines the first generator. and the first discriminator First discriminator The goal is to obtain the real image in the target domain. If the condition is true (marked as 1), then the first generator is... Generated pseudo-images The result is determined to be false (marked as 0); while the first generator... The goal is to make the generated pseudo-image by the first discriminator The result is true (marked as 1).

[0051] Adversarial Generative Loss of Forward Mapping The definition is shown in the following formula: ; in, Represents the mathematical expectation. This indicates that the data follows the distribution of the source domain. The first modality image sample, This indicates that the data follows the distribution of the target domain. The second modality image sample. This term measures the error of the discriminator in distinguishing real samples. This term measures the error of the discriminator in classifying the generated samples. In optimizing the generator... At that time, the goal is to minimize That is, to minimize the distance between the generated sample and the real label.

[0052] Similarly, for the second mode (PET, target domain) From the first modality (MRI, source domain) The reverse mapping process defines the second generator. Second discriminator Adversarial generation loss of inverse mapping The definition is shown in the following formula: in, The term is used to constrain the second discriminator to correctly identify the true source domain image. The term is used to constrain the second discriminator to identify the second generator. The reconstructed pseudo-source domain image.

[0053] Due to the discriminator in this embodiment ( and All of them adopt a dual-view architecture, and the discriminator output in the above formula is... Essentially, it's a combined result of the global and local discriminative branch outputs. Specifically, the least-squares loss of the global and local discriminative branches are calculated separately, and then weighted and summed to obtain the final adversarial generation loss. The global branch loss forces the generator to optimize the overall anatomical structure layout of the image, while the local branch loss penalizes the segmented image patches, forcing the generator to optimize local texture details and eliminate block artifacts. Through this multi-scale adversarial constraint, the generated medical image is ensured to be accurate and consistent across the entire spectrum. Figure 1 Both the consistency and local high-frequency details conform to the characteristic distribution of the real mode.

[0054] This embodiment details a cyclic consistency constraint mechanism for establishing a strict anatomical correspondence between input and output images under conditions of no paired data.

[0055] In the absence of paired training data, relying solely on adversarial generation loss, while enabling generated images to visually approximate the target domain, cannot guarantee that the content of the generated images remains consistent with the input images. For example, the generator might produce an image that conforms to the PET distribution but has completely incorrect anatomical structures, or even exhibit pattern collapse, generating the same PET image regardless of the input MRI image. To address this issue, this embodiment introduces cycle consistency loss, which constrains the image to be reconstructed from the original image after bidirectional transformation, thereby locking in the anatomical content of the image.

[0056] Cyclic consistency constraints involve two processes: forward cyclic consistency and backward cyclic consistency. Forward loop process: for any real image in the source domain (MRI), first through the first generator Convert it into a synthetic image of the target domain. (Synthetic PET); subsequently, the synthesized image It is fed into the second generator as input. We attempt to restore it back to the source domain to obtain the reconstructed image of the source domain. (Reconstructing the MRI). Based on the assumption of the invertibility of the mapping, the source domain reconstructs the image. It should be consistent with the original input image. Get as close as possible at the pixel level.

[0057] Back loop process: for any real image in the target domain (PET), first through the second generator Convert it into a synthetic image of the source domain. (Synthesize MRL); subsequently, use the synthesized image It is fed into the first generator as input. We attempt to restore it back to the target domain to obtain the reconstructed image of the target domain. (Reconstructing PET). Similarly, reconstructing the image in the target domain. It should be consistent with the original input image. Get as close as possible at the pixel level.

[0058] Based on the above process, cycle consistency loss The definition is shown in the following formula: ; in, express Norm (absolute value error). This embodiment uses... Norm rather The norm (mean squared error) is used as a distance metric because... Norms are more robust to outliers in image reconstruction tasks and can effectively avoid over-smoothing (blurring) in generated images, thus better preserving the sharp edge structure information of brain gyri and sulci in medical images.

[0059] By minimizing this cycle consistency loss, the first generator is forced to... Second generator They are inverse mappings to each other. This constraint imposes pixel-level alignment between the input and output images in geometric space, ensuring that the spatial structural information of brain tissue, such as anatomical location, shape, and size, is not distorted or lost during cross-modal style transfer (such as conversion from anatomical imaging MRI to metabolic imaging PET), thereby guaranteeing the anatomical accuracy of the synthesized images in clinical applications.

[0060] See attached document Figure 4 This embodiment details a deep consistency constraint mechanism introduced into the feature encoding space to suppress semantic drift during cross-modal conversion. In unpaired image conversion tasks, relying solely on pixel-level cycle consistency loss (… While it can guarantee the geometric reconstruction of the image, it is difficult to ensure that deep abstract features (such as specific texture patterns or metabolic distribution features of pathological regions) remain unchanged during the mapping process from source domain to target domain and back to source domain. For example, the generator may lose key pathological semantics of the source image in the intermediate generated images, but fill in this information through the generator's illusion when finally reconstructing back to the source domain, thus deceiving the pixel-level loss function. To address this hidden feature loss problem, this embodiment constructs a domain feature consistency loss in the latent feature space.

[0061] The specific steps for calculating this loss are as follows: This embodiment directly utilizes the encoder portion of the generator network as the feature extractor. Specifically, the first generator is selected. The bottleneck layer output or intermediate layer output during the downsampling process of the multi-scale encoder is used as the feature representation. Since the features of this layer have undergone multiple convolutions and downsampling, they can highly abstractly represent the semantic content and domain-invariant features of the input image, while ignoring high-frequency noise.

[0062] Feature comparison process: For the source domain input image pass it through the first generator Converted into a composite image, then processed by a second generator. Reconstructing back to the source domain yields the reconstructed image from the source domain. Then, the original input image... Image reconstructed from source domain Input them again into the feature extractor In the process, the original feature vectors are obtained respectively. and reconstructing feature vectors .

[0063] Similarly, for the target domain input image Calculate the corresponding original feature vector and reconstructed feature vector.

[0064] Distance Metric: To quantify the consistency between feature vectors, this embodiment uses Euclidean distance (L2 distance) as the metric. In the high-dimensional feature space of deep neural networks, although the direction of feature vectors contains important semantic information, in this task scenario, the magnitude of the feature vectors and their absolute position in space also have clear physical meaning. Euclidean distance can accurately capture the geometric differences of features in space, is more sensitive to the overall distribution and numerical changes of features, and is suitable for measuring the point-by-point alignment and reconstruction error of two features in the semantic space.

[0065] Domain Feature Consistency Loss Function Construction: The domain feature consistency loss is used to constrain the consistency between the original medical image and the reconstructed image after iterative generation in the latent feature space. The domain feature consistency loss is obtained by calculating the squared L2 norm of the difference between feature vectors, which is the squared Euclidean distance in the feature space. The specific calculation formula is as follows: ; in, Represents the mathematical expectation. This represents the first modality of the input medical image. Indicates the data distribution; Indicates the first generator. Indicates the second generator. This represents the source domain reconstructed image obtained after forward generation and inverse mapping; Indicates the first in the generator encoder Feature extraction operations of layers, This represents the squared L2 norm of a vector.

[0066] In the above formula, the original feature vector is calculated. With reconstructing feature vectors The squared L2 norm of the difference between the two quantifies their Euclidean distance in the feature space. By minimizing the domain feature consistency loss, the model is forced to retain the deep anatomical information of the input during image transformation, preventing semantic features from drifting in the feature space due to modal differences, and ensuring that the synthesized image and the original image maintain a high numerical approximation in deep semantic expression.

[0067] By minimizing this loss function The model is required to maintain the feature orientation of the image in the latent space throughout the entire closed-loop process of forward generation and reverse reconstruction. This constraint mechanism effectively locks in the deep semantic information of the image, preventing the generator from arbitrarily altering or losing pathological features that are crucial for disease diagnosis during modality transformation, thereby ensuring the semantic consistency and interpretability of the synthesized medical images.

[0068] This embodiment details a specific implementation method that uses a task-guided auxiliary diagnostic discriminator to impose semantic constraints on the generator, ensuring that the synthesized medical images retain pathological features with clinical diagnostic value. In traditional image translation tasks, generators often tend to deceive discriminators to generate visually realistic images, but may overlook subtle pathological changes crucial for disease diagnosis. To overcome this limitation, this embodiment introduces a task-guided auxiliary diagnostic discriminator (Classifier) ​​into the training process and establishes interaction with the generator through a gradient backpropagation mechanism of a classification task.

[0069] The task-guided assisted diagnostic discriminator does not directly classify a single synthetic image, but rather makes inferences based on multimodal feature fusion. First, it acquires the deep anatomical feature map extracted from the source domain input image (first modality, such as MRI) during the encoding stage of the first generator. Simultaneously, the feature map of the synthesized image (synthesizing the second modality, such as synthesizing PET) obtained after the source domain image has been transformed by the first generator is acquired. In order to comprehensively utilize anatomical structural information and synthetic metabolic functional information, and A concatenation operation is performed along the channel dimension to form a joint feature tensor. This fusion strategy ensures that the classifier can simultaneously observe the correspondence between anatomical structures and metabolic intensity, thereby making a more accurate pathological diagnosis.

[0070] Classification Task Execution and Loss Calculation: Joint feature tensor Input task guidance auxiliary diagnostic discriminator After processing through several convolutional and fully connected layers, the final output, via the Softmax activation function, is a predicted probability distribution vector for a specific disease category (such as AD or CN). Task-guided interaction loss A multi-class cross-entropy loss function is used to measure the difference between the predicted distribution and the true label. The calculation formula is as follows: ; Gradient backpropagation and generator optimization: in, This represents the total number of disease categories. It is the unique hot code of the real disease label corresponding to the sample. If the sample belongs to the first... Class, then Otherwise, it is 0. It is a task guidance auxiliary diagnostic discriminator. Predicted sample belongs to the first The probability value of the class.

[0071] This task-guided interaction loss is not only used to update the task-guided auxiliary diagnostic discriminator. The weight parameters are adjusted to improve its classification ability. More importantly, the gradient generated by this loss is passed back to the first generator through the backpropagation algorithm. .

[0072] During the optimization process, the first generator Received from The gradient signal indicates how to adjust the pixel intensity distribution of the generated image to enable the task-guided diagnostic discriminator to make the correct disease classification. This means that the generator is forced to generate discriminative metabolic features in specific anatomical regions of the synthesized image (such as the hippocampus and posterior cingulate cortex). Through this mechanism, the model unifies the two goals of synthesizing realistic images and preserving diagnostic semantics, thereby ensuring that the output medical images are not only visually natural but also practically useful in clinical diagnostic tasks.

[0073] This embodiment describes in detail how to integrate the aforementioned independent loss components into a unified optimization objective, and how to achieve collaborative training of the model by balancing the various constraint modules through hyperparameters.

[0074] To balance visual realism, anatomical consistency, deep semantic feature preservation, and clinical diagnostic value in cross-modal image synthesis, this invention linearly weights and combines adversarial generation loss, cycle consistency loss, domain feature consistency loss, and task-guided interaction loss to construct the final overall objective loss function. The mathematical expression of this function is as follows: ; in, and These represent the adversarial generative losses for forward and inverse mappings, respectively. As basic loss terms, their weights are typically set to 1 to ensure that the generated images conform to the data distribution of the target domain. , and These are the weighted hyperparameters for the cycle consistency loss, domain feature consistency loss, and task-guided interaction loss, respectively. Represents the cycle consistency loss, used to constrain the restoration of geometry; The domain feature consistency loss is used to constrain the directional consistency of deep semantic features. This represents the task-guided interaction loss, used to introduce prior constraints for disease diagnosis.

[0075] The specific values ​​of the hyperparameters mentioned above are crucial to the final performance of the model, as they determine the degree of importance the model places on different constraints during the optimization process.

[0076] Used to control pixel-level differences between the input image and the reconstructed image. Since the numerical range of pixel intensity is usually large, and preserving the anatomical structure from deformation is a primary prerequisite for medical image synthesis, this parameter is usually set to a large value (e.g., 10.0) to impose significant structural constraints and prevent the generator from producing geometric distortions.

[0077] This loss term is used to adjust the strength of the orientation consistency constraint in the deep feature space. By penalizing the Euclidean distance of the feature vectors, it prevents the loss of key semantic information during mode transformation.

[0078] This parameter controls the feedback strength of the task-guided diagnostic discriminator. It needs to be adjusted based on the difficulty of the classification task to ensure the generator receives sufficient gradient signals to preserve pathological features, while avoiding excessive classification loss that could degrade image visual quality (i.e., avoiding adversarial examples with noisy textures).

[0079] Collaborative optimization strategy: Based on the constructed total objective loss function, this embodiment adopts an alternating iterative strategy to train the entire network architecture end-to-end.

[0080] The training process is a minimax game: for the first generator Second generator Its optimization objective is to minimize the total objective loss function. That is, to generate an image that is sufficient to deceive the discriminator, can be accurately restored, and has the correct diagnostic features.

[0081] For the first discriminator Second discriminator Its optimization objective is to maximize the adversarial generation loss, that is, to identify generated images from real images as accurately as possible. This applies to task-guided assisted diagnostic discriminators. Its participation in the calculation At the same time, it also minimizes the classification error based on the real labels to improve its own discrimination ability.

[0082] In practice, an adaptive moment estimation optimizer is used for parameter updates. Momentum parameters are set. (e.g., 0.5) and (For example, 0.999), and set an initial learning rate. During the initial training phase, the learning rate is kept constant, and then a linear decay strategy is used to gradually reduce the learning rate until training ends, ensuring that the model converges to a stable optimal solution in the solution space. Through this multi-objective joint optimization mechanism, the various loss components mutually constrain and complement each other, ultimately obtaining a high-quality cross-modal medical image synthesis model.

[0083] This embodiment details the specific training steps of the cross-modal medical image synthesis model and its inference application in actual clinical scenarios.

[0084] This embodiment employs an alternating optimization approach for end-to-end model training. The training process aims to find the Nash equilibrium point among the generator, discriminator, and task-guided diagnostic discriminator. The specific implementation steps are as follows: First, for the first generator Second generator First discriminator Second discriminator and task guidance auxiliary diagnostic discriminator The weights of all convolutional layers are initialized using a Gaussian distribution with a mean of 0 and a standard deviation of 0.02. Adaptive Moment Estimation (Adam) is selected as the optimization algorithm, and the momentum parameter is set accordingly. It is 0.5. The value is 0.999. The initial learning rate is set to 2 × 10⁻⁶. -4 To prevent oscillations in the later stages of training and to promote model convergence, a linear decay strategy is adopted: the initial learning rate is kept constant for the first 100 training epochs, and then the learning rate is linearly decayed to 0 in the subsequent 100 epochs.

[0085] In each iteration, the parameters of the generator and the task-guided auxiliary diagnostic discriminator are first fixed, and only the discriminator is updated. and .

[0086] A batch of real MRI images were randomly sampled from both the source and target domain datasets. and PET images .

[0087] Using the current first generator Generate synthetic PET images Using the second generator Generate synthetic MRI images .

[0088] To improve the stability of the discriminator training, this embodiment maintains an image buffer to store historically generated images. The discriminator not only calculates the loss based on the currently generated images, but also randomly selects historically generated images from the buffer for discrimination.

[0089] Calculate separately Least-squares adversarial loss for real and synthetic PET images, and Least-squares adversarial loss for real and synthetic MRI images.

[0090] The gradient is calculated through backpropagation and then updated. and The weight parameters enable it to more accurately distinguish generated images from real images.

[0091] Subsequently, the parameters of the discriminator are fixed, and the generator is updated synchronously. , and task-guided auxiliary diagnostic discriminator Perform the forward propagation process: Input the real source domain image. , in turn generate After Reconstruction Input the real target domain image , in turn generate After Reconstruction .

[0092] Simultaneously, feature vectors from the intermediate layer are extracted to calculate the domain feature consistency loss; the source domain image features and synthetic image features are fused and input into the task to guide the auxiliary diagnostic discriminator. Calculate the task-guided interaction loss.

[0093] The overall objective loss function is calculated by combining adversarial generation loss (where the goal is to deceive the discriminator), cycle consistency loss, domain feature consistency loss, and task-guided interaction loss. The total loss relative to the total loss is calculated using the backpropagation algorithm. , and The gradient of the parameters is calculated, and parameter updates are performed. This step enables the generator to simultaneously optimize the images in terms of realism, structural fidelity, and pathological semantics.

[0094] Once the model training is complete and convergence criteria are met, save the first generator. The network weight parameters are deployed in an inference system (such as a medical imaging workstation or a cloud server) for practical cross-modal image synthesis tasks.

[0095] Acquire the first modality of medical images (e.g., sMRI) of the patient to be diagnosed. These images are usually readily available and radiation-free. Perform preprocessing operations on this input image that are strictly consistent with those used in the training phase, including skull dissection to remove non-brain tissue, spatial registration to a standard template space, and intensity normalization based on statistical extrema, to obtain tensor data that meets the network input requirements.

[0096] The preprocessed MRI data is input into a multiscale generator loaded with training weights. The generator only needs to perform a single forward convolution operation to output the corresponding synthetic second modality image (such as synthetic PET) within milliseconds. Because the generator uses a hyperbolic tangent activation function, the raw output data values ​​are distributed between [-1, 1]. Inverse normalization is required to map them back to the standard intensity range of medical images for quantitative analysis.

[0097] The generated synthetic PET images provide complementary information about brain metabolic function. Doctors can directly observe these synthetic images using visualization tools to assess the metabolic activity of specific brain regions. Furthermore, these synthetic images can be directly used as input data to downstream computer-aided diagnostic (CAD) software or quantitative analysis algorithms to calculate quantitative indicators such as the standardized uptake ratio (SUVR). This allows doctors to assist in the early screening and diagnosis of neurodegenerative diseases such as Alzheimer's disease without requiring patients to undergo radioactive tracer injections and PET scans.

[0098] See attached document Figure 5 - Appendix Figure 6 Implementation scenario: Using unpaired structural magnetic resonance (sMRI) data to generate fluorodeoxyglucose positron emission tomography (FDG-PET) images.

[0099] Data preparation and preprocessing: This embodiment uses the publicly available ADNI (Alzheimer's Disease Neuroimaging Initiative) dataset.

[0100] Dataset Construction: sMRI images from 794 subjects were selected as the source domain dataset, and FDG-PET images from another 794 subjects were selected as the target domain dataset. The two datasets were not paired. The datasets included AD patients and normal controls (CN).

[0101] Preprocessing steps: Skull dissection: The BET tool of FSL software was used to remove the skull and non-brain tissues.

[0102] Spatial normalization: All images are nonlinearly registered to the MNI152 standard space and resampled to a voxel resolution of 182×218×182.

[0103] Normalization: Linearly map the pixel intensity of MRI and PET to the [-1, 1] interval.

[0104] Model building details: Generator architecture: Employs a variant of 3DU-Net. The encoder section includes parallel paths. Path A: 7×7×7 convolutional kernel with a stride of 1, capturing the macroscopic structure of the brain.

[0105] Path B: 3×3×3 convolution kernel with a stride of 1, capturing cortical texture details.

[0106] The two outputs are concatenated and then fed into a downsampling layer (a convolution with a stride of 2).

[0107] Discriminator architecture: Global branch: Output a 1×1 scalar to determine the authenticity of the whole brain structure.

[0108] Local branch: Outputs a 16×16×16 matrix, corresponding to the texture realism of a local patch in the brain.

[0109] Task-guided assisted diagnostic discriminator: It adopts a lightweight network architecture as the backbone, takes as input a splice of MRI anatomical features and generated PET metabolic features, and outputs as binary classification probabilities of [AD, CN].

Claims

1. A cross-modal medical image synthesis method based on task guidance and domain feature consistency, characterized in that, Includes the following steps: Obtain a source domain dataset and a target domain dataset, wherein the source domain dataset and the target domain dataset respectively contain unpaired first modality medical images and second modality medical images; A cross-modal medical image synthesis model is constructed, which includes a first generator, a second generator, a first discriminator, a second discriminator, and a task-guided auxiliary diagnostic discriminator; The cross-modal medical image synthesis model is subjected to forward propagation calculation. The first modal medical image is input into the first generator to generate a first synthesized image, and the first synthesized image is input into the second generator to generate a source domain reconstructed image. The second generator and the first generator are used to generate a second synthesized image and a target domain reconstructed image based on the second modal medical image. The domain feature consistency loss is calculated based on the difference in Euclidean distance between the feature vectors of the first modality medical image and the source domain reconstructed image, and between the second modality medical image and the target domain reconstructed image in the generator's preset coding layer. The task-guided interaction loss is calculated by fusing the feature map of the first modality medical image with the feature map of the first synthesized image to obtain a fused feature. The fused feature is then input into the task-guided auxiliary diagnostic discriminator to obtain a predicted probability distribution. The predicted probability distribution is then used to calculate the result based on the real disease label. A total objective loss function is constructed by combining the adversarial generation loss and the cycle consistency loss calculated simultaneously, and the cross-modal medical image synthesis model is iteratively trained using the total objective loss function.

2. The cross-modal medical image synthesis method based on task guidance and domain feature consistency according to claim 1, characterized in that, After obtaining the source domain dataset and the target domain dataset, a preprocessing step is also included: Brain extraction tools were used to dissect the skull in the images and remove non-brain tissue regions; the images were then registered to a standard template space and resampled to a uniform resolution. The image is normalized by intensity, mapping the pixel intensity of the image to the value range of a preset activation function.

3. The cross-modal medical image synthesis method based on task guidance and domain feature consistency according to claim 1, characterized in that, Both the first generator and the second generator employ a multi-scale encoder structure, which includes a first convolutional path and a second convolutional path set in parallel. The first convolutional path uses a convolutional kernel of a first size to extract global spatial structure features, and the second convolutional path uses a convolutional kernel of a second size to extract local texture detail features. The first size is larger than the second size. The feature map output by the first convolutional path and the feature map output by the second convolutional path are concatenated along the channel dimension and then input into the downsampling module.

4. The cross-modal medical image synthesis method based on task guidance and domain feature consistency according to claim 1, characterized in that, Both the first discriminator and the second discriminator employ a dual-view discrimination structure, which includes: The global discrimination branch is used to map the entire image to a scalar score to evaluate the authenticity of the entire image; The local discriminant branch is used to output a feature matrix, where each element corresponds to a local receptive field of the image, and is used to evaluate the texture realism of local regions of the image.

5. The cross-modal medical image synthesis method based on task guidance and domain feature consistency according to claim 1, characterized in that, The task-guided auxiliary diagnostic discriminator includes a feature extraction subnetwork and a classification backbone network; In the step of calculating the task-guided interaction loss, the specific method of feature fusion is as follows: The anatomical feature map of the first modality medical image is extracted, and the metabolic feature map of the first synthesized image is extracted; the anatomical feature map and the metabolic feature map are concatenated in the channel dimension to obtain the fused feature; the fused feature is input into the classification backbone network to obtain the predicted probability distribution.

6. The cross-modal medical image synthesis method based on task guidance and domain feature consistency according to claim 1, characterized in that, The steps for calculating the domain feature consistency loss specifically include: Using the encoder of the first generator as a feature extractor, the original feature vector of the first modality medical image and the reconstructed feature vector of the source domain reconstructed image are extracted respectively. Calculate the L2 norm of the difference between the original feature vector and the reconstructed feature vector, and use the L2 norm as the Euclidean distance between the original feature vector and the reconstructed feature vector. The Euclidean distance is used as the domain feature consistency loss to constrain the first generator and the second generator to maintain the consistency of feature vectors in the feature space.

7. The cross-modal medical image synthesis method based on task guidance and domain feature consistency according to claim 5, characterized in that, The step of calculating the interaction loss for the task also includes: Based on the predicted probability distribution and the real disease label corresponding to the first modality medical image, the classification error is calculated using the multi-class cross-entropy function; The gradient generated by the classification error is passed to the first generator through the backpropagation algorithm, guiding the first generator to retain pathological features that can distinguish diseases during the generation process.

8. The cross-modal medical image synthesis method based on task guidance and domain feature consistency according to claim 1, characterized in that, In the step of constructing the total objective loss function by combining the simultaneously calculated adversarial generation loss and cycle consistency loss: The adversarial generation loss is calculated using a least squares loss function. The adversarial generation loss includes the mean square error generated by the second discriminator in distinguishing between the second modality medical image and the first synthetic image, and the mean square error generated by the first discriminator in distinguishing between the first modality medical image and the second synthetic image. The cyclic consistency loss is calculated using the L1 norm distance formula. The cyclic consistency loss includes the pixel-level absolute error between the first modal medical image and the source domain reconstructed image, and the pixel-level absolute error between the second modal medical image and the target domain reconstructed image.

9. The cross-modal medical image synthesis method based on task guidance and domain feature consistency according to claim 1, characterized in that, The step of iteratively training the cross-modal medical image synthesis model using the total objective loss function includes: The overall objective loss function is constructed by weighted summation. First, fix the parameters of the first generator, the second generator, and the task guidance auxiliary diagnostic discriminator, and update the first discriminator and the second discriminator using the adversarial generation loss; then fix the parameters of the first discriminator and the second discriminator, and synchronously update the first generator, the second generator, and the task guidance auxiliary diagnostic discriminator using the total objective loss function.

10. The cross-modal medical image synthesis method based on task guidance and domain feature consistency according to claim 1, characterized in that, The cross-modal medical image synthesis method also includes an inference application step: The first modality of medical image to be diagnosed is acquired as input data and preprocessing is performed. The preprocessed input data is fed into the first generator after training, and the synthesized second modality medical image is output. The synthesized second modality medical image is subjected to inverse normalization processing to assist in clinical diagnosis or as input for downstream analysis algorithms.

Images