Image generation methods, apparatus, devices, readable storage media, and program products

Abstract

This application relates to an image generation method, apparatus, device, readable storage medium, and program product. The method includes: acquiring a target modal image meeting a first quality requirement and a reference modal image meeting a second quality requirement; wherein the first quality requirement is less than the second quality requirement; decoupling the features of the target modal image and the reference modal image to obtain content features of the reference modal image and attribute features of the target modal image; fusing the attribute features and content features to obtain cross-modal fusion features; acquiring Gaussian noise, using the cross-modal fusion features as conditional input, and denoising the Gaussian noise to obtain implicit features; and decoding the implicit features to obtain a second target modal image meeting the second quality requirement. This method can improve the ability to restore image details, thereby improving the accuracy of the reconstructed image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image generation method, apparatus, device, readable storage medium, and program product. Background Art

[0002] Medical image reconstruction is an important field in medical image processing. It generates images from raw data using various techniques and algorithms to meet the needs of clinical diagnosis and treatment. Traditional medical image reconstruction techniques include atlas-based methods, sparse coding, and traditional machine learning methods. However, these methods are typically time-consuming, labor-intensive, and lack stability and versatility.

[0003] To address these issues, deep learning technology has achieved significant breakthroughs in the field of medical image reconstruction thanks to its continuous development. Related technologies utilize novel techniques such as autoencoders (AEs), convolutional neural networks (CNNs), and generative adversarial networks (GANs) to achieve cross-modal image reconstruction. However, cross-modal reconstruction in these technologies limits the ability to restore image details, resulting in low accuracy of the reconstructed images. Summary of the Invention

[0004] Therefore, it is necessary to provide an image generation method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the accuracy of reconstructed images in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides an image generation method, including:

[0006] Acquire a target modal image that meets a first quality requirement and a reference modal image that meets a second quality requirement; wherein the first quality requirement is less than the second quality requirement;

[0007] Feature decoupling is performed on the target modality image and the reference modality image to obtain the content features of the reference modality image and the attribute features of the target modality image;

[0008] The attribute features and the content features are fused to obtain cross-modal fused features;

[0009] Gaussian noise is obtained, and the cross-modal fusion features are used as conditional inputs to denoise the Gaussian noise, thereby obtaining implicit features.

[0010] The implicit features are decoded to obtain the second target modal image of the second quality requirement.

[0011] In one embodiment, the step of decoupling the target modality image and the reference modality image to obtain the content features of the reference modality image and the attribute features of the target modality image includes:

[0012] The target modality image and the reference modality image are input into a trained multimodal feature decoupling network. The multimodal feature decoupling network is used to decouple features and outputs the content features of the reference modality image and the attribute features of the target modality image.

[0013] In one embodiment, the training method of the multimodal feature decoupling network includes:

[0014] Construct the first total loss function of the autoencoder in the multimodal feature decoupling network, and the second total loss function of the decoupling module in the multimodal feature decoupling network;

[0015] Obtain a first sample image set for training the multimodal feature decoupling network, wherein the first sample image set includes at least two image sets with different quality requirements under multiple modalities;

[0016] For the first image in the first sample image set, a preset image template is used to register it to the individual space of each first image to obtain the corresponding multi-channel initial tissue probability map.

[0017] The encoder is trained based on each first image and the corresponding initial tissue probability map. The training of the encoder is completed when the first function value of the first total loss function is less than the first threshold.

[0018] Obtain a second sample image set, train the decoupling module based on the second sample image set, and complete the training of the decoupling module when the second function value of the second total loss function of the decoupling module is less than the second threshold, thereby obtaining a trained multimodal feature decoupling network.

[0019] In one embodiment, the first total loss function includes an image reconstruction loss function and a tissue probability divergence loss function. The step of training the autoencoder based on each first image and its corresponding initial tissue probability map, and completing the training of the autoencoder when the first function value of the first total loss function is less than a first threshold, includes:

[0020] Each of the first images is input into the encoder, which outputs a feature representation. The feature representation is then input into the decoder to determine the reconstructed image.

[0021] Based on the reconstructed image and the first image, determine the reconstruction loss value of the image reconstruction loss function;

[0022] Determine the target probability distribution and predicted probability distribution of each of the initial tissue probability maps at the current voxel point in the first image;

[0023] Based on the target probability distribution and the predicted probability distribution, determine the organization probability map divergence loss value of the organization probability divergence loss function;

[0024] Based on the reconstruction loss value and the tissue probability map divergence loss value, a first function value of the first total loss function is determined. When the first function value is less than a first threshold, the training of the encoder is completed.

[0025] In one embodiment, training the decoupling module based on the second sample image set, and completing the training of the decoupling module when the second function value of the second total loss function of the decoupling module is less than a second threshold, to obtain a trained multimodal feature decoupling network, includes:

[0026] For each second image in the second sample image set, determine the initial tissue probability map and image features corresponding to each second image;

[0027] Each second image and its corresponding initial tissue probability map are input into the trained encoder to obtain initial content features and initial attribute features.

[0028] The initial content features and the initial attribute features are input into the decoupling module, and the initial attribute features are processed by the attribute feature extraction network in the decoupling module to obtain target attribute features that are unrelated to the content features.

[0029] The initial content features are enhanced by the content feature extraction network in the decoupling module to obtain the target content features;

[0030] The target attribute features and the target content features are fused to obtain new cross-modal fusion features;

[0031] Based on the image features, target attribute features, target content features, initial organization probability map, and cross-modal fusion features, the second function value of the second total loss function of the decoupling module is determined. When the second function value is less than the second threshold, the training of the decoupling module is completed, and a trained multimodal feature decoupling network is obtained.

[0032] In one embodiment, the step of performing feature extraction processing on the initial attribute features through the attribute feature extraction network in the decoupling module to obtain target attribute features unrelated to content features includes:

[0033] The attribute feature extraction network in the decoupling module extracts additional content features from the initial attribute features, and then removes the additional content features from the initial attribute features to obtain target attribute features that are unrelated to the content features.

[0034] The step of enhancing the initial content features through the content feature extraction network in the decoupling module to obtain the target content features includes:

[0035] The initial attribute features and the initial content features are aligned, and the initial content features are enhanced by fusing the additional content features to obtain the target content features.

[0036] In one embodiment, each of the second images and its corresponding initial tissue probability map are input into a trained encoder to obtain initial content features and initial attribute features, including:

[0037] For each second image, an encoder is input to obtain image features. The initial tissue probability map corresponding to the second image is input to the encoder to obtain initial content features.

[0038] Based on the preset relationship between the image, the organization probability map, and the modal attribute map, the initial attribute features are determined according to the preset relationship, the image features, and the initial content features.

[0039] In one embodiment, the step of obtaining Gaussian noise, using the cross-modal fusion features as conditional input, and denoising the Gaussian noise to obtain implicit features, includes:

[0040] Gaussian noise is obtained and used as input. The cross-modal fusion features are used as conditional input and fed into the trained implicit diffusion model for stepwise denoising to obtain implicit features.

[0041] In one embodiment, training the implicit diffusion model includes:

[0042] Construct the third total loss function for the implicit diffusion model;

[0043] A third sample image set is obtained for training the implicit diffusion model, wherein the third sample image set includes multiple sets of images with at least one quality requirement; each set of images includes at least two modalities;

[0044] Gaussian blurring is applied to the third image in the third sample image set to obtain a high-frequency detail image;

[0045] For each image group in the third sample image set, the trained multimodal feature decoupling network is used to decouple the features of the image group to obtain content features and attribute features. The content features obtained from the reference modal image that meets the second quality requirement are decoded to obtain an optimized organization probability map.

[0046] The image features obtained by the encoder in the third image are subjected to progressive noise addition processing to obtain target Gaussian noise;

[0047] Using the target Gaussian noise as the starting point for reverse denoising, and the cross-modal fusion features as control conditions, the data are input into the denoising network of the implicit diffusion model for step-by-step denoising, resulting in noise predictions for each denoising step and the final output prediction image.

[0048] The function value of the third total loss function is determined based on the noise prediction, the high-frequency detail map, the tissue probability map, the content features, the attribute features, and the predicted image. When the function value of the third total loss function is less than the third threshold, the training of the implicit diffusion model is completed, and a trained implicit diffusion model is obtained.

[0049] In one embodiment, the third image is a second target modality sample image that meets the second quality requirement, and the step of performing Gaussian blur processing on the sample image set to obtain a high-frequency detail image includes:

[0050] Gaussian blurring is applied to the second target modality sample image to obtain a low-frequency structure image;

[0051] The high-quality image of the target mode is processed based on the low-frequency structural image to obtain a high-frequency detail image;

[0052] Each image group includes a first target modality sample image that meets the first quality requirement and a second reference modality sample image that meets the second quality requirement. The image group is decoupled using a trained multimodal feature decoupling network to obtain content features and attribute features, including:

[0053] The trained multimodal feature decoupling network is used to extract features from the second reference modality sample image to obtain content features, and the content features are decoded to obtain an optimized organization probability map.

[0054] The trained multimodal feature decoupling network is used to extract features from the first target modality sample image to obtain attribute features.

[0055] In one embodiment, the third total loss function includes a noise prediction loss function, a boundary-aware loss function, and a high-frequency texture loss function. Determining the value of the third total loss function based on the noise prediction, the high-frequency detail map, the tissue probability map, the content features, the attribute features, and the predicted image includes:

[0056] The noise prediction loss value of the noise prediction loss function is determined based on the noise prediction.

[0057] The entropy value of each voxel in the predicted image is calculated based on the tissue probability map, and the error value of each voxel in the predicted image and the second target modality sample image is determined. The boundary sensing loss value of the boundary sensing loss function is determined based on the entropy value and the error value.

[0058] Based on the predicted image and the high-frequency detail map, the high-frequency texture loss value of the high-frequency texture loss function is determined;

[0059] The noise prediction loss value, the boundary perception loss value, and the high-frequency texture loss value are weighted to determine the function value of the third total loss function.

[0060] Secondly, this application also provides an image generation apparatus, comprising:

[0061] The data acquisition module is used to acquire a target modal image that meets a first quality requirement and a reference modal image that meets a second quality requirement; wherein the first quality requirement is less than the second quality requirement;

[0062] A decoupling module is used to decouple the target modal image and the reference modal image by means of features, thereby obtaining the content features of the reference modal image and the attribute features of the target modal image.

[0063] The feature processing module is used to fuse the attribute features and the content features to obtain cross-modal fusion features;

[0064] The image processing module is used to acquire Gaussian noise, take the cross-modal fusion features as conditional input, and perform noise reduction processing on the Gaussian noise to obtain implicit features;

[0065] The implicit features are decoded to obtain the second target modal image of the second quality requirement.

[0066] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above image generation methods.

[0067] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above image generation methods.

[0068] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above image generation methods.

[0069] The aforementioned image generation method, apparatus, computer equipment, computer-readable storage medium, and computer program product achieve feature decoupling of the target modality image meeting the first quality requirement and the reference modality image meeting the second quality requirement in the latent space dimension. This yields content features and attribute features of the modality features that can be shared across modalities, eliminating semantic redundancy in the feature space. Feature structure is guided by structural priors, and cross-modal fusion features are obtained by fusing the decoupled attribute features and content features. The cross-modal fusion features are then used as conditional inputs to denoise Gaussian noise and obtain implicit features. The implicit features are then decoded to generate the second target modality image meeting the second quality requirement, thus improving the accuracy of image reconstruction. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is an application environment diagram of the image generation method in one embodiment;

[0072] Figure 2 This is a flowchart illustrating an image generation method in one embodiment;

[0073] Figure 3 This is a schematic diagram of the structure of a multimodal feature decoupling network based on structural prior guidance in one embodiment;

[0074] Figure 4 This is a flowchart illustrating the training method of a multimodal feature decoupling network in one embodiment;

[0075] Figure 5 This is a schematic diagram of a quality enhancement implicit diffusion model architecture for cross-modal feature fusion in one embodiment;

[0076] Figure 6 This is a flowchart illustrating the training method of an implicit diffusion model in one embodiment;

[0077] Figure 7 This is a flowchart illustrating the image generation method in another embodiment;

[0078] Figure 8 This is a structural block diagram of an image generation device in one embodiment;

[0079] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0081] The combination of multimodal brain imaging technology and intelligent image analysis methods provides a new objective means for the accurate diagnosis and prognostic assessment of brain diseases. Magnetic resonance imaging (MRI) and positron emission tomography (PET), as mainstream neuroimaging modalities, have achieved cross-scale pathological feature analysis from macroscopic structure to molecular function through three-dimensional anatomical morphological characterization, glucose metabolism tracing, and dynamic monitoring of neurotransmitters. They have become the core basis for the clinical diagnosis of neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease.

[0082] However, conventional clinical imaging equipment suffers from significant technical bottlenecks: the isotropic spatial resolution of approximately 1 mm³ in a standard 1.5T MRI system is insufficient to accurately detect microscopic pathological signs such as the shrinkage of the substantia nigra pars compacta, and its sensitivity to sub-millimeter-level pathological changes such as iron ion deposition is inadequate. While ultra-high field 7T MRI can improve resolution to the order of 0.4 mm³, its clinical application is limited by equipment scarcity, the complexity of motion artifact control techniques, and high scanning costs. Although PET imaging can reflect molecular pathological processes such as β-amyloid deposition, its spatial resolution is generally below 2 mm due to limitations in gamma photon detection efficiency and tracer pharmacokinetics, resulting in limited identification of fine structures.

[0083] Deep learning-based image quality enhancement technology offers a new paradigm for solving the aforementioned problems. By establishing a nonlinear mapping relationship between degradation models and high-quality images, this technology can overcome the diffraction limit of physical imaging systems, achieving enhanced visualization of sub-voxel-level pathological features. This not only significantly improves the accuracy and reproducibility of quantitative analysis but also has important clinical value in reducing the risk of radiation exposure and optimizing the allocation of medical resources, laying a technological foundation for advancing the early screening and precise intervention of neurological diseases.

[0084] In recent years, deep learning has made significant breakthroughs in the field of medical image reconstruction, particularly demonstrating important research value in multimodal data collaborative processing. While early single-modal mapping models could improve basic image quality, they failed to effectively integrate complementary biological information from different modalities, limiting the ability to restore details. Current multimodal fusion methods often employ shallow feature stitching strategies, resulting in excessively high cross-modal information coupling and making it difficult to achieve decoupled representation of anatomical features.

[0085] In generative model applications, while Generative Adversarial Networks (GANs) can produce high-fidelity images, their inherent pattern collapse problem limits output diversity, and their network training stability is flawed. Variational Autoencoders (VAEs), despite their fast sampling advantage, often produce images with blurred edges and texture distortion. Diffusion models achieve a balance between image quality and diversity, but their iterative generation mechanism leads to low computational efficiency, making them unsuitable for real-time clinical applications. It is noteworthy that medical image reconstruction places stringent demands on the fidelity of anatomical structures; even subtle morphological deviations can cause characteristic distortions in complex brain regions such as the basal ganglia, leading to clinical misdiagnosis.

[0086] Existing methods generally lack explicit constraints on neuroanatomical structures, resulting in blurred boundaries in reconstructed images at key regions such as the gray-white matter boundary, severely impacting the accuracy of quantitative analysis of brain tissue. This structural distortion limits the applicability of current generative models in the medical field. Therefore, it is necessary to construct a deep representation framework that integrates multimodal complementary information, while introducing anatomical prior constraints to precisely control the reconstruction of brain images, ensuring the topological consistency of biological structures and preventing morphological distortion of key biomarkers (such as substantia nigra iron deposition).

[0087] To address the technical problem of low accuracy in reconstructed images mentioned above, an image generation method is proposed. The image generation method provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 obtains a target modal image that meets a first quality requirement and a reference modal image that meets a second quality requirement from server 104; it decouples the target modal image and the reference modal image to obtain the content features of the reference modal image and the attribute features of the target modal image; it fuses the attribute features and content features to obtain cross-modal fusion features; it then acquires Gaussian noise, uses the cross-modal fusion features as conditional input, and performs denoising processing on the Gaussian noise to obtain implicit features; finally, it decodes the implicit features to obtain the second target modal image that meets the second quality requirement.

[0088] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. The server 104 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0089] In an exemplary embodiment, Figure 2 As shown, an image generation method is provided, which can be applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps 202 to 210. Wherein:

[0090] Step 202: Obtain a target modal image that meets the first quality requirement and a reference modal image that meets the second quality requirement; wherein the index value of the first quality requirement is less than the index value of the second quality requirement.

[0091] The quality requirement index (i.e., quality index) can be determined based on a preset resolution or a preset pixel count. Different quality indices correspond to a preset threshold range, which can be a resolution threshold or a pixel threshold. In this example, resolution is used to distinguish the image quality indices. The preset threshold range corresponding to the first quality requirement is smaller than the preset threshold range corresponding to the second quality requirement. A quality that meets the first quality requirement can be called low quality, and a quality that meets the second quality requirement can be called high quality.

[0092] Images can be from different application scenarios, including medical images and neuroimaging images. This example uses neuroimaging as an illustration. Target modality and reference modality refer to image types generated by different imaging techniques. The target modality refers to the desired image type to be generated or analyzed in multimodal image analysis; this could be magnetic resonance imaging (MRI) or positron emission tomography (PET), etc. The reference modality refers to other imaging modalities used to assist in the analysis of the target modality; this could be computed tomography (CT) or ultrasound imaging, etc. If the target modality is MRI, then the reference modality could be computed tomography (CT).

[0093] The target modal image and the reference modal image are paired data. Paired data can be understood as image data of different modalities and different quality requirements all coming from the same test object. In other words, the target modal image and the reference modal image come from the same test object.

[0094] For example, acquiring a low-quality target modal image I low tar With high-quality reference modal image I highref .

[0095] Step 204: Decouple the target modality image and the reference modality image by feature decoupling to obtain the content features of the reference modality image and the attribute features of the target modality image.

[0096] Feature decoupling can be achieved using a pre-trained multimodal feature decoupling network. This network, guided by structural priors, constructs a dual-path optimization paradigm of latent space decoupling-recombination, establishing modality invariance constraints and anatomical structure fidelity preservation mechanisms. This allows for the separation of cross-modal shared anatomical structural information (i.e., content features) and modality-specific attribute features (e.g., texture / contrast) from multimodal neuroimaging. This eliminates distortions in modality attribute and structural content representation caused by residual anatomical structural information in attribute encoding; it also suppresses the unintended carrying of anatomical structural information by attribute features; thereby improving the interpretability of decoupled features as biomarkers in quantitative analysis of brain diseases.

[0097] Prior knowledge guidance based on tissue structure refers to using known tissue structure information as prior knowledge to guide and optimize the reconstruction process in medical image reconstruction, thereby improving the quality and accuracy of the reconstructed images.

[0098] For example, the target modality image and the reference modality image are input into a trained multimodal feature decoupling network. The feature decoupling is performed through the multimodal feature decoupling network, and the content features of the reference modality image and the attribute features of the target modality image are output.

[0099] Step 206: Fuse attribute features and content features to obtain cross-modal fused features.

[0100] Feature fusion can be achieved by modulating the features of a multimodal feature decoupling network to fuse the decoupled features of multimodal data, thereby obtaining cross-modal fused features.

[0101] Step 208: Obtain Gaussian noise, use the cross-modal fusion features as conditional input, and denoise the Gaussian noise to obtain implicit features.

[0102] For example, Gaussian noise is acquired and used as input, along with cross-modal fusion features as conditional input. This Gaussian noise is then fed into a trained implicit diffusion model for progressive denoising to obtain implicit features. The implicit diffusion model uses cross-modal fusion features as prior conditions, focusing on complex regions that are difficult to reconstruct. It efficiently reconstructs feature maps in a low-dimensional implicit space, improving the quality of neural images. Complex regions that are difficult to reconstruct can be the boundaries between different tissues, such as the gray-white matter boundary.

[0103] The implicit diffusion model includes an encoder, a decoder, and a diffusion model. The core of the diffusion model is a conditional neural network, which can be a U-Net structure. The U-Net captures the spatial continuity and local correlation of the image by repeatedly applying convolution operations.

[0104] It should be noted that, considering the different reconstruction difficulties of different regions in neuroimaging, the loss function in the implicit diffusion model in this embodiment is constructed based on the region selection strategy of structural entropy weighted mapping. It integrates high-frequency residual analysis and tissue boundary detection methods to construct a dual anatomy-driven loss function, which guides the model to adaptively focus on complex regions with high reconstruction difficulty. This allows the model to efficiently complete feature map reconstruction in low-dimensional implicit space and improve the quality of neuroimaging.

[0105] Step 210: Decode the implicit features to obtain the second target modality image with the second quality requirement.

[0106] For example, acquiring a low-quality target modal image I low tar With high-quality reference modal image I high ref The acquired low-quality target modal image I low tar With high-quality reference modal image I high ref The input is fed into a pre-trained multimodal feature decoupling network for feature decoupling and I extraction. high ref Content features and I low tar Attributes and characteristics From Gaussian noise Initially, the fused cross-modal features are used as conditions, input into a cross-attention-based UNet network, and implicit features are gradually generated through denoising. The decoder will Convert to high-quality target modal image I pred .

[0107] The aforementioned image generation method decouples image features in the latent space dimension by taking a target modal image that meets the first quality requirement and a reference modal image that meets the second quality requirement. This yields content features that can be shared across modalities and attribute features that characterize modal features, eliminating semantic redundancy in the feature space. Feature decoupling is guided by structural priors, and the attribute and content features obtained from the decoupling are fused to obtain cross-modal fused features. These cross-modal fused features are then used as conditional inputs to denoise Gaussian noise and obtain latent features. Finally, the latent features are decoded to generate a second target modal image that meets the second quality requirement, thus improving the accuracy of image reconstruction.

[0108] Considering the significant limitations of image spatial decomposition-based brain image feature decoupling methods in related technologies—namely, their content encoders exhibit suboptimal ability to capture brain anatomical structural features, with some anatomical information remaining in the attribute feature space, leading to semantic confusion in the cross-domain (content domain and attribute domain) representation of neuroimages—this paper proposes a multimodal feature decoupling network guided by structural priors to separate the content and attributes of the input multimodal data in the feature latent space.

[0109] In an exemplary embodiment, feature decoupling is performed on a target modal image and a reference modal image to obtain content features of the reference modal image and attribute features of the target modal image. This includes: inputting the target modal image and the reference modal image into a trained multimodal feature decoupling network, performing feature decoupling through the multimodal feature decoupling network, and outputting content features of the reference modal image and attribute features of the target modal image.

[0110] like Figure 3 The diagram shows the structure of a multimodal feature decoupling network guided by structural priors. The network includes an encoder, decoder, self-attention mechanism module, cross-attention mechanism module, and feature modulation. The input consists of image images of different modalities and quality requirements, along with corresponding tissue probability maps. The image input to the model is the image itself, while the other three input images are initial tissue probability maps, which could represent the initial tissue probability maps of cerebrospinal fluid, gray matter, and white matter, respectively. Figure 3 The network architecture diagram shown provides a training method for a multimodal feature decoupling network, such as... Figure 4 As shown, steps 402 to 410 are included, wherein:

[0111] Step 402: Construct the first total loss function of the autoencoder in the multimodal feature decoupling network, and the second total loss function of the decoupling module in the multimodal feature decoupling network.

[0112] It is understandable that a multimodal feature decoupling network consists of two main parts: an autoencoder and a decoupling module. The autoencoder further comprises an encoder and a decoder; the encoder encodes the image into features, and the decoder uses these features to reconstruct the image. Therefore, during training, not only the encoder but also the decoder needs to be trained simultaneously.

[0113] The first total loss function, also known as the reconstruction loss function, is used to train the autoencoder in the multimodal feature decoupling network. The first total loss function includes the image reconstruction loss function and the organization probability divergence loss function. The image reconstruction loss function minimizes the difference between the original input and the reconstructed output. The image reconstruction loss function can be expressed as:

[0114] (1)

[0115] in, x This includes images with different quality requirements in different modalities, and may include a first target modal sample image I that meets the first quality requirement. low tar and the first reference mode sample image I low ref Second target modal sample image I that meets the second quality requirements high tar Second reference mode sample image I high ref .

[0116] Reconstruction losses L recon ∑ represents the sum of reconstruction errors for all sample images. x For all samples x Summation is performed. Enc( x Input data x Convert to the latent space representation. Decoder function Dec(Enc( x Transform the latent space representation back into the original data space to attempt to reconstruct the input data. x L2 norm It is used to calculate the distance between two vectors. Here, it calculates the original input. x and reconstructed data Dec(Enc( x The square root of the sum of the square differences between ))

[0117] The tissue probability divergence loss function is used to compare the similarity between the probability distribution predicted by the model and the true distribution, thereby optimizing the model training. Specifically, it is used to compare the target probability distribution and the predicted probability distribution of the corresponding multi-channel tissue probability map at the current voxel point in the sample image. In this example, considering that the tissue probability map represents the probability distribution of N tissues, Kullback-Leibler (KL) divergence is used for loss calculation. Tissue Probability Divergence Loss Function L kl It can be represented as: (2)

[0118] Where, q label and q pred These represent the current voxel points on the sample image. v The target probability distribution and predicted probability distribution belong to N organizations.

[0119] The first total loss function can be determined based on the image reconstruction loss function and the tissue probability divergence loss function.L ae It can be represented as:

[0120] (3)

[0121] in, This is the first preset coefficient.

[0122] The decoupling module can be a two-stream deep feature decoupling module, including an attribute feature optimization branch (i.e., an attribute feature branch network) and a content feature optimization branch (i.e., a content feature branch network). With this two-stream deep feature decoupling architecture, the initially acquired content and attribute features are further optimized and reconstructed to effectively generate an organization probability map with complete anatomical information, while ensuring that the attribute feature space is strictly separated from content-related interference.

[0123] The second total loss function of the decoupling module includes the implicit self-reconstruction loss function, the implicit cross-reconstruction loss function, the content decoupling loss function, the attribute decoupling loss function, the contrast decoupling loss function, and the attribute consistency loss function.

[0124] The implicit self-reconstruction loss function characterizes the image features of the sample image and the updated attribute features and content features obtained by the decoupling module, respectively. These two features are then processed by the feature modulation module to generate new differences between the image features. (Implicit self-reconstruction loss function) It can be represented as:

[0125] (4)

[0126] y includes high- and low-quality image features of the target mode and the reference mode. L 1-norm It calculates the sum of the absolute values ​​of the vector elements. Represents the sample image features of the model y The difference between the value calculated using the feature modulation modula and the value obtained through the modula. Here, the modula may be a specific function or operation. and It consists of the image features of the sample image, which are output by the decoupling module as attribute features and content features.

[0127] The implicit cross-reconstruction loss function characterizes the loss between high- and low-quality image features from different modalities and the new image features obtained after exchanging and combining updated attribute features and content features acquired through the decoupling module. It can be expressed as equation (5), where , These represent the target modality low-quality image features calculated by the Modula feature modulation and the updated attribute features obtained by the decoupling module, respectively. , , These represent the target modality high-quality image features calculated by the Modula feature modulation, the updated attribute features obtained by the decoupling module, and the updated content features obtained by the decoupling module, respectively. , These represent the reference modality low-quality image features calculated by the Modula feature modulation and the updated attribute features obtained by the decoupling module, respectively. , , These represent the reference modality high-quality image features calculated by the Modula feature modulation, the updated attribute features obtained by the decoupling module, and the updated content features obtained by the decoupling module, respectively:

[0128] (5)

[0129] Content decoupling loss function: constrains the similarity between content features of different modalities and qualities and the features of the initial probability map, as shown in equation (6), where This represents the initial probability map features. and These represent the updated content features of the target modality low-quality image obtained by the decoupling module and the updated content features of the reference modality low-quality image obtained by the decoupling module, respectively. This represents Charbonnier loss.

[0130] (6)

[0131] Attribute decoupling loss function: forces consistent attribute features within the same modality and maximizes the differences in attribute features between different modalities, as shown in equation (7), where and These represent the updated attribute features obtained from the low-quality target modality image after decoupling by the decoupling module, and the updated attribute features obtained from the high-quality target modality image after decoupling by the decoupling module, respectively. and These represent the updated attribute features obtained from the low-quality reference modality image through the decoupling module and the updated attribute features obtained from the high-quality reference modality image through the decoupling module, respectively.

[0132] (7)

[0133] Contrastive decoupling loss function: Enhance the feature decoupling capability through contrastive learning, as shown in Equation (8).

[0134] (8)

[0135] Attribute consistency loss function: constrains the local smoothness of attribute feature maps, so that the gradient change of attribute feature maps is as small as possible during model optimization, thereby achieving the effect of local smoothness, which helps to generate more natural and reasonable neural images. In equation (9), ▽ represents the horizontal gradient and vertical gradient, λ is a coefficient that balances the degree of smoothness, and y includes high and low quality image features of the target modality and the reference modality.

[0136] (9)

[0137] After the autoencoder is trained, its network parameters are fixed, and then the dual-stream deep feature decoupling module is trained. The second total loss function of this module is shown in Equation (10):

[0138] (10)

[0139] Step 404: Obtain a first sample image set for training the multimodal feature decoupling network, wherein the first sample image set includes at least two image groups with different quality requirements under multiple modalities.

[0140] The different modes may include target modes and reference modes, and the quality requirements may include first quality requirements and second quality requirements. That is to say, the target mode includes two target mode images that meet the first quality requirements and the second quality requirements, and the reference mode includes two reference mode images that meet the first quality requirements and the second quality requirements.

[0141] It's important to note that neural network training is always based on training sample data in batches, and the loss function is calculated accordingly. That is, using the first set of sample images as the training sample set, the training sample set can be divided into multiple mini-batches. The model and the neural network are then iteratively trained based on each mini-batch. Therefore, each mini-batch includes several sets of first sample images, and consequently, multiple first target modality sample images, first reference modality sample images, second target modality sample images, and second reference modality sample images.

[0142] Step 406: For the first image in the first sample image set, use a preset image template to register it to the individual space of each first image to obtain the corresponding multi-channel initial tissue probability map.

[0143] The first image includes a first target modal sample image that meets a first quality requirement and a second target modal sample image that meets a second quality requirement, as well as a first reference modal sample image that meets the first quality requirement and a second reference modal sample image that meets the second quality requirement.

[0144] For each training session, the sample images used for training can be registered to the individual space of each sample image using a preset image template to obtain their respective multi-channel initial tissue probability maps.

[0145] For example, a standard brain template (such as MNI152) is registered to the target modality individual space to generate a 3-channel tissue probability map P, corresponding to the probability distributions of cerebrospinal fluid (CSF), gray matter (GM), and white matter (WM), respectively. Here, the tissue probability map P is the initial tissue probability map. That is to say, at this time, only the initial content features of the sample image can be obtained, while the initial attribute features cannot be directly determined.

[0146] Step 408: Train the autoencoder based on each first image and its corresponding initial tissue probability map. The training of the autoencoder is completed when the first function value of the first total loss function is less than the first threshold.

[0147] For example, for each training session, the autoencoder is trained using the sample images used for training and the initial tissue probability maps corresponding to each sample image. The first total loss function value of the autoencoder is calculated, and the parameters of the autoencoder are adjusted according to the first total loss function value. Then, the next batch of training is performed. When the first function value of the first total loss function of the autoencoder is less than a first threshold, the training of the autoencoder is completed.

[0148] Further, in an exemplary embodiment, the autoencoder is trained based on each first image and its corresponding initial tissue probability map. The training of the autoencoder is completed when the first function value of the first total loss function is less than a first threshold, including:

[0149] Each first image is input into the encoder, which outputs a feature representation. The feature representation is then input into the decoder to determine the reconstructed image. Based on the reconstructed image and the first image, the reconstruction loss value of the image reconstruction loss function is determined. The target probability distribution and predicted probability distribution of the current voxel point in each first image belonging to multiple initial tissue probability maps are determined. Based on the target probability distribution and predicted probability distribution, the tissue probability divergence loss value of the tissue probability divergence loss function is determined. Based on the reconstruction loss value and the tissue probability divergence loss value, the first function value of the first total loss function is determined. When the first function value is less than a first threshold, the training of the autoencoder is completed.

[0150] For example, based on the specific calculation methods of the image reconstruction loss function and the tissue probability divergence loss function described above, the reconstruction loss value of the image reconstruction loss function is determined according to the reconstructed image and the first image; the target probability distribution and predicted probability distribution belonging to multiple initial tissue probability maps at the current voxel point in each first image are determined; the tissue probability map divergence loss value of the tissue probability divergence loss function is determined according to the target probability distribution and the predicted probability distribution; and the first function value of the first total loss function is determined according to the reconstruction loss value and the tissue probability map divergence loss value.

[0151] Step 410: Obtain the second sample image set, train the decoupling module based on the second sample image set, and complete the training of the decoupling module when the second function value of the second total loss function of the decoupling module is less than the second threshold, thus obtaining the trained multimodal feature decoupling network.

[0152] In this process, once the autoencoder has been trained, its network parameters are fixed. A second set of sample images is obtained from the prepared training sample data to train the decoupling module, specifically the two-stream deep feature decoupling module. The second set of sample images can be identical to or partially identical to the first set of sample images; no specific limitations are imposed here.

[0153] For example, based on the initial content features and initial attribute features determined by the trained encoder and sample images, the decoupling module is trained. When the function value of the second total loss function of the decoupling module is less than the second threshold, the training of the decoupling module is completed, and a trained multimodal feature decoupling network is obtained.

[0154] In an exemplary embodiment, a decoupling module is trained based on a second sample image set. When the second function value of the second total loss function of the decoupling module is less than a second threshold, the training of the decoupling module is completed, resulting in a trained multimodal feature decoupling network, including:

[0155] For each second image in the second sample image set, determine the initial tissue probability map and image features corresponding to each second image; input each second image and its corresponding initial tissue probability map into the trained encoder to obtain initial content features and initial attribute features; input the initial content features and initial attribute features into the decoupling module, and perform feature extraction processing on the initial attribute features through the attribute feature extraction network in the decoupling module to obtain target attribute features unrelated to content features; enhance the initial content features through the content feature extraction network in the decoupling module to obtain target content features; perform feature fusion on the target attribute features and target content features to obtain new cross-modal fusion features; determine the second function value of the second total loss function of the decoupling module based on the image features, target attribute features, target content features, initial tissue probability map, and cross-modal fusion features; when the second function value is less than the second threshold, the training of the decoupling module is completed, and a trained multimodal feature decoupling network is obtained.

[0156] It is understandable that determining the initial organization probability map based on the above method can only obtain the initial content features of the sample image, while the initial attribute features cannot be directly determined.

[0157] In an exemplary embodiment, each second image and its corresponding initial tissue probability map are input into a trained encoder to obtain initial content features and initial attribute features. This includes: inputting each second image into the encoder to obtain image features; inputting the initial tissue probability map corresponding to the second image into the encoder to obtain initial content features; and determining initial attribute features based on a preset relationship between the image, the tissue probability map, and the modal attribute map, according to the preset relationship, the image features, and the initial content features.

[0158] It is understandable that a sample image I can be decomposed into a tissue probability map P and a modal attribute map A, that is, the image, the tissue probability map, and the modal attribute map satisfy a predefined relationship I=A⊙P, which can be specifically expressed as:

[0159] (11)

[0160] Based on this, the sample images and their corresponding initial tissue probability maps are input to their respective encoders, which output image features and initial content features. Based on this, the relationship satisfied by formula (11) in the image space is transformed to obtain the relationship satisfied by attribute features, content features and image features in the feature space, and thus the initial attribute features can be determined. The initial attribute features f A Initial content features f P and image features f I The relation satisfied in the feature space can be expressed as:

[0161] (12)

[0162] Here, τ is a very small constant to avoid the denominator being 0.

[0163] Specifically, based on image features, target attribute features, target content features, and the initial probability map, the function value of the second total loss function of the decoupling module is determined, including:

[0164] Based on the aforementioned calculation methods for the implicit self-reconstruction function, implicit cross-reconstruction loss function, content decoupling loss function, attribute decoupling loss function, contrastive decoupling loss function, and attribute consistency loss function, the implicit self-reconstruction function value is determined according to the cross-modal fusion features and the image features of the target modality sample images in the second sample image set. The implicit cross-reconstruction loss value is determined according to the image features determined from the second sample image set and the cross-modal fusion features. The content decoupling loss value is determined according to the target content features of different modalities with different quality requirements and their corresponding initial probability map features. The attribute decoupling loss value is determined according to the difference between the initial attribute features of different quality requirements under the same modality. The contrastive decoupling loss value is obtained according to the target content features and target attribute features of different modalities with different quality requirements. The attribute consistency loss is calculated according to the target content features and target attribute features of different modalities with different quality requirements, resulting in the attribute consistency loss value. Based on these loss function values, the function value of the second total loss function of the decoupling module is determined.

[0165] The aforementioned methods construct a two-stream deep feature decoupling architecture based on a structure prior-guided multimodal feature decoupling network. Through a multi-resolution cross-modal joint optimization strategy, decoupling representation of neuroimages is achieved in the latent space dimension, separating content and attributes in the feature latent space. Furthermore, based on feature modulation methods, an anatomical structure-guided cross-modal feature fusion paradigm is established—a dual-path optimization paradigm of latent space decoupling-reconstruction. This paradigm extracts rich anatomical structure prior knowledge through a pre-trained high-quality auxiliary modality encoder, extracts modal attribute features from the target modality image, and fuses the decoupled features of the multimodal data using feature modulation methods. A cross-attention mechanism is then used to achieve precise conditional control, ensuring the structure-attribute consistency of the generated image and helping to reduce distortion.

[0166] Since the initial attribute features obtained are determined based on the registration method, the resulting initial attribute features are not accurate. Therefore, it is necessary to delete other features in the initial attribute features.

[0167] Optionally, in an exemplary embodiment, the initial attribute features are processed by the attribute feature extraction network in the decoupling module to obtain target attribute features that are unrelated to content features. This includes: extracting additional content features from the initial attribute features by the attribute feature extraction network in the decoupling module, deleting the additional content features from the initial attribute features, and obtaining target attribute features that are unrelated to content features.

[0168] For example, based on Figure 3 The network architecture diagram shown uses a self-attention module (SA) to further extract content information from the initial modality attribute features in the attribute feature optimization branch, and removes the extra content features from the original attribute features to generate content-irrelevant attribute features, i.e., target attribute features.

[0169] Accordingly, the initial content features are enhanced by the content feature extraction network in the decoupling module to obtain the target content features, including: aligning the initial attribute features and initial content features, and enhancing the initial content features by fusing additional content features to obtain the target content features.

[0170] For example, cross-attention is used to align the two features, and the additional content information extracted from the attribute feature optimization branch is fused into the content feature optimization branch. By supplementing the details, an enhanced content feature, namely the target content feature, is generated.

[0171] Furthermore, through the two-branch processing, the decoupled network (module) can generate target content features rich in anatomical details. And target attribute features that only represent modality and are unrelated to content. .Will and The input feature modulation module performs feature fusion to obtain new image features, which are then input into the decoder to reconstruct the neural image.

[0172] Compared to the existing technology that directly decouples images, this approach decouples image features and employs a dual-branch optimization method to further enhance decoupling on the basis of initial decoupling. This effectively suppresses information leakage between different feature components obtained from decomposition and improves the interpretability of neuroimaging features.

[0173] Understandably, implicit diffusion models progressively add noise to implicit representations during forward diffusion and achieve high-quality sample generation based on a reverse denoising process. Compared to traditional pixel-level generation paradigms, implicit diffusion models significantly reduce the computational complexity of high-dimensional data reconstruction by utilizing the compact representation characteristics of low-dimensional feature spaces. By extracting semantically consistent deep features from noisy multimodal images as diffusion priors, the fidelity of the generated samples' anatomical structure can be effectively improved. More importantly, these models support precise control of the generation process through conditional embedding mechanisms, providing a theoretical framework for the collaborative reconstruction of multimodal neural images.

[0174] In an exemplary embodiment, Gaussian noise is obtained, and cross-modal fusion features are used as conditional input to denoise the Gaussian noise, resulting in implicit features, including:

[0175] Gaussian noise is obtained and used as input. Cross-modal fusion features are used as conditional input and fed into a trained implicit diffusion model for stepwise denoising to obtain implicit features.

[0176] Among them, the trained implicit diffusion model is a quality-enhanced implicit diffusion model that integrates cross-modal feature fusion, such as... Figure 5 The diagram illustrates the architecture of a quality-enhancing implicit diffusion model for cross-modal feature fusion, including an encoder, decoder, forward denoising, and reverse denoising. The output of the diffusion process is pure Gaussian white noise feature data, which also serves as the input to the denoising process. The white cuboids represent the image representation in the latent feature space, i.e., image features. This model extracts modality-shared content features (e.g., tissue structure information such as the spatial distribution of gray, white matter, and cerebrospinal fluid) and modality-specific attribute features (e.g., texture characteristics such as the contrast of different modal images) through a multimodal feature decoupling network. Then, the feature modulation module integrates the decoupled features from different modalities, and the resulting cross-modal fusion features are used as conditions and injected into the denoising prediction network of the diffusion model to achieve quality-enhanced reconstruction of neuroimages under the constraint of rich anatomical structural information.

[0177] based on Figure 5 The diagram illustrates the architecture of a quality-enhanced implicit diffusion model, providing a training method for such a model. Figure 6 As shown, the following steps are included:

[0178] Step 602: Construct the third total loss function for the implicit diffusion model.

[0179] It should be noted that, in order to guide the model to adaptively focus on complex regions that are difficult to reconstruct, a structure fidelity enhancement strategy was determined. The structure fidelity enhancement strategy includes: designing a region selection strategy based on structural entropy weighted mapping, integrating high-frequency residual analysis and tissue boundary detection methods, and constructing a dual anatomy-driven loss function, i.e., a third total loss function.

[0180] The third overall loss function includes the noise prediction loss function, the boundary awareness loss function, and the high-frequency texture loss function. During the training of the implicit diffusion model, noise is predicted at each denoising step. The resulting noise prediction can be determined based on the UNet architecture, using a cross-attention mechanism to fuse conditional information. This conditional information can be the content features of the reference modality high-quality image after decoupling, determined by the multimodal feature decoupling network used to decouple features from the sample image set, and the attribute features of the target modality low-quality image after decoupling. The noise prediction formula can be expressed as:

[0181] (13)

[0182] Where Zt represents the feature data after adding noise at the t-th time step. This represents the content features after decoupling from the high-quality image of the reference modality. It represents the attribute features after decoupling of the low-quality image of the target modality.

[0183] Based on the noise prediction formula above, the noise prediction loss function can be determined. for:

[0184] (14)

[0185] Building upon this foundation, to better evaluate the structural fidelity of the enhanced reconstructed images, we utilize a reference modality-based tissue probability map to calculate the entropy value of each voxel. The entropy value reflects the uncertainty of voxel tissue affiliation; a higher entropy value indicates that the voxel is more likely to be located in the boundary region of different tissue structures. Therefore, when calculating the boundary-aware loss, the model should focus more on high-entropy voxels to improve the reconstruction quality of boundary regions.

[0186] Boundary-aware loss function It can be represented as:

[0187] (15)

[0188] The entropy value H(x,y,z) can be expressed as:

[0189] (16)

[0190] Where N is the number of organizational categories, i.e., the number of organizational probability maps, and P...i It is the probability of the i-th category. Represents the predicted image. This represents a real sample image.

[0191] Understandably, using the tissue probability map obtained by decoupling from the auxiliary modality high-quality data to calculate the structural entropy as a weight of the reconstruction loss function helps the model actively focus on the tissue boundary region. On the one hand, the auxiliary modality high-quality image is used, and the results are more reliable; on the other hand, each voxel point is assigned a corresponding weight, which solves the alignment problem.

[0192] The high-frequency texture loss function guides the network to better recover the texture information of an image by calculating the difference in texture details between the reconstructed image and the real image. The high-frequency texture loss function can be expressed as:

[0193] (17)

[0194] Where I T label A texture detail image representing a real image, also called a high-frequency detail image, contains high-frequency information in the image, such as the edges of structures and fine details; T pred This represents the texture detail image of the predicted image, i.e., the predicted high-frequency detail map.

[0195] Based on the completion of training of the multimodal feature decoupling network guided by structural priors, a quality-enhanced implicit diffusion model for cross-modal feature fusion is trained. The total loss function of this model is shown in Equation (18):

[0196] (18)

[0197] Step 604: Obtain a third sample image set for training the implicit diffusion model, wherein the third sample image set includes multiple sets of images with at least one quality requirement; each set of images includes at least two modalities.

[0198] The third sample image set is also determined by the accurate training sample image set. The third sample image set can be exactly the same as, partially the same as, or different from the first and second image sample sets.

[0199] Step 606: Perform Gaussian blur processing on the third image of the third sample image set to obtain a high-frequency detail image.

[0200] The third image can be a high-quality target modality image. The high-frequency detail map is obtained by subtracting the corresponding Gaussian blurred version from the high-quality target modality image, thus obtaining the high-frequency components. The determination of the high-frequency detail map includes: performing Gaussian blur processing on the second target modality sample image to obtain a low-frequency structure image; and processing the high-quality target modality image based on the low-frequency structure image to obtain the high-frequency detail map.

[0201] For example, a low-frequency structure image is obtained by adding Gaussian blur noise to the original image (such as a high-quality target modality image). Texture and detail can then be calculated by subtracting the low-frequency structure image from the original image. This is because Gaussian blur smooths the image and removes high-frequency information; therefore, the image obtained by subtracting the blur from the original image mainly retains the high-frequency texture and detail.

[0202] Step 608: For each image group in the third sample image set, the trained multimodal feature decoupling network is used to decouple the features of the image group to obtain content features and attribute features. The content features obtained from the reference modal image that meets the second quality requirement are decoded to obtain the optimized organization probability map.

[0203] The image set includes sample images of different qualities under different modalities. For example, it may include a second reference modal sample image, a first target modal sample image, and a second target modal sample image. The second reference modal sample image can also be called a high-quality reference modal image, and the first target modal sample image can also be called a low-quality target modal image.

[0204] For example, the trained multimodal feature decoupling network is used to extract features from the second reference modality sample image in each image group to obtain content features, and the content features are decoded to obtain an optimized organization probability map; the trained multimodal feature decoupling network is used to extract features from the first target modality sample image to obtain attribute features.

[0205] Furthermore, a pre-trained multimodal feature decoupling network is used to extract content features from the high-quality reference modality image in the sample image set. The content features obtained from the reference modality image that meets the second quality requirement are then input into the decoder to obtain an optimized organization probability map. The pre-trained multimodal feature decoupling network is also used to extract content features from the high-quality reference modality image and attribute features from the low-quality target modality image. These attribute features from the target modality and content features from the reference modality are then input into the feature modulation module to obtain cross-modal fusion features, which are used as prior conditions and injected into the denoising prediction network of the diffusion model.

[0206] Step 610: The image features obtained by the encoder of the third image are subjected to progressive noise addition processing to obtain target Gaussian noise.

[0207] The third image can be a high-quality target modal image, that is, a target modal image that meets the second quality requirement.

[0208] For example, the image features obtained by the encoder from the target modality image in the third sample image set that meets the second quality requirement are determined. These image features are then subjected to forward noise addition, progressively adding Gaussian noise into the low-dimensional implicit feature space (rather than the original image space) to obtain pure Gaussian white noise feature data, i.e., the target Gaussian noise. Here, the time step t∈{1,...,T}. A cosine scheduler is used for noise scheduling to balance training stability and reconstruction quality.

[0209] Step 612: Using the target Gaussian noise as the starting point for reverse denoising and cross-modal fusion features as control conditions, the input is fed into the denoising network of the implicit diffusion model for stepwise denoising, resulting in noise prediction for each step of denoising and the final output prediction image.

[0210] For example, inverse denoising is performed based on the aforementioned forward denoising. Inverse denoising starts with the target Gaussian noise and uses cross-modal fusion features as control conditions. Based on the UNet architecture, a cross-attention mechanism is used to fuse conditional information, resulting in noise predictions for each denoising step and the final predicted image output. For instance, the content features after decoupling from the high-quality reference modality image are used. Attribute features after decoupling (anatomical structure) from low-quality images of the target modality (Modal characteristics) are fused to obtain high-quality fused features as a condition to achieve precise control of the implicit diffusion model for enhancing brain image quality.

[0211] Step 614: Determine the value of the third total loss function based on noise prediction, high-frequency detail map, tissue probability map, content features, attribute features, and prediction image. When the value of the third total loss function is less than the third threshold, the training of the implicit diffusion model is completed, and the trained implicit diffusion model is obtained.

[0212] It is understandable that during the training process of the above diffusion model, the total loss function value during the training process is calculated based on the loss function constructed above, and the model parameters of the diffusion model are updated according to the total loss function.

[0213] For example, based on the aforementioned noise prediction loss function, boundary awareness loss function, and high-frequency texture loss function, the noise prediction loss value of the noise prediction loss function is determined according to noise prediction; the entropy value of each voxel in the predicted image is calculated based on the tissue probability map, and the error value of each voxel in the predicted image and the second target modality sample image is determined; the boundary awareness loss value of the boundary awareness loss function is determined based on the entropy value and the error value; the high-frequency texture loss value of the high-frequency texture loss function is determined based on the predicted image and the high-frequency detail map; the noise prediction loss value, boundary awareness loss value, and high-frequency texture loss value are weighted to determine the function value of the third total loss function. When the function value of the third total loss function is less than the third threshold, the training of the implicit diffusion model is completed, and the trained implicit diffusion model is obtained.

[0214] When determining the high-frequency texture loss value of the high-frequency texture loss function based on the predicted image and the high-frequency detail map, it is necessary to determine the predicted high-frequency detail map of the predicted image through the high-frequency detail map determination method described above. Based on this, the high-frequency texture loss value of the high-frequency texture loss function is determined according to the predicted high-frequency detail map and the high-frequency detail map based on the formula (17) above.

[0215] It should be noted that the aforementioned quality-enhanced implicit diffusion model for cross-modal feature fusion, specifically for medical images, addresses the problem of blurred reconstruction in complex microstructural regions such as the basal ganglia and hippocampus using existing methods. It proposes a dual anatomical-driven loss function: on the one hand, it utilizes a reference modal tissue probability map to construct a boundary-aware loss based on structural entropy weighting, where information entropy is calculated to quantify the reconstruction difficulty of key boundaries such as gray matter and cerebrospinal fluid; on the other hand, it designs a high-frequency texture consistency loss, using Gaussian blur to construct high-frequency detail maps, and enhances the reconstruction accuracy of complex textures in substructural regions such as thalamic nuclei by comparing the target with the generated results.

[0216] The aforementioned quality-enhanced implicit diffusion model for cross-modal feature fusion achieves efficient fusion of cross-modal features in a low-dimensional latent space by constructing a content-attribute decoupling-guided conditional generation mechanism. It utilizes a loss function system with anatomical interpretability, namely, using high-frequency detail maps and tissue probability maps as constraints on the model's loss function. Through a weight allocation mechanism, it guides the model to adaptively focus on complex regions that are difficult to reconstruct, enabling the reconstructed images to achieve diagnostic-grade quality standards in terms of tissue boundaries and texture fidelity.

[0217] It should be noted that the coefficients λ of the first total loss function, the second total loss function, and the third total loss function all range from 0 to 1.

[0218] In an exemplary embodiment, Figure 7 As shown, an image generation method is provided, which can be applied to... Figure 1Taking the terminal in the example, the explanation includes the following steps 702 to 716. Wherein:

[0219] Step 702: Obtain the first sample image set and the second sample image set for training, and train the multimodal feature decoupling network based on the first sample image set and the second sample image set to obtain the trained multimodal feature decoupling network.

[0220] Step 704: Obtain the third sample image set, and determine the high-frequency detail map of the second target modality image and the tissue probability map of the second reference modality sample image in the third sample image set.

[0221] Step 706: Use the trained multimodal feature decoupling network to extract features from the third sample image set to obtain the attribute features of the first target modality image and the content features of the second reference modality image.

[0222] For example, using the pre-trained multimodal feature decoupling network, content features and attribute features are extracted from the high-quality image of the reference modality and the low-quality image of the target modality, respectively. Then, the attribute features of the target modality and the content features of the reference modality are input into the feature modulation module to obtain cross-modal fusion features, which are then used as prior conditions and injected into the denoising prediction network of the diffusion model.

[0223] Step 708: Generate control conditions for the implicit diffusion model based on attribute features and content features.

[0224] Step 710: Forward noise is added to the third sample image set to obtain target noise. Inverse denoising is performed based on control conditions to train the implicit diffusion model and obtain the trained implicit diffusion model.

[0225] Step 712: Obtain the image to be processed, which includes a target modal image that meets the first quality requirement and a reference modal image that meets the second quality requirement.

[0226] Step 714: Decouple the features of the sample image by using the trained multimodal feature decoupling network to obtain the attribute features of the target modality image to be processed and the content features of the reference modality image to be processed.

[0227] Step 716: Starting with Gaussian noise, the fused cross-modal features are used as conditions to input the UNet network based on cross-attention of the implicit diffusion model, gradually denoising to generate implicit features, and decoding the implicit features to obtain a high-quality target modality image.

[0228] The fused cross-modal features are determined by fusing attribute features and content features. It should be noted that the specific implementation in this embodiment can be achieved through the methods defined above, and will not be elaborated upon here.

[0229] In this embodiment, a structure-prior-driven multimodal feature decoupling-generation collaborative architecture is constructed to address core issues in multimodal neuroimage quality enhancement tasks, such as cross-modal feature redundancy, high computational complexity of high-dimensional reconstruction, and anatomical structure distortion. By constructing a dual-stream deep feature decoupling architecture, neuroimage features are separated into cross-modal shared anatomical structure information (content features) and modality-specific attribute features, effectively eliminating semantic redundancy in the feature space. Furthermore, a cross-modal conditional generation framework is designed, employing a feature latent space conditional diffusion model, a content-attribute dual-stream guidance mechanism, and feature modulation methods to achieve cross-modal feature fusion. Based on this, a region selection strategy based on structural entropy weighted mapping is designed, integrating high-frequency residual analysis and tissue boundary detection methods to construct a dual-anatomical-driven loss function, guiding the model to adaptively focus on complex regions with high reconstruction difficulty. In other words, this embodiment enhances image quality through three dimensions: a multimodal feature decoupling mechanism, a cross-modal conditional generation framework, and a structural fidelity enhancement strategy, avoiding structural distortion and ensuring the topological consistency of biological structures.

[0230] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0231] Based on the same inventive concept, this application also provides an image generation apparatus for implementing the image generation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image generation apparatus embodiments provided below can be found in the limitations of the image generation method described above, and will not be repeated here.

[0232] In an exemplary embodiment, Figure 8 As shown, an image generation apparatus is provided, including: a data acquisition module 802, a decoupling module 804, a feature processing module 806, and an image processing module 808, wherein:

[0233] The data acquisition module 802 is used to acquire a target modal image that meets a first quality requirement and a reference modal image that meets a second quality requirement; wherein the first quality requirement is less than the second quality requirement.

[0234] The decoupling module 804 is used to decouple the features of the target modal image and the reference modal image to obtain the content features of the reference modal image and the attribute features of the target modal image.

[0235] The feature processing module 806 is used to fuse attribute features and content features to obtain cross-modal fused features.

[0236] The image processing module 808 is used to acquire Gaussian noise, take the cross-modal fusion features as conditional input, denoise the Gaussian noise to obtain implicit features, and decode the implicit features to obtain a second target modal image that meets the second quality requirements.

[0237] In the aforementioned image generation device, by decoupling the features of the target modality image that meets the first quality requirement and the reference modality image that meets the second quality requirement in the latent space dimension, content features and attribute features of the modality feature that can be used for cross-modal sharing are obtained, semantic redundancy in the feature space is eliminated, feature structure is guided by structural prior, and cross-modal fusion features are obtained by fusing the attribute features and content features obtained by decoupling. The cross-modal fusion features are used as conditional input to denoise Gaussian noise to obtain implicit features. The implicit features are decoded to generate the second target modality image that meets the second quality requirement, thereby improving the accuracy of image reconstruction.

[0238] In an exemplary embodiment, the decoupling module 804 is further configured to input the target modality image and the reference modality image into a trained multimodal feature decoupling network, perform feature decoupling through the multimodal feature decoupling network, and output the content features of the reference modality image and the attribute features of the target modality image.

[0239] In an exemplary embodiment, the image generation apparatus further includes a training module, which is used to construct a first total loss function of the autoencoder in the multimodal feature decoupling network and a second total loss function of the decoupling module in the multimodal feature decoupling network.

[0240] Obtain a first sample image set for training a multimodal feature decoupling network, wherein the first sample image set includes at least two image sets with different quality requirements under multiple modalities;

[0241] For the first image in the first sample image set, a preset image template is used to register it to the individual space of each first image to obtain the corresponding multi-channel initial tissue probability map;

[0242] The encoder is trained based on each first image and its corresponding initial tissue probability map. The training of the autoencoder is completed when the first function value of the first total loss function is less than the first threshold.

[0243] Obtain the second sample image set, train the decoupling module based on the second sample image set, and complete the training of the decoupling module when the second function value of the second total loss function of the decoupling module is less than the second threshold, thus obtaining the trained multimodal feature decoupling network.

[0244] In an exemplary embodiment, the training module is further configured to input each first image into an encoder, output a feature representation, and determine a reconstructed image based on the feature representation through a decoder;

[0245] Based on the reconstructed image and the first image, determine the reconstruction loss value of the image reconstruction loss function;

[0246] Determine the target probability distribution and predicted probability distribution of the current voxel point in each first image, which belong to multiple initial tissue probability maps;

[0247] Based on the target probability distribution and the predicted probability distribution, determine the organization probability map divergence loss value of the organization probability divergence loss function;

[0248] Based on the reconstruction loss value and the organization probability map divergence loss value, the first function value of the first total loss function is determined. When the first function value is less than the first threshold, the training of the autoencoder is completed.

[0249] In an exemplary embodiment, the training module is further configured to determine an initial tissue probability map and image features corresponding to each second image in the second sample image set.

[0250] Each second image and its corresponding initial organization probability map are input into the trained encoder to obtain initial content features and initial attribute features.

[0251] The initial content features and initial attribute features are input into the decoupling module. The attribute feature extraction network in the decoupling module performs feature extraction processing on the initial attribute features to obtain target attribute features that are unrelated to the content features.

[0252] The initial content features are enhanced by the content feature extraction network in the decoupling module to obtain the target content features;

[0253] Feature fusion is performed on target attribute features and target content features to obtain new cross-modal fusion features;

[0254] Based on image features, target attribute features, target content features, initial organization probability map, and cross-modal fusion features, the second function value of the second total loss function of the decoupling module is determined. When the second function value is less than the second threshold, the training of the decoupling module is completed, and a trained multimodal feature decoupling network is obtained.

[0255] In an exemplary embodiment, the training module is further configured to extract additional content features from the initial attribute features through the attribute feature extraction network in the decoupling module, remove the additional content features from the initial attribute features, and obtain target attribute features that are unrelated to the content features.

[0256] The initial content features are enhanced by the content feature extraction network in the decoupling module to obtain the target content features, including:

[0257] The initial attribute features and initial content features are aligned, and additional content features are fused to enhance the initial content features, resulting in the target content features.

[0258] In an exemplary embodiment, the training module is further configured to input each second image into the encoder to obtain image features, and input the initial tissue probability map corresponding to the second image into the encoder to obtain initial content features;

[0259] Based on the preset relationships between images, organization probability maps, and modal attribute maps, initial attribute features are determined according to the preset relationships, image features, and initial content features.

[0260] In an exemplary embodiment, the image processing module 808 is further configured to acquire Gaussian noise, use the Gaussian noise as input, use the cross-modal fusion features as conditional input, and input them into a trained implicit diffusion model for progressive denoising to obtain implicit features.

[0261] In one exemplary embodiment, the training module is also used to construct a third total loss function for the implicit diffusion model;

[0262] Obtain a third sample image set for training the implicit diffusion model, wherein the third sample image set comprises multiple sets of images with at least one quality requirement; each set of images includes at least two modalities;

[0263] Gaussian blurring is applied to the third image in the third sample image set to obtain a high-frequency detail image;

[0264] For each image group in the third sample image set, the trained multimodal feature decoupling network is used to decouple the features of the image group to obtain content features and attribute features. The content features obtained from the reference modal image that meets the second quality requirement are decoded to obtain the optimized organization probability map.

[0265] The image features obtained by the encoder in the third image are subjected to progressive noise addition to obtain target Gaussian noise.

[0266] Starting with the target Gaussian noise as the inverse denoising starting point, and using cross-modal fusion features as control conditions, the noise is input into the implicit diffusion model denoising network for step-by-step denoising, resulting in noise predictions for each denoising step and the final output prediction image.

[0267] The value of the third total loss function is determined based on noise prediction, high-frequency detail map, tissue probability map, content features, attribute features, and prediction image. When the value of the third total loss function is less than the third threshold, the training of the implicit diffusion model is completed, and the trained implicit diffusion model is obtained.

[0268] In an exemplary embodiment, the training module is further configured to perform Gaussian blur processing on the second target modality sample image to obtain a low-frequency structure image;

[0269] High-frequency detail images are obtained by processing the target modality high-quality image based on the low-frequency structure image;

[0270] Each image group includes a first target modality sample image that meets the first quality requirement and a second reference modality sample image that meets the second quality requirement. A trained multimodal feature decoupling network is used to decouple the features of the image group, obtaining content features and attribute features, including:

[0271] The trained multimodal feature decoupling network is used to extract features from the second reference modality sample image to obtain content features, and the content features are decoded to obtain an optimized organization probability map.

[0272] The trained multimodal feature decoupling network is used to extract features from the first target modality sample image to obtain attribute features.

[0273] In an exemplary embodiment, the training module is further configured to determine the noise prediction loss value of the noise prediction loss function based on the noise prediction.

[0274] The entropy value of each voxel in the predicted image is calculated based on the tissue probability map, and the error value of each voxel in the predicted image and the second target modality sample image is determined. The boundary sensing loss value of the boundary sensing loss function is determined based on the entropy value and the error value.

[0275] Based on the predicted image and the high-frequency detail map, determine the high-frequency texture loss value of the high-frequency texture loss function;

[0276] The noise prediction loss, boundary perception loss, and high-frequency texture loss are weighted to determine the value of the third total loss function.

[0277] Each module in the aforementioned image generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0278] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an image generation method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0279] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0280] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0281] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0282] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0283] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0284] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0285] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0286] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image generation method, characterized in that, The method includes: Acquire a target modal image that meets a first quality requirement and a reference modal image that meets a second quality requirement; wherein the first quality requirement is less than the second quality requirement; Feature decoupling is performed on the target modality image and the reference modality image to obtain the content features of the reference modality image and the attribute features of the target modality image; the feature decoupling is performed using a trained multimodal feature decoupling network; The attribute features and the content features are fused to obtain cross-modal fused features; Gaussian noise is obtained, and the Gaussian noise is used as input, while the cross-modal fusion feature is used as conditional input. The input is fed into the trained implicit diffusion model for stepwise denoising to obtain implicit features. The implicit features are decoded to obtain a second target modality image that meets the second quality requirement; The training methods for the multimodal feature decoupling network include: Construct the first total loss function of the autoencoder in the multimodal feature decoupling network, and the second total loss function of the decoupling module in the multimodal feature decoupling network; Obtain a first sample image set for training the multimodal feature decoupling network, wherein the first sample image set includes at least two image sets with different quality requirements under multiple modalities; For the first image in the first sample image set, a preset image template is used to register it to the individual space of each first image to obtain the corresponding multi-channel initial tissue probability map. The autoencoder is trained based on each first image and the corresponding initial tissue probability map. The training of the autoencoder is completed when the first function value of the first total loss function is less than the first threshold. Obtain a second sample image set, train the decoupling module based on the second sample image set, and complete the training of the decoupling module when the second function value of the second total loss function of the decoupling module is less than the second threshold, thereby obtaining a trained multimodal feature decoupling network. The training of the implicit diffusion model includes: Construct the third total loss function for the implicit diffusion model; A third sample image set is obtained for training the implicit diffusion model, wherein the third sample image set includes multiple sets of images with at least one quality requirement; each set of images includes at least two modalities; Gaussian blurring is applied to the third image in the third sample image set to obtain a high-frequency detail image; For each image group in the third sample image set, the trained multimodal feature decoupling network is used to decouple the features of the image group to obtain content features and attribute features. The content features obtained from the reference modal image that meets the second quality requirement are decoded to obtain an optimized organization probability map. The image features obtained by the encoder in the third image are subjected to progressive noise addition processing to obtain target Gaussian noise; Using the target Gaussian noise as the starting point for reverse denoising, and the cross-modal fusion features as control conditions, the data are input into the denoising network of the implicit diffusion model for step-by-step denoising, resulting in noise predictions for each denoising step and the final output prediction image. The function value of the third total loss function is determined based on the noise prediction, the high-frequency detail map, the tissue probability map, the content features, the attribute features, and the predicted image. When the function value of the third total loss function is less than the third threshold, the training of the implicit diffusion model is completed, and a trained implicit diffusion model is obtained.

2. The method according to claim 1, characterized in that, The step of decoupling the target modality image and the reference modality image to obtain the content features of the reference modality image and the attribute features of the target modality image includes: The target modality image and the reference modality image are input into a trained multimodal feature decoupling network. The multimodal feature decoupling network is used to decouple features and outputs the content features of the reference modality image and the attribute features of the target modality image.

3. The method according to claim 1, characterized in that, The first total loss function includes an image reconstruction loss function and a tissue probability divergence loss function. The step of training the autoencoder based on each first image and its corresponding initial tissue probability map, and completing the training of the autoencoder when the first function value of the first total loss function is less than a first threshold, includes: Each of the first images is input into the encoder, which outputs a feature representation. The feature representation is then input into the decoder to determine the reconstructed image. Based on the reconstructed image and the first image, determine the reconstruction loss value of the image reconstruction loss function; Determine the target probability distribution and predicted probability distribution of each of the initial tissue probability maps at the current voxel point in the first image; Based on the target probability distribution and the predicted probability distribution, determine the organization probability map divergence loss value of the organization probability divergence loss function; Based on the reconstruction loss value and the tissue probability map divergence loss value, a first function value of the first total loss function is determined. When the first function value is less than a first threshold, the training of the autoencoder is completed.

4. The method according to claim 3, characterized in that, The process of training the decoupling module based on the second sample image set, and completing the training of the decoupling module when the second function value of the second total loss function of the decoupling module is less than the second threshold, yields a trained multimodal feature decoupling network, including: For each second image in the second sample image set, determine the initial tissue probability map and image features corresponding to each second image; Each second image and its corresponding initial tissue probability map are input into the trained encoder to obtain initial content features and initial attribute features. The initial content features and the initial attribute features are input into the decoupling module, and the initial attribute features are processed by the attribute feature extraction network in the decoupling module to obtain target attribute features that are unrelated to the content features. The initial content features are enhanced by the content feature extraction network in the decoupling module to obtain the target content features; The target attribute features and the target content features are fused to obtain new cross-modal fusion features; Based on the image features, target attribute features, target content features, initial organization probability map, and cross-modal fusion features, the second function value of the second total loss function of the decoupling module is determined. When the second function value is less than the second threshold, the training of the decoupling module is completed, and a trained multimodal feature decoupling network is obtained.

5. The method according to claim 4, characterized in that, The step of extracting target attribute features unrelated to content features by using the attribute feature extraction network in the decoupling module to perform feature extraction processing on the initial attribute features includes: The attribute feature extraction network in the decoupling module extracts additional content features from the initial attribute features, and then removes the additional content features from the initial attribute features to obtain target attribute features that are unrelated to the content features. The step of enhancing the initial content features through the content feature extraction network in the decoupling module to obtain the target content features includes: The initial attribute features and the initial content features are aligned, and the initial content features are enhanced by fusing the additional content features to obtain the target content features.

6. The method according to claim 4, characterized in that, The step of inputting each of the second images and its corresponding initial tissue probability map into the trained encoder to obtain initial content features and initial attribute features includes: For each second image, an encoder is input to obtain image features. The initial tissue probability map corresponding to the second image is input to the encoder to obtain initial content features. Based on the preset relationship between the image, the organization probability map, and the modal attribute map, the initial attribute features are determined according to the preset relationship, the image features, and the initial content features.

7. The method according to claim 1, characterized in that, The third image is a second target modality sample image that meets the second quality requirement. The step of performing Gaussian blur processing on the sample image set to obtain a high-frequency detail image includes: Gaussian blurring is applied to the second target modality sample image to obtain a low-frequency structure image; The high-quality image of the target mode is processed based on the low-frequency structural image to obtain a high-frequency detail image; Each image group includes a first target modality sample image that meets the first quality requirement and a second reference modality sample image that meets the second quality requirement. The image group is decoupled using a trained multimodal feature decoupling network to obtain content features and attribute features, including: The trained multimodal feature decoupling network is used to extract features from the second reference modality sample image to obtain content features, and the content features are decoded to obtain an optimized organization probability map. The trained multimodal feature decoupling network is used to extract features from the first target modality sample image to obtain attribute features.

8. The method according to claim 7, characterized in that, The third total loss function includes a noise prediction loss function, a boundary-aware loss function, and a high-frequency texture loss function. Determining the value of the third total loss function based on the noise prediction, the high-frequency detail map, the tissue probability map, the content features, the attribute features, and the predicted image includes: The noise prediction loss value of the noise prediction loss function is determined based on the noise prediction. The entropy value of each voxel in the predicted image is calculated based on the tissue probability map, and the error value of each voxel in the predicted image and the second target modality sample image is determined. The boundary sensing loss value of the boundary sensing loss function is determined based on the entropy value and the error value. Based on the predicted image and the high-frequency detail map, the high-frequency texture loss value of the high-frequency texture loss function is determined; The noise prediction loss value, the boundary perception loss value, and the high-frequency texture loss value are weighted to determine the function value of the third total loss function.

9. An image generation apparatus, characterized in that, The device includes: The data acquisition module is used to acquire a target modal image that meets a first quality requirement and a reference modal image that meets a second quality requirement; wherein the first quality requirement is less than the second quality requirement; A decoupling module is used to decouple the target modality image and the reference modality image by means of features, thereby obtaining the content features of the reference modality image and the attribute features of the target modality image; the feature decoupling is performed using a trained multimodal feature decoupling network. The feature processing module is used to fuse the attribute features and the content features to obtain cross-modal fusion features; The image processing module is used to acquire Gaussian noise, take the Gaussian noise as input, take the cross-modal fusion feature as conditional input, and input it into the trained implicit diffusion model for stepwise denoising to obtain implicit features; The implicit features are decoded to obtain the second target modality image of the second quality requirement; The training module is used to construct the first total loss function of the autoencoder in the multimodal feature decoupling network, and the second total loss function of the decoupling module in the multimodal feature decoupling network; Obtain a first sample image set for training the multimodal feature decoupling network, wherein the first sample image set includes at least two image sets with different quality requirements under multiple modalities; For the first image in the first sample image set, a preset image template is used to register it to the individual space of each first image to obtain the corresponding multi-channel initial tissue probability map. The autoencoder is trained based on each first image and the corresponding initial tissue probability map. The training of the autoencoder is completed when the first function value of the first total loss function is less than the first threshold. Obtain a second sample image set, train the decoupling module based on the second sample image set, and complete the training of the decoupling module when the second function value of the second total loss function of the decoupling module is less than the second threshold, thereby obtaining a trained multimodal feature decoupling network. The training module is also used to construct the third total loss function of the implicit diffusion model; A third sample image set is obtained for training the implicit diffusion model, wherein the third sample image set includes multiple sets of images with at least one quality requirement; each set of images includes at least two modalities; Gaussian blurring is applied to the third image in the third sample image set to obtain a high-frequency detail image; For each image group in the third sample image set, the trained multimodal feature decoupling network is used to decouple the features of the image group to obtain content features and attribute features. The content features obtained from the reference modal image that meets the second quality requirement are decoded to obtain an optimized organization probability map. The image features obtained by the encoder in the third image are subjected to progressive noise addition processing to obtain target Gaussian noise; Using the target Gaussian noise as the starting point for reverse denoising, and the cross-modal fusion features as control conditions, the data are input into the denoising network of the implicit diffusion model for step-by-step denoising, resulting in noise predictions for each denoising step and the final output prediction image. The function value of the third total loss function is determined based on the noise prediction, the high-frequency detail map, the tissue probability map, the content features, the attribute features, and the predicted image. When the function value of the third total loss function is less than the third threshold, the training of the implicit diffusion model is completed, and a trained implicit diffusion model is obtained.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images