Medical image cross-modal generation method based on double-contrast frequency domain decomposition and adaptive diffusion

Abstract

The invention provides a medical image cross-modal generation method based on double-contrast frequency domain decomposition and adaptive diffusion. The method comprises the steps of obtaining and preprocessing an MRI sample, constructing a k-space decomposition and enhancement process, constructing a k-space double-flow adaptive degradation network, and constructing a k-space double-flow adaptive prediction network. Through a data enhancement technology and an adaptive Gaussian filter bank, targeted separation and enhancement of different frequency components are realized, common edge features are highlighted, and the quality of generated information is improved. A residual noise diffusion model is utilized to generate a high-resolution image with rich details and high diversity while keeping a smooth region and a large-scale structure of the image. An anatomical structure consistency constraint and multi-modal fusion optimization strategy is introduced, complementary information of T1 and T2 weighted MRI is fully utilized, and the generation effect and visual authenticity of medical image cross-modal generation in a complex scene are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically a cross-modal generation method for medical images based on dual-contrast frequency domain decomposition and adaptive diffusion, belonging to the field of image generation. Background Technology

[0002] The field of medical image generation is of great significance for improving the accuracy of disease diagnosis, treatment planning, and medical research. However, existing technologies have significant limitations in terms of multimodal data fusion, image detail accuracy, and the reliability of generated images. Traditional methods are mostly based on single-modal data, such as using only T1 or T2-weighted MRI, which cannot fully utilize the complementary information of different modalities, resulting in insufficient detail and structural integrity in the generated images. In recent years, deep learning has driven the development of multimodal image fusion methods, but existing technologies are prone to inconsistent feature distributions due to differences in imaging parameters during fusion, leading to grayscale shifts or texture conflicts, resulting in artifacts or loss of detail in the generated images.

[0003] Furthermore, existing technologies also have shortcomings in frequency domain processing. Most methods employ Discrete Cosine Transform (DCT) or Fast Fourier Transform (FFT) for global frequency domain decomposition, but these methods lack targeted modeling of the physical meaning of frequency bands. Low-frequency subbands dominate the main contours of the image, while high-frequency subbands contain details and noise. Existing technologies often treat the two interchangeably, using a unified network for processing, making it difficult to coordinate the optimization of low-frequency structure enhancement and high-frequency noise suppression. Diffusion models, such as DDPM, perform excellently in natural image denoising and generation, but their application in medical images is limited by high computational complexity and the lack of multimodal modulation mechanisms. Diffusion models require multiple iterations for denoising, making it difficult to meet the real-time requirements of clinical practice, and existing conditional diffusion models struggle to dynamically fuse dual-contrast information, resulting in the ineffective utilization of modality-specific features. In addition, k-space data offers more usable information compared to image domain data. k-space data is the raw data from the MRI imaging process, containing rich frequency and phase information, which is crucial for restoring image details and structure during image generation. By processing directly in the k-space, this information can be better preserved and utilized, thereby improving the quality and accuracy of image generation. Summary of the Invention

[0004] This invention proposes a cross-modal generation method for medical images based on dual-contrast frequency domain decomposition and adaptive diffusion, aiming to address the shortcomings of existing technologies in frequency domain information utilization and multimodal data fusion. By introducing a residual noise diffusion model, this method can generate high-resolution images with rich detail and high diversity while preserving smooth regions and large-scale structures. Targeted processing of medical image regions in the frequency domain is performed to improve the quality of high-frequency information during generation. Furthermore, anatomical structure consistency constraints and multimodal fusion optimization strategies are introduced, significantly improving the generation effect and visual realism of cross-modal medical image generation in complex scenes.

[0005] Specifically, the steps include the following:

[0006] S1: Acquisition and Preprocessing of MRI Samples

[0007] S2: Constructing the k-space decomposition and enhancement process

[0008] S3: Constructing a k-space two-stream adaptive degenerate network

[0009] S4: Construct a k-space dual-stream adaptive prediction network

[0010] Furthermore, the dataset used in step S1 is the IXISR dataset. This dataset focuses on brain MRI images and covers multiple modalities, including T1-weighted and T2-weighted images. These images are from subjects of different ages, sexes, and health conditions, ensuring broad coverage and diversity of the data. To ensure data consistency and applicability, all images underwent a meticulous preprocessing process, including precise image registration to ensure spatial alignment of anatomical structures, bicubic interpolation to unify image resolution, grayscale normalization to reduce grayscale differences between different images, and cropping or padding operations to ensure consistency in input image size.

[0011] The following section describes the process of constructing and enhancing k-space decomposition in S2:

[0012] In step S2, a k-space decomposition and enhancement process is constructed to improve the quality of high-frequency information during generation.

[0013] S21: Perform two-dimensional fast Fourier transform (FFT) on the input T1 and T2 weighted medical images respectively to transform the images from the spatial domain to the frequency domain representation, and obtain the corresponding complex frequency domain matrix.

[0014] S22: The standard deviation σ of the filter is dynamically generated by the ResSE module, and the frequency importance map is output. An adaptive Gaussian filter bank is designed to process the frequency domain data, thereby achieving targeted separation of different frequency components.

[0015] S23: The high-frequency components of the T1 and T2 images are cross-modulated by complex matrix dot product in the frequency domain to highlight the common edge features, while the low-frequency components are fused using a dynamic weighted fusion strategy.

[0016] S24: Phase constraints are applied to the fused high-frequency components to maintain the spatial consistency of the anatomical structures. This step, through frequency domain decomposition and feature enhancement, achieves effective separation and fusion of high and low frequency components in dual-contrast medical images, providing optimized frequency domain feature inputs for subsequent dual-stream degradation and prediction networks.

[0017] The construction of the k-space two-stream adaptive degradation network in step S3 is described in detail below:

[0018] In step S3, a k-space dual-stream adaptive degradation network is constructed to perform differentiated degradation processing on high-frequency and low-frequency components respectively.

[0019] S31: Generate a high-frequency mask M based on the frequency importance map. H and low-frequency mask M L High-frequency masks highlight high-frequency components at the periphery of the spectrum, while low-frequency masks highlight low-frequency components at the center of the spectrum. The mask design ensures that high-frequency and low-frequency components are effectively distinguished in the frequency domain.

[0020] S32: For the high-frequency components, a Gaussian noise degradation strategy that decreases with each time step is employed. Noise is sampled from a normal distribution and multiplied by a dynamically adjusted noise intensity, then gradually added to the mask-guided high-frequency components to simulate noise effects in actual imaging. This degradation process helps to better recover high-frequency details of the image during subsequent generation.

[0021] S33: For the low-frequency components, progressive residual degradation is achieved through a time-step-dependent Poisson disk sampling mask. The mask preservation ratio gradually increases with iterations, and the k-space low-frequency region is masked by point-by-point multiplication while residual information is calculated. This degradation process helps to better recover the low-frequency structural information of the image in subsequent generation processes.

[0022] S34: To fuse complementary information from both modes, the low-frequency residuals of the T1W image are used as the primary mode, and the low-frequency residuals of the T2W image are used as the secondary mode. A weighted fusion with adjustable weights α is then performed to generate the final low-frequency residuals. This step, through the synergistic effect of high-frequency noise degradation and low-frequency residual degradation, constructs a degradation model that conforms to the characteristics of medical images. This provides a structured learning objective for the subsequent dual-stream prediction network, ensuring that high-frequency details and low-frequency structural information can be effectively recovered during the generation process.

[0023] Finally, the construction of the k-space two-stream adaptive prediction network in step S4 is described in detail below:

[0024] In step S4, a k-space dual-stream adaptive prediction network is constructed, including a residual prediction network and a noise prediction network.

[0025] S41: The residual prediction network is responsible for predicting the residual information in the low-frequency part. It takes the degraded data in the low-frequency part as input and predicts the residual between the original high-resolution image and the degraded low-resolution image by learning the distribution of the data. The noise prediction network is responsible for predicting the noise information in the high-frequency part. It takes the degraded data in the high-frequency part as input and predicts the noise information in the high-frequency part by learning the distribution of the noise.

[0026] S42: By progressively removing noise and recovering high-frequency information, high-frequency details of the image are gradually generated. Finally, the generated k-space data is converted back to the image domain using inverse Fourier transform to obtain the final high-resolution image. During training, a loss function combining k-space loss and phase constraint loss is optimized to ensure prediction accuracy and image quality.

[0027] The beneficial effects of this invention are as follows:

[0028] 1. Data augmentation techniques, such as random cropping, flipping, and rotation, enhance the model's adaptability to different anatomical structures and pathological features, thereby improving its robustness. Targeted processing of medical image regions in the frequency domain is performed using an adaptive Gaussian filter bank and residual attention module to achieve targeted separation and enhancement of different frequency components, especially highlighting common edge features. This improves the quality of high-frequency information during generation, resulting in clearer image details.

[0029] 2. By applying the residual noise diffusion model, high-resolution images with rich details and high diversity can be generated while maintaining smooth regions and large-scale structures in the image, effectively improving the quality of cross-modal generation of medical images. Anatomical structure consistency constraints and multimodal fusion optimization strategies are introduced, fully utilizing the complementary information of T1- and T2-weighted MRI images. This solves the problem of grayscale shift or texture conflict caused by differences in imaging parameters during multimodal fusion in existing technologies, significantly improving the generation effect and visual realism of cross-modal generation of medical images in complex scenes. Attached Figure Description

[0030] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0031] Figure 1 Overall flowchart of cross-modal generation of medical images based on dual-contrast frequency domain decomposition and adaptive diffusion;

[0032] Figure 2Frequency domain processing diagram;

[0033] Figure 3 k-space degradation and prediction network diagram; Detailed Implementation

[0034] The following detailed description provides further details on specific implementation methods.

[0035] To overcome the shortcomings of existing technologies in frequency domain information utilization and multimodal data fusion, we propose a cross-modal generation method for medical images based on dual-contrast frequency domain decomposition and adaptive diffusion. The specific scheme of this invention is as follows:

[0036] S1. In step S1 of this invention, the dataset used is the IXISR dataset. During data preprocessing, medical image processing software or libraries are used to accurately register T1-weighted and T2-weighted images to ensure spatial alignment of anatomical structures. Using the OpenCV or PIL image processing library in Python, bicubic interpolation is used to unify the resolution of all images to the same size to ensure consistency of the input images. The pixel values ​​of the images are normalized to the range [0,1] to reduce grayscale differences between different images. To enhance the model's adaptability to different anatomical structures and pathological features and improve its robustness, data augmentation techniques such as random cropping, flipping, and rotation are applied to the processed images.

[0037] S2. In step S2 of this invention, an adaptive Gaussian filter bank is designed to process frequency domain data in order to separate different frequency components. We first perform a 2D FFT along the spatial dimension to obtain a frequency domain feature map M. The standard deviation σ of the filter is dynamically generated using the ResSE module. This module uses global average pooling and a two-layer fully connected network to learn channel attention weights. Based on the learned channel attention weights and the dynamically generated standard deviation, the frequency domain representation is weighted to generate a frequency importance map, representing the importance of different frequency components. Using the dynamically generated filter, high-low frequency decomposition is performed in the frequency domain, extracting low-frequency and high-frequency components from images T1 and T2 respectively. To enhance feature representation, the high-frequency components of T1 and T2 are cross-modulated, and frequency domain feature fusion is achieved through complex matrix multiplication, highlighting common salient edge features. The low-frequency components are fused using a weighted average strategy, with the weight coefficients dynamically determined by the attention map generated by the ResSE module.

[0038] S21. First, perform a two-dimensional Fast Fourier Transform (FFT) on the input T1 and T2 weighted medical images respectively to obtain the frequency domain feature map M, thus converting the spatial domain image to a frequency domain representation. For image T1 I T1 ∈R H×W and T2 image I T2 ∈R H×WAfter FFT, the corresponding complex frequency domain matrix M is obtained. T1 M T2 ∈R H×W In the frequency domain, the low-frequency components of an image are concentrated in the central region of the spectrum, representing the overall contrast and anatomical structure information of the tissue, while the high-frequency components are distributed in the periphery of the spectrum, corresponding to the texture details of the tissue boundaries and the features of minor lesions.

[0039] S22. To separate different frequency components, design an adaptive Gaussian filter bank to process frequency domain data. For each image, construct a Gaussian low-pass filter H with dynamic standard deviation. Low and high-pass filter H High The key filter parameter, standard deviation σ, is dynamically generated by the ResSE module. The frequency domain feature map M is input into the ResSE module, which first compresses the spatial dimension using global average pooling, then learns channel attention weights through a two-layer fully connected network containing residual connections, ultimately outputting a frequency importance map matching the input size. Based on this, an adaptive standard deviation is calculated.

[0040]

[0041] Where l is the length of the short side of the image, ensuring that the filtering range is adapted to the image scale.

[0042] High- and low-frequency decomposition is performed in the frequency domain using dynamically generated filters. For the T1 image, the low-frequency components are extracted using a Gaussian low-pass filter, while the high-frequency components are obtained using a high-pass filter. The specific formulas are as follows:

[0043]

[0044] Where D(u,v) represents the distance from the center in the frequency domain coordinates.

[0045] The analogical decomposition process of the T2 image uses an independent adaptive parameter σ. T2 To adapt to the feature distribution characteristics of different weighted images.

[0046] S23. To enhance feature representation, the high-frequency components of T1 and T2 are cross-modulated in the frequency domain. Frequency domain feature fusion is achieved through complex matrix dot product, with the specific formula as follows:

[0047]

[0048] Where conj represents the conjugate operation.

[0049] This operation highlights the salient edge features shared by the two weighted images. The low-frequency components are then fused using a weighted average strategy, with the specific formula as follows:

[0050]

[0051] The weighting coefficient α is dynamically determined by the attention map generated by the ResSE module.

[0052] S24. The fused frequency domain features are subjected to residual degradation and noise degradation through a dual-stream adaptive degradation network. A phase correction stage is introduced before degradation to apply phase constraints to the fused high-frequency components in order to maintain the spatial consistency of the anatomical structure.

[0053] S3. In step S3 of this invention, we introduce a masking mechanism to distinguish between high-frequency and low-frequency components, enabling targeted processing in the future. High-frequency and low-frequency masks generated based on the frequency importance map highlight the high-frequency components at the periphery of the spectrum and the low-frequency components at the center of the spectrum, respectively. During high-frequency noise degradation, we use Gaussian noise to simulate the noise effect in actual imaging; the intensity of this noise gradually decreases with time steps. For low-frequency residual degradation, we design a time-step dependent mask whose radius gradually decreases with increasing time steps to simulate a progressive degradation process. The retention ratio of this mask gradually increases with time steps. Poisson disk sampling is used to simulate the physical process of k-space progressive sampling. We apply this mask to the low-frequency parts of T1 and T2 k-space to obtain the degraded low-frequency components. Subsequently, we calculate the low-frequency residual, i.e., the difference between the original low-frequency component and the low-frequency component after masking. This masked low-frequency information will be used in subsequent network training to recover the low-frequency prior of the image. To fully utilize the complementary information of the two modalities, we use the low-frequency residuals of the T1W image as the primary modality and the low-frequency residuals of the T2W image as the secondary modality, and combine them using a weighted method to generate the final low-frequency residual. This weighting method can be adjusted according to specific task requirements to better utilize the complementary information of the two modalities.

[0054] S31. Before degradation processing, a mask guide is introduced to distinguish high-frequency and low-frequency components, ensuring that they can be processed specifically in subsequent processing. A high-frequency mask M is generated based on the frequency importance map. H and low-frequency mask M L High-frequency masks highlight the high-frequency components at the periphery of the spectrum, while low-frequency masks highlight the low-frequency components at the center of the spectrum.

[0055] S32. Regarding step S3, in the high-frequency noise degradation process, we choose Gaussian noise to simulate the noise effect in actual imaging. For Gaussian noise, we use a normal distribution... The sampling noise ∈ , where σ is the noise standard deviation at time step t. The noise intensity λ gradually decreases with time step t, as shown in the following formula:

[0056]

[0057] Where λ is the maximum noise intensity, t is the current time step, and T is the total number of time steps. We multiply the noise ∈ by λ. t This is then added to the high-frequency image in k-space to obtain a degraded high-frequency image. t The specific formula is as follows:

[0058] Image t =Image+λt*∈

[0059] S33. In the low-frequency residual degradation process, we design a time-step dependent mask M. t (The mask radius gradually decreases as t increases, simulating gradual degradation), used to control the proportion of the retained k-space region. Mask M t The retention rate gradually increases with time step t, as shown by the following formula:

[0060]

[0061] Where r max Let be the initial maximum radius (covering the entire central region of k-space), and T be the total number of time steps. The region within the mask (D(u,v) <= r) t Reserved, externally set to zero, simulating the physical process of k-space asymptotic sampling, mask M t We use Poisson disk sampling. We will use the mask M. t When applied to the low-frequency Real in T1 and T2k-space, a degraded low-frequency Real is obtained. t The specific formula is as follows:

[0062] Real t =M t ⊙Real

[0063] Here, ⊙ represents point-by-point multiplication.

[0064] S34. Then, we further calculate the low-frequency residual, the low-frequency residual r t This represents the low-frequency information obscured by the mask, which will be used in subsequent network training to recover the low-frequency prior of the high-resolution image. Specifically, the low-frequency residual is obtained by subtracting the original low-frequency component from the masked low-frequency component, i.e., the portion of the original low-frequency component that is masked. To fully utilize the complementary information of the two modalities, we use the low-frequency residual r of the T1W image... T1 As the dominant mode, the low-frequency residual r of the T2W image T1 As an auxiliary mode, we can combine the residual information of the two modes in a weighted manner to generate the final low-frequency residual r. t :

[0065]

[0066] Here, α is a weighting parameter used to balance the contributions of the two modes. This weighting method can be adjusted according to the needs of specific tasks to fully utilize the complementary information and frequency details of the two modes.

[0067] S4. In the reverse generation process, we start with degraded k-space data and progressively remove noise and recover high-frequency information to generate an image. Starting with the degraded k-space data, including both low-frequency and high-frequency components, for each time step, we first use a residual prediction network to predict the residual information for the low-frequency components, and then use a noise prediction network to predict the noise information for the high-frequency components. By progressively removing noise and recovering high-frequency information, we gradually generate the high-frequency details of the image. Finally, we use inverse Fourier transform to convert the generated k-space data back to the image domain to obtain the final high-resolution image. During training, we optimize a loss function that combines k-space loss and phase constraint loss to ensure prediction accuracy and image quality.

[0068] S41. Starting with the degraded k-space data, which contains both degraded low-frequency and high-frequency components, this degraded data was generated using the adaptive degradation network from the previous steps and includes noise and residual information. For each time step t, the low-frequency component is processed using a residual prediction network. This network learns the distribution of the data and predicts the residual information between the original high-resolution image and the low-resolution image. For each time step t, the high-frequency component is processed using a noise prediction network. This network learns the distribution of noise and predicts the noise information in the high-frequency component. Specifically, it predicts the noise present in the high-frequency component at the current time step and progressively removes this noise to restore the high-frequency details of the image.

[0069] S43. The predicted high-frequency and low-frequency components are combined to form more complete k-space data. This combination process ensures the coordinated optimization of high-frequency details and low-frequency structural information, providing high-quality input for the subsequent inverse Fourier transform. The combined k-space data is then transformed back to the image domain using an inverse Fourier transform to generate the generated image at the current time step. This step maps the data from the frequency domain back to the spatial domain, allowing us to intuitively observe the quality and details of the generated image. During training, we optimize a comprehensive loss function that integrates k-space loss and phase constraint loss to ensure the accuracy of the prediction results and the overall quality of the image. The specific formula is as follows:

[0070] L=λ1*E k-space [||kk HR ||2 ]+λ2*[∠(k)-∠(k LR )]

[0071] Where λ1 and λ2 are weighting parameters used to balance the effects of different loss terms.

[0072] Specifically, the loss term measures the difference between the generated k-space data and the k-space data of the real high-resolution image. This loss term ensures the consistency between the generated result and the real data in the frequency domain, thereby optimizing the generation quality at the frequency domain level. Phase constraints are imposed on the generated high-frequency components to ensure the spatial consistency of anatomical structures. This loss term preserves the phase information of the image, allowing the generated image to better reflect the real anatomical structure in the spatial domain. Weight parameters are introduced to balance the influence of different loss terms. These weight parameters are dynamically adjusted based on the performance during training to achieve fine-tuning of the importance of each loss term. In this way, we can take into account the generation quality in both the frequency and spatial domains during the optimization process, thereby improving the overall effect of cross-modal generation of medical images.

Claims

1. A method for cross-modal generation of medical images based on dual-contrast frequency domain decomposition and adaptive diffusion, characterized in that, Includes the following steps: S1: Acquisition and preprocessing of MRI samples; S2: Constructing the k-space decomposition and enhancement process; S3: Construct a k-space two-stream adaptive degradation network; S4: Construct a k-space dual-stream adaptive prediction network.

2. The method for cross-modal generation of medical images based on dual-contrast frequency domain decomposition and adaptive diffusion according to claim 1, characterized in that... Step S3: In this step, a masking mechanism is introduced to distinguish between high-frequency and low-frequency components. The high-frequency mask emphasizes the high-frequency parts on the periphery of the spectrum, while the low-frequency mask emphasizes the low-frequency parts in the center of the spectrum. For the high-frequency part, a Gaussian noise degradation strategy that decreases with time step is adopted; that is, noise is sampled from a normal distribution, where the noise standard deviation changes with time step t, and the noise intensity gradually decreases. The sampled noise is multiplied by the corresponding coefficient and added to the high frequency in k-space to obtain the degraded high frequency. This degradation process is beneficial for better recovery of high-frequency details in the subsequent generation process. For the low-frequency part, progressive residual degradation is achieved through a time-step dependent Poisson disk sampling mask. The retention ratio of the mask increases with the increase of time step t, while the mask radius increases with the increase of time step t. The mask is reduced in size, preserving the area within the mask and setting the outer region to zero, thus simulating the physical process of k-space progressive sampling. The mask uses a Poisson disk sampling method and is applied to the low-frequency part of the k-space of T1 and T2 to obtain the degraded low-frequency part. Then, the low-frequency residual is further calculated, that is, the original low-frequency component is subtracted from the low-frequency component after masking, to obtain the part of the original low-frequency component that was masked. In order to make full use of the complementary information of the two modes, the low-frequency residual of the T1W image is regarded as the main mode and the low-frequency residual of the T2W image is regarded as the auxiliary mode. Then, the residual information of the two modes is combined by weighting to generate the final low-frequency residual. The weight parameter is used to determine the contribution of the two balanced modes and can be adjusted according to the specific task requirements to effectively utilize the complementary information of the two modes in the frequency details.

3. The method for cross-modal generation of medical images based on dual-contrast frequency domain decomposition and adaptive diffusion according to claim 1, characterized in that... In step S4, the residual information of the low-frequency part and the noise information of the high-frequency part are predicted first through the residual prediction network and the noise prediction network, respectively. Specifically, the residual prediction network takes the degradation data of the low-frequency part as input and predicts the residual information between the original image and the degradation image by learning the distribution of the data; the noise prediction network takes the degradation data of the high-frequency part as input and predicts the noise information of the high-frequency part by learning the distribution of the noise. To progressively remove noise and restore high-frequency information, thereby gradually generating high-frequency details in the image, the process begins with degraded k-space data, including both low-frequency and high-frequency components. For each time step t, a residual prediction network is used to process the low-frequency components, predicting the residual information between the original and degraded images. Simultaneously, a noise prediction network is used to process the high-frequency components, predicting noise information and progressively removing this noise to restore the high-frequency details of the image. Finally, the generated k-space data is transformed back into the image domain using inverse Fourier transform to obtain the final high-resolution image. During training, a loss function combining k-space loss and phase constraint loss is optimized to ensure prediction accuracy and image quality; the specific formula is as follows: L=λ1*E k-space [||k-k HR || 2 ]+λ2*[∠(k)-∠(k LR )] Wherein, λ1 and λ2 are weighting parameters used to balance the influence of different loss terms; In this way, we can take into account the generation quality in both the frequency domain and the spatial domain during the optimization process, thereby improving the overall effect of cross-modal generation of medical images.

Images