Conditional diffusion model-based high-fidelity medical image generation method and device

Abstract

The invention provides a high-fidelity medical image generation method and device based on a conditional diffusion model. The method comprises the steps of forward diffusion and reverse denoising. In each time step of forward diffusion, Gaussian noise is added to the input medical image of the forward diffusion; in each time step of reverse denoising, target noise is generated based on the noise prediction network, and denoising is carried out based on the target noise; in the step of generating the target noise based on a preset noise prediction network, the noise prediction network comprises a coding network, a process processing network and a decoding network; the process processing network comprises a preprocessing module, a semantic boundary enhancement module and a three-dimensional multi-window attention perception module which are arranged in sequence; the semantic boundary enhancement module is provided with two processing channels and a semantic differential convolution unit, and the output ends of the two processing channels are connected with the semantic differential convolution unit; the three-dimensional multi-window perceptual attention module models global dependency features based on a self-attention mechanism.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and apparatus for generating high-fidelity medical images based on a conditional diffusion model. Background Technology

[0002] Medical imaging plays a central role in disease diagnosis, surgical planning, and efficacy evaluation, especially in 3D medical imaging (such as MRI and CT), where doctors can observe the morphology and location of lesions from multiple planes, thus obtaining more comprehensive diagnostic and treatment information. However, the acquisition and use of 3D medical images face multiple challenges. High-quality image acquisition is costly, time-consuming, and requires strict equipment conditions; data involves patient privacy and is subject to legal and ethical restrictions, making large-scale sharing difficult. Furthermore, image annotation requires the participation of professional radiologists, which is time-consuming and costly, resulting in a severe shortage of publicly available, high-quality, and precisely annotated 3D datasets. This data shortage not only limits the training and optimization of AI models in the field of medical imaging but also affects the generalization and implementation of AI systems in clinical practice. Against this backdrop, using generative models to generate high-quality 3D medical images has become an important direction for data augmentation, privacy protection, and AI-assisted diagnosis. While traditional Generative Adversarial Networks (GANs) have been widely used in 2D image generation, they often face problems such as unstable training, pattern collapse, and insufficient ability to model complex spatial structures in 3D tasks, resulting in defects such as spatial discontinuities, structural distortion, and missing details.

[0003] In recent years, diffusion models have gradually demonstrated their advantages in image generation due to their high generation stability, good image quality, and strong detail restoration capabilities, and have shown the potential to outperform GANs in some medical imaging tasks. However, most existing technologies focus on two-dimensional images or unconditional 3D synthesis, and the diffusion models used often employ normally distributed noise. Therefore, they still have shortcomings in generating high-quality, semantically controllable 3D medical images, especially in terms of limited adaptability to cross-organ and cross-modal tasks. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a high-fidelity medical image generation method based on a conditional diffusion model to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a high-fidelity medical image generation method based on a conditional diffusion model, wherein the conditional diffusion model includes a forward diffusion module and a reverse denoising module, and the method includes a forward diffusion process of the forward diffusion module and a reverse denoising process of the reverse denoising module. In each time step of the forward diffusion, Gaussian noise is added to the input medical image for the forward diffusion; In each time step of the reverse denoising, target noise is generated based on a preset noise prediction network, and denoising is performed based on the target noise; In the step of generating target noise based on a preset noise prediction network, the noise prediction network includes an encoding network, a process processing network, and a decoding network arranged in sequence. The process processing network performs noise prediction based on a preprocessing module, a semantic boundary enhancement module, and a three-dimensional multi-window perception attention module set in sequence. The semantic boundary enhancement module is equipped with two processing channels and a semantic differential convolution unit. The output of both processing channels is connected to the semantic differential convolution unit to model long-range dependencies and local structural features in 3D volume data. The three-dimensional multi-window perception attention module models global dependency features based on a self-attention mechanism.

[0006] Using the above approach, this scheme first generates high-fidelity 3D medical images through forward diffusion and reverse denoising processes based on a diffusion model. In the reverse denoising process, a noise prediction network incorporating a semantic boundary enhancement module and a 3D multi-window perception attention module is used to generate target noise. The semantic boundary enhancement module explicitly models the structural differences between semantic masks and noise features, enabling the model to focus on the boundaries and morphological changes of key anatomical regions. The 3D multi-window perception attention module uses a deformable window strategy to jointly model spatial dependencies at different scales, directions, and anatomical structural levels, better capturing global topological structures of brain regions (such as ventricular morphology and gray / white matter distribution) and local tissue texture features (such as cortical hierarchical structures and tumor interiors). In summary, compared to the traditional method using Gaussian noise, this scheme enhances the model's ability to perceive fine-grained anatomical structures and textures, improving its adaptability across organs or modalities.

[0007] In some embodiments of the present invention, the two processing channels of the semantic boundary enhancement module are a first processing channel and a second processing channel, respectively; the first processing channel includes a differential convolutional layer, a convolutional layer and an LN layer arranged in sequence; the second processing channel includes a three-dimensional convolutional layer, a differential convolutional layer, a convolutional layer, an LN layer and a self-multiplying layer arranged in sequence; the semantic differential convolutional unit includes a convolutional layer, a batch normalization layer and a ReLU layer arranged in sequence; the input of the semantic boundary enhancement module and the output of the ReLU layer of the semantic differential convolutional unit are concatenated as the output of the semantic boundary enhancement module.

[0008] In some embodiments of the present invention, the three-dimensional multi-window perception attention module includes a self-attention processing channel and a convolution processing channel. The self-attention processing channel is provided with an axial box, a sagittal box, a coronal box, and a local window. Attention calculations are performed based on the axial box, sagittal box, coronal box, and local window, respectively. The outputs of the attention mechanism calculations for the axial box, sagittal box, coronal box, and local window are concatenated as the output of the self-attention processing channel. The convolution processing channel is sequentially provided with a convolutional layer, a convolutional layer, a GELU layer, a convolutional layer, and an LN layer. The outputs of the self-attention processing channel and the convolution processing channel are concatenated as the output of the three-dimensional multi-window perception attention module.

[0009] In some embodiments of the present invention, in the step of performing attention calculations based on the axial frame, sagittal frame, coronal frame, and local window respectively... For the attention calculation of the axial frame, the attention is calculated separately through the axis in the height direction to match the coordinate axis direction of the image section; For the attention calculation of the sagittal frame, attention is calculated separately through the axis in the lateral direction to enhance the feature representation in the sagittal slice; For the attention calculation of the coronal frame, attention is calculated separately through the depth axis to capture global dependencies in the coronal slice; For the attention calculation of the local window, the attention is calculated separately for each local cube window to capture local detail information.

[0010] In some embodiments of the present invention, the preprocessing module of the process processing network includes residual blocks, attention blocks and residual blocks arranged in sequence.

[0011] In some embodiments of the present invention, the noise prediction network further includes an input module and an output module. The input module concatenates the input of the previous inverse denoising time step with a preset semantic mask and outputs it to the encoding network. The output module includes a GroupNorm layer, a SiLU layer and a 3D convolutional layer arranged in sequence. The output of the three-dimensional multi-window perception attention module is input to the output module, and the target noise is output through the 3D convolutional layer of the output module.

[0012] In some embodiments of the present invention, both the encoding network and the decoding network are provided with four processing layers; The coding network includes a first coding layer, a second coding layer, a third coding layer, and a fourth coding layer; The first coding layer is sequentially configured with a 3D convolutional layer and two residual blocks; Both the second and third coding layers have two residual blocks set sequentially; The fourth coding layer is sequentially configured with residual blocks, attention blocks, residual blocks, and attention blocks; The decoding network includes a first decoding layer, a second decoding layer, a third decoding layer, and a fourth decoding layer; The first decoding layer, the second decoding layer, and the third decoding layer each have three residual blocks arranged sequentially; The fourth decoding layer is sequentially configured with residual blocks, attention blocks, residual blocks, attention blocks, residual blocks, and attention blocks.

[0013] In some embodiments of the present invention, the four processing layers of the encoding network correspond one-to-one with the four processing layers of the decoding network, wherein the first encoding layer corresponds to the first decoding layer, the second encoding layer corresponds to the second decoding layer, the third encoding layer corresponds to the third decoding layer, and the fourth encoding layer corresponds to the fourth decoding layer.

[0014] In some embodiments of the present invention, during the processing of the decoding network, the output of the first encoding layer is concatenated with the output of the second decoding layer as the input of the first decoding layer; the output of the second encoding layer is concatenated with the output of the third decoding layer as the input of the second decoding layer; and the output of the third encoding layer is concatenated with the output of the fourth decoding layer as the input of the third decoding layer.

[0015] A second aspect of the present invention also provides a high-fidelity medical image generation apparatus based on a conditional diffusion model. The apparatus includes a computer device, the computer device including a processor and a memory, the memory storing computer instructions, and the processor executing the computer instructions stored in the memory. When the computer instructions are executed by the processor, the apparatus performs the steps implemented by the method described above.

[0016] A third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned high-fidelity medical image generation method based on a conditional diffusion model.

[0017] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0018] Those skilled in the art will understand that the objectives and advantages achievable with this invention are not limited to those specifically described above, and that the above and other objectives achievable with this invention will become clearer from the following detailed description. Attached Figure Description

[0019] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0020] Figure 1 This is a schematic diagram of the diffusion model in the high-fidelity medical image generation method based on the conditional diffusion model of the present invention; Figure 2 This is a schematic diagram of the overall structure of the noise prediction network in this scheme; Figure 3 This is a schematic diagram of the semantic boundary enhancement module in this scheme; Figure 4 This is a schematic diagram of the structure of the 3D multi-window perception attention module in this solution; Figure 5 This is a schematic diagram of the experimental results for sample 1 in the experiment. Figure 6 This is a schematic diagram of the experimental results for sample 2 in the experiment. Figure 7 This is a schematic diagram of the experimental results for samples 3 and 4. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0022] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0023] To address the aforementioned issues, this solution proposes a novel conditional diffusion model based on existing technology diffusion models. This model aims to improve the detail fidelity and visual quality of generated 3D medical images while maintaining the consistency of anatomical structures. The solution introduces a semantic boundary enhancement module and a 3D multi-window perception attention module, significantly improving the model's texture detail restoration, anatomical boundary recognition, and long-range dependency modeling. This reduces artifacts, enhances edge sharpness, and improves the ability to reconstruct complex anatomical structures (such as brain sulci and gyri).

[0024] like Figure 1 As shown, this invention proposes a high-fidelity medical image generation method based on a conditional diffusion model. The conditional diffusion model includes a forward diffusion module and a reverse denoising module. The method includes a forward diffusion process of the forward diffusion module and a reverse denoising process of the reverse denoising module. In each time step of the forward diffusion, Gaussian noise is added to the input medical image for the forward diffusion; like Figure 2 As shown, in each time step of the reverse denoising, target noise is generated based on a preset noise prediction network, and denoising is performed based on the target noise; In the specific implementation process, the overall framework of this plan is as follows: Figure 1 As shown, this scheme is based on the DDPM model and generates high-fidelity 3D medical images through forward diffusion and backward denoising processes. Building upon this, a mask is incorporated as a conditional control to achieve refined generation of anatomical structures such as tumors and brain sulci. Furthermore, this scheme integrates two core modules—a semantic boundary enhancement module and a 3D multi-window perception attention module—into the denoising network, significantly improving boundary accuracy, texture detail, and structural fidelity. During the forward diffusion process, the clean input medical image is gradually noise-generated over multiple time steps by adding Gaussian noise. At each time step, the noisy image... The generating formula is expressed as: in This represents Gaussian noise sampled from a standard multivariate normal distribution, with parameters... Controlling noise scaling at each step and accumulating the product Define the proportion of the retained signal. This represents the noise scaling factor of the preceding time step s of time step t during the forward diffusion process. In the time step t of the reverse denoising process, the model starts with pure Gaussian noise samples and gradually reconstructs the image through a learned conditional denoising network. The time step t of the reverse denoising process is defined as: in, It is the noise scaling factor for the time step t of the reverse denoising process. The noise is predicted by the denoising network, that is, the predicted noise output by the noise prediction network. It is Gaussian noise sampled at each step. It is the standard deviation of the added noise. Represents the time step of the reverse denoising process The forward noise variance is used to control the amount of noise added during the diffusion process. .

[0025] Furthermore, to ensure semantic controllability, this scheme employs a conditional diffusion strategy: at each time step, the noisy image and the semantic mask are concatenated along the channel dimension. The resulting composite tensor serves as the network input, enabling the model to generate anatomically meaningful outputs guided by the semantic structure. This process effectively models the conditional distribution for predicting the previous denoised image from the current noisy input. Detailed steps for model training and inference are shown in Algorithm 1 in Table 1 and Algorithm 2 in Table 2. This scheme uses a cosine-based... A timetable is defined as: in, It is a smoothing factor. It represents the total number of diffusion steps.

[0026] Table 1 Output: A trained model Table 2 Output: The generated image In this scheme, both the input medical image and the semantic mask are represented as volume data in three-dimensional space: width ( ),high( ) and depth ( To achieve semantic controllability during denoising, this scheme employs a conditional diffusion strategy, concatenating the noisy image with a single-encoded semantic mask along the channel dimension to form a conditional input. Since this framework supports multiple datasets with different imaging modes and label schemes, adjustments will be made accordingly. , and The construction method is as follows. For the BraTS2021 dataset, each input medical image contains four MRI modes (T1, T1ce, T2, and FLAIR), forming a 4-channel volumetric data. After single encoding and background category exclusion, the semantic mask becomes a 4-channel binary volumetric data. At each time step, the noisy image... and Concatenate to generate conditional input For the LiTS dataset, the input is a single-channel 3D CT scan. After a single encoding and removal of the background label, the mask has two channels. (Noisy image) and By splicing, a result was generated. This flexible conditional mechanism allows Integ-DDPM to adapt to different organ types and imaging modalities. While the core diffusion framework remains consistent, certain architectural parameters (such as the number of residual blocks and the number of input / output channels) are adjusted based on the characteristics of each dataset. These adjustments ensure compatibility while preserving the overall design philosophy of conditional semantic guidance.

[0027] In the step of generating target noise based on a preset noise prediction network, the noise prediction network includes an encoding network, a process processing network, and a decoding network arranged in sequence. The process processing network performs noise prediction based on a preprocessing module, a semantic boundary enhancement module, and a three-dimensional multi-window perception attention module set in sequence. The semantic boundary enhancement module is equipped with two processing channels and a semantic differential convolution unit. The output of both processing channels is connected to the semantic differential convolution unit to model long-range dependencies and local structural features in 3D volume data. The three-dimensional multi-window perception attention module models global dependency features based on a self-attention mechanism.

[0028] In the specific implementation process, to enhance the model's ability to perceive fine-grained anatomical structures and textures, this scheme constructs a noise prediction network, integrating a semantic boundary enhancement module and a 3D multi-window perception attention module. The semantic boundary enhancement module improves the model's sensitivity to boundaries by modeling semantic and feature differences; the 3D multi-window perception attention module combines variable window attention and convolutional pathways to enhance the ability to model local and global dependencies.

[0029] Using the above approach, this scheme first generates high-fidelity 3D medical images through forward diffusion and reverse denoising processes based on a diffusion model. In the reverse denoising process, a noise prediction network incorporating a semantic boundary enhancement module and a 3D multi-window perception attention module is used to generate target noise. The semantic boundary enhancement module explicitly models the structural differences between semantic masks and noise features, enabling the model to focus on the boundaries and morphological changes of key anatomical regions. The 3D multi-window perception attention module uses a deformable window strategy to jointly model spatial dependencies at different scales, directions, and anatomical structural levels, better capturing global topological structures of brain regions (such as ventricular morphology and gray / white matter distribution) and local tissue texture features (such as cortical hierarchical structures and tumor interiors). In summary, compared to the traditional method using Gaussian noise, this scheme enhances the model's ability to perceive fine-grained anatomical structures and textures, improving its adaptability across organs or modalities.

[0030] like Figure 3As shown, in some embodiments of the present invention, the two processing channels of the semantic boundary enhancement module are a first processing channel and a second processing channel, respectively; the first processing channel includes a differential convolutional layer, a convolutional layer, and an LN layer arranged in sequence; the second processing channel includes a three-dimensional convolutional layer, a differential convolutional layer, a convolutional layer, an LN layer, and a self-multiplication layer arranged in sequence; the semantic differential convolutional unit includes a convolutional layer, a batch normalization layer, and a ReLU layer arranged in sequence; as shown... Figure 3 As shown, the input of the semantic boundary enhancement module and the output of the ReLU layer of the semantic differential convolution unit are concatenated to form the output of the semantic boundary enhancement module.

[0031] In its implementation, the semantic boundary enhancement module of this scheme aims to enhance the model's sensitivity to semantic differences and structural boundaries between noisy features and conditional guidance. The core of the semantic boundary enhancement module is a semantic differential convolution (SDC) unit, which receives the outputs of two channels: a noisy feature map from the first processing channel of the main network and a semantic guidance map from the second processing channel. To ensure channel compatibility, the processing of the second channel is performed with channel alignment via a 3x3x3 3D ​​convolution. Then, both channels are processed through their own independent 3D convolutions, using learnable differential kernels in their respective differential convolutional layers. and The initialization of these kernels is designed to emphasize orientation-aware structural features.

[0032] The processing of two differential convolutional layers can be represented as: x and g represent the inputs of the differential convolutional layers for the first and second channels, respectively; and These represent the outputs of the differential convolutional layers for the first and second channels, respectively.

[0033] In the specific implementation process, the noisy feature map output by the first processing channel is multiplied and fused with the semantic guidance map output by the second processing channel.

[0034] The fused tensor is then passed through a 3D convolution, followed by batch normalization and ReLU activation: ; in, This represents the output of the semantic boundary enhancement module. This represents the fused tensor obtained by multiplying the output of the first processing channel by the output of the second processing channel. This represents the input to the semantic boundary reinforcement module.

[0035] Using the above scheme, a residual connection will The data is fed back into the processed output, refining local features while preserving the global background. The differential kernel used in the convolution is a 3x3x3 filter, initialized with fixed values ​​(-1 for the center voxel, ±1 for the corners) to extract directional cues. Other weights remain learnable, allowing the module to adapt to the complexity of different anatomical structures during training.

[0036] like Figure 4 As shown, in some embodiments of the present invention, the three-dimensional multi-window perception attention module includes a self-attention processing channel and a convolution processing channel. The self-attention processing channel is provided with an axial box, a sagittal box, a coronal box, and a local window. Attention calculations are performed based on the axial box, sagittal box, coronal box, and local window, respectively. The outputs of the attention mechanism calculations for the axial box, sagittal box, coronal box, and local window are concatenated as the output of the self-attention processing channel. The convolution processing channel is sequentially provided with a convolutional layer, a convolutional layer, a GELU layer, a convolutional layer, and an LN layer. The outputs of the self-attention processing channel and the convolution processing channel are concatenated as the output of the three-dimensional multi-window perception attention module.

[0037] In some embodiments of the present invention, in the step of performing attention calculations based on the axial frame, sagittal frame, coronal frame, and local window respectively... For the attention calculation of the axial frame, the attention is calculated separately through the axis in the height direction to match the coordinate axis direction of the image section; For the attention calculation of the sagittal frame, attention is calculated separately through the axis in the lateral direction to enhance the feature representation in the sagittal slice; For the attention calculation of the coronal frame, attention is calculated separately through the depth axis to capture global dependencies in the coronal slice; For the attention calculation of the local window, the attention is calculated separately for each local cube window to capture local detail information.

[0038] In some embodiments of the present invention, the preprocessing module of the process processing network includes residual blocks, attention blocks and residual blocks arranged in sequence.

[0039] To further enhance spatial modeling and global context understanding, this scheme introduces a 3D multi-window perceptual attention module after the semantic boundary enhancement module. The 3D multi-window perceptual attention module integrates self-attention and convolution to effectively model long-range dependencies and local structural features in 3D volumetric data.

[0040] The 3D multi-window perception attention module will input feature map P Divided into four groups along the channel dimension, each group has Each group has one channel. Each group is assigned a different attention window type: axial strip (along...). ), sagittal bands (along ), coronal bands (along A local cube window and a multi-head self-attention window are used. These windows enable the model to collectively capture global and local spatial patterns. Multi-head self-attention is applied to each group. in, , and They are the first The query, key, and value projection matrix for each window group.

[0041] The outputs of the four attention branches are concatenated: To enable cross-window communication, this solution employs a window translation scheme between adjacent attention layers, translating window partitions. .

[0042] Furthermore, a lightweight convolutional path is applied to the value features. To extract local information from structure perception: Here, LN represents layer normalization, and GELU introduces nonlinearity.

[0043] The outputs of the two paths are passed through learnable scalar weights. and To merge: By adopting the above approach, the model can simultaneously capture long-range dependencies and fine spatial structure, significantly improving performance.

[0044] In some embodiments of the present invention, the noise prediction network further includes an input module and an output module. The input module concatenates the input of the previous inverse denoising time step with a preset semantic mask and outputs it to the encoding network. The output module includes a GroupNorm layer, a SiLU layer and a 3D convolutional layer arranged in sequence. The output of the three-dimensional multi-window perception attention module is input to the output module, and the target noise is output through the 3D convolutional layer of the output module.

[0045] like Figure 2 As shown, in some embodiments of the present invention, both the encoding network and the decoding network are provided with four processing layers; The coding network includes a first coding layer, a second coding layer, a third coding layer, and a fourth coding layer; The first coding layer is sequentially configured with a 3D convolutional layer and two residual blocks; Both the second and third coding layers have two residual blocks set sequentially; The fourth coding layer is sequentially configured with residual blocks, attention blocks, residual blocks, and attention blocks; The decoding network includes a first decoding layer, a second decoding layer, a third decoding layer, and a fourth decoding layer; The first decoding layer, the second decoding layer, and the third decoding layer each have three residual blocks arranged sequentially; The fourth decoding layer is sequentially configured with residual blocks, attention blocks, residual blocks, attention blocks, residual blocks, and attention blocks.

[0046] In some embodiments of the present invention, the four processing layers of the encoding network correspond one-to-one with the four processing layers of the decoding network, wherein the first encoding layer corresponds to the first decoding layer, the second encoding layer corresponds to the second decoding layer, the third encoding layer corresponds to the third decoding layer, and the fourth encoding layer corresponds to the fourth decoding layer.

[0047] In some embodiments of the present invention, during the processing of the decoding network, the output of the first encoding layer is concatenated with the output of the second decoding layer as the input of the first decoding layer; the output of the second encoding layer is concatenated with the output of the third decoding layer as the input of the second decoding layer; and the output of the third encoding layer is concatenated with the output of the fourth decoding layer as the input of the third decoding layer.

[0048] During the pre-training process of the diffusion model in this scheme, ① Use MRIPreprocessor to perform image registration during dataset preprocessing to ensure consistent spatial alignment between different modalities; ② To avoid training order bias caused by numerical class labels in the segmentation mask, the segmentation encoding labels are remapped. For the BraTS2021 dataset, the original segmentation mask contains labels 0 (background), 1 (NCR / NET), 2 (ED), and 4 (ET). To standardize the tumor category, label 4 is remapped to 3, and additional label 4 is introduced to represent the brain region by thresholding the T1 pattern. The final segmentation contains five categories: 0 represents background, 1-3 represent tumor subregions, and 4 represents brain tissue. The segmentation mask in the LiTS dataset contains three original class labels: 0 represents background, 1 represents liver, and 2 represents tumor. To avoid ordinal bias during training, single encoding is applied and the background category is discarded. The resulting semantic mask has two channels: channel 0 corresponds to the liver region, and channel 1 represents the tumor region; ③ Scale the image intensity and normalize it to the specified value. The intervals were defined, and MRI and CT images were cropped.

[0049] Experimental Example To verify the performance of the proposed method in 3D medical image generation, this method randomly selected two samples from the BraTS2021 brain MRI test set, and presented the slice results in the axial, sagittal, and coronal directions, as shown below. Figure 5 and 6 As shown in the figure, the real image, the corresponding segmentation mask, the generation results of existing techniques, and the generation results of the proposed method are presented simultaneously to allow for a direct comparison of the generation quality of different methods.

[0050] To quantitatively assess the quality of the generated 3D medical images, this scheme employs five widely used metrics: MSE, PSNR, MS-SSIM, FID, and VIF.

[0051] Table 3 According to Table 3, in terms of pixel-level fidelity, this scheme achieves the lowest mean square error of 0.03218, significantly outperforming existing technologies Med-DDPM (0.04668) and Pix2PixNIfTI (0.03450). Furthermore, this scheme also achieves the highest PSNR value of 15.9629, indicating that the generated image is closer to the real data in terms of intensity values. From a perceptual quality perspective, the MS-SSIM score of this scheme is 0.66165, lower than Pix2PixNIfTI (0.92110), but still higher than the existing technology Med-DDPM (0.64520), demonstrating better structure preservation. The FID of Integ-DDPM is significantly reduced to 125.83, far lower than the existing technologies Med-DDPM (1522.32) and Pix2PixNIfTI (335.91), indicating that the generated image distribution is closer to the real data. The VIF score also improved to 0.41356, outperforming the other two existing techniques, indicating better image detail preservation. Although the existing Pix2PixNIfTI model achieved the highest MS-SSIM score (0.92110), the generated images appeared overly smooth, lacking the texture details of real images. This phenomenon may lead to an inflated MS-SSIM score, failing to accurately reflect actual visual quality. In contrast, this approach, while maintaining structural coherence and texture information, performs better in both FID and VIF metrics, generating images that visually closer to real samples.

[0052] Qualitative evaluation of the generated images was conducted by experts. The images generated by the existing Pix2PixNIfTI technology exhibit severe distortion, disruption of normal brain anatomy, significant artifacts, loss of realistic brain features, and overall image distortion. The images generated by the existing Med-DDPM technology are of poor quality, exhibiting significant noise, severe loss of detail, blurred brain structure outlines, unclear regional boundaries, and obvious artifacts such as grayscale anomalies. In contrast, the images generated by this proposed solution demonstrate superior structural clarity and anatomical consistency. Brain structures, ventricular morphology, and sulci contours are identifiable to a certain extent, anatomically positioned reasonably, and gray and white matter are largely distinguishable with relatively fewer artifacts. However, compared to realistic images, this solution still suffers from detail loss, such as the disappearance or shallowing of fine branches in the gyri and sulci, blurred gray and white matter boundaries, slightly blurred ventricular edges, and insufficient preservation of tumor boundaries. Based on the combined quantitative analysis results and expert qualitative evaluation, this proposed solution can generate high-quality 3D medical images with clear anatomical structures, reduced artifacts, and strong structural consistency. Although slight blurring still exists in some fine areas such as groove branches and tumor boundaries, the overall image quality is significantly better than existing methods, indicating good prospects for clinical application.

[0053] like Figure 7 As shown, the experimental examples demonstrate the generation results of two liver samples in axial, sagittal, and coronal views. These results include the generated liver images and their corresponding ground truth masks, as well as magnified views of the tumor region. The generated liver images were generated at a resolution of 128×128×128, consistent with the training data. Due to computational resource limitations, the sharpness of some generated images was affected by the resolution—the liver boundaries appeared blurred in some generated images. Although the tumor region could be accurately generated and corresponded to the mask position, the boundary definition was not clear enough. In addition, horizontal stripe artifacts were visible in some sagittal and coronal sections, which may be related to voxel interpolation or model capability limitations and requires further analysis. Despite the presence of local blurring and artifacts, this scheme can still generate structurally sound and semantically consistent abdominal CT images. The overall visual effect of the generated images is satisfactory, and the location information of the tumor region is accurately presented. These results indicate that even under resolution-limited conditions, this scheme still demonstrates good adaptability to novel organ types.

[0054] This invention also provides a high-fidelity medical image generation device based on a conditional diffusion model. The device includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method described above.

[0055] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned high-fidelity medical image generation method based on a conditional diffusion model. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0056] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0057] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0058] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating high-fidelity medical images based on a conditional diffusion model, characterized in that, The conditional diffusion model includes a forward diffusion module and a reverse denoising module, and the steps of the method include the forward diffusion process of the forward diffusion module and the reverse denoising process of the reverse denoising module. In each time step of the forward diffusion, Gaussian noise is added to the input medical image for the forward diffusion; In each time step of the reverse denoising, target noise is generated based on a preset noise prediction network, and denoising is performed based on the target noise; In the step of generating target noise based on a preset noise prediction network, the noise prediction network includes an encoding network, a process processing network, and a decoding network arranged in sequence. The process processing network performs noise prediction based on a preprocessing module, a semantic boundary enhancement module, and a three-dimensional multi-window perception attention module set in sequence. The semantic boundary enhancement module is equipped with two processing channels and a semantic differential convolution unit. The output of both processing channels is connected to the semantic differential convolution unit to model long-range dependencies and local structural features in 3D volume data. The three-dimensional multi-window perception attention module models global dependency features based on a self-attention mechanism.

2. The high-fidelity medical image generation method based on the conditional diffusion model according to claim 1, characterized in that, The semantic boundary enhancement module has two processing channels: a first processing channel and a second processing channel. The first processing channel includes a differential convolutional layer, a convolutional layer, and an LN layer arranged in sequence. The second processing channel includes a three-dimensional convolutional layer, a differential convolutional layer, a convolutional layer, an LN layer, and a self-multiplying layer arranged in sequence. The semantic differential convolutional unit includes a convolutional layer, a batch normalization layer, and a ReLU layer arranged in sequence. The input of the semantic boundary enhancement module and the output of the ReLU layer of the semantic differential convolutional unit are concatenated to obtain the output of the semantic boundary enhancement module.

3. The high-fidelity medical image generation method based on the conditional diffusion model according to claim 1, characterized in that, The 3D multi-window perception attention module includes a self-attention processing channel and a convolution processing channel. The self-attention processing channel is configured with axial boxes, sagittal boxes, coronal boxes, and local windows. Attention calculations are performed based on the axial boxes, sagittal boxes, coronal boxes, and local windows, respectively. The outputs of the attention mechanism calculations for the axial boxes, sagittal boxes, coronal boxes, and local windows are concatenated as the output of the self-attention processing channel. The convolution processing channel is configured with convolutional layers, GELU layers, convolutional layers, and LN layers in sequence. The outputs of the self-attention processing channel and the convolution processing channel are concatenated as the output of the 3D multi-window perception attention module.

4. The high-fidelity medical image generation method based on the conditional diffusion model according to claim 3, characterized in that, In the steps of calculating attention based on the axial frame, sagittal frame, coronal frame, and local window respectively... For the attention calculation of the axial frame, the attention is calculated separately through the axis in the height direction to match the coordinate axis direction of the image section; For the attention calculation of the sagittal frame, attention is calculated separately through the axis in the lateral direction to enhance the feature representation in the sagittal slice; For the attention calculation of the coronal frame, attention is calculated separately through the depth axis to capture global dependencies in the coronal slice; For the attention calculation of the local window, the attention is calculated separately for each local cube window to capture local detail information.

5. The high-fidelity medical image generation method based on the conditional diffusion model according to claim 1, characterized in that, The preprocessing module of the process processing network includes residual blocks, attention blocks, and residual blocks arranged sequentially.

6. The high-fidelity medical image generation method based on the conditional diffusion model according to claim 1, characterized in that, The noise prediction network further includes an input module and an output module. The input module concatenates the input of the previous inverse denoising time step with a preset semantic mask and outputs it to the encoding network. The output module includes a GroupNorm layer, a SiLU layer and a 3D convolutional layer arranged in sequence. The output of the three-dimensional multi-window perception attention module is input to the output module, and the target noise is output through the 3D convolutional layer of the output module.

7. The high-fidelity medical image generation method based on the conditional diffusion model according to claim 1, characterized in that, Both the encoding and decoding networks are configured with four processing layers; The coding network includes a first coding layer, a second coding layer, a third coding layer, and a fourth coding layer; The first coding layer is sequentially configured with a 3D convolutional layer and two residual blocks; Both the second and third coding layers have two residual blocks set sequentially; The fourth coding layer is sequentially configured with residual blocks, attention blocks, residual blocks, and attention blocks; The decoding network includes a first decoding layer, a second decoding layer, a third decoding layer, and a fourth decoding layer; The first decoding layer, the second decoding layer, and the third decoding layer each have three residual blocks arranged sequentially; The fourth decoding layer is sequentially configured with residual blocks, attention blocks, residual blocks, attention blocks, residual blocks, and attention blocks.

8. The high-fidelity medical image generation method based on the conditional diffusion model according to claim 7, characterized in that, The four processing layers of the encoding network correspond one-to-one with the four processing layers of the decoding network, wherein the first encoding layer corresponds to the first decoding layer, the second encoding layer corresponds to the second decoding layer, the third encoding layer corresponds to the third decoding layer, and the fourth encoding layer corresponds to the fourth decoding layer.

9. The high-fidelity medical image generation method based on the conditional diffusion model according to claim 8, characterized in that, During the processing of the decoding network, the output of the first encoding layer is concatenated with the output of the second decoding layer as the input of the first decoding layer; the output of the second encoding layer is concatenated with the output of the third decoding layer as the input of the second decoding layer; and the output of the third encoding layer is concatenated with the output of the fourth decoding layer as the input of the third decoding layer.

10. A high-fidelity medical image generation device based on a conditional diffusion model, characterized in that, The device includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method as described in any one of claims 1 to 9.

Images