Medical image generation method and system based on KAN enhanced multi-phase diffusion

Abstract

This invention belongs to the field of medical image generation technology, and particularly relates to a method and system for generating medical images based on KAN-enhanced multi-temporal diffusion. The method includes: acquiring multi-temporal medical images and structural prior information; extracting temporally specific latent features; generating conditional guidance information by performing cross-modal nonlinear fusion and temporal dependency modeling through a KAN multimodal temporal conditioner and a KAN temporal aggregation module, respectively; generating structural guidance features through a structural guidance consistency module; inputting the conditional guidance information and structural guidance features into a multi-temporal diffusion backbone network to reconstruct the latent features of the target image, and obtaining the target medical image through decoding; and jointly training each module through a hybrid loss function. This invention enhances multi-temporal feature fusion through KAN enhancement, using structural priors to constrain spatial consistency, effectively improving problems such as motion artifacts and structural blurring, and enhancing the anatomical rationality and physiological realism of the generated images.

Description

Technical Field

[0001] This invention belongs to the field of medical image generation technology, and particularly relates to a method and system for generating medical images based on KAN-enhanced multi-temporal diffusion. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Medical imaging plays a crucial role in disease diagnosis, efficacy evaluation, radiotherapy planning, treatment monitoring, and adaptive therapy decision-making. Commonly used imaging modalities such as CT, MRI, CBCT, PET, SPECT, ultrasound, and 4D CT can reflect the anatomical structure, density distribution, functional metabolism, and motion changes of human tissues from different perspectives. However, due to factors such as scanning cost, acquisition time, radiation dose, equipment differences, patient movement, respiratory artifacts, and modal loss, clinically acquired medical images often suffer from insufficient resolution, high noise levels, incomplete temporal phases, missing modal information, blurred structural boundaries, or spatial registration errors, severely limiting their further application in precision medicine.

[0004] In recent years, deep learning-based medical image generation technology has been widely applied to tasks such as cross-modal image synthesis, image quality enhancement, functional image generation, treatment process image prediction, and radiotherapy-assisted decision-making. This mainly includes methods such as convolutional neural networks, generative adversarial networks, and diffusion models. Despite some progress, existing methods still have the following shortcomings: First, traditional feature fusion strategies often employ simple stitching, weighted summation, or multilayer perceptrons, which are insufficient for fully modeling the complex nonlinear relationships between multi-temporal and multimodal medical images. Second, most methods have limited ability to model temporal or phase changes, making it difficult to accurately capture dynamic image features caused by organ movement, anatomical changes during treatment, or functional state alterations. Third, existing generation models mainly rely on image domain reconstruction loss, lacking explicit prior constraints on anatomical structures such as organs, blood vessels, lesions, target areas, or organs at risk, leading to problems such as structural discontinuities, blurred boundaries, local artifacts, or insufficient anatomical plausibility in the generated images. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides a medical image generation method and system based on KAN-enhanced multi-temporal diffusion. The aim is to enhance the fusion of multi-temporal features by leveraging the nonlinear representation capability of KAN and to explicitly constrain the spatial consistency of the generation process with structural priors, thereby improving the spatial consistency, anatomical rationality, clinical interpretability, and physiological authenticity of the generated images.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In the first aspect, a medical image generation method based on KAN-enhanced multi-temporal diffusion is disclosed, including: acquiring multi-temporal medical images and structural prior information related to the target medical image generation task; and using a shared three-dimensional convolutional encoder to independently encode each temporal image to obtain temporal-specific latent features. Conditional guidance information is generated through the KAN multimodal temporal conditioner and the KAN temporal aggregation module; structural guidance consistency information is encoded through the structural guidance consistency module to generate structural guidance features. Conditional guidance information and structural guidance features are input into a multi-temporal diffusion backbone network. Conditional forward noise addition and backward noise reduction are performed in the latent space to gradually reconstruct the latent features of the target medical image. The latent features are then decoded by a pre-trained VAE decoder to generate the target medical image. A generative model is obtained by jointly training the modules using a hybrid loss function.

[0007] Furthermore, multi-temporal medical images include medical images obtained from the same subject at different time points, different scanning phases, or different physiological states; the target medical images include structural medical images, functional medical images, enhanced medical images, medical images of the treatment process, and dose-related medical images.

[0008] Furthermore, the structural prior information includes one or more of the following: organ mask, structural prior mask, lesion mask, target area mask, organ at risk mask, tissue boundary or anatomical contour.

[0009] Furthermore, the KAN temporal aggregation module performs nonlinear fusion of multi-temporal or multi-modal latent features through the B-spline basis function expansion of the KAN layer. The fusion formula is as follows:

[0010] in, The confidence weight for each phase at a specific voxel location. For B-spline basis function extension, For the first Phase features in voxels The eigenvector at that location.

[0011] Furthermore, the structure-guided consistency module utilizes a lightweight 3D structure encoder to extract the global topological features and spatial distribution features of the structure prior mask to generate a structure latent embedding; it eliminates the feature distribution differences between samples through layer normalization; and then maps the structure condition vector to the same dimension as the temporal condition embedding through linear projection.

[0012] Furthermore, the multi-temporal diffusion backbone network performs a forward diffusion process in the latent space, and the process formula is as follows: ; in, The noise variance scheduling parameter, For the current latent vector, Let be the latent noise variable at time t. Let represent the covariance matrix of the Gaussian noise added at time step t.

[0013] Furthermore, in the reverse denoising process, the multi-temporal diffusion backbone network inputs temporal conditional embedding vectors and structurally guided features as conditional information into the denoising network, and guides noise prediction and latent feature reconstruction of the target medical image through cross-attention, feature splicing or feature modulation.

[0014] Secondly, a medical image generation system based on KAN-enhanced multi-temporal diffusion is disclosed, including: The data acquisition module is used to acquire multi-temporal medical images of the same subject, as well as structural prior information related to the target medical image generation task; A shared feature extraction module is used to independently encode each temporal image using a shared 3D convolutional encoder to obtain temporal-specific latent features; The KAN enhanced condition generation module includes a KAN multimodal time conditioner and a KAN time aggregation module, which are used to perform nonlinear fusion and time series modeling on the phase-specific latent features to generate condition guidance information. The structure-guided consistency module is used to encode and modulate the structure prior information to generate structure-guided features; A multi-temporal diffusion backbone network is used to receive the conditional guidance information and the structural guidance features, perform conditional guidance forward noise addition and reverse noise reduction processes in the latent space, and gradually reconstruct the latent feature representation of the target medical image; the latent feature representation is decoded by a pre-trained VAE decoder to generate the target medical image; The training module is used to jointly train the KAN-enhanced conditional generation module, the structure-guided consistency module, and the multi-temporal diffusion backbone network using a hybrid loss function to obtain a trained medical image generation model.

[0015] Thirdly, a computer device is disclosed, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the steps of the method described above.

[0016] Fourthly, a computer-readable storage medium is disclosed having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0017] The above one or more technical solutions have the following beneficial effects: This invention utilizes the KAN multimodal temporal conditioner and KAN temporal aggregation module to perform nonlinear fusion and temporal modeling of multi-temporal and multimodal latent features using KAN's learnable B-spline basis functions. Compared with traditional simple stitching, weighted summation, or multilayer perceptron fusion methods, it can more accurately capture the complex nonlinear dependencies and continuous changing trends between multi-temporal medical images, effectively alleviate phase inaccuracies and motion artifacts caused by physiological movements such as respiratory movements, and significantly improve the anatomical accuracy and dynamic consistency of the generated images.

[0018] This invention explicitly encodes prior structural information such as organ masks, blood vessel masks, lesion masks, or endangered organ masks into structural guidance features through a structural guidance consistency module. These features are then fused with temporal conditional embedding at the channel level and used as a denoising process in a hard-constraint injection diffusion model. This effectively compensates for the lack of explicit anatomical constraints in existing methods, ensuring that the generated target medical image maintains spatial consistency with the real anatomical structure in terms of functional distribution. This avoids the generation of structural discontinuities, blurred boundaries, and anatomically meaningless artifacts, thereby improving the spatial consistency and anatomical rationality of the generated image.

[0019] This invention utilizes a multi-temporal diffusion backbone network to perform condition-guided forward denoising and backward denoising processes in the latent space. It also incorporates a hybrid loss function that includes voxel-level reconstruction loss and gradient preservation loss for joint optimization. This approach ensures voxel-level quantitative accuracy of the generated images while effectively preserving high-frequency structural details such as blood vessel walls and tissue boundaries, avoiding excessive smoothing of the generated results. Ultimately, it outputs target medical images that combine quantitative accuracy, structural clarity, and physiological realism, providing reliable imaging evidence for precise clinical diagnosis and treatment.

[0020] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of a medical image generation method based on KAN-enhanced multi-temporal diffusion according to Embodiment 1 of the present invention; Figure 2 This is a block diagram of a medical image generation method based on KAN-enhanced multi-temporal diffusion according to Embodiment 1 of the present invention; Figure 3 The figure shows the experimental results of the medical image generation method based on KAN-enhanced multi-temporal diffusion according to Embodiment 1 of the present invention. Detailed Implementation

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] Explanation of technical terms: Multitemporal medical imaging refers to medical imaging data obtained from the same subject at different time points, different scanning phases, or different physiological states. This includes, but is not limited to, multi-phase CT, multi-phase MRI, 4D CT, CBCT of the treatment process, dynamic contrast-enhanced imaging, and other medical images with temporal variation characteristics. This type of imaging can reflect morphological changes, density changes, motion changes, or functional changes of tissues and organs over time.

[0027] Example 1 like Figure 1 and Figure 2 As shown, this embodiment discloses a medical image generation method based on KAN-enhanced multi-temporal diffusion, including: Step 1: Acquire multi-temporal medical images and obtain structural prior information related to the target medical image generation task; Acquire multi-temporal medical images of the same subject, along with structural prior information related to the target medical image generation task. Multi-temporal medical images refer to medical images acquired from the same subject at different time points, different scanning phases, or different physiological states. Multimodal medical images may include CT, MRI, CBCT, PET, SPECT, ultrasound images, or combinations thereof.

[0028] The target medical image can be a structural image, a functional image, an enhanced image, a treatment process image, or a dose-related image. To impose anatomical constraints on the generation process of the target medical image, it is also necessary to obtain structural prior information corresponding to the generation task. This information includes one or more of the following: organ mask, structural prior mask, lesion mask, target area mask, organ at risk mask, tissue boundary, or anatomical contour, to constrain the spatial structural consistency and anatomical rationality during the generation process.

[0029] The specific implementation steps of the above data acquisition and processing process include: We collected multi-center, multi-temporal image datasets from clinical settings. Taking the generation of pulmonary ventilation function images based on 4D CT as an example, we obtained multi-phase 4D CT scan data of lung cancer patients and the corresponding SPECT / PET ventilation images (gold standard), and extracted N phase image sequences covering the entire respiratory cycle from end-inspiratory to end-expiratory.

[0030] Automatic segmentation tools (such as TotalSegmentator) are used to segment organs and structures in 4DCT images, generating task-related organ masks (such as lung masks) and structure prior masks (such as blood vessel masks) as the aforementioned structure-guided prior information.

[0031] Based on the acquired data, a multi-temporal-multimodal dataset is constructed, including: initial medical images, organ masks, and structural prior masks.

[0032] Based on this, all images undergo uniform preprocessing: resampling to isotropic voxel spacing (e.g., 2.5×2.5×2.5 mm³), cropping to a fixed size (e.g., 128×128×160 voxels) according to organ masks, and normalizing pixel values ​​to ensure consistency in spatial resolution and intensity distribution of the input data.

[0033] Step 2: Use a shared 3D convolutional encoder to extract independent features from the multi-temporal-multimodal dataset to obtain multi-temporal-multimodal specific latent features.

[0034] To fully extract complementary information from multi-temporal images, this step employs multiple sets of lightweight 3D convolutional encoders with shared weights to independently encode the features of the initial medical image, organ mask, and structural prior mask.

[0035] The organ mask is the organ-level structural prior obtained in step 1 (e.g., a lung mask), used to provide the contour and extent constraints of the target organ; the structural prior mask is the anatomical structural prior (e.g., a blood vessel mask), used to introduce internal topological features. The encoder consists of stacked three-dimensional convolutional layers, normalization layers, and activation functions, capable of extracting phase-specific high-dimensional latent features from different input modalities.

[0036] The specific implementation is as follows: Design three sets of lightweight 3D convolutional encoders with shared weights, denoted as follows: .in, Used for processing initial medical images (such as 4DCT images of each phase). ), through a convolution kernel size of The three-dimensional convolutional layers, batch normalization layers, and activation functions are used to extract phase-specific anatomical structural features and density distribution features, and output CT modal latent features. . Used for processing organ masks (such as lung masks) This focuses on global structural information such as the outline and extent of organs to generate organ mask latent features. . This is used to parse structural prior masks (such as blood vessel masks). This involves extracting fine topological features such as the branching structure and spatial distribution of blood vessels to obtain structural prior mask latent features. ,in Indicates the first One breathing phase.

[0037] The three encoders share weights across different phases, ensuring consistency in feature extraction for the same modality. Meanwhile, the lightweight architecture significantly reduces computational complexity while maintaining feature expressive power, providing a compact and robust latent feature representation for subsequent temporal fusion and conditional modeling.

[0038] Step 3: Generate conditional guidance information using the KAN multimodal time conditioner and the KAN time aggregation module.

[0039] To achieve deep integration and conditional modeling of multi-temporal and multi-modal specific latent features, this step constructs two parallel and collaborative processing branches: the KAN multimodal temporal conditioner and the KAN temporal aggregation module. The two branches process the latent features from the two dimensions of "single-modal temporal dependence" and "cross-modal nonlinear fusion", respectively, and finally generate a unified temporal conditional embedding vector.

[0040] The KAN multimodal temporal conditioner performs cross-modal and cross-phase nonlinear fusion of multi-temporal and multimodal latent features to generate multimodal temporal conditional embeddings. The specific implementation process includes: Multimodal temporal trajectory modeling: for each voxel location in the image (Three-dimensional spatial coordinates) Integrate all Multimodal latent features of each breathing phase are used to construct a time-modal feature trajectory. This feature trajectory contains voxels. Complete information throughout the respiratory cycle: source image modal features reflect changes in tissue density, signal intensity, or spatial morphology at that location over time or phase; organ masking features. It marks whether the location belongs to the target organ parenchyma (such as lung parenchyma) and the dynamic changes of the organ boundary; structural prior mask features. This demonstrates the spatial relationship between the location and internal anatomical structures (such as blood vessels), providing a basis for subsequent function-structure correspondence. Through this trajectory, the model can capture multi-dimensional dynamic information at the voxel level, laying the foundation for accurate modeling of target medical images. In this embodiment, the target medical image is a lung ventilation function image.

[0041] KAN Nonlinear Fusion: To address the problem that traditional multimodal fusion methods (such as simple splicing and weighted summation) are unable to capture complex nonlinear dependencies, this embodiment introduces a Kolmogorov-Arnold network (KAN) layer to replace the traditional multilayer perceptron (MLP) for feature fusion.

[0042] The KAN layer nonlinearly extends the input features using learnable B-spline basis functions, enabling flexible fitting of complex functional relationships across time and modalities. Specifically, for feature trajectories... For each modality and each phase feature, the KAN layer adaptively learns its weight coefficients. (in For phase index, (Based on mode type), and activated via mode-specific B-spline basis functions. The features are then subjected to a nonlinear transformation to ultimately generate multimodal fusion features: ; in, Representation phase and modality The spatiotemporal confidence of learning This represents a learnable activation network based on B-spline basis functions.

[0043] Attention enhancement and projection: To further strengthen the representation of useful information, in multimodal feature fusion... Based on this, a time-channel attention mechanism is introduced. .

[0044] This mechanism dynamically adjusts the weighting of features by calculating their importance scores in the time dimension (each respiratory phase) and the channel dimension (each feature channel): higher attention weights are assigned to phases and channels that show significant changes in features during respiratory motion and are strongly correlated with the generation of the target medical image; and the weights are reduced for phases and channels that are greatly affected by motion artifacts and have low information value.

[0045] The attention-enhanced multimodal fusion features are mapped to the conditional space of the diffusion model through a linear projection layer to obtain the multimodal temporal conditional embedding. :

[0046] in, This represents the multimodal fusion features obtained through KAN nonlinear fusion. This indicates a time-channel attention operation. This represents a linear projection operation. This represents the generated multimodal temporal conditional embedding. This multimodal temporal conditional embedding, as one of the conditional information components, is used to guide the subsequent inverse denoising process of the diffusion model.

[0047] The KAN temporal aggregation module (KTAM) focuses on the temporal dependencies across phases within source image modalities (such as 4DCT). It performs nonlinear temporal modeling and attention-weighted fusion of multi-phase structural latent features to generate temporal conditional embeddings. Specific implementations include: Phase Feature Multiplexing and Organization: Based on the temporal characteristics of multi-phase source images, the latent features of each phase source image extracted by the shared 3D convolutional encoder in step 2 are directly obtained. ,in: ( ) Latent features corresponding to different phases are organized into a multi-phase latent feature sequence according to phase order. The aforementioned latent features have already been encoded in step 2. This step will not repeat feature extraction, but will instead further perform KAN nonlinear time-dependent modeling and attention-weighted fusion to obtain information on tissue structure changes and movement trends under different physiological states.

[0048] KAN time-dependent modeling: Due to the complex non-rigid deformation of tissues and organs during physiological movement, there are non-linear time dependencies between features of different phases, and phase inaccuracies may occur due to irregular movement. Taking pulmonary ventilation function images as an example, due to the complex non-rigid deformation of the lungs during respiration, there are non-linear time dependencies between features of different phases, and phase inaccuracies may occur due to irregular breathing.

[0049] To address this issue, specific phase characteristics are analyzed at the voxel level. Time trajectory (in For the first Phase features in voxels The eigenvectors at the given location are modeled using a KAN layer to determine their nonlinear time dependencies.

[0050] The KAN layer fits the time trajectory using learnable B-spline basis functions, adaptively capturing the continuous changing trends of features in the physiological cycle, rather than simple discrete phase transitions. Simultaneously, the model learns the confidence weights for each phase at specific voxel locations. Phases with minimal motion artifacts and clear anatomical structures are assigned higher weights; phases with severe artifact interference and blurred structures are assigned lower weights, thus generating adaptively weighted temporal fusion features.

[0051] in This involves extending the B-spline basis functions. In this way, KTAM can effectively alleviate the feature inconsistency problem caused by phase misalignment, generating smooth and robust temporal fusion features.

[0052] Temporal attention refinement and projection: To further highlight key information related to the generation of the target medical image in temporal features, a temporal attention mechanism is introduced. Refinement is then performed. First, linear projection is used to... Convert to query vector Key vector Sum value vector ,in Used to characterize the feature targets that need to be focused on at present. Used to calculate the correlation of each phase feature, These are the feature contents that need to be weighted. Then, through calculation... and The transpose of the product, after being normalized by the softmax function, yields the attention scores for each phase feature:

[0053] in For feature dimension, divide by This is to avoid gradient instability caused by excessively large values ​​during calculation. The attention score... and Perform the Hadamard product operation (⊙) to obtain the attention-enhanced features:

[0054] This feature primarily preserves regional information that exhibits significant temporal changes and is strongly correlated with the generation of the target medical image. Finally, for Perform layer normalization ( This process eliminates interference caused by differences in feature distribution, and then uses a linear projection layer ( Mapping this to the conditional space of the diffusion model, the output is the temporal conditional embedding:

[0055] KAN multimodal time conditioner generates Generated with KAN time aggregation module The complementary information in multi-temporal images is captured from two dimensions: "cross-modal nonlinear fusion" and "single-modal temporal dependence". The two together constitute the conditional guidance information of the diffusion model, which is used as the core conditional information input into the subsequent denoising process of the diffusion model, thus guiding the generation of target medical images from different dimensions.

[0056] Step 4: Generate structural guidance features by encoding structural prior information through the structural guidance consistency module.

[0057] To effectively inject structural prior information into the generation process, this step constructs a Structure-Guided Consistency Module (SGCM). This module encodes and normalizes the structural prior mask obtained in step 1 and fuses it with the conditional guidance information generated in step 3 to generate spatially constrained structural guidance features. This module ensures that the generated target medical image maintains spatial consistency with the anatomical structure in terms of functional distribution, avoiding artifacts without anatomical significance. Specific implementation includes: Structural prior coding: Structural prior information can provide spatial constraints for medical image generation. Taking the generation of images of pulmonary ventilation function as an example, there is a close physiological relationship between the topology of pulmonary vessels and ventilation function: densely vascularized areas usually have higher perfusion potential and correspondingly more active ventilation function; sparsely vascularized or missing areas often have weaker ventilation function.

[0058] Based on the aforementioned structural prior information, a lightweight 3D structural encoder is used to encode the structural prior mask obtained in step 1. The lightweight 3D structural encoder consists of shallow 3D convolutional layers and pooling layers, used to extract global topological features, spatial distribution features, and local structural information from the structural prior mask, obtaining the structural latent embedding.

[0059] in, This represents the structural prior mask obtained in step 1. This indicates a lightweight three-dimensional structure encoder. This represents the extracted structural latent embedding. In a specific embodiment of lung ventilation function image generation, the structural prior mask can be a vascular mask.

[0060] Structural Condition Normalization: To ensure that the structural latent embedding matches the conditional guidance information generated in step 3 in terms of feature dimensions, the structural latent embedding is normalized and its dimensions are mapped. First, layer normalization is applied to the structural latent embedding. The process involves processing to reduce the impact of differences in feature distribution among different samples; subsequently, a linear projection layer maps the normalized structural latent embedding to a predefined conditional feature space, generating a structural conditional vector.

[0061] in, Presentation layer normalization operation, This represents the linear projection operation of structural features. This represents the structural condition vector.

[0062] Structure-Time Feature Fusion: In step 3, the KAN multimodal temporal conditioner outputs a multimodal temporal conditional embedding. The KAN time aggregation module outputs time condition embeddings. First, the two types of conditional information are fused to obtain the time-related guiding information corresponding to step 3:

[0063] in, This indicates a splicing operation along the feature channel dimension. This represents a linear projection operation. This represents the time-conditional guided feature obtained after fusing multimodal conditional information and temporal conditional information.

[0064] Subsequently, the time-conditional guidance feature is... With structural condition vector By concatenating along the feature channel dimension, we obtain the joint structure-time features:

[0065] The structure-time joint features simultaneously include multi-temporal dynamic information, multi-modal complementary information, and structural prior information, enabling the subsequent diffusion model to comprehensively utilize temporal variation features and structural spatial constraints during the generation of target medical images.

[0066] Finally, the structure-temporal joint features are mapped to the conditional feature space of the multi-temporal diffusion backbone network through a linear projection layer, resulting in the structure-temporal joint conditional embedding:

[0067] in, This indicates a conditional feature projection operation. This represents the final generated structure-temporal joint conditional embedding. The structure-temporal joint conditional embedding serves as the inverse denoising process of the multi-temporal diffusion backbone network, inputting conditional information to guide the gradual reconstruction of latent features of the target medical image and improve the spatial consistency between the generated image and the structural prior information. Taking the generation of pulmonary ventilation function images as an example, the structural prior mask can employ an intrapulmonary vascular mask, thereby utilizing the spatial distribution and topological information of intrapulmonary vessels to assist in constraining the spatial distribution of the generated ventilation function images.

[0068] Step 5: Input the temporal conditional embedding vector and structure-guided features into the multi-temporal diffusion backbone network, perform conditional guided forward denoising and backward denoising in the latent space, gradually reconstruct the latent features of the target medical image, decode the latent features through a pre-trained VAE decoder, and generate the target medical image.

[0069] A multi-temporal diffusion backbone network (MTDB) is constructed. Using the temporal conditional embedding vector generated in step 3 and the structure-guided features generated in step 4 as conditions, the forward denoising and backward conditional denoising processes of the diffusion model are performed in the latent space to progressively reconstruct the latent feature representation of the target medical image. MTDB consists of three parts: a pre-trained variational autoencoder (VAE), a KAN-enhanced denoising unit, and a conditional injection mechanism. The specific implementation process includes: Pre-trained Variational Autoencoder (VAE) Initialization: To reduce the computational complexity of the diffusion model and improve the efficiency of feature representation in the latent space, a pre-trained variational autoencoder (VAE) is used to map the latent space of the target medical image. The VAE consists of an encoder... and decoder composition.

[0070] Taking the generation of images of pulmonary ventilation function as an example, Used to transmit gold standard ventilation images Encoded as compact latent vectors , It condenses the core functional features of the target image, with a dimensionality far lower than the original image; This is used to decode latent vectors back into the original image space, thereby reconstructing the target image. Before training the overall model, the VAE is trained separately to accurately capture the feature distribution of the target medical image, providing a stable latent space foundation for the subsequent diffusion model.

[0071] Forward diffusion denoising: Following the basic framework of the Denoising Diffusion Probabilistic Model (DDPM), denoising the latent vectors during training is performed... A gradual noise-adding process is performed to simulate the degradation process from latent features of a real target image to pure noise.

[0072] Specifically, at each time step ( , (Total number of noise addition steps), scheduled according to a preset noise variance sequence. To the current latent vector Add Gaussian noise to generate noisy latent vectors The noise addition process follows the probability distribution: ; in, Let be the noise variance scheduling parameter at time step t. For the latent variables of the previous time step, Let be the noisy latent variable at time step t. For an identity matrix that matches the dimensions of the latent variables, Let represent the covariance matrix of the Gaussian noise added at time step t.

[0073] in The initial value is gradually increased to the final value to ensure a smooth noise addition process; The noise intensity varies with Increase and gradually strengthen, when hour, It is approximately pure Gaussian noise. This process enables the model to learn the mapping relationship between noise and true features, providing a training basis for subsequent reverse denoising and reconstruction.

[0074] KAN-enhanced denoising design: The core of the diffusion model is the denoising unit, whose performance directly determines the quality of the generated image. To improve the denoising unit's ability to model complex nonlinear relationships, this step embeds the KAN layer into a traditional 3D UNet structure to construct the KAN-Denoiser. Unlike the static activation functions such as ReLU and GELU used in traditional UNet, the KAN layer in the KAN-Denoiser nonlinearly extends the features through learnable B-spline basis functions, enabling it to more flexibly capture continuous gradient changes, regional transition features, and subtle physiological motion-related changes in the target medical image.

[0075] The KAN layer is inserted into the encoder, decoder, and skip connection paths of UNet, which not only enhances the feature representation capability of a single layer, but also promotes the nonlinear fusion between features at different scales, enabling the model to generate smoother target medical images with more consistent anatomical structures.

[0076] Conditional denoising process: During the inference phase, the inverse process of the diffusion model is performed to denoise the pure Gaussian noise. Begin with the structure-time joint conditional embedding generated in step 4. Guided by the KAN-Denoiser algorithm, noise is gradually reduced, ultimately restoring a clean latent vector consistent with the latent features of the real target image. .

[0077] Specifically, at each time step (from (Iterate in reverse from step 1), first, the noisy latent vector Through feature extraction module Convert to a feature map, then obtain the query vector through linear projection. At the same time, embed the conditions The key vectors are obtained through linear projection. Sum value vector Through calculation and The correlation is used to obtain the conditional attention weights, which are then compared with... We perform a weighted summation to fuse conditional information with latent features:

[0078] in The feature dimension. The fused features. The input is fed into the KAN-Denoiser, and the model learns the conditional posterior distribution. Estimate the noise component in the current latent vector and remove it to obtain the denoised latent vector. The process iterates. After this, the finally recovered clean latent vector is obtained. .

[0079] Target medical image generation: clean latent vectors obtained from the inverse process Decoder input to pre-trained VAE The image resolution is gradually restored through the deconvolution layer and upsampling layer of the decoder, ultimately generating a full-resolution target medical image. Taking the generation of lung ventilation function images as an example, this embodiment generates three-dimensional lung ventilation images. The generated images not only have high consistency with the SPECT / PET gold standard at the voxel level, but also maintain spatial correspondence with the pulmonary vascular structure in terms of functional distribution, and effectively suppress the interference caused by motion artifacts.

[0080] Step 6: Jointly train each module using a hybrid loss function to obtain the generative model.

[0081] To achieve end-to-end joint optimization of the KAN temporal aggregation module, structure-guided consistency module, and multi-temporal diffusion backbone network, this step designs a hybrid loss function. This function constrains the generation of results from two dimensions: voxel-level reconstruction accuracy and structure-level gradient preservation, ensuring the model retains high-frequency structural details while maintaining quantitative accuracy. Specific implementation includes: Voxel-level reconstruction loss: Voxel-level reconstruction accuracy is a fundamental indicator for evaluating the quality of target medical images. To ensure the consistency of the generated image with the gold standard at each voxel location, mean squared error (MSE) is used as the voxel-level reconstruction loss. ; in, The target medical image generated by the model, For the corresponding gold standard image, This represents the square of the L2 norm. This loss function ensures the model learns the quantitative distribution characteristics of the target medical image by minimizing the voxel value difference between the generated image and the gold standard. Taking the generation of images of pulmonary ventilation function as an example... This is the gold standard ventilation image for SPECT / PET. This loss ensures that the generated image accurately reflects the quantitative distribution of lung ventilation function, providing a reliable numerical basis for clinical functional assessment.

[0082] Gradient Preservation Loss: Relying solely on voxel-level MSE loss can easily lead to overly smoothed generated images, losing high-frequency details of internal structural boundaries (such as blood vessel walls, bronchial boundaries, and organ outlines), which are crucial for distinguishing different functional regions or anatomical structures. To address this issue, gradient preservation loss is introduced. By constraining the consistency of the generated image with the gold standard in spatial gradient, high-frequency structural details are preserved.

[0083] in, Indicates along , , Gradient operators for three spatial axes, This represents the L1 norm. The gradient reflects the rate of change of pixel values ​​in an image; gradient values ​​are typically larger in structural boundary regions. Gradient preservation loss minimizes the difference in gradients between the generated image and the gold standard, ensuring that the generated image accurately reproduces the boundary transition features of functional regions or anatomical structures, avoiding regional blurring caused by over-smoothing.

[0084] Total loss function: To balance voxel-level reconstruction accuracy and structural coherence, the two losses mentioned above are weighted and summed to form a unified reconstruction consistency loss (RCL):

[0085] in, and These are preset weighting coefficients. Experiments have verified that when... , Achieving optimal balance: Set larger weights to ensure voxel-level quantitative accuracy; By setting appropriate weights, structural details are not lost. During training, the KAN temporal aggregation module, structure-guided consistency module, and multi-temporal diffusion backbone network are jointly optimized using this loss function as the objective, resulting in a well-trained medical image generation model.

[0086] To further demonstrate the effectiveness of the method proposed in this embodiment, the quantitative performance of different algorithms in reconstructing lung ventilation images was compared on the 4DCT-SPECT dataset. Through comparison... The KDiffVent algorithm achieved the best performance across all evaluation metrics, including MS-SSIM, PSNR, Dice, and ASD. In particular, its significantly lower LPIPS score indicates that KDiffVent's visual quality is closer to that of real lung ventilation images, consistent with subsequent qualitative analysis results. Table 1 shows a quantitative performance comparison of different algorithms for reconstructing lung ventilation images on the 4DCT-SPECT dataset.

[0087] Table 1. Quantitative comparison of different methods for generating images of pulmonary ventilation function

[0088] like Figure 3 As shown, a qualitative comparison of various lung ventilation image reconstruction methods is performed on the 4DCT-SPECT dataset. Figure 3 Using four typical cases as examples, this paper horizontally compares the original CT images, the SPECT gold standard (GT), and ventilation images generated by methods such as Jac, Hu, 3DUnet, DFPN, ​​cGAN, and CWDN with those generated by the method in this embodiment. Visualization results clearly show that, compared to other methods, the ventilation maps generated by the method in this embodiment are closer to the gold standard (GT column) in terms of spatial consistency of functional distribution and smoothness of regional transitions. For example, in case 2, the method in this embodiment avoids the local tomographic artifacts of the Jac method and corrects the overall color cast problem of the Hu method. Simultaneously, the boundaries of its functional regions match the CT anatomical structures (CT column) more accurately, effectively preserving the detailed differences in heterogeneous regions within the lungs. This intuitive comparison further verifies the superiority of the method in this embodiment in qualitative indicators, demonstrating its ability to more accurately reproduce the functional distribution characteristics of the gold standard ventilation images.

[0089] Example 2 Based on the method described in Embodiment 1, the purpose of this embodiment is to provide a medical image generation system based on KAN-enhanced multi-temporal diffusion, comprising: The data acquisition module is used to acquire multi-temporal medical images of the same subject, as well as structural prior information related to the target medical image generation task; A shared feature extraction module is used to independently encode each temporal image using a shared 3D convolutional encoder to obtain temporal-specific latent features; The KAN enhanced condition generation module includes a KAN multimodal time conditioner and a KAN time aggregation module, which are used to perform nonlinear fusion and time series modeling on the phase-specific latent features to generate condition guidance information. The structure-guided consistency module is used to encode and modulate the structure prior information to generate structure-guided features; A multi-temporal diffusion backbone network is used to receive the conditional guidance information and the structural guidance features, perform conditional guidance forward noise addition and reverse noise reduction processes in the latent space, and gradually reconstruct the latent feature representation of the target medical image; the latent feature representation is decoded by a pre-trained VAE decoder to generate the target medical image; The training module is used to jointly train the KAN-enhanced conditional generation module, the structure-guided consistency module, and the multi-temporal diffusion backbone network using a hybrid loss function to obtain a trained medical image generation model.

[0090] Example 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0091] Example 4 The purpose of this embodiment is to provide a computer-readable storage medium.

[0092] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above-described method.

[0093] The steps and methods involved in the devices of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0094] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0095] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A medical image generation method based on KAN-enhanced multi-temporal diffusion, characterized in that, include: Acquire multi-temporal medical images and obtain structural prior information related to the target medical image generation task; A shared 3D convolutional encoder is used to independently encode each temporal image to obtain temporal-specific latent features; Conditional guidance information is generated through the KAN multimodal temporal conditioner and the KAN temporal aggregation module; structural guidance consistency information is encoded through the structural guidance consistency module to generate structural guidance features. Conditional guidance information and structural guidance features are input into a multi-temporal diffusion backbone network, and conditional guidance forward denoising and reverse denoising are performed in the latent space to gradually reconstruct the latent features of the target medical image. The latent features are decoded using a pre-trained VAE decoder to generate the target medical image; A generative model is obtained by jointly training the modules using a hybrid loss function.

2. The medical image generation method based on KAN-enhanced multi-temporal diffusion as described in claim 1, characterized in that, Multi-temporal medical images include medical images of the same subject at different time points, different scanning phases, or different physiological states; the target medical images include structural medical images, functional medical images, enhanced medical images, medical images of the treatment process, and dose-related medical images.

3. The medical image generation method based on KAN-enhanced multi-temporal diffusion as described in claim 1, characterized in that, The structural prior information includes one or more of the following: organ mask, structural prior mask, lesion mask, target area mask, organ at risk mask, tissue boundary or anatomical contour.

4. The medical image generation method based on KAN-enhanced multi-temporal diffusion as described in claim 1, characterized in that, The KAN temporal aggregation module performs nonlinear fusion of multi-temporal or multi-modal latent features through the B-spline basis function expansion of the KAN layer. The fusion formula is as follows: in, The confidence weight for each phase at a specific voxel location. For B-spline basis function extension, for Phase features in voxels The eigenvector at that location.

5. The medical image generation method based on KAN-enhanced multi-temporal diffusion as described in claim 1, characterized in that, The structure-guided consistency module uses a lightweight 3D structure encoder to extract the global topological features and spatial distribution features of the structure prior mask to generate the structure latent embedding; it eliminates the feature distribution differences between samples through layer normalization; and then maps the structure condition vector to the same dimension as the temporal condition embedding through linear projection.

6. The medical image generation method based on KAN-enhanced multi-temporal diffusion as described in claim 1, characterized in that, The multi-temporal diffusion backbone network performs a forward diffusion process in the latent space, as shown in the following formula: ; in, The noise variance scheduling parameter, For the current latent vector, Let be the latent noise variable at time t. Let represent the covariance matrix of the Gaussian noise added at time step t.

7. The medical image generation method based on KAN-enhanced multi-temporal diffusion as described in claim 1, characterized in that, In the reverse denoising process, the multi-temporal diffusion backbone network inputs temporal conditional embedding vectors and structurally guided features as conditional information into the denoising network, and guides noise prediction and latent feature reconstruction of the target medical image through cross-attention, feature splicing or feature modulation.

8. A medical image generation system based on KAN-enhanced multi-temporal diffusion, characterized in that, include: The data acquisition module is used to acquire multi-temporal medical images of the same subject, as well as structural prior information related to the target medical image generation task; A shared feature extraction module is used to independently encode each temporal image using a shared 3D convolutional encoder to obtain temporal-specific latent features; The KAN enhanced condition generation module includes a KAN multimodal time conditioner and a KAN time aggregation module, which are used to perform nonlinear fusion and time series modeling on the phase-specific latent features to generate condition guidance information. The structure-guided consistency module is used to encode and modulate the structure prior information to generate structure-guided features; A multi-temporal diffusion backbone network is used to receive the conditional guidance information and the structural guidance features, perform conditional guidance forward noise addition and reverse noise reduction processes in the latent space, and gradually reconstruct the latent feature representation of the target medical image; the latent feature representation is decoded by a pre-trained VAE decoder to generate the target medical image; The training module is used to jointly train the KAN-enhanced conditional generation module, the structure-guided consistency module, and the multi-temporal diffusion backbone network using a hybrid loss function to obtain a trained medical image generation model.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the medical image generation method based on KAN-enhanced multi-temporal diffusion as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the medical image generation method based on KAN-enhanced multi-temporal diffusion as described in any one of claims 1-7.

Images