A medical image generation method, system, product and medium based on a diffusion model

Abstract

A method, system, product, and medium for generating medical images based on a diffusion model are disclosed. The method includes: acquiring initial latent variables and a dynamic optical surface image sequence; extracting spatiotemporal features from the dynamic optical surface image sequence to obtain a surface motion feature vector; iteratively denoising the latent variables corresponding to each time step; performing wavelet decomposition on the latent variables corresponding to the current time step to obtain low-frequency and high-frequency components; acquiring a temporal embedding vector and encoding the surface motion feature vector into a conditional embedding vector; determining the target low-frequency and target high-frequency components based on the temporal and conditional embedding vectors; performing inverse wavelet transform on the target low-frequency and target high-frequency components to obtain the latent variables corresponding to the next time step; and generating a three-dimensional CT image of the target object by obtaining the target latent variables when the iterative denoising process meets a preset evolution termination condition. This application improves the accuracy of generated medical images.

Description

Technical Field

[0001] This application relates to the field of medical image technology, specifically to a method, system, product, and medium for generating medical images based on a diffusion model. Background Technology

[0002] In the field of modern precision radiotherapy (such as tumor radiotherapy), accurately obtaining the real-time location and morphology of the patient's internal anatomical structures (especially tumors and organs at risk) during the respiratory cycle is crucial to ensuring precise delivery of radiotherapy doses. Three-dimensional CT (Computed Tomography) images can provide high-resolution information on internal anatomical structures and are the "gold standard" for developing and adjusting radiotherapy plans.

[0003] Existing technologies typically utilize optical surface monitoring systems to acquire dynamic optical surface images of patients in real time (such as three-dimensional point clouds or depth maps of abdominal respiratory fluctuations). Then, traditional deep learning generative models (such as Generative Adversarial Networks (GANs) or conventional Convolutional Neural Networks (CNNs)) are used to establish a mapping relationship from external surface motion data to internal three-dimensional CT images, thereby synthesizing real-time internal three-dimensional CT images of the patient without increasing radiation.

[0004] However, there exists an extremely nonlinear cross-modal mapping relationship between the optical undulations of the patient's external surface and the complex changes in the internal three-dimensional anatomical structure, with significant differences in their data distributions. Traditional image generation models, when dealing with this complex cross-modal mapping, often only learn a coarse overall data distribution. This results in severe limitations in the image quality of the generated internal three-dimensional CT images: on the one hand, the global anatomical structure of the image (such as the outline of large organs) is prone to distortion or deformation; on the other hand, key local details of the image (such as the tiny edges of tumors and vascular textures) are often very blurry, exhibiting severe artifacts. This problem of global structural distortion and loss of local details means that the synthesized three-dimensional CT images cannot meet the stringent requirements of high-precision anatomical localization in clinical radiotherapy, reducing the accuracy of the generated medical images. Summary of the Invention

[0005] This application provides a medical image generation method, system, product, and medium based on a diffusion model, which solves the technical problems of global structural distortion and local detail loss in the process of generating three-dimensional CT images, thereby improving the accuracy of generated medical images.

[0006] The first aspect of this application provides a medical image generation method based on a diffusion model, the method comprising: Acquire initial latent variables for image generation and a dynamic optical surface image sequence of the target object, wherein the initial latent variables are sampled from a preset prior distribution; Spatiotemporal feature extraction is performed on the dynamic optical body surface image sequence to obtain the body surface motion feature vector; Based on the pre-trained Schrödinger bridge diffusion model, the latent variables corresponding to each time step are iteratively denoised. The denoising process for each time step includes the following steps: Wavelet decomposition is performed on the latent variable corresponding to the current time step to obtain low-frequency components and high-frequency components, wherein the latent variable corresponding to the first time step is the initial latent variable; Obtain the temporal embedding vector of the current time step, and encode the body surface motion feature vector into a conditional embedding vector; Based on the time embedding vector and the conditional embedding vector, the low-frequency component and the high-frequency component are respectively subjected to conditional fusion processing to obtain the target low-frequency component and the target high-frequency component. Perform inverse wavelet transform on the target low-frequency component and the target high-frequency component to obtain the latent variable corresponding to the next time step of the current time step; The latent variable corresponding to the iterative denoising process that meets the preset evolution termination condition is determined as the target latent variable. The target latent variable is input into the pre-trained image decoder to obtain the three-dimensional CT image of the target object.

[0007] Optionally, spatiotemporal feature extraction is performed on the dynamic optical body surface image sequence to obtain a body surface motion feature vector, specifically including: Spatial features are extracted from each phase image in the dynamic optical body surface image sequence to obtain the spatial dimension features corresponding to each phase image; The spatial dimension features corresponding to each of the time-phase images are sequentially input into a preset long short-term memory network for temporal feature extraction to obtain the body surface motion feature vector.

[0008] Optionally, spatial features are extracted from each temporal image in the dynamic optical surface image sequence to obtain the spatial dimension features corresponding to each temporal image, specifically including: Each of the aforementioned temporal images undergoes initial convolution processing to obtain an initial spatial feature map, which includes multiple feature nodes; Traverse each feature node in the initial spatial feature map and define a local spatial neighborhood centered on the current feature node; Extract the target feature value of the current feature node and the reference feature values ​​of adjacent feature nodes in the local spatial neighborhood; Calculate the spatial feature difference between the target feature value and the reference feature value, and determine the local anomaly confidence of the current feature node based on the spatial feature difference, thereby obtaining the local anomaly confidence of each feature node; Based on the local anomaly confidence level corresponding to each feature node, a feature mask weight matrix matching the size of the initial spatial feature map is generated. The initial spatial feature map is multiplied by the feature mask weight matrix to obtain the corrected feature map. Spatial pooling is performed on the corrected feature map to obtain the spatial dimension features corresponding to each of the temporal images.

[0009] Optionally, based on the time embedding vector and the conditional embedding vector, conditional fusion processing is performed on the low-frequency component and the high-frequency component respectively to obtain the target low-frequency component and the target high-frequency component, specifically including: The time embedding vector and the conditional embedding vector are jointly mapped using a first mapping function, and the mapping result is added to and fused with the low-frequency component to obtain the target low-frequency component. The calculation formula is as follows: ,in, For the target low-frequency component, A t For the low-frequency component, φ A The first mapping function; The time embedding vector and the conditional embedding vector are jointly mapped using a second mapping function, and the mapping result is added to and fused with the high-frequency component to obtain the target high-frequency component. The calculation formula is as follows: ,in, For the target high-frequency component, D t For the high-frequency component, φ D Let e ​​be the second mapping function. t Let e ​​be the time embedding vector. z The conditional embedding vector is denoted as .

[0010] Optionally, before the step of obtaining the initial latent variables for image generation and the dynamic optical surface image sequence of the target object, the method further includes: Obtain a training sample set, which includes multiple sets of training samples, each set of training samples including a dynamic optical surface image sequence and a three-dimensional CT image of the sample. The initial latent variables of the samples, which follow a preset noise distribution, and the sequence of dynamic optical surface images of the samples are input into the initial Schrödinger bridge diffusion model to be trained to obtain the predicted three-dimensional CT image. Based on the predicted 3D CT image and the sample 3D CT image, the Schrödinger bridge loss, regularization loss and adversarial loss are calculated respectively. Based on the Schrödinger bridge loss, the regularization loss, and the adversarial loss, the network parameters of the initial Schrödinger bridge diffusion model are updated until the network parameters meet the preset convergence condition, thus obtaining the pre-trained Schrödinger bridge diffusion model.

[0011] Optionally, before the step of sequentially inputting the spatial dimension features corresponding to each of the temporal images into a preset long short-term memory network for temporal feature extraction, the method further includes: According to the time series, the temporal feature change rate between the spatial dimension features corresponding to the temporal images in the target temporal image group is calculated. The target temporal image group consists of adjacent first temporal images and second temporal images, and the temporal sequence of the first temporal image precedes that of the second temporal image. If the change rate of the temporal feature is greater than the preset motion mutation threshold, then the spatial dimension feature corresponding to the second temporal image is determined to be a transient abnormal feature, and a corresponding temporal blocking identifier is generated for the transient abnormal feature; If the rate of change of the temporal feature is less than or equal to the preset motion mutation threshold, then the spatial dimension feature corresponding to the second temporal image is determined to be a smooth motion feature.

[0012] Optionally, the spatial dimension features corresponding to each of the aforementioned temporal images are sequentially input into a preset long short-term memory network for temporal feature extraction to obtain the body surface motion feature vector, specifically including: During the process of sequentially inputting the spatial dimension features corresponding to each of the aforementioned temporal images into the preset long short-term memory network, if the currently input spatial dimension feature carries the temporal blocking identifier, a dynamic state compensation mechanism is triggered. The dynamic state compensation mechanism includes: decaying the input gate activation value of the preset long short-term memory network for the current time step corresponding to the currently input spatial dimension feature to a preset lower limit value, and extracting the hidden layer memory state of the preset long short-term memory network at the previous time step corresponding to the previously input spatial dimension feature. Based on the hidden layer memory state of the previous time step and the preset temporal decay coefficient, the hidden layer memory state of the current time step is reconstructed through autoregression to obtain the reconstructed memory state. Based on the reconstructed memory state, the temporal feature extraction of subsequent phase images continues until the spatial dimension features corresponding to each phase image have been extracted in a temporal manner, thus obtaining the body surface motion feature vector.

[0013] In a second aspect, embodiments of this application provide a diffusion-based medical image generation system, which includes one or more processors and a memory; the memory is coupled to the one or more processors and is used to store computer program code, which includes computer instructions, and the one or more processors call the computer instructions to cause the diffusion-based medical image generation system to perform the method described in the first aspect and any possible implementation thereof.

[0014] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a diffusion-based medical image generation system, cause the diffusion-based medical image generation system to perform the method described in the first aspect and any possible implementation thereof.

[0015] Fourthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a diffusion-based medical image generation system, cause the diffusion-based medical image generation system to perform the method described in the first aspect and any possible implementation thereof.

[0016] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: 1. In the iterative denoising process of the Schrödinger bridge diffusion model, the latent variables are decomposed using wavelet decomposition, and the surface motion feature vector is used as a conditional embedding vector, which is conditionally fused with the low-frequency and high-frequency components respectively. Therefore, the Schrödinger bridge model can accurately fit the highly nonlinear cross-modal mapping relationship between surface undulations and internal anatomical structures. Simultaneously, wavelet decomposition is used for frequency domain decoupling, allowing the low-frequency component representing the global anatomical structure and the high-frequency component representing local details to be independently guided by surface motion features. This overcomes the technical problem of global structural distortion and loss of local details caused by traditional generative models that can only learn coarse data distributions, thus achieving high-fidelity reconstruction of macroscopic contours and microscopic textures, meeting the requirements of high-precision anatomical localization in clinical radiotherapy, and improving the accuracy of generated medical images.

[0017] 2. When acquiring the motion feature vector of the body surface, the spatial feature difference between the current feature node in the initial spatial feature map and its neighboring feature nodes in the local spatial neighborhood is calculated to determine the confidence level of local anomalies and generate a feature mask weight matrix. This matrix is ​​then multiplied with the initial spatial feature map to obtain a corrected feature map. The pooled spatial dimension features are then input into a preset long short-term memory network for temporal feature extraction. Therefore, by comparing the feature differences in the local neighborhood, local spatial anomalies (such as sensor noise or local occlusion interference) in the body surface image can be accurately identified. The feature mask weight matrix is ​​used to adaptively attenuate and filter the anomaly nodes. This solves the technical problem in existing technologies where spatial feature extraction is easily affected by local noise, leading to the accumulation and amplification of erroneous features in the temporal network, thus reducing the image generation quality. This ensures the purity of the spatial dimension features, providing a reliable data foundation for subsequent temporal feature extraction and further improving the accuracy and robustness of the generated medical images.

[0018] 3. By jointly mapping the temporal embedding vector and the conditional embedding vector using the first and second mapping functions respectively, and then adding and fusing the mapping results with the low-frequency and high-frequency components respectively, the target low-frequency and high-frequency components are obtained. This joint mapping enables the injection of body surface motion features to have time-aware capabilities, allowing for dynamic adjustment of the conditional guidance strategy based on the feature states at different denoising stages of the diffusion model. Simultaneously, independent mapping functions achieve customized feature modulation of low-frequency macroscopic structures and high-frequency microscopic details, and the conditional information is smoothly injected while preserving the original frequency domain features through addition and fusion. This process effectively solves the technical problems of existing technologies where the conditional injection method is singular and static, leading to inaccurate cross-modal feature alignment and easy destruction of original image features. Furthermore, it achieves accurate matching of body surface motion states and internal anatomical structures in different frequency domains and at different denoising stages, further improving the accuracy of generated medical images.

[0019] 4. By calculating the temporal feature change rate between spatial dimension features of adjacent temporal images, a temporal blocking identifier is generated for transient abnormal features exceeding a preset motion mutation threshold. When input into a preset long short-term memory network, a dynamic state compensation mechanism is triggered for features carrying this identifier. This mechanism attenuates the input gate activation value at the current time step and performs autoregressive reconstruction based on the hidden layer memory state of the previous time step to obtain a reconstructed memory state. Therefore, by using change rate monitoring, sudden abnormal movements of the patient during the acquisition process (such as coughing or irregular breathing) can be accurately captured. The forced attenuation of the input gate activation value effectively blocks the contamination of the network's internal memory by abnormal features. At the same time, the autoregressive reconstruction of historical states smoothly fills the temporal gaps at the moment of abnormality. This solves the technical problem in existing technologies where sudden irregular movements of the patient cause temporal feature breaks, leading to severe artifacts or prediction failures in the generated medical images. This ensures the temporal continuity of the body surface motion feature vector and its strong anti-interference robustness, further improving the accuracy of the generated medical images. Attached Figure Description

[0020] Figure 1 This is a block diagram of the training and generation stages of the Schrödinger bridge diffusion model in the embodiments of this application; Figure 2 This is a flowchart illustrating a medical image generation method based on a diffusion model in an embodiment of this application. Figure 3 This is a schematic diagram of the network structure for spatiotemporal feature extraction in an embodiment of this application; Figure 4 This is a schematic diagram of the conditional wavelet residual network structure inside the Schrödinger bridge diffusion model in the embodiments of this application; Figure 5 This is a schematic diagram of the overall architecture and data flow of the medical image generation method based on the diffusion model in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of a medical image generation system based on a diffusion model in an embodiment of this application; Explanation of reference numerals in the attached figures: 201, Central Processing Unit; 202, Read-Only Memory; 203, Random Access Memory; 204, Bus; 205, Input / Output Interface; 206, Input Section; 207, Output Section; 208, Storage Section; 209, Communication Section; 210, Driver; 211, Removable Media. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0022] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0023] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0024] Figure 1 This is a flowchart illustrating the training and generation stages of the Schrödinger bridge diffusion model in this application embodiment; the following is in conjunction with... Figure 1 The training and generation of the Schrödinger bridge diffusion model are explained in detail.

[0025] From an overall technical architecture perspective, this application aims to acquire real-time dynamic optical surface image sequences of patients and use them as conditional guidance to perform iterative denoising in the latent space using a pre-trained Schrödinger bridge diffusion model combined with wavelet decomposition, ultimately generating high-fidelity three-dimensional CT images of the patient's interior. However, in order for the Schrödinger bridge diffusion model to possess the ability to accurately fit the highly nonlinear cross-modal mapping relationship between external surface undulations and complex internal anatomical structures, the initial network must be sufficiently trained using historical sample data before performing the actual image generation task. Therefore, prior to step S101, the method of this embodiment further includes a pre-training stage for the Schrödinger bridge diffusion model, specifically including the following steps: obtaining a training sample set, the training sample set including multiple sets of training samples, each set of training samples including a sample dynamic optical surface image sequence and a sample three-dimensional CT image; inputting the sample initial latent variables that follow a preset noise distribution and the sample dynamic optical surface image sequence into the initial Schrödinger bridge diffusion model to be trained to obtain a predicted three-dimensional CT image; calculating the Schrödinger bridge loss, regularization loss, and adversarial loss based on the predicted three-dimensional CT image and the sample three-dimensional CT image, respectively; updating the network parameters of the initial Schrödinger bridge diffusion model based on the Schrödinger bridge loss, the regularization loss, and the adversarial loss until the network parameters meet a preset convergence condition, thereby obtaining the pre-trained Schrödinger bridge diffusion model.

[0026] To enable the model to learn the highly nonlinear cross-modal mapping relationship between external surface undulations and complex internal anatomical structures, the system acquires a training sample set. This training sample set includes multiple sets of training samples, each consisting of a sequence of dynamic optical surface images and a 3D CT image. Specifically, in clinical practice, the system collects 4D CT images of lung cancer patients during the simulated localization phase, along with corresponding dynamic optical surface point cloud data. A B-spline deformation registration algorithm is used to spatially align the corresponding phases of the 3D surface images with the phases of the 4D CT scans, forming paired or unpaired training data. Simultaneously, Gaussian noise data augmentation technology is used to expand the limited data. This multi-dimensional data collection and augmentation process ensures the diversity and sufficiency of the training data, providing a solid data foundation for the model's subsequent generalization ability. For example, the system can collect data from 400 clinical lung cancer cases covering different stages and respiratory patterns, expanding it to three times the original data to construct a comprehensive training sample set.

[0027] Based on the constructed training sample set, to overcome the limitations of traditional diffusion models that rely on Gaussian prior assumptions in unpaired cross-modal image conversion tasks, the system inputs the initial latent variables of the samples, which follow a preset noise distribution, and the sample dynamic optical surface image sequence into the initial Schrödinger bridge diffusion model to be trained, thereby obtaining the predicted 3D CT image. The initial latent variables of the samples are high-dimensional noise tensors randomly sampled from the preset prior noise distribution. They serve as the physical starting point for the inverse generation process of the diffusion model. Guided by the spatiotemporal conditions provided by the sample dynamic optical surface image sequence, they are gradually denoised and "sculpted" into a predicted 3D CT image with actual anatomical structure by the model.

[0028] The core idea of ​​Schrödinger's bridge theory originates from thermodynamics and aims to find a smooth, optimal stochastic path that is "closest" to a reference stochastic process in a thermodynamic sense. Specifically, this principle first describes the dynamic evolution path of the probability density through a stochastic differential equation, expressed as: Where, b(t, x) t ) represents the drift term, σ represents the diffusion coefficient, and W represents the diffusion coefficient. t This is standard Brownian motion. Combining the model training and generation architecture, we know that during the diffusion process, the system inputs the original image data (denoted as x0) into the model, and then processes it through a neural network (denoted as q). φ The target image is predicted at different time steps (denoted as x1), and noisy images at different time steps are generated step by step by combining Brownian motion noise (denoted as x). t_i This completes the forward propagation of the model and outputs the predicted 3D CT image (i.e. the generated fake image).

[0029] After obtaining the predicted 3D CT image through model forward propagation, in order to accurately evaluate the generation quality and guide network optimization, the system calculates the Schrödinger bridge loss, regularization loss, and adversarial loss based on the predicted 3D CT image and the sample 3D CT image, respectively. Specifically, to find the optimal propagation path, the system calculates the Schrödinger bridge loss, the formula of which is expressed as: In this formula, For noisy image x t_i The mean (expectation) of the distance between the generated fake image x1 and the generated fake image x1 is used to make the diffusion process tend to the optimal transmission and minimize the transportation cost; Let H be the information entropy of the fake image probability, which serves as the entropy regularization term to measure path deviation. The joint distribution of the generated fake image and the conditional distribution is expressed as: The edge distribution of the generated fake image is represented as: Meanwhile, to constrain the range of image transformation and ensure the consistency of content generated across modalities, the system calculates a regularization loss, expressed by the following formula: , where R is a differentiable similarity scalar function (e.g., a simple downsampling network) used to process the original image x0 and the generated fake image x1 to calculate feature similarity. Furthermore, to constrain the endpoint of the path, the system introduces a discriminator network to calculate adversarial loss, i.e., the discriminator loss in classic generative adversarial networks, used to distinguish whether the generated fake image belongs to the target domain of the real 3D CT image (denoted as π1 distribution).

[0030] Based on the Schrödinger bridge loss, regularization loss, and adversarial loss calculated above, in order to continuously approach the optimal cross-modal mapping capability of the model, the system performs backpropagation and updates on the network parameters of the initial Schrödinger bridge diffusion model until the network parameters meet the preset convergence condition, thus obtaining the pre-trained Schrödinger bridge diffusion model. Specifically, the system first calculates the global total loss by weighting the above three losses according to preset weight coefficients. Then, it uses the backpropagation algorithm to calculate the gradient of this total loss with respect to the network parameters, and uses a preset deep learning optimizer (such as the Adam optimizer) combined with a dynamically decaying learning rate to fine-tune and update the network parameters such as convolutional kernel weights and biases along the gradient descent direction. During continuous iterative training, the system monitors the network state in real time to determine whether the preset convergence condition is met. The preset convergence conditions specifically include two scenarios: First, the total loss value of the model on the validation set changes by less than a preset minimum threshold within a preset number of iteration cycles (epochs) (i.e., the loss curve tends to stabilize, the model performance no longer significantly improves, and it has reached a local or global optimum); second, the overall number of iterations of the model training reaches a preset maximum iteration threshold (to prevent the model from getting stuck in infinite training or overfitting). The preset minimum threshold and maximum iteration threshold are both determined by those skilled in the art based on multiple pre-experiment results using historical sample data, hardware computing power limitations, and the actual clinical requirements for image generation accuracy. When the system detects that either of the above convergence conditions is met, it determines that the model has fully learned the optimal evolution law from surface data to internal CT images. At this point, the system automatically stops gradient calculation and parameter updates, and extracts, solidifies, and saves the network parameters (i.e., weight file) at the current moment, thus formally obtaining the pre-trained Schrödinger bridge diffusion model. For example, in actual training, the system sets the maximum iteration cycle to 500 times. If, during the 420th iteration, the system finds that the total loss decreases by less than 0.001 for 20 consecutive iterations, the convergence condition is triggered, the system terminates the training early, and saves the model weights at this time. The saved model then has the ability to generate high-fidelity images that can be directly applied to the clinical reasoning stage.

[0031] Figure 2 This is a schematic flowchart of a medical image generation method based on a diffusion model in an embodiment of this application.

[0032] Please see Figure 2 This application provides an embodiment of a medical image generation method based on a diffusion model, the method comprising: S101. Obtain the initial latent variables for image generation and the dynamic optical surface image sequence of the target object, wherein the initial latent variables are sampled from a preset prior distribution; After completing the pre-training of the Schrödinger bridge diffusion model, the system enters the actual medical image inference and generation stage. In order to obtain real-time guidance information for predicting internal anatomical structures without increasing the patient's radiation dose, the system performs the steps of acquiring initial latent variables for image generation and dynamic optical surface image sequences of the target object.

[0033] In practice, the target audience is typically lung cancer patients undergoing radiotherapy. The system uses an optical surface monitoring device installed in the treatment room to non-invasively and continuously capture real-time images of the patient's chest and abdomen, acquiring three-dimensional point clouds or depth maps containing multiple consecutive time points, forming a dynamic optical surface image sequence. Simultaneously, to provide a physical starting point for image generation in the diffusion model, the system samples initial latent variables from a preset prior distribution. These initial latent variables are essentially high-dimensional pure noise matrices. In the generation principle of the diffusion model, the model needs to start from a completely disordered random noise state and gradually remove noise to reconstruct a meaningful image. The preset prior distribution is usually set to a standard normal distribution, and the system uses a random number generation algorithm to extract values ​​from this distribution to construct the initial latent variables. Using the initial latent variables as a "blank canvas" and the dynamic optical surface image sequence as a "conditional paintbrush," the system can precisely guide the generation process of internal three-dimensional CT images using the external surface motion state, completely avoiding the secondary radiation hazards of traditional real-time CT scans.

[0034] For example, when a lung cancer patient is lying on a radiotherapy bed, a binocular vision camera on the ceiling of the machine room continuously captures the three-dimensional contour of the patient's chest surface at a frequency of 10 frames per second, forming a dynamic optical surface image sequence. At the same time, the system backend generates a random noise tensor with a size matching the target CT feature map as an initial latent variable through a Gaussian noise generator. The system uses the initial latent variable and the dynamic optical surface image sequence together as the input basis for subsequent denoising generation steps.

[0035] S102. Perform spatiotemporal feature extraction on the dynamic optical body surface image sequence to obtain the body surface motion feature vector; After obtaining the dynamic optical surface image sequence and initial latent variables, in order to accurately capture the dynamic mapping relationship between the patient's surface undulations and internal anatomical structures, the system extracts spatiotemporal features from the dynamic optical surface image sequence to obtain a surface motion feature vector. The dynamic optical surface image sequence not only contains the three-dimensional spatial contour of the patient's chest and abdomen at a given moment, but also contains the dynamic temporal patterns that evolve with the respiratory cycle. Simply relying on a single frame image cannot accurately predict the movement trajectory of the internal tumor. Therefore, the system first needs to perform spatial dimension feature mining on each independent frame in the dynamic optical surface image sequence, and then input the mined spatial features into a Long Short-Term Memory (LSTM) network in chronological order for temporal correlation analysis. The LSTM network is a recurrent neural network specifically designed for processing sequential data. It incorporates input gates, forget gates, and output gates, effectively memorizing historical respiratory information over long periods and filtering out irrelevant local noise. By deeply fusing spatial and temporal features, the system finally outputs a surface motion feature vector, providing precise motion state guidance for the subsequent diffusion denoising process, specifically including steps S1021-S1022: S1021. Spatial features are extracted from each phase image in the dynamic optical body surface image sequence to obtain the spatial dimension features corresponding to each phase image. Because clinically acquired optical surface images are often affected by changes in ambient light, slight obstruction by patient clothing, or sensing errors in the monitoring equipment itself, single-frame temporal images frequently contain local noise or abrupt changes. If these noisy temporal images are directly input into the subsequent Long Short-Term Memory (LSTM) network, the abrupt changes will be mistakenly identified as actual respiratory movements by the network, severely affecting the accuracy of the surface motion feature vectors. Therefore, the system needs to establish an anomaly filtering mechanism based on local neighborhood comparison. Specifically, this may include the following steps: Each of the aforementioned temporal images undergoes initial convolution processing to obtain an initial spatial feature map, which includes multiple feature nodes; Traverse each feature node in the initial spatial feature map and define a local spatial neighborhood centered on the current feature node; Extract the target feature value of the current feature node and the reference feature values ​​of adjacent feature nodes in the local spatial neighborhood; Calculate the spatial feature difference between the target feature value and the reference feature value, and determine the local anomaly confidence of the current feature node based on the spatial feature difference, thereby obtaining the local anomaly confidence of each feature node; Based on the local anomaly confidence level corresponding to each feature node, a feature mask weight matrix matching the size of the initial spatial feature map is generated. The initial spatial feature map is multiplied by the feature mask weight matrix to obtain the corrected feature map. Spatial pooling is performed on the corrected feature map to obtain the spatial dimension features corresponding to each of the temporal images.

[0036] Figure 3 This is a schematic diagram of the network structure for spatiotemporal feature extraction in an embodiment of this application. The following is a combination of... Figure 3 Step S1021 will be described in detail.

[0037] To extract physically meaningful patient surface contour edges and texture information from the raw optical surface pixel array, the system performs initial convolution processing on each temporal image to obtain an initial spatial feature map. Since single spatial convolution is insufficient to capture complex, subtle surface deformations, the system adds wavelet convolution frequency domain feature learning to the existing spatial convolution feature learning. Specifically, the system first uses a wavelet transform function to map the input temporal image to the wavelet domain, separating low-frequency components representing global chest and abdominal undulations and high-frequency components representing local skin folds or marker details. Subsequently, to process the three-dimensional surface data containing depth information, the system divides a complete respiratory cycle of the patient into multiple consecutive time steps, such as from time step t=1 to time step t=10 as shown in the attached figure. In each time step, the system uses a three-dimensional convolutional neural network (3D CNN) to extract features from the wavelet domain components of the current temporal image. As shown in the figure, the 3D convolutional neural network contains multiple cascaded convolutional layers, namely layers C1, C2, C3, C4, and C5. The system employs a 5x5 3D convolutional kernel in layer C1 and a 3x3 3D ​​convolutional kernel in subsequent layers C2 to C5, performing sliding window operations in 3D space. As the network depth increases, the spatial size of the feature map gradually decreases (e.g., spatial resolution decreases from 55 to 27, and then further to 13), while the number of channels gradually increases (e.g., the number of channels increases from 256 to 384), thus mapping the voxel space data to a high-dimensional feature space. Finally, the system inversely transforms the processed frequency domain features back to the spatial domain, outputting the initial spatial feature map. The initial spatial feature map output after wavelet convolution is a 3D tensor, comprising multiple feature nodes. Each feature node is essentially a numerical unit of the initial spatial feature map in a specific spatial coordinate system. Each feature node corresponds to a specific-sized 3D receptive field in the original temporal image, representing the basic anatomical features of a local area on the body surface of a lung cancer patient.

[0038] After obtaining the initial spatial feature map containing a massive number of feature nodes, the system traverses each feature node in the initial spatial feature map to establish a spatial reference system for comparative analysis, and delineates a local spatial neighborhood centered on the current feature node. Because the respiratory fluctuations on the chest and abdomen of lung cancer patients during simulation positioning and radiotherapy exhibit high continuity and smoothness in physical space, any abrupt numerical changes are highly likely to be caused by noise from optical monitoring equipment, slight obstruction by patient clothing, or reflections from the edge of the treatment bed. Therefore, the system sequentially locks the feature nodes in the initial spatial feature map as the current feature node according to spatial coordinate order, and extracts a fixed-size 3D window in three-dimensional space as the local spatial neighborhood. The role of the local spatial neighborhood is to delineate the surrounding area that is closest to the current feature node in physical location, providing a background reference for subsequent anomaly detection. For example, the system uses the current feature node as the center coordinate and expands outwards to delineate a 3x3x3 3D ​​space as the local spatial neighborhood.

[0039] After defining the local spatial neighborhood, the system needs to obtain specific numerical values ​​for quantization calculations. Therefore, the system extracts the target feature value of the current feature node and the reference feature values ​​of adjacent feature nodes within the local spatial neighborhood. The target feature value represents the intensity of the surface undulation features at the center location currently being detected, while the reference feature values ​​represent the intensity of the normal surface undulation features surrounding the center location. The system directly reads the floating-point number corresponding to the current feature node from the tensor matrix of the initial spatial feature map as the target feature value, and simultaneously reads the set of floating-point numbers corresponding to all adjacent feature nodes within the local spatial neighborhood (excluding the center location) as reference feature values. For example, in the aforementioned 3x3x3 local spatial neighborhood, the system extracts the value at the very center as the target feature value and extracts the 26 values ​​surrounding the center as reference feature values.

[0040] After extracting the specific values ​​of the center and surrounding areas, in order to accurately identify noise points that do not conform to normal respiratory physiology, the system calculates the spatial feature difference between the target feature value and the reference feature value, and determines the local anomaly confidence of the current feature node based on the spatial feature difference, thus obtaining the local anomaly confidence of each feature node. In specific implementation, the system first calculates the average of all reference feature values, then calculates the absolute difference between the target feature value and the average value, using the absolute difference as the spatial feature difference. The larger the spatial feature difference, the more incongruous the fluctuation state of the center position is with the surrounding chest and abdominal environment. Subsequently, the system inputs the spatial feature difference into a Sigmoid activation function for normalization mapping, outputting a probability value between 0 and 1, which is defined as the local anomaly confidence. The local anomaly confidence is a custom quantitative indicator used to intuitively reflect the probability that the current feature node belongs to device noise or occlusion interference. For example, if the target feature value is 0.9, and the average value of the 26 surrounding reference feature values ​​is 0.2, the spatial feature difference between the two is extremely large. After function mapping, the local anomaly confidence level may be as high as 0.95, which means that the current feature node is very likely to be a noise point.

[0041] After evaluating the anomaly probabilities at all locations, to mathematically construct a filter capable of automatically masking noise, the system generates a feature mask weight matrix matching the size of the initial spatial feature map based on the local anomaly confidence level corresponding to each feature node. The system subtracts the local anomaly confidence level of each feature node from the value 1 to obtain the retention weight for each feature node. The physical meaning of the retention weight is that nodes with higher anomaly probabilities have retention weights closer to 0; normal nodes with lower anomaly probabilities have retention weights closer to 1. The system arranges the retention weights of all feature nodes according to the original spatial coordinates of the initial spatial feature map, combining them into a three-dimensional tensor to form the feature mask weight matrix. The feature mask weight matrix acts like a smart filter, precisely indicating which locations in the initial spatial feature map need to have their information weakened and which locations need to have their information fully preserved.

[0042] After constructing the intelligent filter, the system performs substantive noise reduction and filtering operations. The system performs a dot product operation between the initial spatial feature map and the feature mask weight matrix to obtain a corrected feature map. The dot product operation involves multiplying each feature node value in the initial spatial feature map element-wise with the corresponding weight in the feature mask weight matrix. Through this dot product operation, noise values ​​in the initial spatial feature map that are judged to have a high probability of being abnormal are multiplied by a very small weight close to 0, thus significantly suppressing or even eliminating their numerical value; while normal body contour feature values ​​are multiplied by a weight close to 1, thus being completely preserved. After the global dot product operation, the system outputs a corrected feature map that eliminates local mutation interference, is smoother, and fully conforms to the real human anatomy.

[0043] After obtaining the purified surface features, the data dimensionality of the corrected feature map is still too large. Directly inputting it into the subsequent Long Short-Term Memory (LSTM) network would overload computational resources and easily lead to overfitting. Therefore, the system performs spatial pooling on the corrected feature map to obtain the spatial dimensionality features corresponding to each phase image. Spatial pooling is a classic feature dimensionality reduction technique. The system typically uses a global average pooling algorithm to average the feature values ​​in each channel of the corrected feature map, compressing the three-dimensional spatial matrix into a one-dimensional feature vector. Spatial pooling not only significantly reduces the number of network parameters but also enhances the robustness of features to small spatial translations.

[0044] Because Long Short-Term Memory (LSTM) networks retain historical states, sudden actions such as coughing, swallowing, or involuntary muscle spasms during the acquisition of dynamic optical surface image sequences can cause drastic jumps in the spatial dimension features of adjacent time-phase images. If these drastically changed spatial dimension features are directly input into the LTM network, the sudden noise will be amplified and propagated by the network's cell states, severely contaminating the hidden states of all subsequent time steps. This ultimately leads to the output surface motion feature vector deviating from the actual respiratory trajectory. Therefore, the system must rigorously examine the smoothness of feature evolution between adjacent time-phase images before extracting temporal features, identifying and isolating abrupt frames that do not conform to normal physiological respiratory rhythms in advance, ensuring that the input sequence received by the LTM network maintains temporal continuity and stability. Specifically, this may include the following steps: According to the time series, the temporal feature change rate between the spatial dimension features corresponding to the temporal images in the target temporal image group is calculated. The target temporal image group consists of adjacent first temporal images and second temporal images, and the temporal sequence of the first temporal image precedes that of the second temporal image. If the change rate of the temporal feature is greater than the preset motion mutation threshold, then the spatial dimension feature corresponding to the second temporal image is determined to be a transient abnormal feature, and a corresponding temporal blocking identifier is generated for the transient abnormal feature; If the rate of change of the temporal feature is less than or equal to the preset motion mutation threshold, then the spatial dimension feature corresponding to the second temporal image is determined to be a smooth motion feature.

[0045] In its implementation, the system slides along the time axis, binding adjacent temporal images together to form a target temporal image group. This group consists of an adjacent first and second temporal image, with the first image preceding the second. The system extracts the spatial dimension features corresponding to the first and second temporal images. Since these features are high-dimensional vectors after pooling dimensionality reduction, the system uses the Euclidean distance algorithm to calculate the absolute distance between the two vectors in the feature space. This absolute distance is then divided by the acquisition time interval between the two temporal images to obtain the temporal feature change rate. The temporal feature change rate is a custom dynamic indicator used to accurately measure the severity of changes in the surface undulations of the patient's chest and abdomen per unit time. By calculating the temporal feature change rate, the system can transform minute muscle tremors or violent coughing movements, which are difficult to detect with the naked eye, into intuitive mathematical values. For example, the system combines the first phase image at time step t=2 and the second phase image at time step t=3 into a target phase image group, calculates the Euclidean distance between two one-dimensional vectors of length 64, and finds that the temporal feature change rate is 0.15.

[0046] After obtaining specific numerical indicators, the system needs to perform a safety assessment on these indicators. If the rate of change of the temporal feature exceeds a preset motion mutation threshold, the spatial dimension feature corresponding to the second temporal image is determined to be a transient abnormal feature, and a corresponding temporal blocking identifier is generated for the transient abnormal feature. The preset motion mutation threshold is a safety limit value pre-set by the system backend. This threshold is determined by statistically analyzing the maximum deformation rate upper limit of a large number of lung cancer patients under stable breathing conditions. The system compares the calculated rate of change of the temporal feature with the preset motion mutation threshold. If the rate of change exceeds the threshold, it indicates that the patient experienced a violent action such as coughing, swallowing, or involuntary convulsions at the moment the second temporal image was acquired. The system immediately determines the spatial dimension feature corresponding to the second temporal image to be a transient abnormal feature. Transient abnormal features represent contaminated data that does not conform to normal physiological respiratory rhythms. To prevent contaminated data from entering subsequent networks, the system generates a corresponding temporal blocking identifier for the transient abnormal feature. A temporal blocking flag is essentially a special mask instruction. Its function is to force the Long Short-Term Memory (LSTM) network to close its input gates or reset its cell state when the network processes the current time step, thereby completely cutting off the backward propagation path of abnormal information. For example, if the preset motion mutation threshold is 0.3, and the calculated temporal feature change rate is as high as 0.8, the system determines that the spatial dimension feature at time step t=3 is a transient abnormal feature and generates a temporal blocking flag with a value of "1" for time step t=3.

[0047] Corresponding to the above-mentioned abnormal interception situations, if the rate of change of temporal features is less than or equal to the preset motion mutation threshold, the spatial dimension features corresponding to the second temporal image are determined to be smooth motion features. When the comparison results show that the rate of change of temporal features is within the safe range of the preset motion mutation threshold, it indicates that the evolution process from the first temporal image to the second temporal image fully conforms to the normal respiratory fluctuation pattern of lung cancer patients, and no sudden muscle jumps have occurred. At this time, the system directly determines that the spatial dimension features corresponding to the second temporal image are smooth motion features. Smooth motion features refer to safe feature data that has high temporal continuity and can truly reflect the movement trajectory of the tumor in the target area. By determining smooth motion features, the system issues a pass to normal respiratory sequences, ensuring that smooth motion features can be input into the long short-term memory network without hindrance for deep temporal correlation analysis, thereby ensuring the purity and accuracy of the final output surface motion feature vector. For example, if the calculated temporal feature change rate is 0.15, which is significantly less than the preset motion mutation threshold of 0.3, the system determines that the spatial dimension feature at time step t=3 is a smooth motion feature, allowing the Long Short-Term Memory Network to normally receive and process the data at time step t=3.

[0048] S1022. The spatial dimension features corresponding to each of the time phase images are sequentially input into a preset long short-term memory network for temporal feature extraction to obtain the body surface motion feature vector.

[0049] Since the system has already generated temporal blocking markers for sudden abnormal actions in the pre-processing steps, a thorny contradiction arises when the pre-set Long Short-Term Memory (LSTM) network receives spatial dimension features sequentially: directly removing the abnormal features carrying the temporal blocking markers from the sequence would cause a break in the entire time series, preventing the LSTM network from maintaining continuous internal state evolution; conversely, forcibly inputting abnormal features into the LSTM network would severely contaminate its cell states. Therefore, the system must establish a dynamic compensation mechanism within the LSTM network that can both isolate noise and fill temporal gaps. Specifically, this can include the following steps: During the process of sequentially inputting the spatial dimension features corresponding to each of the aforementioned temporal images into the preset long short-term memory network, if the currently input spatial dimension feature carries the temporal blocking identifier, a dynamic state compensation mechanism is triggered. The dynamic state compensation mechanism includes: decaying the input gate activation value of the preset long short-term memory network for the current time step corresponding to the currently input spatial dimension feature to a preset lower limit value, and extracting the hidden layer memory state of the preset long short-term memory network at the previous time step corresponding to the previously input spatial dimension feature. Based on the hidden layer memory state of the previous time step and the preset temporal decay coefficient, the hidden layer memory state of the current time step is reconstructed through autoregression to obtain the reconstructed memory state. Based on the reconstructed memory state, the temporal feature extraction of subsequent phase images continues until the spatial dimension features corresponding to each phase image have been extracted in a temporal manner, thus obtaining the body surface motion feature vector.

[0050] The spatial dimension features output after spatial pooling are used as the input variable (denoted as Xt) for the current time step and directly input into the corresponding time node of the preset Long Short-Term Memory (LSTM) network. The preset LSTM network contains a forget gate, an input gate, and an output gate. In the normal temporal feature extraction process, the preset LSTM network receives the input variable Xt for the current time step and combines it with the hidden state (denoted as ht-1) and cell state (denoted as Ct-1) passed from the previous time step, performing complex matrix multiplication and nonlinear activation operations, ultimately outputting the hidden state (denoted as ht) and the updated cell state (denoted as Ct) for the current time step. However, during the process of sequentially inputting the spatial dimension features corresponding to each temporal image into the preset LSTM network, if the currently input spatial dimension feature carries a temporal blocking indicator, it indicates that the currently input spatial dimension feature is abnormal mutation data caused by the patient's cough or convulsions. To prevent abnormal mutation data from contaminating the internal state of the preset LSTM network, the system triggers a dynamic state compensation mechanism. The dynamic state compensation mechanism is a fault-tolerant strategy specifically designed to address temporal breaks. Its core principle is to physically block abnormal inputs and fill the gaps using historical healthy states. Specifically, the dynamic state compensation mechanism involves the system decaying the input gate activation value of the preset Long Short-Term Memory (LSTM) network at the current time step, corresponding to the spatial dimension features of the current input, to a preset lower limit. The input gate activation value is a dynamically calculated ratio between 0 and 1 in the LSTM network; it acts as a "valve opening" controlling the inflow of information, determining how much new data can be allowed to pass through and be written into the network's long-term memory. The preset lower limit is an extremely small value infinitely close to zero. Decaying the input gate activation value to the preset lower limit means the system completely closes the input channel at the current time step, forcibly rejecting the input variable Xt carrying the temporal blockage flag from entering the preset LSTM network. Simultaneously, the system extracts the hidden layer memory state of the preset LSTM network at the previous time step, corresponding to the spatial dimension features of the previous input. The hidden layer memory state of the previous time step specifically includes the hidden state ht-1 and cell state Ct-1 passed from the previous time step, representing the patient's last stable and healthy respiratory fluctuations before the abnormal movement occurred. For example, assuming the input variable X3 at time step t=3 carries a temporal blocking flag, the system immediately modifies the input gate activation value of time step t=3 to 0.0001, completely blocking the inflow of input variable X3, and simultaneously extracts the hidden state h2 and cell state C2 output from time step t=2.

[0051] After successfully isolating abnormal noise and extracting historical health states, the system performs autoregressive reconstruction of the hidden layer memory state at the current time step. This is because the preset long short-term memory network must output a state at the current time step to pass on to the next time step; otherwise, the entire time series would be broken. The system uses the hidden layer memory state from the previous time step and a preset temporal decay coefficient to obtain the reconstructed memory state. The preset temporal decay coefficient is an empirical value derived from statistics on the smoothness of stable respiratory cycles in a large number of lung cancer patients. It is used to simulate the physical process of the natural continuation and slight decay of historical respiratory states over time in the absence of new external stimuli. Autoregressive reconstruction refers to the derivation process that does not rely on the current external input variable Xt, but only on the historical states of the preset long short-term memory network itself for self-evolution calculation. In specific implementation, the system multiplies the extracted hidden state ht-1 and cell state Ct-1 from the previous time step by the preset temporal decay coefficient, bypassing the normal input gate calculation logic, and directly generates the hidden state ht and the updated cell state Ct at the current time step. The generated hidden state ht and cell state Ct are collectively referred to as the reconstructed memory state. Through autoregressive reconstruction, the system effectively fills the gaps on the timeline when real and valid inputs are missing, maintaining the continuity of temporal features in the mathematical dimension. For example, with a preset temporal decay coefficient of 0.95, the system multiplies the hidden state h2 and cell state C2 at time step t=2 by 0.95 to deduce the reconstructed hidden state h3 and reconstructed cell state C3 at time step t=3, and uses the reconstructed hidden state h3 and reconstructed cell state C3 as the reconstructed memory states.

[0052] After reconstructing the state at the current time step, to obtain the complete global motion pattern, the system continues to extract temporal features from subsequent time-phase images based on the reconstructed memory state, until the spatial dimension features corresponding to each time-phase image have all been extracted temporally, resulting in a body surface motion feature vector. In specific implementation, the system passes the reconstructed hidden state ht and cell state Ct of the current time step to the next time step. In the next time step, if the received input variable Xt+1 represents normal smooth motion features, the preset long short-term memory network resumes normal matrix multiplication and nonlinear activation operations, combining the reconstructed memory state and input variable Xt+1 to generate a new hidden state ht+1 and cell state Ct+1. For example... Figure 3As shown, through sequential cyclic processing from time step t=1 to time step t=10, the preset long short-term memory network concatenates the spatial dimension features of all temporal images, and finally outputs spatial-temporal features that fuse global motion patterns at the last time step, using these spatial-temporal features as the body surface motion feature vector. Through the aforementioned dynamic compensation and continuous extraction mechanism, the system ensures that even when local optical body surface data is severely contaminated, it can still output a highly robust and accurate body surface motion feature vector, providing extremely precise motion state guidance for the subsequent denoising generation process of the Schrödinger bridge diffusion model. For example, the system passes the hidden state h3 and cell state C3 reconstructed at time step t=3 to time step t=4. Time step t=4 receives the normal input variable X4. The preset long short-term memory network combines the hidden state h3, cell state C3, and input variable X4 to calculate the hidden state h4 and cell state C4. The preset long short-term memory network continues to calculate cyclically until time step t=10, finally outputting a body surface motion feature vector containing the dynamic patterns of the complete respiratory cycle.

[0053] S103. Based on the pre-trained Schrödinger bridge diffusion model, iterative denoising processing is performed on the latent variables corresponding to each time step in sequence. Specifically, the following operations are performed during the denoising processing of each time step: S103-a. Perform wavelet decomposition on the latent variable corresponding to the current time step to obtain low-frequency components and high-frequency components, wherein the latent variable corresponding to the first time step is the initial latent variable. S103-b: Obtain the temporal embedding vector of the current time step, and encode the body surface motion feature vector into a conditional embedding vector; S103-c: Based on the time embedding vector and the conditional embedding vector, conditional fusion processing is performed on the low-frequency component and the high-frequency component respectively to obtain the target low-frequency component and the target high-frequency component. S103-d, Perform wavelet inverse transform on the target low-frequency component and the target high-frequency component to obtain the hidden variable corresponding to the next time step of the current time step; After successfully acquiring the surface motion feature vector that incorporates global motion patterns, the system enters the core medical image generation stage. Based on a pre-trained Schrödinger bridge diffusion model, it iterative denoising is performed on the latent variables corresponding to each time step. Since traditional diffusion models typically denoise in a single pixel space or latent space, they struggle to simultaneously maintain the consistency of global anatomical structures and the clarity of local tissue textures. Furthermore, traditional diffusion models are difficult to accurately map unpaired clinical data. Therefore, the system introduces a wavelet transform frequency domain processing mechanism into the denoising network of the Schrödinger bridge diffusion model, as described in steps S103-a to S103-d.

[0054] S103-a. Perform wavelet decomposition on the latent variable corresponding to the current time step to obtain low-frequency components and high-frequency components, wherein the latent variable corresponding to the first time step is the initial latent variable. Traditional diffusion models typically perform global denoising on latent variables in a single spatial dimension. This single-dimensional denoising approach often results in blurred edges in the generated medical images, making it difficult to simultaneously capture the global anatomical contours of the patient's chest and abdomen as well as the subtle textures of local tumor tissue. To overcome the shortcomings of single-dimensional denoising, the system introduces wavelet decomposition technology. Latent variables are high-dimensional tensor matrices used within the diffusion model to represent the image data. In the first time step, the latent variables corresponding to the first time step are the initial latent variables sampled from a predefined prior distribution, which are essentially a pure Gaussian noise matrix. In subsequent time steps, the latent variables corresponding to the current time step are the intermediate state matrices of the denoised output from the previous time step.

[0055] In practice, the system uses a preset wavelet basis function (e.g., the Haar wavelet basis) to perform a frequency domain transformation on the latent variables corresponding to the current time step. Through this frequency domain transformation, the system separates the latent variables at the current time step into two independent parts: low-frequency components and high-frequency components. The low-frequency components are approximate signals obtained after the latent variables pass through a low-pass filter. These components mainly contain gently changing background information and global macroscopic structures in the image. The high-frequency components, on the other hand, are detail signals obtained after the latent variables pass through a high-pass filter. These components typically contain matrices in multiple directions, including horizontal, vertical, and diagonal directions, and mainly capture edges, contours, and local abrupt textures in the image. By performing wavelet decomposition on the latent variables at the current time step, the system successfully decouples the complex image generation task into a dual-track task of macroscopic structure generation and microscopic detail generation. This allows subsequent conditional fusion processing to apply motion guidance of varying intensities to features at different frequencies, thereby significantly improving the clarity and accuracy of the anatomical structures in the final generated 3D CT image. For example, suppose the latent variable corresponding to the current time step is a three-dimensional tensor with a spatial dimension of 64 x 64 x 64. The system uses wavelet basis functions to decompose the three-dimensional tensor and outputs a low-frequency component with a spatial dimension of 32 x 32 x 32, as well as multiple high-frequency components with the same spatial dimension of 32 x 32 x 32 (corresponding to detailed features in different spatial directions).

[0056] S103-b: Obtain the temporal embedding vector of the current time step, and encode the body surface motion feature vector into a conditional embedding vector; After wavelet decomposition, the Schrödinger Bridge diffusion model needs to determine its specific stage in the denoising evolution process and requires external guidance on the patient's respiratory status. Therefore, the system performs the operation of obtaining the temporal embedding vector of the current time step and encoding the body surface motion feature vector into a conditional embedding vector. The temporal embedding vector is a high-dimensional mathematical feature used to characterize the denoising progress, indicating the noise intensity level currently faced by the Schrödinger Bridge diffusion model. In specific implementation, the system uses a sine-cosine positional encoding function to map the scalar value of the current time step. This function, through combinations of trigonometric functions of different frequencies, transforms a single time step value into a temporal embedding vector with rich dimensional information.

[0057] Meanwhile, to ensure that the Schrödinger Bridge diffusion model generates medical images strictly according to the patient's actual physical movement trajectory, the system extracts the surface motion feature vectors output by the Long Short-Term Memory network in the previous step. Since the data dimension of the surface motion feature vectors usually does not match the feature dimension within the Schrödinger Bridge diffusion model, the system utilizes a pre-defined multilayer perceptron network to perform dimensional transformation and feature reshaping on the surface motion feature vectors, encoding them into conditional embedding vectors. The conditional embedding vector is essentially a "blueprint" translated into neural network language, containing the patient's chest and abdominal undulation patterns and target area displacement information at specific respiratory phases. By acquiring the temporal embedding vector and the conditional embedding vector, the system provides precise coordinate references and content guidance for subsequent frequency domain feature fusion, ensuring that the Schrödinger Bridge diffusion model not only knows "how much noise needs to be removed" but also understands "what kind of anatomical structure needs to be generated ultimately."

[0058] For example, suppose the Schrödinger bridge diffusion model has a total of 1000 denoising time steps, and the current time step is step 500. The system inputs the value 500 into the sine-cosine position encoding function, outputting a time embedding vector of length 256. Simultaneously, the system inputs a body motion feature vector of length 64, output from the Long Short-Term Memory network, into a multilayer perceptron network. The multilayer perceptron network performs nonlinear mapping and dimensionality expansion on the 64-length vector, outputting a conditional embedding vector of length 256, thus perfectly adapting to the fusion calculation requirements of subsequent steps.

[0059] S103-c: Based on the time embedding vector and the conditional embedding vector, conditional fusion processing is performed on the low-frequency component and the high-frequency component respectively to obtain the target low-frequency component and the target high-frequency component. Because the low-frequency components output by wavelet decomposition carry the global macroscopic anatomical structure of the patient's chest and abdomen, while the high-frequency components contain the microscopic details of local organ edges and lesion textures, the low-frequency and high-frequency components play drastically different physical roles in medical image generation. Using a uniform fusion strategy could easily lead to distortion of the macroscopic structure or loss of microscopic details. Therefore, the system must perform customized conditional fusion processing on the low-frequency and high-frequency components based on temporal embedding vectors and conditional embedding vectors, respectively. Specifically, the system constructs independent mathematical mapping channels within the Schrödinger bridge diffusion network. A specially designed mapping function is used to jointly transform the temporal embedding vector and conditional embedding vector into feature matrices that perfectly match the dimensions of the frequency domain components. Then, a residual addition mechanism is used to precisely superimpose the transformed feature matrices onto the low-frequency and high-frequency components, respectively. Through the above independent mapping and residual superposition operations, the system achieves a smooth transition and deep penetration of externally guided conditions into the target low-frequency and high-frequency components without destroying the original frequency domain information. The specific formula derivation and fusion steps are described below.

[0060] The time embedding vector and the conditional embedding vector are jointly mapped using a first mapping function, and the mapping result is added to and fused with the low-frequency component to obtain the target low-frequency component. The calculation formula is as follows: ,in, For the target low-frequency component, A t For the low-frequency component, φ A The first mapping function; The time embedding vector and the conditional embedding vector are jointly mapped using a second mapping function, and the mapping result is added to and fused with the high-frequency component to obtain the target high-frequency component. The calculation formula is as follows: ,in, For the target high-frequency component, D t For the high-frequency component, φ D Let e ​​be the second mapping function. t Let e ​​be the time embedding vector. z The conditional embedding vector is denoted as .

[0061] For the low-frequency components representing the global macroscopic anatomical structure, the system performs joint mapping of the temporal embedding vector and conditional embedding vector through a first mapping function, and then adds the mapping result to the low-frequency components to obtain the target low-frequency components. Since the low-frequency components determine the overall contour and basic morphology of the patient's chest and abdomen, directly replacing the low-frequency components with external guidance information would cause the image's basic structure to collapse. Therefore, the system adopts a residual addition fusion strategy. Specifically, the first mapping function is a pre-built multilayer perceptron or convolutional network module within the Schrödinger bridge diffusion model, specifically designed to handle smooth global features. The system inputs the temporal embedding vector and conditional embedding vector into the first mapping function. The first mapping function first concatenates or performs cross-attention calculation on the temporal embedding vector and conditional embedding vector, and then maps the concatenated or calculated feature matrix into a guidance tensor with the same spatial dimension as the low-frequency components. Next, the system calculates according to formula A... t '=A t +φ A (e t ,e z The system performs element-wise addition of the guiding tensor and the original low-frequency components. Through this addition and fusion operation, the system successfully injects the macroscopic deformation rules required for the current denoising stage into the low-frequency components while preserving the original global structure of the latent variables. This ensures that the output target low-frequency components accurately match the patient's current respiratory phase.

[0062] For example, suppose that the temporal embedding vector and the conditional embedding vector are both one-dimensional arrays of length 256, and the low-frequency component is a three-dimensional matrix of dimension 32 by 32 by 32 by 32. The first mapping function transforms the two one-dimensional arrays into a three-dimensional guiding matrix of the same dimension 32 by 32 by 32 by 32. The system adds the values ​​of the three-dimensional guiding matrix to the corresponding positions of the low-frequency component to obtain the final target low-frequency component.

[0063] After guiding the macroscopic structure of the low-frequency components, the system uses a second mapping function to jointly map the temporal embedding vector and conditional embedding vector for the high-frequency components representing the edges of local organs and the texture of lesions. The mapping result is then added to and fused with the high-frequency components to obtain the target high-frequency components. Since the microscopic details contained in the high-frequency components exhibit significantly different patterns of change during respiratory motion compared to the global contour, and local tumors often involve complex non-rigid deformations, using multiple mapping channels would result in blurred tumor edges. Therefore, the system employs a second mapping function, independent of the first mapping function, specifically to handle detailed features. This second mapping function is another set of neural network modules optimized for high-frequency mutation signals.

[0064] In practical operation, the system inputs the temporal embedding vector and the conditional embedding vector into a second mapping function. The second mapping function extracts deep features related to local high-frequency deformations from the temporal and conditional embedding vectors and transforms these deep features into detail guidance tensors that match the dimensions of the high-frequency components. Subsequently, the system calculates according to formula D... t =D t +φ D (e t ,e z The system guides the detail tensor and performs element-wise superposition with the original high-frequency components. Through independent mapping and superposition, the system applies exclusive micro-motion constraints to the high-frequency components, ensuring that the output target high-frequency components can still maintain extremely sharp tissue boundaries and realistic lesion textures after denoising.

[0065] For example, for the high-frequency component matrices in the horizontal, vertical and diagonal directions generated by wavelet decomposition, the second mapping function generates three corresponding detail guidance matrices based on the time embedding vector and the conditional embedding vector. The system adds the detail guidance matrices to the high-frequency component matrices in the corresponding directions, thereby accurately reconstructing the minute anatomical details of the target object in each spatial direction.

[0066] Figure 4 This is a schematic diagram of the conditional wavelet residual network structure inside the Schrödinger bridge diffusion model in this application embodiment.

[0067] Combination Figure 4 As shown, the core carrier for iterative denoising in the Schrödinger bridge diffusion model is the conditional wavelet residual network. In the left-hand backbone of the conditional wavelet residual network, from top to bottom, the network consists of an initial 7x7 convolutional layer, multiple 3x3 convolutional layers, and a downsampling layer (labeled "down" in the figure). The initial convolutional and downsampling layers are mainly used for preliminary spatial feature extraction and spatial dimensionality compression of the input latent variables. The feature matrix after dimensionality compression then enters multiple cascaded wavelet residual blocks in the middle of the network for in-depth frequency domain feature mining. After processing all wavelet residual blocks, the feature matrix enters the bottom upsampling network for progressive restoration of spatial resolution, finally outputting the denoised latent variables.

[0068] To further reveal how external guiding conditions are injected into frequency domain features, such as Figure 4As shown in the magnified area on the right, each wavelet residual block incorporates a unique dual-path fusion structure. The main path of the wavelet residual block contains two layers of cascaded wavelet transform convolutions (labeled WTConv in the figure). Unlike conventional spatial domain convolutions, wavelet transform convolutions can independently and precisely extract low-frequency and high-frequency components from latent variable decomposition in the frequency domain. Wavelet transform convolutions form the underlying physical implementation of the first and second mapping functions mentioned earlier. At the end of the wavelet residual block, the system sets a feature addition node (the ⊕ symbol in the figure). This node receives the feature matrix processed by the wavelet transform convolution and simultaneously receives the external input temporal embedding vector (the time-related embeddings labeled in the figure; in actual medical image generation tasks, the input also includes a conditional embedding vector encoded by body surface motion feature vectors). At the feature addition node, the system performs element-wise addition and fusion of the temporal embedding vector, the conditional embedding vector, and the feature matrix. The physical operation process shown in the magnified image on the right is precisely the system's execution of the formula. and The underlying network operation logic. Through the aforementioned conditional wavelet residual network architecture, the system ensures that each iteration of the denoising process accurately follows the patient's real-time respiratory trajectory.

[0069] S103-d, Perform wavelet inverse transform on the target low-frequency component and the target high-frequency component to obtain the hidden variable corresponding to the next time step of the current time step; After performing conditional fusion processing on the low-frequency and high-frequency components respectively, the system obtains the target low-frequency and high-frequency components fused with external motion guidance information. Since the denoising network of the Schrödinger bridge diffusion model requires the input data to be a complete tensor matrix in a unified spatial dimension during cross-time step iterations, the target low-frequency and high-frequency components, being in the frequency domain, cannot be directly used as the input basis for the next denoising stage. Therefore, the system must perform inverse wavelet transform on the target low-frequency and high-frequency components to obtain the latent variables corresponding to the next time step. Mathematically, the inverse wavelet transform is the strict inverse operation of wavelet decomposition in the preceding steps. The core principle of the inverse wavelet transform is to use a pre-defined synthesis filter bank to upsample and spatially reconstruct signals in different frequency bands.

[0070] In practice, the system synchronously inputs the target low-frequency component and the target high-frequency component (including horizontal, vertical, and diagonal components) into the wavelet inverse transform module. Following the reconstruction rules of orthogonal wavelet bases, the wavelet inverse transform module aligns the macroscopic anatomical information carried by the target low-frequency component with the microscopic tissue detail information carried by the target high-frequency component in terms of spatial dimensions and performs pixel-level superposition and fusion, thereby restoring the originally separated frequency domain feature matrix into a complete spatial domain high-dimensional tensor. The spatial domain high-dimensional tensor restored by the wavelet inverse transform is the latent variable corresponding to the next time step. By performing the wavelet inverse transform operation, the system not only successfully restores the original spatial resolution of the latent variables, but more importantly, it perfectly locks the patient's actual respiratory motion patterns within the reconstructed latent variables. As the denoising time steps progress, each newly generated latent variable contains less random noise than the previous one and more closely approximates the patient's actual physiological state at a specific respiratory phase in terms of anatomical morphology, thus ensuring the accuracy of the entire Schrödinger bridge diffusion model evolution trajectory.

[0071] For example, suppose that after conditional fusion processing, the system obtains a target low-frequency component with a spatial dimension of 32x32x32, and three target high-frequency components with the same spatial dimension of 32x32x32. The system uses the inverse wavelet transform algorithm to upsample and synthesize these four components, ultimately outputting a complete three-dimensional tensor with a spatial dimension restored to 64x64x64. The system uses this complete three-dimensional tensor with a dimension of 64x64x64 as the latent variable for the next time step and inputs it into the next denoising loop for further processing.

[0072] S104. When the iterative denoising process satisfies the preset evolution termination condition, the latent variable corresponding to the target latent variable is determined, and the target latent variable is input into the pre-trained image decoder to obtain the three-dimensional CT image of the target object.

[0073] After performing multiple rounds of iterative denoising within the Schrödinger bridge diffusion model, the system needs to determine whether the denoising process has completely eliminated Gaussian noise in the initial latent variables and transform the high-dimensional mathematical matrix into a medical image that doctors can intuitively interpret.

[0074] The preset evolution termination condition is a time step threshold pre-set by the system backend. This condition typically indicates that the Schrödinger bridge diffusion model has completed all prescribed denoising iterations. In practice, the system monitors the current time step value in real time. When the system detects that the current time step has reached the preset evolution termination condition, it means that the random noise within the latent variables has been completely removed, and the latent variables have evolved into a pure feature matrix that incorporates the patient's actual respiratory movement patterns. At this point, the system immediately stops the iterative loop and directly uses the latent variable output from the last wavelet inverse transform as the target latent variable. Although the target latent variable already possesses clear anatomical structural features, it remains in a highly compressed latent space dimension and cannot be directly displayed as a visual image. To bridge the semantic gap between the latent space and the real physical pixel space, the system extracts the target latent variable and inputs it into a pre-trained image decoder. The pre-trained image decoder is an inverse mapping module built on a 3D convolutional neural network. Before the system's official operation, the pre-trained image decoder has undergone extensive parameter optimization using numerous real medical images.

[0075] After receiving the target latent variables, the pre-trained image decoder re-unfolds the highly condensed feature matrix into a voxel matrix with true physical dimensions and grayscale values ​​through layer-by-layer 3D deconvolution and upsampling operations. Following spatial reconstruction by the pre-trained image decoder, the system finally outputs a 3D CT image of the target object. Through these decoding operations, the system not only perfectly recreates the 3D spatial morphology of the patient's thoracic and abdominal organs, but also, because the entire generation process is strictly constrained by the body surface motion feature vectors, the final generated 3D CT image can accurately match the patient's true respiratory phase at the moment of specific optical surface image acquisition, thus providing extremely reliable dynamic imaging evidence for subsequent precision radiotherapy of lung cancer.

[0076] For example, assuming the Schrödinger bridge diffusion model has a total denoising step count of 1000, when the system detects that the time step has been traced back to step 0, the system determines that the iterative denoising process meets the preset evolution termination condition and extracts the target latent variable with a spatial dimension of 64x64x64 from the output of step 0. Subsequently, the system inputs the target latent variable with a dimension of 64x64x64 into a pre-trained image decoder. The pre-trained image decoder undergoes multiple 3D deconvolutions for amplification, and finally outputs a 3D CT image of the target object with a spatial resolution of 256x256x256, which includes the CT density of real human tissue.

[0077] Figure 5 This is a schematic diagram of the overall architecture and data flow of the medical image generation method based on the diffusion model in the embodiments of this application, combined with Figure 5As shown, it is mainly divided into three core logical modules: spatiotemporal feature extraction, iterative denoising generation, and model training constraints.

[0078] First, in order to accurately capture the physical deformation patterns of patients at different respiratory stages, the system acquires a sequence of dynamic optical surface images of the target object. For example... Figure 5 As shown in the top left corner, the system inputs images from each time step t=1 to t=10 in the dynamic optical body surface image sequence into a 3D convolutional neural network (i.e., the 3D CNN labeled in the figure) for spatial feature extraction, outputting spatial dimensional features stripped of redundant background. Subsequently, the system inputs these spatial dimensional features sequentially into a Long Short-Term Memory (LSTM) temporal processing network (i.e., the LSTM labeled in the figure). The LSTM network performs in-depth contextual analysis and anomaly compensation on the spatial dimensional features, ultimately outputting a body surface motion feature vector that incorporates global motion patterns. This body surface motion feature vector will serve as the core guiding coordinates in the subsequent image generation process.

[0079] After acquiring the body surface motion feature vectors, the system activates a pre-trained Schrödinger bridge diffusion model. The system samples a pure noise matrix from a pre-defined prior distribution as the initial latent variable (corresponding to...). Figure 5 The x0~π0 input terminals on the left side). The system inputs the initial hidden variables to the core denoising network of the Schrödinger bridge diffusion model (i.e., q marked in the figure). φ (x1|x t_j The core denoising network, as shown in the dashed box module, employs a classic U-shaped architecture. The left side of the core denoising network contains multiple convolutional layers (CIR) and downsampling layers (DownSample) to compress the spatial dimension of latent variables. The middle section contains multiple wavelet residual blocks (WTResBlock) for frequency domain feature processing, and the right side contains multiple upsampling layers (UpSample) to restore image resolution. During the layer-by-layer operation of the core denoising network, the system extracts the current time step t. j And convert it into a temporal embedding vector, while simultaneously converting the body surface motion feature vector (corresponding to Figure 5 The z-transform path (marked in the diagram) is encoded as a conditional embedding vector. The system synchronously injects the temporal embedding vector and the conditional embedding vector into each intermediate wavelet residual block. Within the wavelet residual block, the system performs wavelet decomposition on the latent variables and uses the temporal embedding vector and the conditional embedding vector to conditionally fuse the decomposed low-frequency and high-frequency components. Finally, the processed latent variables are output through inverse wavelet transform.

[0080] After a complete forward computation, the core denoising network outputs the clean image features predicted at the current time step (i.e., x1(x) marked in the figure). t_jHowever, diffusion models cannot generate the final high-resolution image in one step; they must rely on rigorous mathematical iterative mechanisms. Figure 5 The complex mathematical formulas and adders (⊕) at the bottom illustrate the system's nonlinear evolution equations based on Schrödinger bridge theory, which incorporate the hidden variable x at the current time step. t_j Predicted clean image features x1(x t_j and according to a specific variance σ j+1 The sampled Gaussian noise N(0, σ) j+1 I) Perform a weighted summation calculation. Through this weighted summation calculation, the system derives the latent variable x corresponding to the next time step. t_j+1 The system will assign the hidden variable x corresponding to the next time step. t_j+1 The noise is then re-input into the core denoising network to start a new loop until the noise inside the latent variables is completely eliminated.

[0081] Furthermore, to ensure that the Schrödinger bridge diffusion model can output medical images that conform to real human anatomy, such as... Figure 5 As shown by the dashed path on the right, the system introduces a triple loss function constraint during the model training phase. The system performs pixel-level comparisons between the model-predicted 3D CT images and real sample 3D CT images, calculating a regularization loss to constrain macroscopic structural consistency, a Schrödinger bridge loss to constrain the smoothness of the denoising trajectory, and an adversarial loss to improve the realism of local lesion textures. The system integrates the Schrödinger bridge loss, regularization loss, and adversarial loss to update the network parameters through backpropagation, thereby endowing the model with powerful cross-modal generation capabilities. Finally, in the actual inference phase, when the iterative denoising process meets the preset evolution termination condition, the system uses an image decoder to transform the target latent variables into a 3D CT image of the target object, achieving a precise mapping from external optical surface signals to internal 3D anatomical structures.

[0082] Please see Figure 6 This is a schematic diagram of the structure of a medical image generation system based on a diffusion model in an embodiment of this application.

[0083] It should be noted that, Figure 6 The structure of a diffusion-based medical image generation system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0084] like Figure 6As shown, a diffusion-based medical image generation system includes a central processing unit 201, which can perform various appropriate actions and processes according to a program stored in a read-only memory 202 or a program loaded from a storage section 208 into a random access memory 203, such as performing the methods described in the above embodiments. The random access memory 203 also stores various programs and data required for system operation. The central processing unit 201, the read-only memory 202, and the random access memory 203 are interconnected via a bus 204. An input / output interface 205 is also connected to the bus 204.

[0085] The following components are connected to the input / output interface 205: an input section 206 including audio input devices, push-button switches, etc.; an output section 207 including an LCD display, audio output devices, indicator lights, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 209 performs communication processing via a network such as the Internet. A drive 210 is also connected to the input / output interface 205 as needed. A removable medium 211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 210 as needed so that computer programs read from it can be installed into the storage section 208 as needed.

[0086] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 209, and / or installed from removable medium 211. When the computer program is executed by central processing unit 201, it performs the various functions defined in the present invention.

[0087] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, flash memory, optical fiber, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0089] Specifically, a medical image generation system based on a diffusion model in this embodiment includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, it implements a medical image generation method based on a diffusion model provided in the above embodiment.

[0090] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in a diffusion-based medical image generation system described in the above embodiments; or it may exist independently and not incorporated into the diffusion-based medical image generation system. The storage medium carries one or more computer programs that, when executed by a processor of the diffusion-based medical image generation system, cause the diffusion-based medical image generation system to implement the diffusion-based medical image generation method provided in the above embodiments.

Claims

1. A medical image generation method based on a diffusion model, characterized in that, The method includes: Acquire initial latent variables for image generation and a dynamic optical surface image sequence of the target object, wherein the initial latent variables are sampled from a preset prior distribution; Spatiotemporal feature extraction is performed on the dynamic optical body surface image sequence to obtain the body surface motion feature vector; Based on the pre-trained Schrödinger bridge diffusion model, the latent variables corresponding to each time step are iteratively denoised. The denoising process for each time step includes the following steps: Wavelet decomposition is performed on the latent variable corresponding to the current time step to obtain low-frequency components and high-frequency components, wherein the latent variable corresponding to the first time step is the initial latent variable; Obtain the temporal embedding vector of the current time step, and encode the body surface motion feature vector into a conditional embedding vector; Based on the time embedding vector and the conditional embedding vector, the low-frequency component and the high-frequency component are respectively subjected to conditional fusion processing to obtain the target low-frequency component and the target high-frequency component. Perform inverse wavelet transform on the target low-frequency component and the target high-frequency component to obtain the latent variable corresponding to the next time step of the current time step; The latent variable corresponding to the iterative denoising process that meets the preset evolution termination condition is determined as the target latent variable. The target latent variable is input into the pre-trained image decoder to obtain the three-dimensional CT image of the target object.

2. The method according to claim 1, characterized in that, The step of extracting spatiotemporal features from the dynamic optical body surface image sequence to obtain a body surface motion feature vector specifically includes: Spatial features are extracted from each phase image in the dynamic optical body surface image sequence to obtain the spatial dimension features corresponding to each phase image; The spatial dimension features corresponding to each of the time-phase images are sequentially input into a preset long short-term memory network for temporal feature extraction to obtain the body surface motion feature vector.

3. The method according to claim 2, characterized in that, The step of extracting spatial features from each temporal image in the dynamic optical surface image sequence to obtain the spatial dimension features corresponding to each temporal image specifically includes: Each of the aforementioned temporal images undergoes initial convolution processing to obtain an initial spatial feature map, which includes multiple feature nodes; Traverse each feature node in the initial spatial feature map and define a local spatial neighborhood centered on the current feature node; Extract the target feature value of the current feature node and the reference feature values ​​of adjacent feature nodes in the local spatial neighborhood; Calculate the spatial feature difference between the target feature value and the reference feature value, and determine the local anomaly confidence of the current feature node based on the spatial feature difference, thereby obtaining the local anomaly confidence of each feature node; Based on the local anomaly confidence level corresponding to each feature node, a feature mask weight matrix matching the size of the initial spatial feature map is generated. The initial spatial feature map is multiplied by the feature mask weight matrix to obtain the corrected feature map. Spatial pooling is performed on the corrected feature map to obtain the spatial dimension features corresponding to each of the temporal images.

4. The method according to claim 1, characterized in that, The step of performing conditional fusion processing on the low-frequency component and the high-frequency component based on the time embedding vector and the conditional embedding vector to obtain the target low-frequency component and the target high-frequency component specifically includes: The time embedding vector and the conditional embedding vector are jointly mapped using a first mapping function, and the mapping result is added to and fused with the low-frequency component to obtain the target low-frequency component. The calculation formula is as follows: ,in, For the target low-frequency component, A t For the low-frequency component, φ A The first mapping function; The time embedding vector and the conditional embedding vector are jointly mapped using a second mapping function, and the mapping result is added to and fused with the high-frequency component to obtain the target high-frequency component. The calculation formula is as follows: ,in, For the target high-frequency component, D t For the high-frequency component, φ D Let e ​​be the second mapping function. t Let e ​​be the time embedding vector. z The conditional embedding vector is denoted as .

5. The method according to claim 1, characterized in that, Prior to the step of obtaining the initial latent variables for image generation and the dynamic optical surface image sequence of the target object, the method further includes: Obtain a training sample set, which includes multiple sets of training samples, each set of training samples including a dynamic optical surface image sequence and a three-dimensional CT image of the sample. The initial latent variables of the samples, which follow a preset noise distribution, and the sequence of dynamic optical surface images of the samples are input into the initial Schrödinger bridge diffusion model to be trained to obtain the predicted three-dimensional CT image. Based on the predicted 3D CT image and the sample 3D CT image, the Schrödinger bridge loss, regularization loss and adversarial loss are calculated respectively. Based on the Schrödinger bridge loss, the regularization loss, and the adversarial loss, the network parameters of the initial Schrödinger bridge diffusion model are updated until the network parameters meet the preset convergence condition, thus obtaining the pre-trained Schrödinger bridge diffusion model.

6. The method according to claim 2, characterized in that, Before the step of sequentially inputting the spatial dimension features corresponding to each of the temporal images into a preset long short-term memory network for temporal feature extraction according to the time series, the method further includes: According to the time series, the temporal feature change rate between the spatial dimension features corresponding to the temporal images in the target temporal image group is calculated. The target temporal image group consists of adjacent first temporal images and second temporal images, and the temporal sequence of the first temporal image precedes that of the second temporal image. If the change rate of the temporal feature is greater than the preset motion mutation threshold, then the spatial dimension feature corresponding to the second temporal image is determined to be a transient abnormal feature, and a corresponding temporal blocking identifier is generated for the transient abnormal feature; If the change rate of the temporal feature is less than or equal to the preset motion mutation threshold, then the spatial dimension feature corresponding to the second temporal image is determined to be a smooth motion feature.

7. The method according to claim 2, characterized in that, The step of sequentially inputting the spatial dimension features corresponding to each of the temporal images into a preset long short-term memory network for temporal feature extraction to obtain the body surface motion feature vector specifically includes: During the process of sequentially inputting the spatial dimension features corresponding to each of the aforementioned temporal images into the preset long short-term memory network, if the currently input spatial dimension feature carries a temporal blocking identifier, a dynamic state compensation mechanism is triggered. The dynamic state compensation mechanism includes: decaying the input gate activation value of the preset long short-term memory network for the current time step corresponding to the currently input spatial dimension feature to a preset lower limit value, and extracting the hidden layer memory state of the preset long short-term memory network at the previous time step corresponding to the previously input spatial dimension feature. Based on the hidden layer memory state of the previous time step and the preset temporal decay coefficient, the hidden layer memory state of the current time step is reconstructed through autoregression to obtain the reconstructed memory state. Based on the reconstructed memory state, the temporal feature extraction of subsequent phase images continues until the spatial dimension features corresponding to each phase image have been extracted in a temporal manner, thus obtaining the body surface motion feature vector.

8. A medical image generation system based on a diffusion model, characterized in that, The diffusion-based medical image generation system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the diffusion-based medical image generation system to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on a diffusion-based medical image generation system, the diffusion-based medical image generation system performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on a diffusion-based medical image generation system, the diffusion-based medical image generation system performs the method as described in any one of claims 1-7.

Images