Image registration method and system for 4d respiratory movement synthesis

Abstract

To provide an image registration method and system for 4D respiratory movement synthesis which can effectively generate and acquire a sequence image having movement continuity between two respiration states and is very useful for research of lung part image registration or research of a disease during other respiratory processes.SOLUTION: A registration method includes a process for learning space deformation information between a moving image and a fixed image and generating an intermediate state, a process for generating a velocity field on the basis of the intermediate state and the moving image and generating some deformation fields after solving the velocity field, and a process for distorting the moving image in each deformation field by different degrees and generating a time frame image of a continuous trace in order to realize a purpose for 4D respiratory movement synthesis.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the technical field of image registration, and in particular to an image registration method for 4D respiratory motion synthesis, an image registration apparatus for 4D respiratory motion synthesis, a computer device, a computer-readable storage medium and a computer program product. [Background technology]

[0002] The discussion in this section merely provides background information related to the present invention and does not necessarily constitute prior art.

[0003] In the field of lung registration, the registration task faces significant challenges due to the complex, nonlinear, and large deformation caused by respiratory motion. To address this issue, 4D CT sequence images provide abundant temporal information, and the registration process is guided by this prior information. This approach effectively solves the respiratory registration problem. 4D CT combines traditional CT scanning technology with time sequence information to provide three-dimensional image data of dynamic changes in lung organs or tissues over time. However, 4D CT is difficult to acquire due to issues such as the high cost of image acquisition and the potential harm of radiation to the human body. Medical image generation technology can solve the data shortage problem and better support clinical diagnosis and treatment.

[0004] Research into medical image generation based on deep learning techniques has made significant progress. One of the currently most widely used methods is based on generative adversarial networks (GANs). GAN models can automatically learn features from image data through a training process and generate new images that resemble the original. However, GAN models may produce artificial features that contradict anatomical structures. Another method of image generation is based on deformable image registration, which generates new images by distorting moving images using a generated smooth deformation field. Research on this method has mostly focused on generating three-dimensional images. Diffusion models have been proposed for application in the field of medical image generation. In this method, a diffusion module estimates intermediate states and then feeds several intermediate states obtained by interpolation into a deformation module to generate time frames along a continuous trajectory. However, simply generating successive time frames using intermediate states obtained by simple interpolation does not ensure structural motion continuity in the generated images. While the above methods have solved the image generation problem to some extent, they also have certain limitations. GAN models generate artificial features that are not medically desirable, research on methods based on deformable image registration has mostly focused on 3D images, and methods based on diffusion models lack time-continuous interpretability due to simple interpolation of intermediate states. Summary of the Invention

[0005] To solve the shortcomings of the prior art, the present invention provides an image registration method and system for 4D respiratory motion synthesis, which can effectively generate and acquire sequence images with motion continuity between two respiratory states, which is very useful for studying lung image registration or other diseases during respiratory process.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides an image registration method for 4D respiratory motion synthesis.

[0008] Image registration methods for 4D respiratory motion synthesis include: A process of learning spatial transformation information between the moving image and the fixed image and generating an intermediate state; A process of generating a velocity field based on the intermediate state and the moving image, solving the velocity field, and then generating several deformation fields; For 4D respiratory motion synthesis between moving and fixed images (the moving and fixed images form an image pair), the process includes distorting the moving images to different degrees with each deformation field to generate time-frame images of a continuous trajectory.

[0009] As a further limitation of the first aspect of the present invention, the process of generating a velocity field based on the intermediate state and the moving image includes: It involves a process of estimating the velocity field from the moving image to the fixed image based on the intermediate states.

[0010] As a further limitation of the first aspect of the present invention, the process of learning spatial transformation information between the moving image and the fixed image and generating an intermediate state includes: The process involves adding deterministic random Gaussian noise to a fixed image to control interference removal in the backward diffusion process, learning a Markov transformation from the Gaussian noise to a data distribution, and generating an intermediate state with spatial transformation information from the moving image to the fixed image.

[0011] As a further limitation of the first aspect of the present invention, a fully convolutional neural network is utilized to generate the velocity field based on the intermediate states and moving images.

[0012] As a further limitation of the first aspect of the present invention, the velocity field is solved by the method of Euler integration before generating several deformation fields.

[0013] As a further limitation of the first aspect of the present invention, the process of distorting the moving image to a different extent with each deformation field to generate time frame images of the continuous trajectory comprises: This involves sending the deformation field and the movement image to a spatial transformation network to obtain a sequence of images with motion continuity.

[0014] As a further limitation of the first aspect of the present invention, the total loss function is the sum of the diffusion loss, the similarity measure between the moving and fixed images, the smoothness loss and the anti-folding loss.

[0015] In a second aspect, the present invention provides an image registration apparatus for 4D respiratory motion synthesis.

[0016] Image registration system for 4D respiratory motion synthesis an intermediate state generation unit configured to learn spatial transformation information between the moving image and the fixed image and generate an intermediate state; a deformation field generation unit configured to generate a velocity field based on the intermediate state and the moving image, and to generate several deformation fields after solving the velocity field; and a continuous image generation unit configured to distort the moving image to different degrees with each deformation field to generate continuous trajectory time-frame images for 4D respiratory motion synthesis between the moving image and the fixed image.

[0017] In a third aspect, the present invention provides a method for producing a method of manufacturing a semiconductor device comprising: a processor adapted to execute a computer program; and a computer-readable storage medium having a computer program stored therein, the computer program, when executed by the processor, realizing the image registration method for 4D respiratory motion synthesis according to the first aspect of the present invention.

[0018] In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon a computer program, said computer program being suitable for being loaded by a processor to perform the image registration method for 4D respiratory motion synthesis according to the first aspect of the invention.

[0019] In a fifth aspect, the present invention provides a computer program product comprising a computer program which, when executed by a processor, implements the image registration method for 4D respiratory motion synthesis according to the first aspect of the present invention.

[0020] Compared with the prior art, the present invention has the following advantageous effects:

[0021] This invention innovatively proposes an image registration method for 4D respiratory motion synthesis, in which a diffusion module learns spatial deformation information between moving and fixed images and generates intermediate states. The intermediate states are fed into a deformation module to generate a flow field together with the moving image. The flow field is then solved using the Euler integral method, and the generated deformation fields can distort the moving image to different degrees to generate continuous time frames of trajectories. This solution can effectively generate intermediate time images between two respiratory states, which is very useful for studying lung image registration or other respiratory diseases. The model proposed in this invention has wide applicability and can provide new tools and techniques in the field of medical image processing.

[0022] The present invention can effectively ensure the model accuracy and improve the registration accuracy by using the sum of the diffusion loss, the similarity measurement between the moving image and the fixed image, the smoothness loss, and the anti-folding loss as the total loss function.

[0023] Advantages of additional aspects of the invention will be set forth in part in the description that follows, and in part will be obvious from the description, or may be learned by practice of the invention.

[0024] The drawings in the specification that form a part of this invention are intended to facilitate a better understanding of the invention, and the illustrative examples of the invention and their descriptions are intended to help interpret the invention and are not intended to constitute an unduly limiting view of the invention. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a schematic flowchart of an image registration method for 4D respiratory motion synthesis provided in Example 1 of the present invention; [Figure 2] FIG. 1 is a principle diagram of the image registration method for 4D respiratory motion synthesis provided in Example 1 of the present invention. [Figure 3] FIG. 1 is a structural schematic diagram of a fully convolutional network model provided in Example 1 of the present invention. [Figure 4] FIG. 1 is a schematic diagram of an image registration device for 4D respiratory motion synthesis provided in Example 2 of the present invention. [Figure 5] 1 is a structural schematic diagram of an electronic device provided in Example 3 of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0026] In the following the invention will be further explained with reference to the figures and examples.

[0027] It should be pointed out that the following detailed description is all exemplary and is intended to further explain the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art.

[0028] Unless inconsistent, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0029] Example 1: In this embodiment, a diffeomorphic image registration method for 4D respiratory motion synthesis is proposed. Below, we first briefly introduce the technical terms and related concepts of this processing scheme. Medical image registration refers to the process of performing a spatial transformation or series of spatial transformations on one medical image to spatially match corresponding points on another medical image. Such a match means that the same anatomical points on the human body have the same spatial locations on the two matching images. The result of the registration should be such that all anatomical points on the two images, or at least all points of diagnostic significance and surgical interest, match.

[0030] Denoising diffusion probabilistic models (DDPMs) are parameterized Markov chains trained using variational inference. Denoising diffusion probabilistic models (hereafter referred to as diffusion models) are a type of deep generative model that typically includes two processes: a forward diffusion process and a reverse dediffusion process. The forward and reverse Markov chains are configured with a finite number of time steps. The forward diffusion process is a parameter-free Markov chain, while the reverse dediffusion process requires a learning algorithm to train the model.

[0031] Random Gaussian noise, also known as white noise or random noise, is a random signal or interference that follows a Gaussian (normal) distribution, characterized by a constant power spectral density at all frequencies, exhibiting random fluctuations of equal energy at different frequencies. From a practical perspective, Gaussian noise is a random variation or perturbation that occurs in various systems and processes. It is present in many natural phenomena, such as atmospheric interference, thermal noise in electronic circuits, and even background noise in communication channels. Gaussian noise can also be artificially added to signals or data for various purposes, such as testing or simulating real-world environmental conditions. Graphically, Gaussian noise appears as a random pattern of values, centered around a mean or average, with fewer extreme values. Such a distribution is mathematically represented by a Gaussian probability density function.

[0032] Fully Convolutional Networks (FCNs) are a framework for image semantic segmentation proposed by Jonathan Long et al. in their 2015 paper "Fully Convolutional Networks for Semantic Segmentation." They represent pioneering research in the application of deep learning to semantic segmentation. FCNs replace the fully connected layers behind traditional CNNs with convolutional layers, resulting in a heat map output instead of categories. To address the image size reduction problem caused by convolution and pooling, FCNs use an upsampling method to restore the image size. The FCN network structure is primarily divided into two parts: a fully convolutional part and a deconvolutional part. The fully convolutional part is similar to some classic CNN networks (e.g., VGG, ResNet, etc.) used for feature extraction. The deconvolutional part obtains the original size of the semantic segmented image through upsampling. The input of an FCN can be a color image of any size, and the output is the same size as the input.

[0033] Spatial Transformer Networks (STN networks) are neural network modules that can be inserted into any existing convolutional neural network (CNN) to enhance the model's tolerance to geometric transformation perturbations. The main function of STN is to perform spatial transformations within the network, i.e., to automatically perform desired geometric transformations such as scaling, shearing, and rotation on the data as it passes through the network, allowing the network to better adapt to the geometry of the input data.

[0034] The Euler integral focuses on the physical quantities of fluid particles flowing at each spatial point in a flow field, and is related to both time and spatial position, and can ultimately obtain the flow state of the entire flow field through calculation.

[0035] The ReLU function is a common activation function. Its calculation is generally performed after convolution. Therefore, it belongs to the nonlinear activation functions, similar to the tanh function and sigmoid function. The inverse of the ReLU function is always equal to 1 in the positive part. Therefore, the use of the ReLU function in deep networks will not lead to the problems of vanishing or exploding gradients. In addition, the ReLU function has a fast calculation speed, which speeds up network training.

[0036] Trilinear interpolation is a method for linear interpolation of three-dimensional discretely sampled data. This method not only considers two-dimensional interpolation problems, but also adds a third dimension (z-axis), meaning that the interpolation must consider not only the x- and y-direction sample points but also the z-direction sample points. The results of trilinear interpolation are independent of the order of the interpolation calculations; the final result is the same regardless of the order of the dimensions. Furthermore, trilinear interpolation can be considered as linear B-spline interpolation of three-dimensional tensors. It has a wide range of applications in fields such as numerical analysis, data analysis, and computer graphics, including edge detection and Gaussian filtering in image processing, and gradient histogram correction in feature extraction such as HOG feature extraction.

[0037] Downsampling, also known as decimation in the field of digital signal processing, is a technique in multirate digital signal processing or the process of reducing a signal sampling rate, usually to reduce data transmission speed or data size, and is complementary to interpolation, which increases the sampling frequency.

[0038] Specifically, the image registration method for 4D respiratory motion synthesis proposed in this invention connects two lung images of extreme respiratory states by generating several intermediate time frames. This model is composed of a diffusion module and a deformation module. The diffusion module learns spatial deformation information between the moving image and the fixed image and generates an intermediate state. The intermediate state is fed into the deformation module, which generates a flow field together with the moving image. The flow field is solved using the Euler integral method, and the generated several deformation fields distort the moving image to different degrees to generate a continuous trajectory of time frames. This method can effectively generate intermediate time frames between two respiratory states, which is very useful for studying lung image registration or other respiratory diseases. The model proposed in this invention has wide applicability and can provide new tools and techniques in the field of medical image processing.

[0039] Specifically, as shown in Figure 1, it includes the following processes: S1: Learn the spatial transformation information between the moving image and the fixed image and generate an intermediate state. S2: Generate a velocity field based on the intermediate state and moving images, solve the velocity field, and then generate several deformation fields. S3: To achieve registration between the moving and fixed images, the moving image is distorted to different degrees in each deformation field to generate time-frame images of the continuous trajectory.

[0040] As shown in Figure 2, M represents the moving image, F represents the fixed image, and x t represents the fixed image with deterministic perturbation, and is sent to the diffusion module. After that, an intermediate state z and a moving image M are generated and sent to the deformation module. The deformation module estimates the velocity field from the moving image to the fixed image based on the intermediate state z. The deformation time from the moving image to the fixed image is t. To simplify the training process and calculation, t is set to unit 1, and the total time is divided into several time steps. The subsequent several deformation fields can be solved by a single time step Euler integral (i.e., the method marked with a circle C in Figure 2), and are represented by I1, I2, I3, I wis the generated image.

[0041] In S1, the specific process is realized by the diffusion module, and includes: The diffusion module is realized based on the denoising diffusion probability model, which combines the concepts of diffusion process and probability modeling. Its basic principle is to carry out a process called forward diffusion process, which introduces random Gaussian noise into the original image, and then learn to remove noise from the image with added noise through the backward diffusion process. Let the original data be x0~q(x), and continue to add perturbation noise. The distribution of x at time t can be expressed by formula (1-1).

number

number

[0042] As the time step t approaches a sufficiently large T, x T The distribution of x0 is derived from the distribution of x0, and the backward noise removal process p(x0|x T ) cannot be calculated directly from a formula, so we use a neural network model to learn the parameterized Gaussian process, define the Gaussian process as a trainable Markov chain, and use the neural network to solve the distribution of x0 until we solve the distribution of x0, as shown in formula (1-3). t Based on the distribution of x t-1To summarize the above, the backward denoising process of DDPM learns the distribution of p θ (x t-1 |x t ) where θ is a parameter of the network model.

number

number

[0043] In S2, the specific process is realized by the transformation module, which includes: After obtaining the intermediate state z, it is sent to a deformation module along with the translation image. This module is essentially a combination of a fully convolutional neural network (FCN), an ordinary differential equation solver, and a differentiable spatial transformation network, similar to a U-net network structure. The FCN network structure is similar to the U-Net network structure, as shown in Figure 3. It consists of a three-layer encoder / decoder. The first layer of downsampling uses a single convolutional layer with a kernel size of 3x3x3. After initial feature mapping of the input, it uses a ReLU activation function to calculate and downsample the feature map to half its original size through trilinear interpolation until it reaches the bottom layer. The bottom layer contains two convolutional layers with a kernel size of 3x3x3. After passing through the bottom layer, the feature map is sent to the decoder. Each layer of the decoder is connected to the feature map of the encoder by skip connection. Each layer of the decoder has two consecutive convolutional layers with a kernel size of 3×3×3, and upsampling by trilinear interpolation to double the size of the original. Finally, it passes through one convolutional layer with a kernel size of 5×5×5 to convert the velocity field V from the moving image to the fixed image. MF Generate.

[0044] In the model proposed in this invention, the velocity field V MF Assuming that (ν) is a static velocity field and the total displacement vector field from the moving image to the fixed image is φ, the differential equation shown in formula (1-5) holds.

number

number

number

[0045] The proposed model is optimized according to the following loss function, which includes the diffusion loss of the diffusion model, the similarity function between the registered image and the fixed image, the smoothness constraint of the deformation field, and the anti-folding constraint:

number

[0046]

number

number

[0047] Example 2: As shown in FIG. 4, the second embodiment of the present invention is as follows: an intermediate state generation unit configured to learn spatial transformation information between the moving image and the fixed image and generate an intermediate state; a deformation field generation unit configured to generate a velocity field based on the intermediate state and the moving image, and to generate several deformation fields after solving the velocity field; and a continuous image generation unit configured to distort the moving image to different degrees in each deformation field and generate time-frame images of a continuous trajectory to achieve registration between the moving image and the fixed image.

[0048] The intermediate state generation unit is more specifically realized by a diffusion module, and includes: The basic principle of the diffusion module is to perform a process called forward diffusion, which introduces random Gaussian noise into the original image, and then learn to remove noise from the noisy image using the backward diffusion process. Let the original data be x0~q(x), and continue to add perturbation noise. The distribution of x at time t can be expressed by formula (2-1).

number

number

[0049] As the time step t approaches a sufficiently large T, x T The distribution of x0 is derived from the distribution of x0, and the backward noise removal process p(x0|x T) cannot be calculated directly from a formula, so we use a neural network model to learn the parameterized Gaussian process, define the Gaussian process as a trainable Markov chain, and use the neural network to solve the distribution of x0 until we solve the distribution of x0, as shown in formula (2-3). t Based on the distribution of x t-1 To summarize the above, the backward denoising process of DDPM learns the distribution of p θ (x t-1 |x t ) where θ is a parameter of the network model.

number

number

[0050] The specific process of the deformation field generation unit is realized by the deformation module, and includes the following: After obtaining the intermediate state z, it is fed into a deformation module together with the translation image, which is essentially a combination of a fully convolutional neural network (FCN) similar to a U-shaped network structure, an ordinary differential equation solver, and a differentiable spatial transformation network. In the model proposed in this invention, the velocity field V MF Assuming that (ν) is a static velocity field and the total displacement vector field from the moving image to the fixed image is φ, the differential equation shown in formula (2-5) holds.

number

[0051] The continuous image generation unit specifically includes the following: To generate a time frame between two extreme images, the large deformation from the moving image to the fixed image is divided into several gradual small deformations, and the small deformations are set to an equal single time step size in the total time, and four time step sizes are set, and formula (2-6) can be obtained based on the Euler integration method.

number

number

[0052] In this embodiment, the whole process is preferably realized by the overall innovative proposal of the network model, and the network model proposed in the present invention is optimized according to the following loss function, including the diffusion loss of the diffusion model and the similarity function between the registered image and the fixed image, the smoothness constraint and the anti-folding constraint of the deformation field.

number

[0053]

number

number

[0054] It should be understood that each of the above units may be configured as one or several other units, either separately or as a whole, or one (or several) of the units may be further divided into several smaller units, thereby achieving the same operation without affecting the realization of the technical effect of the embodiments of the present application. The above units are divided based on logical functions, and in actual applications, the function of one unit may be realized by several units, or the functions of several units may be realized by one unit. In other embodiments of the present application, the image processing device may include other units, and in actual applications, these functions may be realized with the assistance of other units, or may be realized by several units working together.

[0055] According to another embodiment of the present application, the system described in this embodiment and the image processing method of the embodiment of the present application can be configured and realized by executing a computer program (including program code) capable of executing each step of the corresponding method described in embodiment 1 on a general-purpose computing device such as a computer including processing elements and memory elements such as a central processing unit (CPU), random access memory (RAM), and read only memory (ROM). The computer program can be recorded, for example, on a computer-readable recording medium and can be loaded into and executed by the above-mentioned computing device via the computer-readable recording medium.

[0056] Example 3: 5, a fourth embodiment of the present invention provides an electronic device including a processor 1001, a communication interface 1002, and a computer-readable storage medium 1003. Here, the processor 1001, the communication interface 1002, and the computer-readable storage medium 1003 may be connected by a bus or other means.

[0057] Here, the communication interface 1002 is used for transmitting and receiving data, the computer-readable storage medium 1003 may be stored in a memory of the electronic device, the computer-readable storage medium 1003 is for storing a computer program, the computer program includes program commands, and the processor 1001 is for executing the program commands stored in the computer-readable storage medium 1003.

[0058] The processor 1001 (also referred to as CPU (Central Processing Unit)) is the computing core and control core of the electronic device, and is suitable for implementing one or more commands, specifically, for implementing the process or corresponding function of the method described in Example 1 by loading and executing one or more commands.

[0059] Example 4: This embodiment provides a computer-readable storage medium (Memory), which is a storage device in an electronic device for storing programs and data. It can be understood that the computer-readable storage medium here may include a storage medium built into the electronic device, and may also include an expansion storage medium supported by the electronic device. The computer-readable storage medium provides a storage space in which a processing system of the electronic device is stored.

[0060] The storage space may further store one or more instructions suitable for being loaded and executed by the processor, which may be one or more computer programs (including program code). The computer-readable storage medium may be a high-speed RAM memory, a non-volatile memory such as at least one disk memory, or optionally at least one computer-readable storage medium located remotely from the processor.

[0061] In one embodiment, the computer-readable storage medium stores one or more commands, and the processor loads and executes the one or more commands stored in the computer-readable storage medium to achieve the corresponding steps in the method embodiment described in Example 1 above.

[0062] Example 5: This embodiment provides a computer program product or a computer program, the computer program product or the computer program including computer instructions stored in a computer-readable storage medium, a processor of an electronic device reading the computer instructions from the computer-readable storage medium and executing the computer instructions, thereby causing the electronic device to perform the method of embodiment 1.

[0063] Those skilled in the art will recognize that each example unit and algorithm step described with reference to the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. A skilled engineer may implement the described functions in different ways for each specific application, and such implementation should not be considered outside the scope of the present application.

[0064] The above embodiments may be implemented, in whole or in part, in software, hardware, firmware, or any combination thereof. When implemented in software, they may be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer commands. When loaded and executed on a computer, the computer program commands produce, in whole or in part, processes or functions according to the embodiments of the present disclosure. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer commands may be stored on or transmitted via a computer-readable storage medium. The computer commands may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, radio, microwave, etc.) methods. The computer-readable storage medium may be any available medium accessible by a computer or a data processing device, including a server, data center, etc., integrated with one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)).

[0065] The above is merely a preferred embodiment of the present invention, and is not intended to limit the present invention. Those skilled in the art can make various modifications and variations to the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall all be included in the protection scope of the present invention.

Claims

1. A process of learning spatial deformation information between a moving image and a fixed image and generating an intermediate state, the specific process being realized by a diffusion module, including: The diffusion module is implemented based on the denoising diffusion probability model, and converts the original data into x ０ ∼q(x), and by continuing to add perturbation noise, the distribution of x at time t can be expressed by formula (1-1): [Equation 21] However, β ｔ are predefined hyperparameters that determine the variance and mean of the Gaussian noise added in the forward diffusion process, and q(x ｔ |x ｔ－１ ) is the forward diffusion process to extract the original data x ０ is a distribution obtained by adding Gaussian noise to x at any one time step. ｔ The distribution of is obtained by DDPM as shown in formula (1-2), [Equation 22] ε ０ is a known Gaussian distribution following (0,I), i.e., a perturbation of the forward diffusion process, The backward noise removal process of DDPM is p θ (x ｔ－１ |x ｔ ) where θ is a parameter of the network model, [Equation 23] However, μ θ (x ｔ , t) is the mean learned by the neural network model, [0000] is a fixed variance, and the known x ｔ A process in which the diffusion module learns by comparing the moving image and the fixed image and obtains an intermediate state z containing spatial deformation information of the mapping from the moving image to the fixed image; A process of generating a velocity field according to the intermediate state and the moving image, solving the velocity field by the Euler integration method, and then generating several deformation fields, the specific process is realized by a deformation module, and includes: After obtaining the intermediate state z, it is sent to a deformation module together with the moving image. This module is essentially a combination of a fully convolutional neural network, an ordinary differential equation solver, and a differentiable spatial transformation network, which is essentially similar to a U-net network structure. The fully convolutional neural network structure is similar to a U-net network structure. The network is composed of a three-layer encoder / decoder. The first layer of downsampling uses one convolutional layer with a kernel size of 3x3x3. After initial feature mapping of the input, it is calculated using the ReLU activation function. The feature map is downsampled to half its original size through trilinear interpolation until it reaches the bottom layer, which includes two convolutional layers with a kernel size of 3x3x3. After passing through the bottom layer, the feature map is sent to the decoder. Each layer of the decoder is connected to the feature map of the encoder by skip connection. Each layer of the decoder has two consecutive convolutional layers with a kernel size of 3x3x3, and upsampling by trilinear interpolation to double the size of the original. Finally, it passes through one convolutional layer with a kernel size of 5x5x5 to generate the velocity field V from the moving image to the fixed image. ＭＦ Generate The deformation time from the moving image to the fixed image is t, and the total time is divided into several time steps, and the subsequent several deformation fields are solved by a single time step Euler integration method; and a process of distorting the moving image to different degrees with each deformation field to generate continuous trajectory time-frame images for 4D respiratory motion synthesis between the moving image and the fixed image.

1. A method for image registration for 4D respiratory motion synthesis, comprising:

2. The process of generating a velocity field based on intermediate states and moving images is It includes a process for estimating the velocity field from the moving image to the fixed image based on the intermediate state. The image registration method for 4D respiratory motion synthesis according to claim 1 .

3. The process of learning spatial transformation information between moving and fixed images and generating intermediate states is as follows: This involves adding deterministic random Gaussian noise to a fixed image to control interference removal in the backward diffusion process, learning a Markov transformation from the Gaussian noise to a data distribution, and generating an intermediate state with spatial transformation information from the moving image to the fixed image. The image registration method for 4D respiratory motion synthesis according to claim 1 .

4. Use a fully convolutional neural network to generate a velocity field based on intermediate states and moving images, or solve the velocity field using the Euler integral method and then generate several deformation fields. The image registration method for 4D respiratory motion synthesis according to claim 1 .

5. The process of distorting the moving image to different degrees in each deformation field and generating time-frame images of the continuous trajectory is This involves sending the deformation field and the translation image to a spatial transformation network to obtain a sequence of images with motion continuity. The image registration method for 4D respiratory motion synthesis according to claim 1 .

6. The total loss function is the sum of the diffusion loss, the similarity measure between the moving image and the fixed image, the smoothness loss, and the anti-folding loss. The image registration method for 4D respiratory motion synthesis according to claim 1 .

7. An intermediate state generation unit configured to learn spatial deformation information between a moving image and a fixed image and generate an intermediate state, the specific process being realized by a diffusion module, and including: The diffusion module is realized based on the denoising diffusion probability model, which converts the original data into ０ ∼q(x), and by continuing to add perturbation noise, the distribution of x at time t can be expressed by formula (2-1): [Equation 25] However, β ｔ are predefined hyperparameters that determine the variance and mean of the Gaussian noise added in the forward diffusion process, and q(x ｔ |x ｔ－１ ) is the forward diffusion process to extract the original data x ０ is a distribution obtained by adding Gaussian noise to x at any one time step. ｔ The distribution of is obtained by DDPM as shown in formula (2-2), [Equation 26] ε ０ is a known Gaussian distribution following (0,I), i.e., a perturbation of the forward diffusion process, The backward noise removal process of DDPM is p θ (x ｔ－１ |x ｔ ) where θ is a parameter of the network model, [0000] However, μ θ (x ｔ , t) is the mean learned by the neural network model, [0000] is a fixed variance, and the known x ｔ an intermediate state generation unit that compares the moving image and the fixed image by using the above and the diffusion module learning to obtain an intermediate state z that includes spatial deformation information of the mapping from the moving image to the fixed image; A deformation field generation unit configured to generate a velocity field according to an intermediate state and a moving image, and generate several deformation fields after solving the velocity field, the specific process being realized by a deformation module, including: After obtaining the intermediate state z, it is sent to a deformation module together with the moving image. This module is essentially a combination of a fully convolutional neural network, an ordinary differential equation solver, and a differentiable spatial transformation network, which is essentially similar to a U-net network structure. The fully convolutional neural network structure is similar to a U-net network structure. The network is composed of a three-layer encoder / decoder. The first layer of downsampling uses one convolutional layer with a kernel size of 3x3x3. After initial feature mapping of the input, it is calculated using the ReLU activation function. The feature map is downsampled to half its original size through trilinear interpolation until it reaches the bottom layer, which includes two convolutional layers with a kernel size of 3x3x3. After passing through the bottom layer, the feature map is sent to the decoder. Each layer of the decoder is connected to the feature map of the encoder by skip connection. Each layer of the decoder has two consecutive convolutional layers with a kernel size of 3x3x3, and upsampling by trilinear interpolation to double the size of the original. Finally, it passes through one convolutional layer with a kernel size of 5x5x5 to generate the velocity field V from the moving image to the fixed image. ＭＦ Generate A deformation field generation unit that divides the total time into several time steps, where the deformation time from the moving image to the fixed image is t, and solves the subsequent several deformation fields using a single time step Euler integration method; a sequential image generation unit configured to distort the moving image to different degrees with each deformation field to generate sequential trajectory time-frame images for 4D respiratory motion synthesis between the moving image and the fixed image; 1. An image registration apparatus for 4D respiratory motion synthesis, comprising:

8. a processor adapted to execute a computer program; a computer-readable storage medium having a computer program stored therein, the computer program implementing the image registration method for 4D respiratory motion synthesis according to any one of claims 1 to 6 when executed by the processor; and 1. A computer device comprising:

9. 10. A computer-readable storage medium having stored thereon a computer program, the computer program being suitable for being loaded by a processor to perform the image registration method for 4D respiratory motion synthesis according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, the computer program being executed by a processor to implement the image registration method for 4D respiratory motion synthesis according to any one of claims 1 to 6.

Images