Method and system for generating lung dr images to ct respiratory motion images

Abstract

The present application relates to the technical field of image generation, in particular to a method and system for generating a lung DR image into a CT respiratory motion image, the method of the present application first acquires DR data and CT data of multiple patients, forms a generalization data set after preprocessing, trains a generative adversarial network to obtain a generalization generative adversarial network; then, using the multi-phase CT data and DR data of a historical respiratory cycle of a target patient before surgical intervention, a specific data set is obtained, the generalization generative adversarial network is trained to obtain a specific generative adversarial network suitable for the target patient, which greatly reduces the generation deviation caused by individual differences and improves the matching degree of the generated image and the real physiological state of the patient; finally, during the interventional surgery and radiotherapy, only the real-time DR data of the lung of the target patient needs to be obtained, and the real-time 4D CT data of the patient can be obtained, so that the lesion positioning and targeted treatment are more timely and accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image generation, in particular to a method and system for generating CT respiratory motion images based on lung DR images. BACKGROUND

[0002] Computed Tomography (CT) is a medical imaging technique that uses X-rays and computers to generate high-resolution images of human tissue structures. Compared with Digital Radiography (DR, also known as X-ray film), CT has higher contrast and clarity, and can clearly display tissue structures and lesions in the form of three-dimensional voxel images. CT is particularly effective in visualizing bone and calcified tissue, and is therefore commonly used to diagnose fractures, bone tumors, and vascular lesions. In addition, low-dose CT is still the gold standard for early lung cancer screening.

[0003] 4D CT, on the other hand, is a multi-phase CT that is continuously taken over a period of time, and can show changes in human tissue structures due to respiratory motion. This feature of 4D CT supports clinicians in surgical planning, radiotherapy design, and precise lesion localization. Despite these advantages, CT also has significant limitations. The ionizing radiation dose generated during CT imaging is much higher than that of digital radiography (by tens to hundreds of times), increasing the risk of cancer. The scanning process itself is time-consuming, limiting its real-time application in surgical guidance and emergency situations. Therefore, developing methods to obtain CT images faster with lower radiation has become an important focus of current research. SUMMARY

[0004] The purpose of the present application is to provide a method and system for generating CT respiratory motion images based on lung DR images.

[0005] The technical solution of the present application is as follows:

[0006] A method for generating CT respiratory motion images based on lung DR images, comprising the following operations:

[0007] S1, obtaining DR data and CT data of multiple patient lungs, and forming a generalized data set after preprocessing, training a generative adversarial network to obtain a generalized generative adversarial network;

[0008] The processing operation in the training of the generative adversarial network is as follows: based on the X-ray image and the corresponding edge feature map, the X-ray shallow information feature, the X-ray middle information feature, and the X-ray deep information feature are obtained; the X-ray deep information feature is spliced with the deep edge feature corresponding to the edge feature map, and a channel enhancement feature is obtained through multi-channel feature fusion processing; the X-ray image is obtained based on the DR data and the CT data;

[0009] After expanding the channel enhancement features into three dimensions, the attention enhancement features are obtained through several non-local attention processing;

[0010] After cross attention processing based on residual connection, the attention enhancement features, the X-ray shallow information features, the X-ray middle information features and the X-ray deep information features are reconstructed into pseudo-CT data;

[0011] When the training loss value of each pseudo-CT data and the corresponding real CT data is less than the loss threshold, the training is completed, and a general generative adversarial network is obtained;

[0012] S2, obtain the multi-phase CT data of the target patient's lung containing one historical respiratory cycle, register the first phase CT data in the multi-phase CT data to other phase CT data to obtain a multi-phase registration displacement field, and then expand the displacement field through sampling to transform other phase CT data to obtain a plurality of CT training data. After preprocessing, the specific data set is formed with the target patient's DR data;

[0013] The general generative adversarial network is trained using the specific data set to obtain a specific generative adversarial network;

[0014] S3, real-time acquisition of the DR data of the target patient's lung, and real-time CT generation data is obtained by processing the specific generative adversarial network.

[0015] In S1, the method for obtaining the X-ray shallow information features, the X-ray middle information features and the X-ray deep information features is as follows: the X-ray image and the corresponding edge feature map are respectively processed by two-dimensional convolution to obtain X-ray convolution map and edge convolution map; the X-ray convolution map and the edge convolution map are spliced and down-sampled, and then subjected to multi-channel feature fusion processing for multiple times to obtain X-ray shallow information features; the edge convolution map is subjected to multi-channel feature fusion processing for multiple times to obtain edge shallow information features; the X-ray shallow information features and the edge shallow information features are spliced and down-sampled, and then subjected to multi-channel feature fusion processing for multiple times to obtain X-ray middle information features; the edge shallow information features are subjected to multi-channel feature fusion processing for multiple times to obtain edge middle information features; the X-ray middle information features and the edge middle information features are spliced and down-sampled, and then subjected to multi-channel feature fusion processing for multiple times to obtain X-ray deep information features.

[0016] The operation of multi-channel feature fusion processing is as follows: after the spliced down-sampled map is subjected to convolution processing in different channels, it is subjected to splicing processing, layer normalization processing, convolution processing and multi-layer perception processing to obtain nonlinear features; the shallow nonlinear features and the spliced down-sampled map are subjected to element-by-element addition to obtain X-ray information features, which are used for performing the next multi-channel feature fusion processing.

[0017] The operation of the non-local attention in S1 is: the channel enhancement three-dimensional feature is subjected to internal normalization processing to obtain an internal normalized feature; the internal normalized feature is subjected to three-dimensional convolution processing of different scales, and then subjected to element-by-element multiplication and non-linear processing to obtain a convolution fusion non-linear feature; the internal normalized feature is subjected to three-dimensional convolution processing, and then subjected to element-by-element multiplication with the convolution fusion non-linear feature to obtain a convolution fusion feature; the convolution fusion feature and the internal normalized feature are subjected to element-by-element addition to obtain an initial attention enhancement feature, which is used to perform the next non-local attention processing.

[0018] The training loss in S1 includes a generator loss and a discriminator loss; the generator loss includes a reconstruction loss of pseudo CT data, a projection loss of the pseudo CT data and corresponding real CT data, a perception loss, and an adaptive motion region loss.

[0019] The projection loss is calculated by the following formula:

[0020] ,

[0021] is the projection loss, 、 、 are projection functions of transverse, sagittal and coronal planes respectively, 、 are real CT data and pseudo CT data respectively, is a norm 1 processing, is an expectation function.

[0022] The operation of the preprocessing in S1 is: resampling the CT data and cutting out the central region to obtain CT key data; aligning the CT key data with the DR data in space to obtain an X-ray image; performing three-linear interpolation processing and bilinear interpolation on the CT data and the X-ray image respectively, and adjusting them to the same size to obtain a data pair; all data pairs form a generalization data set.

[0023] A lung DR image to CT respiratory motion image generation system based on the lung DR image to CT respiratory motion image generation method described above, comprising:

[0024] The generalization generative adversarial network generation module is used for acquiring DR data and CT data of multiple patient lungs, and forms a generalization data set after preprocessing, and trains the generative adversarial network to obtain a generalization generative adversarial network; the processing operation in the generative adversarial network is: based on the X-ray image and the corresponding edge feature map, X-ray shallow information features, X-ray middle information features and X-ray deep information features are acquired; the X-ray deep information features are spliced with deep edge features corresponding to the edge feature map, and channel enhancement features are obtained through multi-channel feature fusion processing; the X-ray image is obtained based on the DR data and the CT data; after the channel enhancement features are expanded into three dimensions, attention enhancement features are obtained through several times of non-local attention processing; after the attention enhancement features, the X-ray shallow information features, the X-ray middle information features and the X-ray deep information features are processed through cross attention based on residual connection, and three-dimensional reconstruction is performed, pseudo CT data are obtained; when the training loss value of each pseudo CT data and the corresponding real CT data is less than a loss threshold value, the training is ended, and the generalization generative adversarial network is obtained;

[0025] The specific generative adversarial network generation module is used for acquiring multiple CT data containing a historical respiratory cycle of the lung of a target patient, registering the first CT data in the multiple CT data into other CT data to obtain a multiple registration displacement field, transforming other CT data using the expanded displacement field after sampling expansion to obtain multiple CT training data, and forming a specific data set after preprocessing with the DR data of the target patient; the generalization generative adversarial network is trained using the specific data set to obtain a specific generative adversarial network.

[0026] The CT data generation module is used for acquiring DR data of the lung of a target patient in real time, and obtaining CT generation data in real time through the specific generative adversarial network.

[0027] A lung DR image to CT respiratory motion image generation device based on the lung DR image to CT respiratory motion image generation method.

[0028] A computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the lung DR image to CT respiratory motion image generation method.

[0029] The present application has the following advantages:

[0030] The application provides a lung DR image to CT respiratory motion image generation method, first, DR data and CT data of multiple patients are acquired, after pretreatment, a generalization data set is formed, a generative adversarial network is trained, and a generalization generative adversarial network is obtained; then, a specific data set is acquired by using multi-phase CT data and DR data of a historical respiratory cycle of a target patient before surgical intervention, the generalization generative adversarial network is trained, and a specific generative adversarial network suitable for the target patient is obtained, the generation deviation caused by individual differences is greatly reduced, and the matching degree of the generated image and the real physiological state of the patient is improved; finally, during the intervention surgery and radiotherapy, only DR data of the lung of the target patient needs to be acquired in real time, and real-time 4D CT data of the patient can be obtained, so that lesion positioning and targeted treatment are more timely and accurate.

[0031] The application provides a lung DR image to CT respiratory motion image generation method, a generative adversarial network is designed, the feature fusion capability of an X-ray image and an edge feature map can be enhanced, high-quality real-time CT images can be acquired through X-ray films while the patient avoids suffering from excessive radiation during the acquisition of the guide medical image, the generation effect is controlled according to the training loss, the structural similarity of the generated CT data and the real CT data is high, and specific details of lung tissue, such as lung blood vessels, lung tuberculosis lesion areas and bronchi, can be presented, the respiratory motion process presented is very close to the real respiratory process, and the method can assist a radiotherapy doctor in accurate lesion positioning and targeted treatment. DETAILED DESCRIPTION

[0032] The embodiment provides a lung DR image to CT respiratory motion image generation method, and the method comprises the following operations:

[0033] S1, DR data and CT data of the lung of multiple patients are acquired, after pretreatment, a generalization data set is formed, a generative adversarial network is trained, and a generalization generative adversarial network is obtained;

[0034] S2, multi-phase CT data containing a historical respiratory cycle of the lung of a target patient are acquired, first-phase CT data in the multi-phase CT data is registered to other-phase CT data, a multi-phase registration displacement field is obtained, after sampling expansion, the expanded displacement field is used to transform other-phase CT data, a plurality of CT training data are obtained, and after pretreatment, DR data of the target patient and the CT training data form a specific data set; the generalization generative adversarial network is trained by using the specific data set, and a specific generative adversarial network is obtained;

[0035] S3, DR data of the lung of the target patient are acquired in real time, and CT generation data are obtained in real time by processing the DR data through the specific generative adversarial network.

[0036] Specific step details are as follows.

[0037] S1, acquire DR data and CT data of multiple patients' lungs, form a generalization data set after preprocessing, train a generative adversarial network, and obtain a generalization generative adversarial network.

[0038] Acquire DR data and CT data of multiple patients' lungs, form a generalization data set after preprocessing, train a generative adversarial network, and obtain a generalization generative adversarial network, realize the use of group generalization model to reduce the training time of patient-specific model, and improve the robustness of patient-specific model.

[0039] First, acquire DR data and CT data of multiple patients' lungs, and obtain multiple data pairs after preprocessing of DR data and CT data of each patient, forming a generalization data set.

[0040] Among them, the preprocessing operation of DR data and CT data is as follows: using SimpleITK library to convert the original DICOM format DR data and CT data to NIFTI format, resampling the CT data to a resolution of 1x1x1mm 3 , cutting out the center area, cutting the area volume to 320x320x320mm 3 , obtaining CT key data; based on DRR technology, the CT key data is spatially aligned with the DR data to obtain an X-ray image; the CT data is resized to 256x256x256mm 3 , the X-ray image is resized to 256x256x256mm 3 , and finally the CT data and X-ray image of the same size are stored in the h5py file in matrix form to obtain the data pair, and all data pairs form a generalization data set.

[0041] Then, train the generative adversarial network using the generalization data set to realize the generalization function of generating CT data from DR data, and obtain a generalization generative adversarial network.

[0042] The processing operation steps in training the generative adversarial network are as follows.

[0043] Step 1, based on the X-ray image and the corresponding edge feature map, obtain X-ray shallow information features, X-ray middle information features and X-ray deep information features.

[0044] The method for obtaining the X-ray shallow information feature, the X-ray middle information feature and the X-ray deep information feature is specifically as follows: the X-ray image and the corresponding edge feature map are respectively subjected to two-dimensional convolution processing to obtain an X-ray convolution map and an edge convolution map; the X-ray convolution map and the edge convolution map are subjected to splicing and then down-sampling, and are subjected to multi-channel feature fusion processing for multiple times (preferably 3 times) to obtain an X-ray shallow information feature; the edge convolution map is subjected to multi-channel feature fusion processing for multiple times (preferably 3 times) to obtain an edge shallow information feature; the X-ray shallow information feature and the edge shallow information feature are subjected to splicing and then down-sampling, and are subjected to multi-channel feature fusion processing for multiple times (preferably 3 times) to obtain an X-ray middle information feature; the edge shallow information feature is subjected to multi-channel feature fusion processing for multiple times (preferably 3 times) to obtain an edge middle information feature; and the X-ray middle information feature and the edge middle information feature are subjected to splicing and then down-sampling, and are subjected to multi-channel feature fusion processing for multiple times (preferably 27 times) to obtain an X-ray deep information feature.

[0045] The operation of the multi-channel feature fusion is as follows: after the splicing down-sampling map is subjected to convolution processing in different channels, the splicing down-sampling map is subjected to splicing processing, layer normalization processing, convolution processing and multi-layer perception processing to obtain a nonlinear feature; the shallow nonlinear feature and the splicing down-sampling map are subjected to element-by-element addition to obtain an X-ray information feature, which is used for performing the next multi-channel feature fusion processing.

[0046] The splicing down-sampling map is obtained based on the X-ray convolution map and the edge convolution map being subjected to splicing and then down-sampling, or the X-ray shallow information feature and the edge shallow information feature being subjected to splicing and then down-sampling, or the X-ray middle information feature and the edge middle information feature being subjected to splicing and then down-sampling.

[0047] In the convolution processing in different channels, the result of the convolution processing in the first channel remains unchanged, that is, the splicing down-sampling map is not processed; the convolution processing in the second channel is achieved through 3x3 convolution; and the convolution processing in the third channel and the convolution processing in the fourth channel are both achieved through orthogonal strip convolution.

[0048] The convolution processing in different channels is achieved through the following formula:

[0049] ,

[0050] ,

[0051] ,

[0052] ,

[0053] ,

[0054] The splicing down-sampling mapx performing a splitting operation with different allocation ratios, 、 、 、 respectively a first segmentation map, a second segmentation map, a third segmentation map, a fourth segmentation map, 、 、 、 respectively a first channel feature map, a second channel feature map, a third channel feature map, a fourth channel feature map,

[0055] is a cross strip convolution, extracting multi-channel deep features, 、 、 respectively a 3x3 convolution, an 11x1 convolution, a 1x11 convolution, is a concatenation operation.

[0056] Step 2, the X-ray deep information feature is spliced with the deep edge feature corresponding to the edge feature map, and a channel enhanced feature is obtained through multi-channel feature fusion processing; the X-ray image is obtained based on DR data and CT data.

[0057] The method for obtaining the deep edge feature corresponding to the edge feature map is: the edge middle layer information feature is processed through multi-channel feature fusion processing multiple times (preferably 3 times) to obtain an edge deep layer information feature.

[0058] Step 3, the channel enhanced feature is expanded into three dimensions to obtain a channel enhanced three-dimensional feature; the channel enhanced three-dimensional feature is processed through non-local attention multiple times (preferably 2 times) to obtain an attention enhanced feature.

[0059] The operation of non-local attention processing is: the channel enhanced three-dimensional feature is processed through inner normalization to obtain an inner normalized feature; after the inner normalized feature is processed through three-dimensional convolution of different scales, element-wise multiplication and non-linear processing (preferably a softmax function) are performed to obtain a convolution fusion non-linear feature; the inner normalized feature is processed through three-dimensional convolution, and the convolution fusion non-linear feature is processed through element-wise multiplication to obtain a convolution fusion feature; the convolution fusion feature and the inner normalized feature are added element by element to obtain an initial attention enhanced feature, which is used to perform the next non-local attention processing.

[0060] Step 4, after the attention enhanced feature, the X-ray shallow information feature, the X-ray middle layer information feature, and the X-ray deep information feature are processed through cross attention based on residual connection, three-dimensional reconstruction is performed to obtain pseudo-CT data.

[0061] The operation of the cross-attention processing based on the residual connection is specifically: the attention enhanced feature and the X-ray shallow information feature are processed by cross-attention processing to obtain a first cross-attention feature; the first cross-attention feature and the X-ray middle information feature are processed by cross-attention processing to obtain a second cross-attention feature; and the second cross-attention feature and the X-ray deep information feature are processed by cross-attention processing to obtain a third cross-attention feature, which is used to perform the operation of three-dimensional reconstruction.

[0062] Taking the second cross-attention feature as an example, the operation of the cross-attention processing is: after the X-ray middle information feature is expanded in dimension to become a three-dimensional feature, the key feature and the value feature are obtained, and the query feature of the first cross-attention feature is processed by attention processing, three-dimensional convolution processing based on the residual connection, and up-sampling to obtain the second cross-attention feature.

[0063] Step 5: When the training loss of each pseudo CT data and the corresponding real CT data is less than the loss threshold, the training is ended, and the generalization generative adversarial network is obtained.

[0064] The training loss includes a generator loss and a discriminator loss.

[0065] The generator loss includes a reconstruction loss of the pseudo CT data, a projection loss of the pseudo CT data and the corresponding real CT data, a perception loss, and an adaptive motion region loss.

[0066] The projection loss is calculated by the following formula:

[0067] ,

[0068] is the projection loss, 、 、 are projection functions of the transverse plane, the sagittal plane and the coronal plane respectively, 、 are the real CT data and the pseudo CT data respectively, is a norm 1 processing, is an expectation function.

[0069] The perception loss is calculated by the following formula:

[0070] ,

[0071] is the perception loss, is a network processing used to extract texture and structure information of an image, is a norm 2 processing.

[0072] The adaptive motion region loss is calculated by the following formula:

[0073] ,

[0074] Adaptive motion region loss, 4D CT data of a patient, containing CT data at multiple time points within a breathing cycle, pseudo-CT data to real CT data displacement field, normative processing.

[0075] The discriminator loss includes an adversarial loss and a decoding loss.

[0076] S2, obtaining multi-phase CT data (10-phase CT data) of a target patient containing a historical breathing cycle, registering the first-phase CT data in the multi-phase CT data to other phase CT data to obtain multi-phase registration displacement fields (1080 displacement fields), expanding the displacement fields after sampling (which can be realized by a principal component analysis model PCA-MM model), transforming other phase CT data using the expanded displacement fields to obtain multiple CT training data (1080 CT data to simulate the human morphology existing in the complete breathing process of the patient), and after preprocessing, forming a specific data set with the DR data of the target patient; training the generalization generative adversarial network using the specific data set to obtain a specific generative adversarial network applicable to the target patient, so that the generative adversarial network fits the individual breathing regularity and anatomical characteristics of the target patient, greatly reduces the generation deviation caused by individual differences, improves the matching degree of the generated image and the real physiological state of the patient, and further improves the accuracy and reliability of the subsequent clinical application based on the generated image.

[0077] S3, in the clinical process, the DR data (X-ray image) of the target patient can be obtained in real time, processed by the specific generative adversarial network, and CT generated data can be obtained in real time, so that the lesion positioning and targeted treatment are more timely and accurate.

[0078] To verify the generation effect of the embodiment, the following is realized.

[0079] Experimental setup. The experimental development environment used is pytorch2.0.1, on an NVIDIA RTX 4090 graphics processor, using the Adam optimizer, the learning rate is 0.0002, the batch size is set to 1, in the pre-training stage, a diverse group of patient data is trained for 100 rounds, about 15 hours, in the transfer learning stage, it is trained for 20 rounds, about 3 hours, the experimental parameters are shown in Table 1, and the implementation results are shown in Table 2.

[0080] Table 1 Summary of experimental parameters

[0081]

[0082] Table 2 Summary of experimental results

[0083]

[0084] The embodiment also provides a system for generating a lung DR image into a CT respiratory motion image, which is used to implement the method for generating a lung DR image into a CT respiratory motion image.

[0085] The general generative adversarial network generation module is configured to obtain DR data and CT data of lungs of a plurality of patients, form a generalization data set after preprocessing, train the generative adversarial network, and obtain a general generative adversarial network. The processing operation in the generative adversarial network is as follows: based on an X-ray image and a corresponding edge feature map, X-ray shallow information features, X-ray middle information features, and X-ray deep information features are obtained; the X-ray deep information features are spliced with deep edge features corresponding to the edge feature map, and multi-channel feature fusion processing is performed to obtain channel enhancement features; the X-ray image is obtained based on the DR data and the CT data; after the channel enhancement features are expanded into three dimensions, a plurality of times of non-local attention processing are performed to obtain attention enhancement features; after the attention enhancement features, the X-ray shallow information features, the X-ray middle information features, and the X-ray deep information features are processed based on cross attention of residual connection, three-dimensional reconstruction is performed to obtain pseudo-CT data; when a training loss value of each pseudo-CT data and corresponding real CT data is less than a loss threshold value, the training is ended, and the general generative adversarial network is obtained.

[0086] The specific generative adversarial network generation module is configured to obtain a plurality of CT data of a target patient's lung in a historical respiratory cycle, register first CT data in the plurality of CT data to other CT data to obtain a plurality of registration displacement fields, expand the plurality of registration displacement fields by sampling, transform the other CT data using the expanded displacement fields to obtain a plurality of CT training data, and form a specific data set after preprocessing the plurality of CT training data and DR data of the target patient. The general generative adversarial network is trained using the specific data set to obtain a specific generative adversarial network.

[0087] The CT data generation module is configured to obtain DR data of a target patient's lung in real time, and obtain CT generation data in real time by processing the DR data using the specific generative adversarial network.

[0088] The embodiment also provides a device for generating a lung DR image into a CT respiratory motion image, which includes a processor and a memory. When the processor executes a computer program stored in the memory, the method for generating a lung DR image into a CT respiratory motion image is implemented.

[0089] The embodiment also provides a computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to realize the above-mentioned generation method from a lung DR image to a CT respiratory motion image.

[0090] The generation method from a lung DR image to a CT respiratory motion image provided by the embodiment first acquires DR data and CT data of a plurality of patients, forms a generalization data set after preprocessing, trains a generative adversarial network to obtain a generalization generative adversarial network; then, acquires a specific data set by using multi-phase CT data and DR data of a historical respiratory cycle of a target patient before surgical intervention, trains the generalization generative adversarial network to obtain a specific generative adversarial network suitable for the target patient, greatly reduces the generation deviation caused by individual differences, and improves the matching degree of the generated image and the real physiological state of the patient; finally, during the intervention surgery and radiotherapy, only DR data of the lung of the target patient needs to be acquired in real time, so that real-time 4D CT data of the patient can be obtained, so that the lesion positioning and targeted treatment are more timely and accurate.

[0091] The generation method from a lung DR image to a CT respiratory motion image provided by the embodiment designs a generative adversarial network, which can enhance the feature fusion capability of the X-ray image and the edge feature map, so that the patient can avoid suffering from excessive radiation in the process of acquiring the guiding medical image, can acquire high-quality real-time CT images through X-ray films at the same time, and can control the generation effect according to the training loss, so that the structural similarity between the generated CT data and the real CT data is high, and the specific details of the lung tissue, such as lung blood vessels, lung tuberculosis lesion areas and bronchi, can be presented, and the respiratory motion process presented is very close to the real respiratory process, which can assist the radiotherapy doctor in accurate positioning of the lesion and targeted treatment.

Claims

1. A method for generating respiratory motion images from lung DR images to CT images, characterized in that, This includes the following operations: S1. Obtain DR and CT data of lungs from multiple patients, preprocess them to form a generalized dataset, train the generative adversarial network, and obtain the generalized generative adversarial network. The processing operations in training the generative adversarial network are as follows: based on the X-ray image and the corresponding edge feature map, obtain the shallow X-ray information features, the middle X-ray information features, and the deep X-ray information features; concatenate the deep X-ray information features with the deep edge features corresponding to the edge feature map, and obtain the channel enhancement features through multi-channel feature fusion processing; the X-ray image is obtained based on DR data and CT data; After extending the channel enhancement features to three dimensions, the attention enhancement features are obtained through several nonlocal attention processes. Attention-enhanced features, along with shallow X-ray, mid-layer X-ray, and deep X-ray information features, are processed by cross-attention based on residual connections and then reconstructed in three dimensions to obtain pseudo-CT data. Training ends when the training loss value of each pseudo-CT data and the corresponding real CT data is less than the loss threshold, and a generalized generative adversarial network is obtained. S2. Obtain multi-phase CT data of the target patient's lungs within a historical respiratory cycle. Register the first phase CT data in the multi-phase CT data to the other phase CT data to obtain the multi-phase registration displacement field. After sampling and augmentation, use the augmented displacement field to transform the other phase CT data to obtain multiple CT training data. Combine these with the target patient's DR data and, after preprocessing, form a specific dataset. A generalized generative adversarial network is trained using a specific dataset to obtain a specific generative adversarial network. S3. Real-time acquisition of DR data of the target patient's lungs, which is then processed by a specific generative adversarial network to obtain CT-generated data in real time.

2. The method for generating CT respiratory motion images from lung DR images according to claim 1, characterized in that, In S1, the methods for obtaining shallow X-ray information features, mid-layer X-ray information features, and deep X-ray information features are as follows: The X-ray image and the corresponding edge feature map are respectively processed by two-dimensional convolution to obtain the X-ray convolution map and the edge convolution map; After the X-ray convolutional image and the edge convolutional image are stitched together, they are downsampled and then subjected to multiple multi-channel feature fusion processes to obtain the shallow X-ray information features. The edge convolutional map is processed through multiple multi-channel feature fusion processes to obtain shallow edge information features; the shallow X-ray information features and the shallow edge information features are concatenated and downsampled, and then processed through multiple multi-channel feature fusion processes to obtain the mid-layer X-ray information features; The shallow edge information features are processed through multiple multi-channel feature fusion processes to obtain the mid-edge edge information features; the mid-edge X-ray information features and the mid-edge edge information features are spliced ​​together and then downsampled, and processed through multiple multi-channel feature fusion processes to obtain the deep X-ray information features.

3. The method for generating CT respiratory motion images from lung DR images according to claim 1 or 2, characterized in that, The operation of multi-channel feature fusion processing is as follows: The stitched downsampled image is convolved in different channels, and then processed by stitching, layer normalization, convolution, and multilayer perceptron to obtain nonlinear features. The shallow nonlinear features are added to the stitched downsampled image element by element to obtain X-ray information features, which are used to perform the next multi-channel feature fusion process.

4. The method for generating CT respiratory motion images from lung DR images according to claim 1, characterized in that, The non-local attention operation in S1 is as follows: The enhanced 3D features of the channel are then processed by internal normalization to obtain internally normalized features; After the internally normalized features are processed by 3D convolution at different scales, they are multiplied element-wise and nonlinearly processed to obtain convolutionally fused nonlinear features. After the inner normalized features are processed by 3D convolution, they are multiplied element-wise with the convolutional fusion nonlinear features to obtain the convolutional fusion features. The convolutional fusion features are then added element-wise with the inner normalized features to obtain the initial attention enhancement features, which are used to perform the next nonlocal attention processing.

5. The method for generating CT respiratory motion images from lung DR images according to claim 1, characterized in that, In S1, the training loss includes generator loss and discriminator loss; generator loss includes reconstruction loss of pseudo-CT data, projection loss of pseudo-CT data and corresponding real CT data, perception loss, and adaptive motion region loss.

6. The method for generating CT respiratory motion images from lung DR images according to claim 5, characterized in that, The projection loss is calculated using the following formula: ， For projection loss, , , These are the projection functions for the cross-section, sagittal plane, and coronal plane, respectively. , These are real CT data and pseudo CT data, respectively. For paradigm 1 processing, Let be the expected function.

7. The method for generating CT respiratory motion images from lung DR images according to claim 1, characterized in that, In S1, the preprocessing operation is as follows: The CT data is resampled and the central region is cropped to obtain the key CT data. The key CT data and DR data are spatially aligned to obtain the X-ray image. The CT data and X-ray image are then subjected to trilinear interpolation and bilinear interpolation respectively, and adjusted to the same size to obtain data pairs. All data pairs form a generalized dataset.

8. A system for generating respiratory motion images from lung DR images to CT images, used to implement the method for generating respiratory motion images from lung DR images to CT images as described in claim 1, characterized in that, include: The generalized generative adversarial network generation module is used to acquire DR and CT data of the lungs of multiple patients. After preprocessing, a generalized dataset is formed, which is then used to train the generative adversarial network to obtain the generalized generative adversarial network. The processing operations in training the generative adversarial network are as follows: Based on the X-ray image and the corresponding edge feature map, obtain the shallow X-ray information features, the mid-layer X-ray information features, and the deep X-ray information features; concatenate the deep X-ray information features with the deep edge features corresponding to the edge feature map, and obtain the channel enhancement features through multi-channel feature fusion processing; the X-ray image is obtained based on DR data and CT data; after expanding the channel enhancement features into three dimensions, it is processed through several non-local attention processes to obtain the attention enhancement features; Attention-enhanced features, along with shallow, mid, and deep X-ray information features, are processed through cross-attention based on residual connections and then reconstructed in 3D to obtain pseudo-CT data. Training ends when the training loss value of each pseudo-CT data and its corresponding real CT data is less than the loss threshold, resulting in a generalized generative adversarial network. The specific generative adversarial network generation module is used to acquire multi-phase CT data of the lungs of the target patient within a historical respiratory cycle. The first phase CT data in the multi-phase CT data is registered to the other phase CT data to obtain a multi-phase registration displacement field. After sampling and augmentation, the augmented displacement field is used to transform the other phase CT data to obtain multiple CT training data. These data, along with the target patient's DR data, are preprocessed to form a specific dataset. A generalized generative adversarial network is trained using a specific dataset to obtain a specific generative adversarial network. The CT data generation module is used to acquire DR data of the lungs of the target patient in real time, and then process it with a specific generative adversarial network to obtain CT generated data in real time.

9. A device for generating respiratory motion images from lung DR images to CT images, characterized in that, The system includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the method for generating lung DR images to CT respiratory motion images as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the method for generating lung DR images to CT respiratory motion images as described in any one of claims 1-7.

Images