Method and system for reconstructing three-dimensional ct from a single two-dimensional x-ray perspective image using reinforcement learning

Abstract

The application provides a method and system for reconstructing three-dimensional CT from a single two-dimensional X-ray perspective image by using reinforcement learning, and belongs to the technical field of image reconstruction, and comprises the following steps: acquiring an X-ray perspective image; processing the acquired X-ray perspective image by using a pre-trained reconstruction model to obtain a reconstructed three-dimensional CT image; wherein the reconstruction model is obtained by training a training set, and each set of training data comprises a three-dimensional CT at a t1 moment, a three-dimensional CT at a t2 moment and an X-ray perspective image corresponding to the CT at the t2 moment. By using the powerful modeling and solving capability of reinforcement learning, the three-dimensional CT reconstruction accuracy and reliability are improved; the three-dimensional CT reconstruction accuracy and speed can be further improved, which is of great significance for reducing the imaging dose, improving the imaging quality and improving the cancer treatment level.

Description

Technical Field

[0001] This invention relates to the field of image reconstruction technology, and more specifically to a method and system for reconstructing three-dimensional CT scans from a single two-dimensional X-ray fluoroscopic image using reinforcement learning. Background Technology

[0002] Reconstructing computed tomography (CT) images from single-frame 2D X-ray fluoroscopy images is a novel 3D imaging technique. Its core principle is to utilize prior knowledge from individuals or groups to perform regression modeling on the relationship between single-frame 2D X-ray fluoroscopy images and 3D CT images, and then use this model to reconstruct 3D CT images from the single-frame 2D X-ray fluoroscopy images. Compared with traditional methods, this method has broad application prospects in radiotherapy due to its shorter reconstruction time and higher accuracy. During the pre-treatment positioning phase, this method can be used to reconstruct 3D CT images, producing higher image quality than commonly used cone-beam CT (CBCT) with an imaging dose less than one percent of that of CBCT. During treatment, real-time reconstruction of 3D CT images using this method allows for the acquisition of information about organ movement during treatment, which is crucial for achieving high-precision automated gated radiotherapy, tumor-tracking radiotherapy, and real-time adaptive radiotherapy. After radiotherapy, the actual dose distribution of the patient can be reconstructed using the 3D CT images reconstructed using this method and compared with the planned dose distribution, thus providing a more effective assessment of treatment efficacy.

[0003] Existing methods for reconstructing 3D CT images from a single X-ray fluoroscopy image can be divided into two categories: iterative optimization-based methods and regression modeling-based methods. Iterative methods transform the reconstruction problem into a registration problem, using an iterative optimization process to find the 3D CT image that produces the most similar projection to the input single X-ray fluoroscopy image. However, the relationship between the single X-ray fluoroscopy image and the 3D CT image is complex, and the optimizer in this type of method often gets trapped in local optima, resulting in low registration accuracy and efficiency. Regression modeling-based methods model the relationship between the 2D X-ray fluoroscopy image and the 3D CT image, and then use the constructed model to directly regenerate the 3D CT image from the X-ray fluoroscopy image. This type of method avoids iterative optimization, thus ensuring reconstruction efficiency. However, currently, deep learning networks are mainly used as the reconstruction model, which often suffers from overfitting due to the difficulty in obtaining sufficient training data. Furthermore, the reconstruction accuracy of this type of method still needs further improvement.

[0004] Existing methods based on optimization registration are inefficient and prone to getting trapped in local optima, leading to decreased reconstruction accuracy. Because the mapping relationship between a single X-ray fluoroscopy image and 3D CT is highly complex, traditional iterative registration algorithms often get stuck in local optima during optimization, affecting reconstruction efficiency and effectiveness, and failing to meet clinical needs. Existing methods based on depth regression modeling directly estimate the relationship between a single X-ray fluoroscopy image and 3D CT, requiring a large amount of high-quality training data, which is difficult to obtain. Therefore, in practical applications, overfitting often leads to decreased reconstruction accuracy. Furthermore, existing depth regression modeling methods have poor interpretability; for scenarios requiring rapid reconstruction, manual verification is difficult, thus the confidence level of the reconstruction needs improvement. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for reconstructing three-dimensional CT scans from a single two-dimensional X-ray fluoroscopic image using reinforcement learning, so as to solve at least one of the technical problems existing in the background art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for reconstructing a three-dimensional CT scan based on a single two-dimensional X-ray fluoroscopic image using reinforcement learning, comprising:

[0008] Obtain X-ray fluoroscopic images;

[0009] The acquired X-ray fluoroscopic images are processed using a pre-trained reconstruction model to obtain reconstructed 3D CT images. The reconstruction model is trained using a training set, and each training set includes a 3D CT at time t1, a 3D CT at time t2, and the corresponding X-ray fluoroscopic image at time t2. The 3D CT at time t1 is the reference CT, the X-ray fluoroscopic image at time t2 is the input data, and the 3D CT at time t2 is the training label.

[0010] As a further limitation of the first aspect of the present invention, the training process of the reconstruction model is as follows: State variable initialization: n sets of training data are randomly selected from the training set to input the perspective image I. proj With reference CT, i.e. CT0, construct the initial state S0=(I projAction generation: The state variables are input into the policy network, which performs feature extraction, feature transformation, and deformation field reconstruction on the state variables, and outputs the deformation field obtained based on the current state variables; Update state variables and output reward value: The deformation field is input into the reconstruction environment, which updates the state variables and outputs the reward value; The updated state variables and reward value are input into the policy network again, and the parameters of the policy network are updated and optimized based on the obtained reward function value; Repeated iterations and optimizations are performed until no further updates are needed to obtain the trained reconstruction model.

[0011] As a further limitation of the first aspect of the present invention, in the training of the reconstruction model, a Markov quadruple is used to model the CT reconstruction process, and the elements of the quadruple are defined as follows: State: State S at step t. t Defined as "the initial single X-ray fluoroscopic image I input". proj Compared with current reconstructed CT images t The joint matrix of ", i.e. S t =(I proj CT t ); where I proj For a fixed input two-dimensional X-ray image, CT t The 3D CT image generated after the t-th iteration; the core function of the state is to reflect the "matching state between the current CT and X-ray fluoroscopy images," providing a basis for subsequent action decisions; Action: Action a at step t. t It is a deformation field used for CT scans. t Make local or global adjustments to improve I proj Matching degree; State transition: Given the state S at step t t With action a t Next round state S t+1 It can be determined by the reconstruction function recon, i.e., S. t+1 =(I proj CT t+1 ), including CT t+1 =recon(CT t , a t The recon function is based on the deformation field a. t CT t Perform deformation to ensure the generated CT scan t+1 Adhering to anatomical continuity; Reward: The reward function is used to quantify action a. t The effect of improving reconstruction accuracy.

[0012] As a further limitation of the first aspect of the present invention, a policy network P is trained. θ Determine the action based on the current state, i.e., a t=P θ (S t In order to obtain P θ The objective function is as follows:

[0013] ;

[0014] in, It refers to strategy. Indicates that in state S t Next, perform action a t The probability distribution; This represents the trajectory generated by the interaction between the intelligent agent and the environment; For all strategies The expected value of the generated trajectory; optimization of the reinforcement learning network is achieved by finding the θ that maximizes J; in the network application phase, the initial state S0 is generated based on the reference CT and a single X-ray fluoroscopic image, and then... The state variables are continuously updated iteratively, and when the reward item r... t When the changes cease, a reconstructed CT image is obtained.

[0015] As a further limitation of the first aspect of the present invention, the three-dimensional CT reconstruction process only includes a policy network, comprising: firstly, inputting a reference CT and a single X-ray fluoroscopic image into the reconstruction environment; the reconstruction environment has two main functions: one is to generate the state variables for the next iteration based on the state transition operations defined in the four-element group; the other is to generate the state variables and the reward value based on the reward value defined in the four-element group; the reconstruction environment generates the current state S. t and single-step reward r t S t With r t The inputs are passed to the policy network, and the executor performs actions based on these inputs and outputs. t If the reward value no longer increases, output the current reconstructed CT; otherwise, output a. t Enter the rebuild environment again and start the next cycle.

[0016] As a further limitation of the first aspect of the present invention, a three-dimensional CT reconstruction model comprising a policy network and a value network is fed into a reconstruction environment module, using CT and single X-ray fluoroscopic images as initial inputs; in the reconstruction environment, the current state S is generated based on these two types of inputs. t and single-step reward r t Subsequently, state S t and single-step reward r t They will be passed to the strategy respectively. The included executor and evaluator components; for the executor, based on the received S t and r tAfter internal network processing, the system outputs the corresponding action; the evaluator then evaluates the input S. t and r t Perform an evaluation, calculate and output Q(S) t , a t This output is fed back to the policy network to help it adjust subsequent action generation strategies; if the reward value no longer increases, the current reconstructed CT is output; otherwise, a t The data is then input into the reconstruction environment again, entering the next cycle, thereby driving the entire CT reconstruction process to continuously iterate and optimize, so that the results of each reconstruction can gradually approach a more ideal state.

[0017] Secondly, the present invention provides a system for reconstructing a three-dimensional CT scan based on a single two-dimensional X-ray fluoroscopic image using reinforcement learning, comprising:

[0018] The acquisition module is used to acquire X-ray fluoroscopic images;

[0019] The processing module is used to process the acquired X-ray fluoroscopic images using a pre-trained reconstruction model to obtain reconstructed three-dimensional CT images. The reconstruction model is trained using a training set, and each set of training data includes a three-dimensional CT at time t1, a three-dimensional CT at time t2, and the corresponding X-ray fluoroscopic image at time t2. The three-dimensional CT at time t1 and the X-ray fluoroscopic image at time t2 are the training input data, and the three-dimensional CT at time t2 is the training label.

[0020] Thirdly, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the method for reconstructing a three-dimensional CT scan using reinforcement learning based on a single two-dimensional X-ray fluoroscopic image as described in the first aspect.

[0021] Fourthly, the present invention provides a computer device including a memory and a processor, the processor and the memory communicating with each other, the memory storing program instructions executable by the processor, and the processor calling the program instructions to execute the method for reconstructing a three-dimensional CT scan using reinforcement learning based on a single two-dimensional X-ray fluoroscopic image as described in the first aspect.

[0022] Fifthly, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the method of reconstructing a three-dimensional CT based on a single two-dimensional X-ray fluoroscopic image using reinforcement learning as described in the first aspect.

[0023] The beneficial effects of this invention are: by utilizing the powerful modeling and solving capabilities of reinforcement learning, the accuracy and reliability of 3D CT reconstruction can be improved; it can further enhance the reconstruction accuracy and speed of 3D CT, which is of great significance for reducing imaging dose, improving imaging quality, and improving the level of cancer treatment.

[0024] The advantages of additional aspects of the invention will be set forth more clearly in the following description or will be learned by practice of the invention. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a system framework diagram of reconstructing a three-dimensional CT scan using reinforcement learning based on a single two-dimensional X-ray fluoroscopic image, as described in an embodiment of the present invention.

[0027] Figure 2 This is a flowchart of a 3D CT reconstruction process that includes only a policy network, as described in an embodiment of the present invention.

[0028] Figure 3 This is a flowchart of a 3D CT reconstruction process that includes only a policy network and a value network, as described in an embodiment of the present invention. Detailed Implementation

[0029] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0030] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0031] It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as here.

[0032] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0033] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0034] To facilitate understanding of the present invention, the present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0035] Those skilled in the art should understand that the accompanying drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily essential for implementing the present invention.

[0036] This invention combines the advantages of reinforcement learning with those of iterative optimization and deep learning methods to form a novel method for reconstructing three-dimensional CT images from a single X-ray fluoroscopy image, and constructs a system for reconstructing three-dimensional CT images from a single X-ray fluoroscopy image.

[0037] This paper proposes a method for reconstructing 3D CT scans from single X-ray fluoroscopic images using reinforcement learning. Reinforcement learning employs a deep learning network as an optimizer, leveraging its powerful modeling capabilities to improve optimization performance and efficiency, thus avoiding the accuracy degradation caused by getting trapped in local optima in traditional iterative optimization methods. Furthermore, reinforcement learning updates parameters using a single iteration, while traditional deep learning requires batches of data; this difference in training units makes reinforcement learning less dependent on data, effectively avoiding overfitting due to insufficient data. Finally, this invention adds anatomical constraints to the reinforcement learning environment, thereby improving the robustness and interpretability of the reconstruction process and significantly enhancing the reliability of the method in the absence of manual verification. A system for reconstructing 3D CT scans from single X-ray fluoroscopic images using reinforcement learning is also proposed, providing reconstruction functionality through both standalone tools and support libraries that can be embedded in other systems. On one hand, when working as a standalone tool, the system features a user-friendly interface. By inputting the paths to a reference CT image and an auxiliary reference CT image, the system can complete the reconstruction. On the other hand, as a support library, the system provides interface functions that can be directly called by other systems and returns the reconstructed CT image directly from memory, achieving efficient data transfer. Finally, the system is also equipped with logging functions to ensure that each reconstruction is accurate, secure, and can be reviewed.

[0038] Example 1

[0039] In this embodiment 1, a system for reconstructing a three-dimensional CT scan based on a single two-dimensional X-ray fluoroscopic image using reinforcement learning is provided. The system includes: an acquisition module for acquiring the X-ray fluoroscopic image; and a processing module for processing the acquired X-ray fluoroscopic image using a pre-trained reconstruction model to obtain a reconstructed three-dimensional CT image. The reconstruction model is trained using a training set, and each set of training data includes a three-dimensional CT scan at time t1, a three-dimensional CT scan at time t2, and the corresponding X-ray fluoroscopic image at time t2. The three-dimensional CT scan at time t1 and the X-ray fluoroscopic image at time t2 are the training input data, and the three-dimensional CT scan at time t2 is the training label.

[0040] In this embodiment, the above-described system is used to implement a method for reconstructing a three-dimensional CT scan from a single two-dimensional X-ray fluoroscopic image using reinforcement learning. The method includes: acquiring an X-ray fluoroscopic image using an acquisition module; processing the acquired X-ray fluoroscopic image using a pre-trained reconstruction model using a processing module to obtain a reconstructed three-dimensional CT image; wherein the reconstruction model is trained using a training set, and each set of training data includes a three-dimensional CT scan at time t1, a three-dimensional CT scan at time t2, and the X-ray fluoroscopic image corresponding to the CT scan at time t2; wherein the three-dimensional CT scan at time t1 and the X-ray fluoroscopic image at time t2 are the training input data, and the three-dimensional CT scan at time t2 is the training label.

[0041] The training process of the reconstruction model is as follows: State variable initialization: n sets of "X-ray fluoroscopy image-real CT" paired data are randomly selected from the training set. The initial reconstructed CT is generated with the real CT as a reference, and the initial state S0=(Iproj, CT0) is constructed; Action generation: The state variables are input into the policy network. The policy network performs feature extraction, feature transformation and deformation field reconstruction on the state variables, and outputs the deformation field obtained based on the current state variables; Update state variables and output reward value: The deformation field is input into the reconstruction environment. The reconstruction environment updates the state variables and outputs the reward value; The updated state variables and reward value are input into the policy network again. The parameters of the policy network are updated and optimized based on the obtained reward function value; Iteration and optimization are repeated until no further updates are needed to obtain the trained reconstruction model.

[0042] In the training of the reconstruction model, Markov quadruples are used to model the CT reconstruction process. The elements of the quadruples are defined as follows: State: State S at step t. t Defined as "Input single X-ray fluoroscopic image I" proj Compared with current reconstructed CT images t The joint matrix of ", i.e. S t =(I proj CT t ); where I proj For a fixed input two-dimensional X-ray image, CT t The 3D CT image generated after the t-th iteration; the core function of the state is to reflect the "matching state between the current CT and X-ray images," providing a basis for subsequent action decisions; Action: Action a at step t. t It is a deformation field used for CT scans. t Make local or global adjustments to improve I proj Matching degree; State transition: Given the state S at step t t With action a t Next round state S t+1It can be determined by the reconstruction function recon, i.e., S t+1 =(I proj CT t+1 ), including CT t+1 =recon(CT t , a t The recon function is based on the deformation field a. t CT t Perform deformation to ensure the generated CT scan t+1 Adhering to anatomical continuity; Reward: The reward function is used to quantify action a. t The effect of improving reconstruction accuracy.

[0043] Train a policy network P θ Determine the action based on the current state, i.e., a t =P θ (S t In order to obtain P θ The objective function is as follows:

[0044] ;

[0045] in, It refers to strategy. Indicates that in state S t Next, perform action a t The probability distribution; This represents the trajectory generated by the interaction between the intelligent agent and the environment; For all strategies The expected value of the generated trajectory; optimization of the reinforcement learning network is achieved by finding the θ that maximizes J; in the network application phase, the initial state S0 is generated based on the reference CT and a single X-ray fluoroscopic image, and then... The state variables are continuously updated iteratively, and when the reward item r... t When the changes cease, a reconstructed CT image is obtained.

[0046] The 3D CT reconstruction workflow, which only includes a policy network, consists of the following steps: First, a reference CT scan and a single X-ray fluoroscopic image are input into the reconstruction environment. The reconstruction environment serves two main purposes: one is to generate the state variables for the next iteration based on the state transition operations defined in the quaternion set; the other is to generate the state variables and reward value based on the reward value defined in the quaternion set. The reconstruction environment then generates the current state S. t and single-step reward r t S t With r t The inputs are passed to the policy network, and the executor performs actions based on these inputs and outputs. t If the reward value no longer increases, output the current reconstructed CT; otherwise, output a.t Enter the rebuild environment again and start the next cycle.

[0047] A 3D CT reconstruction model, incorporating a policy network and a value network, is fed into the reconstruction environment module, using CT and single-X-ray fluoroscopic images as initial inputs. Within the reconstruction environment, the current state S is generated based on these two types of inputs. t and single-step reward r t Subsequently, state S t and single-step reward r t They will be passed to the strategy respectively. The included executor and evaluator components; for the executor, based on the received S t and r t After internal network processing, the system outputs the corresponding action; the evaluator then evaluates the input S. t and r t Perform an evaluation, calculate and output Q(S) t , a t This output is fed back to the policy network to help it adjust subsequent action generation strategies; if the reward value no longer increases, the current reconstructed CT is output; otherwise, a t The data is then input into the reconstruction environment again, entering the next cycle, thereby driving the entire CT reconstruction process to continuously iterate and optimize, so that the results of each reconstruction can gradually approach a more ideal state.

[0048] Example 2

[0049] In this embodiment, a model for reconstructing a 3D CT scan from a single X-ray fluoroscopy image using reinforcement learning is first proposed. This model is trained using the following method: each set of training data includes a 3D CT scan at time t1, a 3D CT scan at time t2, and the corresponding X-ray fluoroscopy image at time t2. The 3D CT scan at time t1 and the X-ray fluoroscopy image at time t2 are the training input data, and the 3D CT scan at time t2 is the training label. The training process is as follows: First, state variable initialization: n sets of "X-ray fluoroscopy image - real CT" paired data are randomly selected from the training set. An initial reconstructed CT (CT0, which can be obtained by slightly randomly deforming the real CT) is generated using the real CT as a reference, constructing the initial state S0=(Iproj, CT0). Second, action generation: the state variables are input into the policy network. The policy network performs feature extraction, feature transformation, and deformation field reconstruction on the state variables, outputting the deformation field obtained based on the current state variables. Third, updating state variables and outputting reward values: the deformation field is input into the reconstruction environment. The reconstruction environment updates the state variables and outputs reward values. The updated state variables and reward values ​​are input into the policy network again, and the parameters of the policy network are updated and optimized based on the obtained reward function value. If the reward value can be increased, the process returns to step two and iterates repeatedly; if no further updates are needed, the process proceeds to the next step. Step four: Output the reconstructed CT: Based on the final state variables, the reconstructed 3D CT is obtained.

[0050] In this embodiment, CT reconstruction from a single X-ray fluoroscopy image is achieved using reinforcement learning. This transforms CT reconstruction into a sequential decision-making process involving interaction between an agent (an agent refers to a policy network or a policy network and a value network) and the "reconstruction environment" (the reconstruction environment refers to the mathematical rules that output reward values ​​and update state variables based on generated actions). This embodiment uses Markov quadruples to model the CT reconstruction process. The elements of the quadruples are defined as follows:

[0051] State: State S at step t t Defined as "Input single X-ray fluoroscopic image I" proj Compared with current reconstructed CT images t The joint matrix of ", i.e. S t =(I proj CT t ). Among them, I proj For a fixed input two-dimensional X-ray image, CT t The state is the 3D CT image generated after the t-th iteration; the core function of the state is to reflect the "matching state between the current CT and X-ray images", providing a basis for subsequent action decisions.

[0052] Action: Step t, action a t It is a deformation field used for CT scans. t Make local or global adjustments to improve I proj The degree of matching.

[0053] State transition: Given the state S at step t t With action a t Next round state S t+1 It can be determined by the reconstruction function recon, i.e., S t+1 =(I proj CT t+1 ), including CT t+1 =recon(CT t , a t The recon function is based on the deformation field a. t CT t Perform deformation to ensure the generated CT scan t+1 It conforms to the continuity of anatomical structure.

[0054] Reward: The reward function is used to quantify action a. t Improvement in reconstruction accuracy

[0055] Based on the above definition, this embodiment requires training a policy network P. θ Determine the action based on the current state, i.e., a t =P θ (S t In order to obtain P θ The objective function is as follows:

[0056]

[0057] in, It refers to strategy. Indicates that in state S t Next, perform action a t The probability distribution. This represents the trajectory generated by the interaction between the intelligent agent and the environment. For all strategies The expected value of the generated trajectory. The reinforcement learning network can be optimized by finding the θ that maximizes J. In the network application phase, the initial state S0 is generated based on the reference CT scan and a single X-ray fluoroscopic image, and then... The state variables are continuously updated iteratively, and when the reward item r... t When the changes cease, a reconstructed CT image can be obtained.

[0058] As can be seen, using reinforcement learning to build a reconstruction model combines the advantages of iterative optimization and deep learning: on the one hand, it utilizes the powerful modeling capabilities of deep learning models to accurately model the relationship between each iterative action and the objective function; on the other hand, it abandons the existing strategy of solving regression models in one step and adopts a strategy of gradually approximating the optimal solution to improve reconstruction accuracy.

[0059] like Figure 1 As shown in this embodiment, based on the trained model described above, a system for reconstructing 3D CT scans from a single X-ray fluoroscopic image using reinforcement learning is proposed. This system consists of three modules: a data input module, a reinforcement learning model personalization and parameter adjustment module, and a 3D CT reconstruction module based on the reinforcement learning model. First, the data input module mainly includes a data input submodule and a data management submodule. The data input submodule obtains the input data required for reconstruction (specifically, the 3D CT scan at time t1, a 3D CT scan at time t2, and the corresponding X-ray fluoroscopic image at time t2) primarily through network data packets or local file reading. The data management module archives and manages the input data, the reconstructed CT image, and the logs generated during the reconstruction process for easy retrieval and use.

[0060] The reinforcement learning model personalization and parameter setting module mainly includes a model personalization fine-tuning submodule, a personalized model management submodule, and a model parameter setting submodule. The model personalization fine-tuning submodule can specifically fine-tune the model weights based on individual patient data (assuming the policy network has n layers, the fine-tuning method mainly involves training only the last l layers of the policy network, freezing the weights of the remaining nl layers; l can generally be 1-3), thereby improving reconstruction accuracy and effect. The personalized model management submodule is mainly responsible for the storage, deletion, and loading of various personalized models. The model parameter setting module is mainly used to set the parameters involved in the model application stage, including the weights of different reward items, the maximum number of iterations, etc.

[0061] The reinforcement learning-based 3D CT reconstruction module comprises a 3D CT reconstruction submodule, a reconstruction log recording submodule, and a reconstruction result output submodule. The 3D CT reconstruction submodule primarily reconstructs 3D CT images based on the input data (specifically, the 3D CT images at time t1 and the X-ray fluoroscopic images at time t2) and the corresponding model and parameters. The reconstruction log recording submodule records the changes in various reward items during the reconstruction process, providing the necessary data foundation for subsequent model performance analysis. The reconstruction result output submodule is responsible for outputting the reconstructed CT images in DICOM format via network data or local files.

[0062] like Figure 2As shown, this embodiment also proposes a 3D CT reconstruction process that only includes a policy network. First, the reference CT and single X-ray fluoroscopic image are used as inputs into the reconstruction environment. The reconstruction environment has two main functions: one is to generate the state variables for the next iteration based on the state transition operations defined in the four-element group. The specific operations are as follows:

[0063] S t+1 =(I proj CT t+1 )

[0064] Among them, CT t+1 =recon(CT t , a t Here, the recon function utilizes the deformation field a t CT t Perform deformation. Another step is to generate state variables and reward values ​​based on the reward values ​​defined in the four-element group. This can be defined using the following function:

[0065]

[0066] Where P represents the forward projection function, which calculates the forward projection of the input CT image.

[0067] Reconstruct the environment to generate the current state S t and single-step reward r t S t With r t The inputs are passed to the policy network, and the executor (the policy network itself) performs actions a based on these inputs. t (Presented in Deformation Field DVF format), if the reward value no longer increases, output the current reconstructed CT; otherwise, output a. t Re-enter the data into the rebuild environment and proceed to the next loop. Regarding the objective function, you can directly... Calculate the gradient to optimize the policy parameters:

[0068] ;

[0069] Indicates benchmark relative to Find the gradient. The discount factor for rewards is a constant. t raised to the power of t; k is a temporary operator in the operation, its value is from the current t to T, where T is the maximum number of iterations. This represents the reward value at step k. express k to the power of t.

[0070] Using this method, a reinforcement learning policy network for reconstructing 3D CT scans from a single X-ray fluoroscopic image can be obtained.

[0071] like Figure 3 As shown, this embodiment also proposes a 3D CT reconstruction model that includes a policy network and a value network. CT and single X-ray fluoroscopic images are used as initial inputs and fed into the reconstruction environment module. In the reconstruction environment, the current state S is generated based on these two types of inputs. t and single-step reward r t Subsequently, state S t and single-step reward r t They will be passed to the strategy respectively. It includes the executor and evaluator sections. For the executor, it depends on the received S... t and r t After internal network computation, the corresponding action is output (presented in the form of a deformation field DVF). The evaluator will then evaluate the input S... t and r t Perform an evaluation, calculate and output Q(S) t , a t This output is fed back to the policy network to help it adjust subsequent action generation strategies. If the reward value no longer increases, the current reconstructed CT is output; otherwise, a... t The data is then input into the reconstruction environment again, entering the next cycle, thereby driving the entire CT reconstruction process to continuously iterate and optimize, so that the results of each reconstruction can gradually approach a more ideal state.

[0072] Example 3

[0073] This embodiment 3 provides a non-transitory computer-readable storage medium for storing computer instructions. When executed by a processor, the computer instructions implement the method described above for reconstructing a three-dimensional CT scan using reinforcement learning from a single two-dimensional X-ray fluoroscopic image. The method includes:

[0074] Obtain X-ray fluoroscopic images;

[0075] The acquired X-ray fluoroscopic images are processed using a pre-trained reconstruction model to obtain reconstructed three-dimensional CT images. The reconstruction model is trained using a training set, and each training set includes a three-dimensional CT at time t1, a three-dimensional CT at time t2, and the corresponding X-ray fluoroscopic image at time t2. The three-dimensional CT at time t1 and the X-ray fluoroscopic image at time t2 are the training input data, and the three-dimensional CT at time t2 is the training label.

[0076] Example 4

[0077] This embodiment 4 provides a computer device, including a memory and a processor, wherein the processor and the memory communicate with each other, and the memory stores program instructions that can be executed by the processor. The processor calls the program instructions to execute the method described above for reconstructing a three-dimensional CT scan using reinforcement learning from a single two-dimensional X-ray fluoroscopic image. The method includes:

[0078] Obtain X-ray fluoroscopic images;

[0079] The acquired X-ray fluoroscopic images are processed using a pre-trained reconstruction model to obtain reconstructed three-dimensional CT images. The reconstruction model is trained using a training set, and each training set includes a three-dimensional CT at time t1, a three-dimensional CT at time t2, and the corresponding X-ray fluoroscopic image at time t2. The three-dimensional CT at time t1 and the X-ray fluoroscopic image at time t2 are the training input data, and the three-dimensional CT at time t2 is the training label.

[0080] Example 5

[0081] This embodiment 5 provides an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the method described above for reconstructing a three-dimensional CT scan using reinforcement learning from a single two-dimensional X-ray fluoroscopic image. The method includes:

[0082] Obtain X-ray fluoroscopic images;

[0083] The acquired X-ray fluoroscopic images are processed using a pre-trained reconstruction model to obtain reconstructed three-dimensional CT images. The reconstruction model is trained using a training set, and each training set includes a three-dimensional CT at time t1, a three-dimensional CT at time t2, and the corresponding X-ray fluoroscopic image at time t2. The three-dimensional CT at time t1 and the X-ray fluoroscopic image at time t2 are the training input data, and the three-dimensional CT at time t2 is the training label.

[0084] In summary, this invention presents a novel method and system for reconstructing 3D CT scans from a single X-ray fluoroscopic image. By leveraging the powerful modeling and solving capabilities of reinforcement learning, the accuracy and reliability of 3D CT reconstruction are improved. Reconstructing CT scans from a single X-ray fluoroscopic image is not only a current research hotspot but also allows for the acquisition of a patient's 3D anatomical information without hardware modifications. The provided technical specifications not only outline a comprehensive and flexible technical framework and basic methods for reconstructing CT scans from a single X-ray fluoroscopic image but also describe specific implementation details, such as methods for reconstructing 3D CT scans from a single X-ray fluoroscopic image using different reinforcement learning models. The method and system provided by this invention can further improve the reconstruction accuracy and speed of 3D CT, which is of great significance for reducing imaging dose, improving imaging quality, and enhancing cancer treatment.

[0085] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solutions disclosed in the present invention, various modifications or variations that can be made by those skilled in the art without creative effort should be included within the scope of protection of the present invention.

Claims

1. A method for reconstructing a three-dimensional CT scan from a single two-dimensional X-ray fluoroscopic image using reinforcement learning, characterized in that, include: Obtain X-ray fluoroscopic images; The acquired X-ray fluoroscopic images are processed using a pre-trained reconstruction model to obtain reconstructed three-dimensional CT images. The reconstruction model is trained using a training set, and each training set includes a three-dimensional reference CT at time t1, a three-dimensional CT at time t2, and the corresponding X-ray fluoroscopic image at time t2. The three-dimensional reference CT at time t1 and the X-ray fluoroscopic image at time t2 are the training input data, and the three-dimensional CT at time t2 is the training label. The training process of the reconstruction model is as follows: State variable initialization: randomly select n sets of training data from the training set to input the X-ray perspective image I. proj With reference CT, i.e. CT0, construct the initial state S0=(I proj Action generation: The state variables are input into the policy network, which performs feature extraction, feature transformation, and deformation field reconstruction on the state variables, and outputs the deformation field obtained based on the current state variables; Update state variables and output reward value: The deformation field is input into the reconstruction environment, which updates the state variables and outputs the reward value; The updated state variables and reward value are input into the policy network again, and the parameters of the policy network are updated and optimized based on the obtained reward function value; Repeated iterations and optimizations are performed until no further updates are needed to obtain the trained reconstruction model; In the training of the reconstruction model, Markov quadruples are used to model the CT reconstruction process. The elements of the quadruples are defined as follows: State: State S at step t. t Defined as "Input single X-ray perspective image I" proj Compared with current reconstructed CT images t The joint matrix of ", i.e. S t =(I proj CT t ); where I proj For a fixed input two-dimensional X-ray image, CT t The 3D CT image generated after the t-th iteration; the core function of the state is to reflect the "current matching state between the CT and X-ray images," providing a basis for subsequent action decisions; Action: Action a at step t. t It is a deformation field used for CT scans. t Make local or global adjustments to improve I proj Matching degree; State transition: Given the state S at step t t With action a t Next round state S t+1 Determined by the reconstruction function recon, i.e., S t+1 =(I proj CT t+1 ), including CT t+1 =recon(CT t , a t The recon function is based on the deformation field a. t CT t Perform deformation to ensure the generated CT scan t+1 Adhering to anatomical continuity; Reward: The reward function is used to quantify action a. t The effect of improving reconstruction accuracy.

2. The method for reconstructing a three-dimensional CT scan using reinforcement learning based on a single two-dimensional X-ray fluoroscopic image according to claim 1, characterized in that, Train a policy network P θ Determine the action based on the current state, i.e., a t =P θ (S t In order to obtain P θ The objective function is as follows: ； in, It refers to strategy; This represents the trajectory generated by the interaction between the intelligent agent and the environment; For all strategies The expected value of the generated trajectory; optimization of the reinforcement learning network is achieved by finding the θ that maximizes J; in the network application phase, the initial state S0 is generated based on the reference CT and a single X-ray fluoroscopic image, and then... The state variables are continuously updated iteratively, and when the reward item r... t When the changes cease, a reconstructed CT image is obtained.

3. The method for reconstructing a three-dimensional CT scan using reinforcement learning based on a single two-dimensional X-ray fluoroscopic image according to claim 2, characterized in that, The 3D CT reconstruction workflow, which only includes a policy network, consists of the following steps: First, a reference CT scan and a single X-ray fluoroscopic image are input into the reconstruction environment. The reconstruction environment serves two main purposes: one is to generate the state variables for the next iteration based on the state transition operations defined in the quaternion set; the other is to generate the state variables and reward value based on the reward value defined in the quaternion set. The reconstruction environment then generates the current state S. t and single-step reward r t S t With r t The data is passed to the policy network, and the executor takes actions based on these inputs and outputs. t If the reward value no longer increases, output the current reconstructed CT; otherwise, output a. t Enter the rebuild environment again and start the next cycle.

4. The method for reconstructing a three-dimensional CT scan using reinforcement learning based on a single two-dimensional X-ray fluoroscopic image according to claim 2, characterized in that, A 3D CT reconstruction model, incorporating a policy network and a value network, is fed into the reconstruction environment module, using CT and single-X-ray fluoroscopic images as initial inputs. Within the reconstruction environment, the current state S is generated based on these two types of inputs. t and single-step reward r t Subsequently, state S t and single-step reward r t They will be passed to the strategy respectively. The included executor and evaluator components; for the executor, based on the received S t and r t After internal network processing, the system outputs the corresponding action; the evaluator then evaluates the input S. t and r t Perform an evaluation, calculate and output Q(S) t , a t This output is fed back to the policy network to help it adjust subsequent action generation strategies; if the reward value no longer increases, the current reconstructed CT is output; otherwise, a t The data is then input into the reconstruction environment again, entering the next cycle, thereby driving the continuous iteration and optimization of the entire CT reconstruction process, so that the results of each reconstruction can gradually approach a more ideal state.

5. A system for reconstructing a three-dimensional CT scan from a single two-dimensional X-ray fluoroscopic image using reinforcement learning, based on the method described in any one of claims 1-4, characterized in that, include: The acquisition module is used to acquire X-ray fluoroscopic images; The processing module is used to process the acquired X-ray fluoroscopic images using a pre-trained reconstruction model to obtain reconstructed three-dimensional CT images. The reconstruction model is trained using a training set, and each set of training data includes a three-dimensional reference CT at time t1, a three-dimensional CT at time t2, and the X-ray fluoroscopic image corresponding to the CT at time t2. The three-dimensional reference CT at time t1 and the X-ray fluoroscopic image at time t2 are the training input data, and the three-dimensional CT at time t2 is the training label.

6. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the method for reconstructing a three-dimensional CT scan using reinforcement learning based on a single two-dimensional X-ray fluoroscopic image as described in any one of claims 1-4.

7. A computer device, characterized in that, The method includes a memory and a processor, the processor and the memory communicating with each other, the memory storing program instructions executable by the processor, and the processor calling the program instructions to execute the method for reconstructing a three-dimensional CT scan using reinforcement learning based on a single two-dimensional X-ray fluoroscopic image as described in any one of claims 1-4.

8. An electronic device, characterized in that, include: The electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the method for reconstructing a three-dimensional CT scan using reinforcement learning based on a single two-dimensional X-ray fluoroscopic image as described in any one of claims 1-4.

Images