Four-dimensional CT image reconstruction methods, devices, medical equipment, and storage media
By constructing a four-dimensional CT image reconstruction model using implicit neural representation (INR) and utilizing learnable feature encoding modules and static template images, the model addresses the issues of insufficient learning ability and low computational efficiency of high-resolution data in four-dimensional CT image reconstruction, achieving higher accuracy and wider application.
Patent Information
- Application Number
- CN202310841681.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-10
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-07-10
AI Technical Summary
Existing four-dimensional CT image reconstruction methods have poor learning ability when processing high-resolution data, limited angle reconstruction has limitations, large computational load and low efficiency, and cannot meet the needs of practical applications.
An implicit neural representation (INR) is used to construct a reconstruction model. The time and space coordinates are converted into spatiotemporally coupled features through a learnable feature encoding module. Combined with static template images, image reconstruction is performed, which reduces computational consumption and improves reconstruction accuracy and applicability.
It improves the accuracy of high temporal resolution reconstruction, expands the application range of the model, meets practical needs, and reduces computational complexity.
Smart Images

Figure CN116740215B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of imaging technology, and in particular to a method, apparatus, medical device, and storage medium for reconstructing four-dimensional CT (Computed Tomography) images. Background Technology
[0002] 4DCT (4-Dimensional Computed Tomography) technology can image moving objects and is used in the medical field to image areas such as the lungs and heart during movement; INR (implicit neural representation) is a data representation method based on neural networks in the field of deep learning, which is suitable for representing multidimensional data such as images.
[0003] In related technologies, an INR-based 4DCT image representation method can be used. In the first method, INR is used to represent static images, and a traditional polynomial with discrete matrix parameters is used to represent the four-dimensional motion field. The feature encoding module in the INR is a non-learnable fixed-frequency encoding function that transforms the four-dimensional spatiotemporal coordinates of the complex image into a feature vector, which is then input into a learnable MLP (multi-layer perceptron) to obtain image values. In the second method, for the four-dimensional motion field, a high-quality 4DCT reconstructed image of the scanned object is first acquired as a prior dynamic image. The motion field between each phase in the prior dynamic image is extracted, and the principal components of motion are extracted from it using PCA (Principal Component Analysis). Then, using INR, which cannot be learned by a feature encoding module, the time coordinates are converted into weight vectors of each principal component. The principal components are linearly weighted to obtain the motion field of each phase from the static image to the prior image. The 4D (4-Dimensional) reconstructed image is obtained through deformation motion transformation. During the training process, on the one hand, the 4D reconstructed image is reprojected according to the scanning relationship to compare with the projection obtained during the actual scan to calculate the loss function. On the other hand, the INR of the static image is compared with the data of specific phases in the projection obtained during the actual scan in the image domain and projection domain to calculate the loss function.
[0004] However, the methods in related technologies have limited representation capabilities, making it difficult to represent complex, high-resolution dynamic images, and difficult to fit motion fields with complex motion, high spatial resolution, and temporal resolution; they are also computationally inefficient, making it difficult to process high-resolution data, and limiting application scenarios, thus failing to meet practical needs. Summary of the Invention
[0005] This application provides a four-dimensional CT image reconstruction method, apparatus, medical device, and storage medium to solve the problems in related technologies, such as the poor learning ability of four-dimensional CT image reconstruction to high-resolution data, the limitations of finite angle reconstruction, and the large amount of computation and low computational efficiency of the reconstruction model, which makes the application range of the model in actual scenarios narrow and unable to meet the actual use needs.
[0006] The first aspect of this application provides a method for reconstructing a four-dimensional CT image, comprising the following steps: acquiring projection data of a four-dimensional computed tomography (CT) scan to be reconstructed; extracting the temporal and spatial coordinates of a target from the projection data of the four-dimensional CT to be reconstructed; inputting the temporal and spatial coordinates into a pre-constructed reconstruction model, and outputting a reconstructed image of the four-dimensional CT to be reconstructed, wherein the reconstruction model includes a motion field INR composed of implicit neural representations (INRs) and a static template image INR, the motion field INR generating deformation motion offset and spatiotemporal coupling features based on the temporal and spatial coordinates, and the static template image INR performing image reconstruction based on the spatial coordinates after deformation motion offset and / or the spatiotemporal coupling features to obtain the reconstructed image of the four-dimensional CT to be reconstructed.
[0007] Optionally, the time coordinates are the time points of physical projection data acquisition, or the motion phase signals recorded by motion monitoring sensors; the motion field INR includes a first feature encoding module, a second feature encoding module, a coupling module, and a first neural network module, wherein the first feature encoding module is used to encode the time coordinates to obtain time features; the second feature encoding module is used to encode the spatial coordinates to obtain spatial features; the coupling module is used to couple the time features and the spatial features to obtain spatiotemporal coupling features; and the first neural network module is used to convert the spatiotemporal coupling features into deformation motion offsets.
[0008] Optionally, the calculation formula for the spatiotemporal coupling feature is:
[0009]
[0010] Where M is the sports field INR, e M This is a spatiotemporal coupled feature encoding module; Let t be a spatial vector and t be time. s(t) is the spatial feature vector, s(t) is the temporal feature vector, and χ is the coupling module.
[0011] Optionally, the formula for calculating the deformation motion offset is:
[0012]
[0013] Where, φ MThis is the first neural network module. This represents the offset of the deformation motion.
[0014] Optionally, the static template image INR includes a third feature encoding module, a second neural network module, and a non-deformation-time-varying module. The third feature encoding module encodes the spatial coordinates after the deformation motion offset to obtain template spatial features. The non-deformation-time-varying module merges the spatiotemporal coupling features obtained from the motion field INR with the template spatial features with only deformation motion when the target to be reconstructed has time-varying effects caused by factors other than deformation motion, to obtain template spatial features with additional non-deformation-time-varying characteristics. The second neural network module reconstructs the reconstructed image of the four-dimensional CT to be reconstructed based on the template spatial features with only deformation motion or with additional non-deformation-time-varying characteristics.
[0015] Optionally, for the target V1 with deformation motion in the four-dimensional CT image to be reconstructed, the image reconstruction representation is as follows:
[0016]
[0017] Where h is the third feature encoding module, The image reconstruction expression for a target V1 with deformation motion from a 4D CT scan is given. This is the second neural network encoding module;
[0018] For the target V2 in the four-dimensional CT image to be reconstructed, which has additional non-deformation that changes over time, the image reconstruction representation is as follows:
[0019]
[0020] in, For the image reconstruction expression of a target V2 with additional non-deformation that varies over time in a four-dimensional CT, h is the third feature encoding module. This is the second neural network encoding module; The vector is transformed through the non-deformation time-varying module. and Expressions that combine and concatenate elements.
[0021] A second aspect of this application provides a four-dimensional CT image reconstruction device, comprising: an acquisition module for acquiring projection data of a four-dimensional computed tomography (CT) scan to be reconstructed; an extraction module for extracting the time coordinates and spatial coordinates of a target from the projection data of the four-dimensional CT scan to be reconstructed; and a reconstruction module for inputting the time coordinates and spatial coordinates into a pre-constructed reconstruction model and outputting a reconstructed image of the four-dimensional CT scan to be reconstructed. The reconstruction model includes a motion field INR composed of implicit neural representations (INRs) and a static template image INR. The motion field INR generates deformation motion offset and spatiotemporal coupling features based on the time coordinates and spatial coordinates. The static template image INR performs image reconstruction based on the spatial coordinates after deformation motion offset and / or the spatiotemporal coupling features to obtain the reconstructed image of the four-dimensional CT scan to be reconstructed.
[0022] Optionally, the time coordinates are the time points of physical projection data acquisition, or the motion phase signals recorded by motion monitoring sensors; the motion field INR includes a first feature encoding module, a second feature encoding module, a coupling module, and a first neural network module, wherein the first feature encoding module is used to encode the time coordinates to obtain time features; the second feature encoding module is used to encode the spatial coordinates to obtain spatial features; the coupling module is used to couple the time features and the spatial features to obtain spatiotemporal coupling features; and the first neural network module is used to convert the spatiotemporal coupling features into deformation motion offsets.
[0023] Optionally, the calculation formula for the spatiotemporal coupling feature is:
[0024]
[0025] Where M is the sports field INR, e M This is a spatiotemporal coupled feature encoding module; Let t be a spatial vector and t be time. s(t) is the spatial feature vector, and s(t) is the temporal feature vector; This is a coupled module.
[0026] Optionally, the formula for calculating the deformation motion offset is:
[0027]
[0028] Where, φ M This is the first neural network module. This represents the offset of the deformation motion.
[0029] Optionally, the static template image INR includes a third feature encoding module, a second neural network module, and a non-deformation-time-varying module. The third feature encoding module encodes the spatial coordinates after the deformation motion offset to obtain template spatial features. The non-deformation-time-varying module merges the spatiotemporal coupling features obtained from the motion field INR with the template spatial features with only deformation motion when the target to be reconstructed has time-varying effects caused by factors other than deformation motion, to obtain template spatial features with additional non-deformation-time-varying characteristics. The second neural network module reconstructs the reconstructed image of the four-dimensional CT to be reconstructed based on the template spatial features with only deformation motion or with additional non-deformation-time-varying characteristics.
[0030] Optionally, for the target V1 with deformation motion in the four-dimensional CT image to be reconstructed, the image reconstruction representation is as follows:
[0031]
[0032] Where h is the third feature encoding module, The image reconstruction expression for a target V1 with deformation motion from a 4D CT scan is given. This is the second neural network module.
[0033] For the target V2 in the four-dimensional CT image to be reconstructed, which has additional non-deformation that changes over time, the image reconstruction representation is as follows:
[0034]
[0035] in, For the image reconstruction expression of a target V2 with additional non-deformation that varies over time in a four-dimensional CT, h is the third feature encoding module. This is the second neural network encoding module; The vector is transformed through the non-deformation time-varying module. and Expressions that combine and concatenate elements.
[0036] A third aspect of this application provides a medical device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the four-dimensional CT image reconstruction method as described in the above embodiments.
[0037] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the four-dimensional CT image reconstruction method as described in the above embodiments.
[0038] Therefore, this application has at least the following beneficial effects:
[0039] By constructing and applying a reconstruction model containing a feature encoding module with learnable features, the model can directly calculate at any single location without first calculating the complete static image and four-dimensional motion field before performing deformation operations. This reduces computational costs, lowers the difficulty of solving the problem, overcomes the limitations of finite angle reconstruction, and improves the accuracy of high temporal resolution reconstruction. At the same time, it increases the applicability of the model to different application scenarios, thereby improving the user experience and meeting practical needs.
[0040] This solves the problems in related technologies, such as the poor learning ability of four-dimensional CT image reconstruction to high-resolution data, the limitations of finite angle reconstruction, and the large amount of computation and low efficiency of reconstruction models, which narrow the application range of models in real-world scenarios and cannot meet the needs of actual use.
[0041] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0042] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0043] Figure 1 This is a flowchart of a four-dimensional CT image reconstruction method according to an embodiment of this application;
[0044] Figure 2 This is a flowchart of a four-dimensional CT image reconstruction method according to an embodiment of this application;
[0045] Figure 3 This is a schematic diagram of a cone-beam CT system according to an embodiment of this application;
[0046] Figure 4 This is a schematic diagram illustrating an implementation method for learnable feature encoding according to an embodiment of this application;
[0047] Figure 5 This is a schematic diagram of a four-dimensional CT image reconstruction device according to an embodiment of this application;
[0048] Figure 6 This is a schematic diagram of the structure of a medical device according to an embodiment of this application. Detailed Implementation
[0049] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0050] X-ray CT imaging technology has wide applications in medical, security inspection, and industrial non-destructive testing fields. 3DCT (3-Dimensional Computed Tomography) technology can only image stationary objects, while the recently developed 4DCT technology can image moving objects and is used in the medical field for imaging areas such as the lungs and heart during motion. Existing 4DCT reconstruction methods typically divide a complete motion cycle into several phases and select the phase with the smaller motion amplitude for imaging. For the phase of interest, to ensure image quality, the reconstruction algorithm generally requires the acquisition angle to be distributed across a complete 0° to 180°+ sector angle to avoid finite angle artifacts. This is usually achieved by setting a sufficiently long scan time to cover multiple motion cycles. However, long scan times result in higher doses, and when the projection acquired for the same phase includes multiple motion cycles, motion mismatch can introduce additional motion artifacts.
[0051] For 4DCT, reducing the scan time to cover only a single motion cycle of the moving object to be reconstructed can significantly reduce the dose, improve scanning efficiency, and reduce motion artifacts caused by multi-cycle motion mismatch. However, reducing the scan time will lead to a reduction in the angular range scanned for each phase. Therefore, in the extreme case where the scan time covers only a single motion cycle, the reconstruction suffers from a severe finite-angle ill-condition. Furthermore, when dividing the same motion cycle into phases, the shorter the phase width, the less motion artifacts are contained in the phase image, and thus the higher the achievable temporal resolution.
[0052] However, at the same CT scanner rotation speed, the shorter the phase width, the smaller the angular range of the image scanned within that time window. CT scanner gantry rotation speed is limited; therefore, when pursuing high temporal resolution reconstruction, each time phase of the 3D scanned object can only correspond to a very small angular range. This reconstruction problem constitutes a very serious finite-angle ill-conditioned reconstruction problem. Therefore, the finite-angle reconstruction problem in 4D CT reconstruction is a key bottleneck issue in shortening scan time and improving the temporal resolution of reconstructed images.
[0053] To address the finite-angle reconstruction problem in 4DCT reconstruction, it is necessary to utilize the correlations between images to be reconstructed at different time phases. In existing 4DCT reconstruction algorithms, one approach mines the temporal and spatial correlations between images of different phases, and between dynamic images and prior images, to design prior regularization terms for iterative reconstruction algorithms. Another approach assumes that images of all phases can be obtained from a static image through deformation motion transformation of a motion field, thus designing an algorithm for jointly reconstructing static images and 4D motion fields. However, while these methods differ in their algorithms, they all represent static images as discrete 3D matrices and motion fields as discrete 4D matrices or parameterized models with low-dimensional discrete matrix parameters. In the discrete matrix representations of static images and motion fields, the correlations between different locations are weak, failing to fully model temporal and spatial correlations. Therefore, relying solely on algorithms is insufficient to further improve the performance of the methods. Solving the finite angle problem in 4DCT can reduce the dimension of the problem to be solved from 4D to a lower dimension, thus fully limiting the size of the solution space. Therefore, in related technologies, dynamic image models can be established, and 4DCT reconstruction algorithms can be designed specifically to solve the finite angle reconstruction problem in 4DCT reconstruction more effectively.
[0054] A new neural network-based data representation method called Implicit Neural Representation (INR) has emerged in the field of deep learning, suitable for representing multidimensional data such as images. In INR, a multidimensional data point is defined as a continuous field. For any coordinate in the data, it is first converted into a feature vector through a feature encoding module, and then the feature vector is converted into a value at that location through a neural network. INR can not only model discrete images as a relatively smooth continuous field, but also ensures that the image values at each location are obtained through feature transformation modules and neural network processing, resulting in higher correlation between locations than in discrete matrix forms. Therefore, the INR approach can be used to model dynamic objects in 4DCT, introducing stronger spatiotemporal correlation and more effective dimensionality reduction at the data representation level. Furthermore, models built using this new method allow for more flexible and deeper interaction between data representation and reconstruction algorithms, improving algorithm performance. Additionally, common deep learning methods often require training on several datasets before being applied to new test data, leading to a generalization problem of training-test mismatch. Most INR methods do not require training on any dataset. They can be trained directly on the real-time data obtained in the application. The usage is similar to traditional iterative reconstruction algorithms, so there is no generalization problem and the cost of collecting training data in the early stage is eliminated.
[0055] In related technologies, two INR-based 4DCT image representation methods can be used. In the first method, INR is used to represent static images, and a traditional polynomial with discrete matrix parameters is used to represent the four-dimensional motion field. The feature encoding module in the INR is an unlearnable fixed-frequency encoding function that transforms the four-dimensional spatiotemporal coordinates of the complex image into a feature vector, which is then input into a learnable MLP to obtain the image values. This method has several drawbacks: the feature encoding module in the static image INR is not learnable, limiting the representational power of the INR and making it difficult to represent complex, high-resolution dynamic images; the four-dimensional motion field is still represented by a traditional polynomial with discrete matrix parameters, which has limited representational power and is difficult to fit motion fields with complex motion, high spatial resolution, and temporal resolution; the interaction between the static image INR and the motion field is accomplished through traditional deformation transformation, and the 4D single-point pixel value is determined by the static image value and motion field value of all points in a large neighborhood after deformation transformation. Therefore, the processing of each ray in each training step involves calculating the discrete form of the complete static image and the four-dimensional motion field from the model. This approach does not fully utilize the characteristics of the INR, cannot limit the calculation to the range of directly related pixels, has low computational efficiency, and is difficult to handle high-resolution data; the method can only learn dynamic objects generated by deformation motion and cannot handle a wider range of motion.
[0056] The second method represents static images in the same way as the first, using INR, which the feature encoding module cannot learn. For the four-dimensional motion field, a high-quality 4DCT reconstructed image of the scanned object is first acquired as a priori dynamic image. The motion field between each phase in the prior dynamic image is extracted, and principal components of motion are extracted using PCA. Then, using an INR that the feature encoding module cannot learn, the time coordinates are converted into weight vectors for each principal component. The principal components are then linearly weighted to obtain the motion field at each time phase from the static image to the prior image. The 4D reconstructed image is obtained through deformation motion transformation. During training, on the one hand, the 4D reconstructed image is reprojected according to the scanning relationship to compare with the projection obtained during the actual scan to calculate the loss function. On the other hand, the static image INR is compared with the data of specific time phases in the projection obtained during the actual scan in the image domain and projection domain to calculate the loss function. This method has several drawbacks: First, like the first method, the static image INR has limited representational power and is difficult to represent complex, high-resolution dynamic images. Second, the motion field requires prior acquisition of previous 4DCT data of the scanned object to extract the principal components of the motion field, limiting its application scenarios. Furthermore, the method's accuracy is relatively low when the motion components differ significantly between two scans. Third, using only INR to represent the linear weighted weights of the principal components in the motion field is problematic because both the principal components themselves are unlearnable and the feature encoding modules in the INR of the weights are also unlearnable. Therefore, the representational power of the motion field INR is also limited, making it difficult to represent complex motion fields. Additionally, the model's calculation process for individual dynamic image pixel values is similar to the first method. To calculate a single pixel value, the static image values and motion field values of all points within a large neighborhood must first be calculated from the model before deformation transformation. Therefore, the training process is also similar to the first method, involving the calculation of the global static image and motion field, as well as the projection and reconstruction of the global image. This results in high computational cost, low efficiency, and difficulty in using high-resolution data. Finally, the method still assumes that motion only involves deformation, further limiting its application scenarios.
[0057] In view of the aforementioned background technologies, four-dimensional CT image reconstruction has poor learning ability for high-resolution data, limited angle reconstruction has limitations, and the reconstruction model has a large computational load and low computational efficiency, resulting in a narrow application range of the model in practical scenarios and failing to meet practical needs. Therefore, this application provides a four-dimensional CT image reconstruction method. The following description, with reference to the accompanying drawings, outlines embodiments of the four-dimensional CT image reconstruction method, apparatus, medical device, and storage medium of this application.
[0058] Specifically, Figure 1 This is a schematic flowchart of a four-dimensional CT image reconstruction method provided in an embodiment of this application.
[0059] like Figure 1As shown, this four-dimensional CT image reconstruction method includes the following steps:
[0060] In step S101, the projection data of the four-dimensional computed tomography CT scan to be reconstructed is obtained.
[0061] It is understood that the embodiments of this application may first obtain the projection data of the reconstructed four-dimensional computed tomography scan, so as to facilitate the use of the projection data in subsequent embodiments; wherein, the obtained projection data may include time coordinates, spatial coordinates, motion field INR composed of implicit neural representation INR and static template image INR, etc., which will be described in detail in the following embodiments and will not be repeated here.
[0062] In step S102, the time coordinates and spatial coordinates of the target in the projection data of the four-dimensional CT to be reconstructed are extracted.
[0063] It is understandable that, such as Figure 2 As shown, in this embodiment of the application, the time coordinates and spatial coordinates can be extracted from the projection data obtained in step S101, so as to facilitate the use of the data in subsequent embodiments, such as coupling spatial features and time features as spatiotemporal coupling features, etc., which will not be elaborated here.
[0064] In step S103, the time coordinates and spatial coordinates are input into the pre-constructed reconstruction model, and the reconstructed image of the four-dimensional CT to be reconstructed is output. The reconstruction model includes a motion field INR composed of implicit neural representations INR and a static template image INR. The motion field INR generates deformation motion offset and spatiotemporal coupling features based on the time coordinates and spatial coordinates. The static template image INR performs image reconstruction based on the spatial coordinates and / or spatiotemporal coupling features after deformation motion offset, so as to obtain the reconstructed image of the four-dimensional CT to be reconstructed.
[0065] It is understandable that, such as Figure 2 As shown, the embodiments of this application can pre-build a four-dimensional CT reconstruction model, which includes two main modules: as follows Figure 2 The sports field INR shown in Figure (1) and as shown in Figure (2) Figure 2 The static template image INR shown in Figure (2) is used in the design of the motion field INR. Spatial and temporal features are coupled and processed as spatiotemporal coupling features. In the design of the static template image INR, the deformation motion mechanism and the optional non-deformation time-varying mechanism are modeled. In the model of this application embodiment, multiple INRs are used as basic units. Each INR has a learnable feature encoding module, which encodes spatial features, temporal features and template spatial features respectively.
[0066] In this embodiment, the time coordinate is the time point of the physical projection data acquisition, or the motion phase signal recorded by the motion monitoring sensor; the motion field INR includes a first feature encoding module, a second feature encoding module, a coupling module, and a first neural network module, wherein the first feature encoding module is used to encode the time coordinate to obtain time features; the second feature encoding module is used to encode the spatial coordinate to obtain spatial features; the coupling module is used to couple the time features and spatial features to obtain spatiotemporal coupling features; and the first neural network module is used to convert the spatiotemporal coupling features into deformation motion offset.
[0067] In this embodiment, at least one neural network module can be selected to realize the transformation of spatiotemporal coupling features, such as an MLP module. In the following embodiments, the MLP module will be used as an example for specific explanation.
[0068] It is understood that embodiments of this application may use a learnable feature encoding module, which converts input coordinates into feature vectors, while the neural network module, comprising several layers of a neural network (in embodiments of this application, an MLP module can be used as the neural network module), receives the vector output by the feature encoding module as input and outputs a new vector or value. Specifically, the INR used in this application incorporates a learnable feature encoding module, which includes parameters that can be adjusted during training, thus enhancing the representational power of the INR. Figure 3 As shown, learnable feature encoding modules can be implemented in various forms commonly used in various fields, such as fixed frequency encoding + learnable MLP, querying coordinate values in learnable feature matrices, or querying coordinate values in a set of learnable feature matrices with different resolutions.
[0069] In this embodiment of the application, the formula for calculating the spatiotemporal coupling characteristic is as follows:
[0070]
[0071] Where M is the sports field INR, e M This is a spatiotemporal coupled feature encoding module; Let t be a spatial vector and t be time. s(t) is the spatial feature vector, and s(t) is the temporal feature vector; This is a coupled module.
[0072] In this embodiment, the formula for calculating the deformation motion offset is:
[0073]
[0074] Where, φ M This is the first neural network module. This represents the offset of the deformation motion.
[0075] Understandably, the motion field INR includes a learnable spatiotemporally coupled feature encoding module e M and a neural network module φ M The time and space coordinates are converted into deformation motion offsets; the spatiotemporal coupled feature encoding module e M Modeling it as a coupling of spatial and temporal features involves combining spatial feature vectors. The time feature vector s(t) is coupled to the time feature vector through some kind of coupling module Transformed into a new feature vector; coupling module It can be a fixed operation, such as a simple concatenation operation or element-wise multiplication; or it can be a function with learnable parameters, such as concatenating two vectors and then inputting the result into a neural network. Among these, spatial feature vectors... Both the temporal feature vector s(t) and the spatial feature vector employ learnable feature encoding designs. They can take the same or different forms, for example, spatial feature vectors. The approach involves querying within a set of multi-resolution feature matrices, while the temporal feature vector s(t) is encoded using a fixed frequency and a learnable MLP.
[0076] In this embodiment of the application, the static template image INR includes a third feature encoding module, a second neural network module, and a non-deformation time-varying module. The third feature encoding module is used to encode the spatial coordinates after deformation motion offset to obtain template spatial features.
[0077] The non-deformation time-varying module is used to merge the spatiotemporal coupling features obtained from the motion field INR with the template space features with only deformation motion when the target to be reconstructed has time-varying effects caused by factors other than deformation motion, to obtain template space features with additional non-deformation time-varying effects; the second neural network module is used to reconstruct the reconstructed image of the four-dimensional CT to be reconstructed based on the template space features with only deformation motion or with additional non-deformation time-varying effects.
[0078] It is understood that, in the embodiments of this application, a 4D dynamic image can be decoupled into a static template image INR (denoted as V) and a motion field INR (denoted as M). The relationship between the two includes necessary deformation motion effects and unnecessary non-deformation time-varying effects; such as Figure 2 As shown, the attenuation coefficient for the 4D object to be reconstructed in t is the spatial coordinate of a point on the object, while t is the time coordinate.
[0079] In the relationship between deformation and motion, for object μ, the model assumes that μ has a specific time t and position. On The value can correspond to the spatial location in the static template image INR. The value on The sports field INR at time t and position The value on Modeled from point to The deformation motion offset. Due to the time t and position The object value is only related to its position after a certain motion, and this relationship models the deformation motion.
[0080]
[0081]
[0082] The embodiments of this application can also establish a second model in the additional relationship of non-deformation time-varying effects: assuming The value on depends not only on querying the position after a deformation in the static template image INR. It also depends on The static template image INR receives more than just the static template image itself. As input, it also receives representatives eigenvectors As an additional input.
[0083]
[0084] In this embodiment of the application, for a target V1 with deformation motion in the four-dimensional CT image to be reconstructed, the image reconstruction representation is as follows:
[0085]
[0086] Where h is the third feature encoding module, The image reconstruction expression for a target V1 with deformation motion from a 4D CT scan is given. This is the second neural network module.
[0087] For a target V2 in the four-dimensional CT image to be reconstructed that has additional non-deformation time-varying characteristics, the image reconstruction representation is as follows:
[0088]
[0089] in, For the image reconstruction expression of a target V2 with additional non-deformation that varies over time in four-dimensional CT, h is the third feature encoding module, and φ is the third feature encoding module. V2This is the second neural network encoding module; The vector is transformed through the non-deformation time-varying module. and Expressions that combine and concatenate elements.
[0090] It is understandable that the feature encoding module in the static template image INR is designed with learnable parameters; the deformation-only INR form of V1 is expanded into a learnable template third feature encoding module h and a neural network module φ. V .
[0091] Therefore, in this embodiment of the application, after training the loss function, any set of spatial and temporal coordinates to be queried from any set of four-dimensional CT scans can be substituted into the model for inference to obtain the reconstructed location value; wherein, the method for defining the loss function in the training process of this embodiment of the application can be specifically as follows:
[0092] The loss function includes both a data fidelity term and an optional multi-level regularization term. Obtaining the data fidelity term involves comparing the model's predicted values with the true projected data, which can be achieved in at least one way. For example, distance functions that can be used include, but are not limited to, the l-norm. Weighted least squares, conditional probability, and other methods are beneficial for denoising CT projection noise modeling. p represents the model's predicted value, and p represents the true label.
[0093] Multi-level regularization is implemented by simultaneously constraining the properties of intermediate variables in the model, including the outputs of the motion field INR and the template object INR, which are substituted into the smoothing representation function and the edge extraction function. The sparsity, smoothness and other properties of the above outputs are improved by optimizing these feature measures. The functions used include, but are not limited to, the aforementioned l-norm, as well as the commonly used total variation function in the field.
[0094] The parameter training process of the method embodiment in this application can be specifically described as follows:
[0095] (1) Obtain the 4DCT projection of the 4D object to be reconstructed. Each time point corresponds to a projection angle.
[0096] (2) In each iteration step of the training, a subset of projection data points (containing different timestamps and coordinates on the two-dimensional projection map) is randomly selected from the four-dimensional CT projection according to a certain rule. Different rules can be followed, such as: completely random or each batch of data points has the same timestamp.
[0097] (3) Calculate the path of the X-ray corresponding to each projection data point; select several query spatial coordinates in the overlapping area of the X-ray path and the reconstructed area, and query the predicted value of the proposed model under the spatial coordinates and timestamp.
[0098] (4) Based on the line integral rendering equation of X-ray projection, calculate the projection value of the proposed model prediction result at each projection data point position, that is, the projection value predicted by the model.
[0099] (5) Substitute the projected values predicted by the model, the true projected labels, and the intermediate variables predicted by the model into the loss function, and train the model in the method through backpropagation of error.
[0100] (6) After training, any set of spatial and temporal coordinates to be queried in the four-dimensional CT can be substituted into the model for reasoning to obtain the reconstructed value of that location.
[0101] The four-dimensional CT image reconstruction method of this application will be described below with a specific embodiment:
[0102] (1) This embodiment can be applied to any scanning mode or system architecture of 4DCT without loss of generality; the following process will take a cone-beam circular orbit CT imaging system as an example. Figure 3 As shown, in this system, the flat panel detector and the light source move around the object being scanned at a certain rotation speed. The center of rotation is set as the origin of the coordinate system. The flat panel detector is always perpendicular to the XY plane, and the Z coordinate of the light source is 0.
[0103] In this embodiment, the object being scanned is the patient's lung region, and a single respiratory cycle is 5 seconds (s). The gantry rotation speed is set to 2.5s per revolution, so during a continuous 5-second scan, the gantry rotates 2 revolutions, scanning a total of 720 degrees; 360 evenly distributed viewing angles are acquired in each revolution, for a total of 720 viewing angles, with each viewing angle corresponding to a time window of 5 / 720s in length.
[0104] Assuming the cone-beam projection data obtained by the detector is in, Let C and D represent the number of columns and rows of a single projection, respectively, and V represent the total number of viewpoints. The time-varying attenuation coefficient of the object to be reconstructed at a certain point can be represented by a field. It means that, among them, y is the spatial coordinate in the object coordinate system, and for convenience, it is assumed that x, y, z are all normalized to [0,1]; t is the time coordinate (time stamp) during the scanning period, and for convenience, it is assumed that t is normalized to [0,1].
[0105] In applications such as medical diagnosis, for ease of visualization, It is often stored and displayed in discrete form, as shown in the following formula:
[0106]
[0107] Assume the physical volume of the object to be reconstructed is S.x ×S y ×S z The scanning time is T scan Then the ideal spatial resolution that this discrete form can represent is Ideal time resolution is For continuous forms There are no formal limitations on temporal or spatial resolution. In this embodiment, a continuous form of the attenuation coefficient can be used. After the model is trained, in actual use, the discrete linear attenuation coefficient map of the object to be reconstructed can be obtained by reasoning the model prediction value of each discrete grid point position.
[0108] (2) Learnable features can take many forms; in this embodiment, one such form can be used. Figure 4 The examples shown are "fixed frequency coding + learnable MLP" (Form 1) and "a set of learnable feature matrices at different resolutions" (Form 3); among them, Figure 4 This is an example of learning feature encoding implementation in the basic unit "INR with learnable features". Form 3, "a set of learnable feature matrices at different resolutions", consists of a series of "learnable feature matrices" in Form 2.
[0109] In Form 1, for example, for each dimension of the input coordinates (,y), it is first transformed into a frequency domain vector through a fixed frequency coding function. For example, x is transformed into...
[0110] γ(x)=[sin(2 0 x),sin(2 1 x),…,sin(2 K-1 x),cos(2 0 x),cos(2 1 x),…,cos(2 K- 1 x)],
[0111] Here, the integer K is the order of the frequency domain vector, which can be set to 16 in this embodiment. Next, the frequency domain vectors of each dimension are concatenated into a large vector, such as [γ(x), γ(y)], and input into a parameter-learnable MLP, which is then transformed into a new vector f(x,y) of length F, representing the feature vector. In this embodiment, the parameter-learnable MLP is a neural network with two fully connected layers. The first layer has a ReLU activation function, while the second layer does not.
[0112] In form 2, for example, it includes a parameter-learnable matrix. The first two dimensions correspond to the dimensions of the input coordinates, and the size F of the last dimension is the number of channels in the feature matrix. Assuming the input coordinates (,y) have been normalized to [0,1], we first scale each dimension to [0,N-1], obtaining... Where N is the dimension of the parameter-learnable matrix. Next, neighboring integer grid points are found. For the two-dimensional coordinates in this embodiment, there are four neighboring integer grid points. Next, the values of these four neighboring integer grid points are found in the parameter-learnable matrix. Finally, based on the d-dimensional coordinates... Given the positional relationship between the points and four adjacent integer grid points, perform d-linear interpolation (bilinear interpolation in this example) to obtain the value of (,y) in the matrix. It is a vector of length F, representing the eigenvector.
[0113] In Form 3, for example, there are L combinations of Form 2 with different feature matrix resolutions, where L represents the resolution level, which can be set to 16 in this embodiment. Different resolutions mean that the first d dimensions of the matrix are different. For example, in this embodiment, the first two dimensions of the first matrix are 16, and the first two dimensions of the last matrix are 512. The input coordinates (,y) are input into these L Form 2 feature encodings respectively, resulting in L feature vectors of length F. Finally, these are concatenated to obtain a multi-resolution feature vector of length L×F.
[0114] In particular, for the large-sized eigenma matrices present in forms 2 and 3, storage space can be compressed using certain methods. For example, to compress the first d dimensions of a large d+1 dimensional matrix into a single dimension of size T, thereby reducing the storage space required by the matrix... For storage, the following hash coordinate mapping function is used to map the input d-dimensional coordinates to an integer index between [0, T-1]:
[0115]
[0116] in, This represents the bitwise XOR operation, π1,…, d For each distinct, sufficiently large prime number. The value of the input coordinate in H is then looked up using a new integer index. Although different d-dimensional coordinates may be mapped to the same integer index, i.e., a "hash collision" occurs, using a hash coordinate mapping function can effectively reduce the matrix storage size with only a small performance loss. For example, in this embodiment, T = 2 can be set. 19 For feature matrices with a total size greater than T in the first d dimensions, compression is performed in the manner described above.
[0117] (3) Define the motion field INR of the model:
[0118] One embodiment of this application may be defined as follows: Figure 2 The motion field INR in the model shown in Figure (1)
[0119] In this embodiment, the coupling module χ in the motion field INR is configured as an element-wise multiplication of the temporal feature vector and the spatial feature vector. To perform this operation, the temporal and spatial feature vectors are set to have the same length in this example. After the time coordinate t is normalized to [0,1], it is input into a learnable feature encoding module. The learnable feature encoding adopts form 1 described in section 8-2 above, where the frequency encoding parameter K = 16. The MLP contains two fully connected layers, the first layer being 32×64 in size and the second layer being 64×32 in size, thus encoding the time coordinate into a feature vector s(t) of length 32. Spatial coordinates... After normalization to [0,1], a learnable feature encoding module is input. The learnable feature encoding adopts form 3 described in section 8-2 above, which includes 16 feature matrices with different resolutions, and the size of the i-th matrix is N. i ×N i ×N i ×F, where {N} i} i=1,…,16 Given a geometric sequence from 32 to 1024, F = 2. Furthermore, the hash encoding described in section 8-2 is used to compress the number of parameters in the matrix, with parameter T = 2. 19 π1 = 1, π2 = 2654435761, π3 = 805459861. Therefore, the spatial coordinates are encoded into a feature vector of length 32. s(t) and The spatiotemporal coupling feature vector is obtained through the coupling module χ (element-by-element multiplication in this embodiment). The input is an MLP defined by three convolutional layers with sizes of 32×64, 64×64, and 64×3. The first two layers are followed by a ReLU activation function. The final output is a vector of length 3. This represents the offset of the deformation motion.
[0120] (4) Define the static template image INR of the model:
[0121] One embodiment of this application may be defined as follows: Figure 2 The static template image INR is shown in Figure (1).
[0122] Regarding the connection between deformation motion, first, based on the results of the previous step... To calculate the coordinates after deformation. The input is then fed into a learnable feature encoding module, where the learnable feature encoding adopts form 3 as described in section 8-2 above. The specific definition is the same as the learnable feature encoding of spatial coordinates in the motion field INR. The output of the module is a vector of length 32, which represents the template spatial features. To include an optional non-deformable time-varying relationship, the intermediate variable of the motion field INR in 8-3 will be... Shorted to the aforementioned template space features Then, they are concatenated into a new vector of length 64, which serves as the input for the subsequent MLP; otherwise, only template space features are considered. This vector serves as the input to the subsequent MLP. Finally, the resulting vector of length α is input into an MLP defined by three convolutional layers of size α×64, 64×64, and 64×1, with the first two layers followed by a ReLU activation function. The final result is a floating-point number for reconstructing the image.
[0123] (5) Method training process:
[0124] The first step is to obtain the 4DCT projection obtained from the sample cone-beam CT system scan, with a size of [size missing]. The number of viewpoints, V = 720, is the total number of timestamps.
[0125] The second step involves setting the training duration to 500 epochs, with each epoch containing 720 iterations. For each iteration, one frame is randomly selected from the 720 projection frames, and 1024 projection data points are randomly selected on the C×D two-dimensional projection data. The X-ray path (the line connecting the detector and the light source) of each projection data point is calculated to ensure its overlap with the reconstructed region. Several spatial coordinate points to be queried are then selected on these line segments according to a specific method. For example, if the absolute value of the X-coordinate of the light source is greater than its absolute value of the Y-coordinate, query points are selected on all overlapping line segments at the same X-coordinate interval, with the interval set to the pixel size in the X direction of the region to be reconstructed. Otherwise, select the query points on all overlapping line segments at the same Y-coordinate interval, with the interval set to the Y-direction pixel size of the region to be reconstructed. Input the spatial coordinates and timestamps of all query points into the aforementioned defined model to obtain a series of predicted values for the line attenuation coefficient.
[0126] The third step is to calculate the projection values of the proposed model prediction results at each projection data point location based on the line integral rendering equation of CT projection, i.e., the projection values predicted by the model. For example, for a line segment on the X-ray path corresponding to any projection data point, the predicted value of the line attenuation coefficient of each query point is multiplied by the distance between adjacent query points, and then summed to obtain the predicted projection value.
[0127] The fourth step involves substituting the model-predicted projected values, the true projected labels, and the model-predicted intermediate variables into the loss function. For example, the loss function includes a data fidelity term and a regularization term. The data fidelity term is defined as the 1-norm of the difference between the model-predicted projected values and the true projected labels, and the regularization term is defined as the final output of the motion field INR in step 8-3, which is the deformation motion offset. The 2-norm. Taking a data fidelity term with a weight of 1 and a regularization term with a weight of 0.1 as an example, the loss function result is substituted into the deep learning optimizer, and the model parameters are updated through backpropagation; in this embodiment, the Adam optimizer can be used, and the learning rate can be set to 3e-3.
[0128] (6) 4DCT inference after training:
[0129] After training, this embodiment can be discretized according to the 4D image visualization requirements. Calculate the 4D coordinates of each pixel. For a given dimension, if the size is N, then the N coordinates of that dimension are: The 4D coordinates of each pixel are input into the trained model to obtain the predicted line attenuation coefficient for that pixel. Finally, all values are arranged by index into a discrete 4D image form.
[0130] In summary, the four-dimensional CT image reconstruction method proposed in this application can directly calculate any single location by constructing and applying a reconstruction model containing a feature encoding module with learnable features. This eliminates the need to first calculate the complete static image and four-dimensional motion field before performing deformation operations, reducing computational costs, lowering the difficulty of solving the problem, overcoming the limitations of finite angle reconstruction, and thus improving the accuracy of high temporal resolution reconstruction. At the same time, it increases the applicability of the model to different application scenarios, thereby improving the user experience and meeting practical needs.
[0131] Next, the four-dimensional CT image reconstruction apparatus according to the embodiments of this application is described with reference to the accompanying drawings.
[0132] Figure 5 This is a block diagram of a four-dimensional CT image reconstruction device according to an embodiment of this application.
[0133] like Figure 5 As shown, the four-dimensional CT image reconstruction device 10 includes: an acquisition module 100, an extraction module 200, and a reconstruction module 300.
[0134] The system includes an acquisition module 100 for acquiring projection data of the four-dimensional computed tomography (CT) scan to be reconstructed; an extraction module 200 for extracting the time and spatial coordinates of the target from the projection data of the four-dimensional CT scan to be reconstructed; and a reconstruction module 300 for inputting the time and spatial coordinates into a pre-constructed reconstruction model and outputting a reconstructed image of the four-dimensional CT scan to be reconstructed. The reconstruction model includes a motion field INR composed of implicit neural representations (INRs) and a static template image INR. The motion field INR generates deformation motion offset and spatiotemporal coupling features based on the time and spatial coordinates. The static template image INR performs image reconstruction based on the spatial coordinates and / or spatiotemporal coupling features after deformation motion offset, thereby obtaining a reconstructed image of the four-dimensional CT scan to be reconstructed.
[0135] In this embodiment, the time coordinate is the time point of the physical projection data acquisition, or the motion phase signal recorded by the motion monitoring sensor; the motion field INR includes a first feature encoding module, a second feature encoding module, a coupling module, and a first neural network module, wherein the first feature encoding module is used to encode the time coordinate to obtain time features; the second feature encoding module is used to encode the spatial coordinate to obtain spatial features; the coupling module is used to couple the time features and spatial features to obtain spatiotemporal coupling features; and the first neural network module is used to convert the spatiotemporal coupling features into deformation motion offset.
[0136] In this embodiment of the application, the formula for calculating the spatiotemporal coupling characteristic is as follows:
[0137]
[0138] Where M is the sports field INR, e M This is a spatiotemporal coupled feature encoding module; Let t be a spatial vector and t be time. s(t) is the spatial feature vector, and s(t) is the temporal feature vector; This is a coupled module.
[0139] In this embodiment, the formula for calculating the deformation motion offset is:
[0140]
[0141] Where, φ M For neural network modules, This represents the offset of the deformation motion.
[0142] In this embodiment, the static template image INR includes a third feature encoding module, a second neural network module, and a non-deformation-time-varying module. The third feature encoding module is used to encode the spatial coordinates after deformation motion offset to obtain template spatial features. The non-deformation-time-varying module is used to merge the spatiotemporal coupling features obtained from the motion field INR with the template spatial features with only deformation motion when the target to be reconstructed has time-varying effects caused by factors other than deformation motion, to obtain template spatial features with additional non-deformation-time-varying features. The second neural network module is used to reconstruct the reconstructed image of the four-dimensional CT to be reconstructed based on the template spatial features with only deformation motion or with additional non-deformation-time-varying features.
[0143] In this embodiment of the application, for a target V1 with deformation motion in the four-dimensional CT image to be reconstructed, the image reconstruction representation is as follows:
[0144]
[0145] Where h is the third feature encoding module, The image reconstruction expression for a target V1 with deformation motion from a 4D CT scan is given. This is the second neural network module;
[0146] For a target V2 in the four-dimensional CT image to be reconstructed that has additional non-deformation time-varying characteristics, the image reconstruction representation is as follows:
[0147]
[0148] in, For the image reconstruction expression of a target V2 with additional non-deformation that varies over time in four-dimensional CT, h is the third feature encoding module, and φ is the third feature encoding module. V2 This is the second neural network encoding module; The vector is transformed through the non-deformation time-varying module. and Expressions that combine and concatenate elements.
[0149] It should be noted that the foregoing explanation of the four-dimensional CT image reconstruction method embodiment also applies to the four-dimensional CT image reconstruction device of this embodiment, and will not be repeated here.
[0150] The four-dimensional CT image reconstruction device proposed in the embodiments of this application can directly calculate any single location by constructing and applying a reconstruction model containing a feature encoding module with learnable features. This eliminates the need to first calculate the complete static image and four-dimensional motion field before performing deformation operations, reducing computational consumption, lowering the difficulty of solving the problem, overcoming the limitations of finite angle reconstruction, thereby improving the accuracy of high temporal resolution reconstruction. At the same time, it increases the applicability of the model to different application scenarios, thereby improving the user experience and meeting practical needs.
[0151] Figure 6 A schematic diagram of the structure of a medical device provided in an embodiment of this application. The medical device may include:
[0152] The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.
[0153] When the processor 602 executes the program, it implements the four-dimensional CT image reconstruction method provided in the above embodiments.
[0154] Furthermore, medical equipment also includes:
[0155] Communication interface 603 is used for communication between memory 601 and processor 602.
[0156] The memory 601 is used to store computer programs that can run on the processor 602.
[0157] The memory 601 may include high-speed RAM (Random Access Memory) and may also include non-volatile memory, such as at least one disk storage.
[0158] The memory 601 is used to store computer programs that can run on the processor 602.
[0159] The memory 601 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.
[0160] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0161] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.
[0162] The processor 602 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.
[0163] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described four-dimensional CT image reconstruction method.
[0164] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0165] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0166] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0167] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.
[0168] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0169] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.
Claims
1. A method for reconstructing four-dimensional CT images, characterized in that, Includes the following steps: Acquire projection data from the four-dimensional computed tomography CT scan to be reconstructed; Extract the time and spatial coordinates of the target from the projection data of the four-dimensional CT to be reconstructed; The time coordinates and spatial coordinates are input into a pre-constructed reconstruction model, and the reconstructed image of the four-dimensional CT to be reconstructed is output. The reconstruction model includes a motion field INR composed of implicit neural representations INR and a static template image INR. The motion field INR generates deformation motion offset and spatiotemporal coupling features based on the time coordinates and spatial coordinates. The static template image INR performs image reconstruction based on the spatial coordinates after deformation motion offset and / or the spatiotemporal coupling features to obtain the reconstructed image of the four-dimensional CT to be reconstructed.
2. The four-dimensional CT image reconstruction method according to claim 1, characterized in that, The time coordinates are derived from the time points of the projected data acquisition, or from the motion phase signals recorded by motion monitoring sensors; the motion field INR includes a first feature encoding module, a second feature encoding module, a coupling module, and a first neural network module, wherein... The first feature encoding module is used to encode the time coordinates to obtain time features; The second feature encoding module is used to encode the spatial coordinates to obtain spatial features; The coupling module is used to couple the temporal features and the spatial features to obtain spatiotemporal coupled features; The first neural network module is used to convert the spatiotemporal coupling features into deformation motion offsets.
3. The four-dimensional CT image reconstruction method according to claim 2, characterized in that, The calculation formula for the spatiotemporal coupling feature is as follows: ), in, For spatial vectors, t For time, For spatial feature vectors, This is a time feature vector; This is a coupled module.
4. The four-dimensional CT image reconstruction method according to claim 3, characterized in that, The formula for calculating the deformation motion offset is: , in, This is the first neural network module. This represents the offset of the deformation motion.
5. The four-dimensional CT image reconstruction method according to claim 1, characterized in that, The static template image INR includes a third feature encoding module, a second neural network module, and a non-deformation time-varying module, wherein... The third feature encoding module is used to process the spatial coordinates after the deformation motion offset. Encoding yields template space features for deformation-only motion; The non-deformation time-varying module is used to merge the spatiotemporal coupling features obtained by the motion field INR with the template space features with only deformation motion when the target to be reconstructed has time-varying effects caused by factors other than deformation motion, to obtain template space features with additional non-deformation time-varying effects. The second neural network module is used to reconstruct the reconstructed image of the four-dimensional CT to be reconstructed based on the template spatial features that are either deformable only or have additional non-deformable time-varying features.
6. The four-dimensional CT image reconstruction method according to claim 5, characterized in that, For the target with deformation motion in the four-dimensional CT image to be reconstructed The image reconstruction representation is as follows: , in, This is the third feature encoding module. For targets with deformation motion The image reconstruction expression for four-dimensional CT, This is the second neural network module; For the target in the four-dimensional CT image to be reconstructed that exhibits additional non-deformation over time... The image reconstruction representation is as follows: , in, For targets with additional non-deformation that change over time The image reconstruction expression for four-dimensional CT, This is the third feature encoding module. This is the second neural network module; The vector is transformed through the non-deformation time-varying module. and Expressions that combine and concatenate elements.
7. A four-dimensional CT image reconstruction device, characterized in that, include: The acquisition module is used to acquire the projection data of the four-dimensional computed tomography CT scan to be reconstructed; The extraction module is used to extract the time and spatial coordinates of the target in the projection data of the four-dimensional CT to be reconstructed; The reconstruction module is used to input the time coordinates and spatial coordinates into a pre-constructed reconstruction model and output the reconstructed image of the four-dimensional CT to be reconstructed. The reconstruction model includes a motion field INR composed of implicit neural representations INR and a static template image INR. The motion field INR generates deformation motion offset and spatiotemporal coupling features based on the time coordinates and spatial coordinates. The static template image INR performs image reconstruction based on the spatial coordinates after deformation motion offset and / or the spatiotemporal coupling features to obtain the reconstructed image of the four-dimensional CT to be reconstructed.
8. The four-dimensional CT image reconstruction device according to claim 7, characterized in that, The time coordinates are derived from the time points of the projected data acquisition, or from the motion phase signals recorded by motion monitoring sensors; the motion field INR includes a first feature encoding module, a second feature encoding module, a coupling module, and a first neural network module, wherein... The first feature encoding module is used to encode the time coordinates to obtain time features; The second feature encoding module is used to encode the spatial coordinates to obtain spatial features; The coupling module is used to couple the temporal features and the spatial features to obtain spatiotemporal coupled features; The first neural network module is used to convert the spatiotemporal coupling features into deformation motion offsets.
9. The four-dimensional CT image reconstruction device according to claim 8, characterized in that, The calculation formula for the spatiotemporal coupling feature is as follows: ), in, It is characterized by spatiotemporal coupling. For spatial vectors, t For time, For spatial feature vectors, This is a time feature vector; This is a coupled module.
10. The four-dimensional CT image reconstruction device according to claim 9, characterized in that, The formula for calculating the deformation motion offset is: , in, This is the first neural network module. This represents the offset of the deformation motion.
11. The four-dimensional CT image reconstruction device according to claim 7, characterized in that, The static template image INR includes a third feature encoding module, a second neural network module, and a non-deformation time-varying module, wherein... The third feature encoding module is used to process the spatial coordinates after the deformation motion offset. Encoding is performed to obtain template space features; The non-deformation time-varying module is used to merge the spatiotemporal coupling features obtained by the motion field INR with the template space features with only deformation motion when the target to be reconstructed has time-varying effects caused by factors other than deformation motion, to obtain template space features with additional non-deformation time-varying effects. The second neural network module is used to reconstruct the reconstructed image of the four-dimensional CT to be reconstructed based on the template spatial features that are either deformable only or have additional non-deformable time-varying features.
12. The four-dimensional CT image reconstruction device according to claim 11, characterized in that, For the target with deformation motion in the four-dimensional CT image to be reconstructed The image reconstruction representation is as follows: , in, This is the third feature encoding module. For targets with deformation motion The image reconstruction expression for four-dimensional CT, This is the second neural network module; For the target in the four-dimensional CT image to be reconstructed that exhibits additional non-deformation over time... The image reconstruction representation is as follows: , in, For targets with additional non-deformation that change over time The image reconstruction expression for four-dimensional CT, This is the third feature encoding module. This is the second neural network module; The vector is transformed through the non-deformation time-varying module. and Expressions that combine and concatenate elements.
13. A medical device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the four-dimensional CT image reconstruction method as described in any one of claims 1-6.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the four-dimensional CT image reconstruction method as described in any one of claims 1-6.
