Method and system for performing three-dimensional reconstruction on sparse-angle cone beam projection data

Abstract

The invention mainly relates to the technical field of ray imaging. In order to solve the problems that in the prior art, when three-dimensional reconstruction is carried out on cone beam projection data, the requirement for the computing power of equipment is high, and the operation process is complex, the invention provides a method and device for carrying out three-dimensional reconstruction on the cone beam projection data with sparse angles, and the core idea is that the method comprises the following steps: collecting the cone beam projection data of a target detection object; preprocessing the collected cone beam projection data of the sparse angle to respectively obtain an image coding feature vector and a sparse convolution feature vector; splicing the image coding feature vector and the sparse convolution feature vector, and inputting the spliced image coding feature vector and sparse convolution feature vector into a decoding network to obtain a decoding feature vector; inputting the decoding feature vector into an output network, and outputting a shape three-dimensional matrix and a radiation absorption value three-dimensional matrix of the detected object by the output network; and filtering the shape three-dimensional matrix and the radiation absorption value three-dimensional matrix based on a set threshold to obtain a three-dimensional reconstruction result of the cone beam projection data.

Description

Technical Field

[0001] The present invention mainly relates to the field of ray imaging technology, and in particular to a method and system for three-dimensional reconstruction of sparse angle cone beam projection data. Background Art

[0002] Cone beam computed tomography (CBCT) was born in the 1990s and has been widely used in the fields of medical imaging and industrial detection technology. In medical imaging detection, the main advantages of CBCT compared to traditional parallel beam and fan beam CT are low radiation dose, high resolution and few artifacts. Once it came out, it quickly became an important tool in the field of medical imaging, especially in stomatology and radiotherapy. The cone beam 3D reconstruction task refers to the use of cone beam projection data from multiple angles to reconstruct the 3D data of the corresponding part of the patient, which reflects the patient's tissue structure morphology and tissue density. Sparse angle cone beam 3D reconstruction refers to the reconstruction using a small amount of cone beam projection data. Traditional 3D reconstruction algorithms are mainly divided into several categories of methods, such as filtered back projection algorithm (FDK), algebraic iteration algorithm, total variation algorithm and Krylov subspace algorithm.

[0003] The filtered back-projection algorithm mainly uses the Fourier slice theorem for reconstruction; it is simple to implement and works well when there are many projection angles, but the reconstruction effect is not clear enough under sparse angle conditions, and the image quality is easily affected by noise. The algebraic iterative algorithm gradually approximates the result through multiple iterations, can handle incomplete data and noise, has better noise resistance and artifact suppression capabilities, and is currently the gold standard algorithm for three-dimensional reconstruction. Typical reconstruction algorithms include ART, SART, OS_SART, etc.; but the disadvantages of the iterative algorithm are that the calculation is large, time-consuming, and requires more computing resources, which limits its practical application. The total variation algorithm introduces total variation constraints on the basis of the filtered back-projection algorithm, and optimizes the reconstructed image by minimizing the total variation of the image. Typical algorithms include ASD_POCS, etc. The disadvantage is that it is sensitive to initial conditions and requires multiple iterations to achieve the ideal effect. The Krylov subspace algorithm is a method for solving linear equations. It approximates the true solution by iteratively constructing a residual vector. It is suitable for solving large-scale linear systems. Typical algorithms include LSQR and CGLS. Its disadvantages are that it is sensitive to initial conditions and the number of iterations, has high computational complexity, and takes a long time to rebuild.

[0004] At present, deep learning models have achieved considerable success in the fields of natural language, image recognition, speech synthesis, etc. due to their advantages such as high accuracy, fast computing speed and ability to process complex data. Therefore, some deep learning models have also emerged in the field of cone beam 3D reconstruction. At present, the algorithms for cone beam 3D reconstruction using deep learning can be roughly divided into two categories: enhancement algorithms based on deep learning and direct reconstruction algorithms based on deep learning. The enhancement algorithm based on deep learning first uses the traditional reconstruction algorithm to perform 3D reconstruction, and then slices the reconstruction results into pictures and uses the deep learning model to enhance them one by one. The models used include generation models such as cycle generative adversarial networks (Cycle-GAN) and diffusion models, or super-resolution models. The direct reconstruction algorithm based on deep learning does not use the traditional algorithm for reconstruction, but directly obtains the final reconstruction result. The models used include multi-layer perceptron, neural radiation field model (NeRF), Guassian Splatting model, etc. However, most direct reconstruction algorithms based on deep learning have high requirements for computing power, and practical applications have high requirements for device computing power. At the same time, due to the complexity of the model, most of them are difficult to process large-size projection data, which will lead to problems such as data explosion and model dimensionality disaster. Summary of the invention

[0005] Technical Problems to be Solved by the Invention

[0006] A method and system for three-dimensional reconstruction of sparse angle cone beam projection data are provided to solve the problem that the prior art has high requirements on equipment computing power and complex calculation process for sparse angle cone beam three-dimensional reconstruction.

[0007] The technical solution adopted by the present invention to solve the above technical problems

[0008] In one aspect, the present invention provides a method for three-dimensional reconstruction of sparse angle cone beam projection data, comprising:

[0009] Step S1: collecting cone beam projection data of the target detection object, and preprocessing the collected sparse angle cone beam projection data;

[0010] Step S2: extracting the image coding feature vector of the preprocessed cone beam projection data;

[0011] Step S3: performing three-dimensional reconstruction on the preprocessed cone beam projection data based on a filtered back projection algorithm to obtain a preliminary reconstructed three-dimensional matrix, and obtaining a sparse convolution feature vector based on the preliminary reconstructed three-dimensional matrix;

[0012] Step S4: concatenate the image coding feature vector and the sparse convolution feature vector and input them into the decoding network to obtain a decoding feature vector;

[0013] Step S5: input the decoded feature vector into the output network, and the output network outputs the three-dimensional matrix of the shape of the target detection object and the three-dimensional matrix of the radiation absorption value;

[0014] Step S6: filtering the shape three-dimensional matrix and the radiation absorption value three-dimensional matrix based on a set threshold value to obtain a three-dimensional reconstruction result of the cone beam projection data of the target detection object.

[0015] Furthermore, the preprocessing in step S1 includes: scaling, data normalization, outlier removal, noise addition, padding and coordinate transformation.

[0016] Further, step S2 includes:

[0017] Select an image coding network and set the length of the image coding feature vector output by the image coding network;

[0018] stacking the preprocessed cone beam projection data according to the collection angle of the cone beam projection data;

[0019] The stacked conical projection data is input into the image coding network, and the image coding network outputs the image feature coding vector according to the set image coding feature vector length value.

[0020] Furthermore, the image encoding network is any one of Unet, Segnet or AutoEncoder deep learning networks.

[0021] Furthermore, the method for obtaining the preliminary reconstructed three-dimensional matrix in step S3 includes:

[0022] Performing filtering on the pre-processed cone beam projection data;

[0023] Performing back-projection calculation on the filtered cone beam projection data, and back-projecting each cone beam projection data onto a plane according to the cone beam projection data acquisition angle;

[0024] All the planes obtained by back-projection are superimposed to obtain a preliminary reconstructed three-dimensional matrix.

[0025] Furthermore, in step S3, the preliminary reconstructed three-dimensional matrix is ​​input into a backbone network module to obtain a sparse convolution feature vector; the backbone network module includes a plurality of structural levels composed of a combination of a 3D sparse convolution layer, an attention layer and a pooling layer.

[0026] Furthermore, the decoding network in step S4 includes a 3D sparse convolution layer, a 3D dilute deconvolution layer, a pooling layer and a linear layer.

[0027] Furthermore, the output network in step S5 includes a convolutional layer and a linear layer, and the linear layer is provided with two output heads, which are respectively used to output a three-dimensional matrix of the shape of the target detection object and a three-dimensional matrix of the radiation absorption value.

[0028] Further, step S6 includes: first using a preset threshold to filter the shape three-dimensional matrix to obtain the target detection object shape, and then using the filtered target detection object shape to filter the radiation absorption value three-dimensional matrix to obtain a three-dimensional reconstruction result of the cone beam projection data of the target detection object.

[0029] On the other hand, the present invention further provides a system for performing three-dimensional reconstruction on sparse angle cone beam projection data, the system for performing three-dimensional reconstruction on sparse angle cone beam projection data comprising:

[0030] A preprocessing module, used for preprocessing the collected sparse angle cone beam projection data;

[0031] An image coding module, used for extracting an image coding feature vector of the preprocessed cone beam projection data;

[0032] A filter back projection algorithm module performs three-dimensional reconstruction on the pre-processed cone beam projection data based on the filter back projection algorithm to obtain a preliminary reconstructed three-dimensional matrix;

[0033] The backbone network module obtains the diluted convolution feature vector based on the obtained preliminary reconstructed three-dimensional matrix;

[0034] A decoding module, including a decoding network, for concatenating the image encoding feature vector and the diluted convolution feature vector, and inputting the concatenated feature vector into the decoding network to obtain a decoding feature vector;

[0035] An output module, including an output network, wherein the output network is provided with two output heads, and is used to output a three-dimensional matrix of a shape of the target detection object and a three-dimensional matrix of a radiation absorption value according to a decoding feature vector output by the decoding module;

[0036] The filtering module is used to filter the shape three-dimensional matrix according to a preset threshold value to obtain the shape of the target detection object, and then use the filtered target detection object shape to filter the radiation absorption value three-dimensional matrix to obtain the three-dimensional reconstruction result of the cone beam projection data of the target detection object.

[0037] Beneficial Effects of the Invention

[0038] The method and system for performing three-dimensional reconstruction on sparse angle cone-beam projection data described in the present invention adopt an image coding network and a diluted convolutional network in deep learning to perform three-dimensional reconstruction on sparse angle cone-beam projection data. It can fully integrate the two-dimensional features of the projection data and the three-dimensional features corresponding to the object, does not use additional auxiliary information, can process large-size projection data, does not have high requirements on the computing power and other performance of the equipment, and can effectively improve the data processing efficiency of the three-dimensional reconstruction process of sparse angle cone-beam projection data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flow chart of a method for three-dimensional reconstruction of sparse angle cone beam projection data according to the present invention;

[0040] Figure 2 It is a schematic diagram of a device for three-dimensional reconstruction of sparse angle cone beam projection data according to the present invention. DETAILED DESCRIPTION

[0041] In one aspect, the present invention provides a method for three-dimensional reconstruction of sparse angle cone beam projection data, the specific steps are as follows: Figure 1 shown.

[0042] Step S1: setting the cone beam projection data acquisition parameters of the target detection object, specifically including the resolution of the cone beam detector, the size of the three-dimensional reconstruction result, the source-to-object distance (DSO), the source-to-detector distance (DSD), the projection angle, etc. After setting the above-mentioned cone beam projection data acquisition parameters, the cone beam projection data of the target detection object is acquired based on the cone beam detector, and the collected sparse angle cone beam projection data of M angles are pre-processed, wherein the collection angles of the cone beam projection data are no more than 30, and the pre-processing of the collected sparse angle cone beam projection data specifically includes: scaling the size of the cone beam projection data, normalizing the data, removing outliers, adding noise, filling and coordinate transformation processing into a standard image that can be recognized by the next image encoding network and filter projection algorithm.

[0043] Step S2: extract the preprocessed cone beam projection data and input it into the image coding network, and the image coding network outputs the image coding feature vector of the preprocessed cone beam projection data.

[0044] In this embodiment, the image encoding network is any one of AutoEncoder network, SegNet, and Unet.

[0045] Step S3: Perform three-dimensional reconstruction on the preprocessed cone beam projection data based on a filtered back projection algorithm to obtain a preliminary reconstructed three-dimensional matrix, and obtain a sparse convolution feature vector based on the preliminary reconstructed three-dimensional matrix.

[0046] The method of initially reconstructing the three-dimensional matrix includes:

[0047] The preprocessed cone beam projection data are filtered; the filtered cone beam projection data are back-projected, and each cone beam projection data is back-projected onto a plane according to the angle at which the cone beam projection data is collected; all the planes obtained by back-projection are superimposed to obtain a preliminary reconstructed three-dimensional matrix.

[0048] The preliminary reconstructed three-dimensional matrix is ​​input into the backbone network module to obtain a sparse convolution feature vector. The backbone network module includes multiple structural layers composed of 3D sparse convolution layers, attention layers and pooling layers. The number of structural layers of the backbone network can be flexibly set according to needs. The reconstructed three-dimensional matrix is ​​input into the backbone network. After the input data passes through each layer of the backbone network stack, a sparse convolution feature vector is obtained. The length of the output sparse convolution feature vector needs to be set in advance in the backbone network model.

[0049] Step S4: The image coding feature vector and the sparse convolution feature vector are concatenated and input into a decoding network to obtain a decoding feature vector. The decoding network is composed of a combination of multiple 3D sparse convolution layers, 3D sparse deconvolution layers and pooling layers.

[0050] Step S5: The decoded feature vector is input into the output network, and the output network outputs the three-dimensional matrix of the shape and the three-dimensional matrix of the radiation absorption value of the target detection object; the output module is composed of convolution and linear layers, and the final output has two output heads, which are used to output the three-dimensional matrix of the shape and the three-dimensional matrix of the radiation absorption value respectively.

[0051] Step S6 includes: firstly filtering the shape three-dimensional matrix using a preset threshold to obtain the shape of the target object, and then filtering the radiation absorption value three-dimensional matrix using the filtered target object shape to obtain a three-dimensional reconstruction result of the cone beam projection data of the target object.

[0052] On the other hand, based on the above method for performing three-dimensional reconstruction on sparse angle cone beam projection data, the present invention also provides a system for performing three-dimensional reconstruction on sparse angle cone beam projection data, such as Figure 2 As shown, the system includes a preprocessing module, an image encoding module, a filtered back projection algorithm module, a backbone network module, a decoding module, an output module, and a filtering module;

[0053] The preprocessing module is used to preprocess the collected sparse angle cone beam projection data;

[0054] The image coding module is used to extract the image coding feature vector of the preprocessed cone beam projection data;

[0055] The filtered back projection algorithm module performs three-dimensional reconstruction on the pre-processed cone beam projection data based on the filtered back projection algorithm to obtain a preliminary reconstructed three-dimensional matrix;

[0056] The backbone network module obtains a diluted convolution feature vector based on the obtained preliminary reconstructed three-dimensional matrix;

[0057] The decoding module includes a decoding network, which is used to concatenate the image coding feature vector and the diluted convolution feature vector, and input the concatenated feature vector into the decoding network to obtain a decoding feature vector.

[0058] The output module includes an output network, wherein the output network is provided with two output heads, and is used to output a three-dimensional matrix of the shape and a three-dimensional matrix of the radiation absorption value of the target detection object according to the decoded feature vector output by the decoding module;

[0059] The filtering module is used to filter the shape three-dimensional matrix according to a preset threshold to obtain the shape of the target detection object, and then use the filtered target detection object shape to filter the radiation absorption value three-dimensional matrix to obtain the three-dimensional reconstruction result of the cone beam projection data of the target detection object.

Claims

1. A method for three-dimensional reconstruction of sparse angle cone beam projection data, characterized in that: include: Step S1: collecting cone beam projection data of the target detection object, and preprocessing the collected sparse angle cone beam projection data; Step S2: extracting the image coding feature vector of the preprocessed cone beam projection data; Step S3: performing three-dimensional reconstruction on the preprocessed cone beam projection data based on a filtered back projection algorithm to obtain a preliminary reconstructed three-dimensional matrix, and obtaining a sparse convolution feature vector based on the preliminary reconstructed three-dimensional matrix; Step S4: concatenate the image coding feature vector and the sparse convolution feature vector and input them into the decoding network to obtain a decoding feature vector; Step S5: input the decoded feature vector into the output network, and the output network outputs the three-dimensional matrix of the shape of the target detection object and the three-dimensional matrix of the radiation absorption value; Step S6: filtering the shape three-dimensional matrix and the radiation absorption value three-dimensional matrix based on a set threshold value to obtain a three-dimensional reconstruction result of the cone beam projection data of the target detection object.

2. The method for three-dimensional reconstruction of sparse angle cone beam projection data according to claim 1, characterized in that: The preprocessing in step S1 includes: scaling, data normalization, outlier removal, noise addition, filling and coordinate transformation.

3. The method for three-dimensional reconstruction of sparse angle cone beam projection data according to claim 1, characterized in that: Step S2 includes: Select an image coding network and set the length of the image coding feature vector output by the image coding network; stacking the preprocessed cone beam projection data according to the collection angle of the cone beam projection data; The stacked conical projection data is input into the image coding network, and the image coding network outputs the image feature coding vector according to the set image coding feature vector length value.

4. The method for three-dimensional reconstruction of sparse angle cone beam projection data according to claim 3, characterized in that: The image encoding network is any one of Unet, Segnet or AutoEncoder deep learning networks.

5. The method for three-dimensional reconstruction of sparse angle cone beam projection data according to claim 1, characterized in that: The method for obtaining the preliminary reconstructed three-dimensional matrix in step S3 includes: Performing filtering on the pre-processed cone beam projection data; Performing back-projection calculation on the filtered cone beam projection data, and back-projecting each cone beam projection data onto a plane according to the cone beam projection data acquisition angle; All the planes obtained by back-projection are superimposed to obtain a preliminary reconstructed three-dimensional matrix.

6. The method for three-dimensional reconstruction of sparse angle cone beam projection data according to claim 1, characterized in that: In step S3, the preliminary reconstructed three-dimensional matrix is ​​input into a backbone network module to obtain a sparse convolution feature vector; the backbone network module includes a plurality of structural levels composed of a combination of a 3D sparse convolution layer, an attention layer and a pooling layer.

7. The method for three-dimensional reconstruction of sparse angle cone beam projection data according to any one of claims 1 to 6, characterized in that: The decoding network in step S4 includes a 3D sparse convolution layer, a 3D dilute deconvolution layer, a pooling layer and a linear layer.

8. The method for three-dimensional reconstruction of sparse angle cone beam projection data according to any one of claims 1 to 6, characterized in that: The output network in step S5 includes a convolutional layer and a linear layer. The linear layer is provided with two output heads, which are used to output a three-dimensional matrix of the shape of the target detection object and a three-dimensional matrix of the radiation absorption value, respectively.

9. The method for three-dimensional reconstruction of sparse angle cone beam projection data according to any one of claims 1 to 6, characterized in that: Step S6 includes: firstly filtering the shape three-dimensional matrix using a preset threshold to obtain the shape of the target object, and then filtering the radiation absorption value three-dimensional matrix using the filtered target object shape to obtain the three-dimensional reconstruction result of the cone beam projection data of the target object.

10. A system for three-dimensional reconstruction of sparse angle cone beam projection data, used to implement the method for three-dimensional reconstruction of sparse angle cone beam projection data according to any one of claims 1 to 9, characterized in that: include: A preprocessing module, used for preprocessing the collected sparse angle cone beam projection data; An image coding module, used for extracting an image coding feature vector of the preprocessed cone beam projection data; A filter back projection algorithm module performs three-dimensional reconstruction on the pre-processed cone beam projection data based on the filter back projection algorithm to obtain a preliminary reconstructed three-dimensional matrix; The backbone network module obtains the diluted convolution feature vector based on the obtained preliminary reconstructed three-dimensional matrix; The decoding module includes a decoding network, which is used to concatenate the image encoding feature vector and the diluted convolution feature vector, and input the concatenated feature vector into the decoding network to obtain a decoding feature vector; The output module includes an output network, and the output network is provided with two output heads, which are respectively used to output a three-dimensional matrix of the shape of the target detection object and a three-dimensional matrix of the radiation absorption value according to the decoding feature vector output by the decoding module; The filtering module is used to filter the shape three-dimensional matrix according to a preset threshold value to obtain the shape of the target detection object, and then use the filtered target detection object shape to filter the radiation absorption value three-dimensional matrix to obtain the three-dimensional reconstruction result of the cone beam projection data of the target detection object.

Images