A learning-based tomographic imaging and reconstruction method

Abstract

The application provides a learning-based tomographic imaging and reconstruction method to measure the scene density distribution in a light reuse manner. During imaging, the light source emits light according to the intensity obtained through pre-learning, the light from different directions reaches the sensor after being absorbed and attenuated by the scene, and the density information of the scene is obtained by calculating and reconstructing the measurement value. The light emission intensity and the reconstruction algorithm are obtained by learning through a neural network. The method models the CT imaging process as a linear fully connected layer, and the weight corresponds to the light emission intensity of the light source during imaging; the reconstruction algorithm is modeled as a nonlinear neural network, which can be optimized according to the characteristics of the scanning geometry. The method requires a small amount of collected data, and the calculation and reconstruction do not require strong prior assumptions, realizes efficient and high-quality CT acquisition and reconstruction, and can be applied to high-speed dynamic scene three-dimensional reconstruction.

Description

Technical Field

[0001] This invention relates to the field of computed tomography (CT) technology, and more particularly to imaging and reconstruction methods. Background Technology

[0002] Computed tomography (CT) is an important research area in medical imaging and computer vision graphics. This technology reconstructs the internal structure of objects by measuring the amount of light absorbed by a scene in different directions. It has a wide range of applications, including medical imaging, industrial monitoring, aviation security, and cultural relic preservation.

[0003] In particular, applying tomography to dynamic scenes has immense scientific and practical value. For example, mechanical inspection and medical diagnosis both require 3D reconstruction of high-speed dynamic scenes. However, extending traditional tomography to dynamic scenes presents a key challenge: high-quality reconstruction results often rely on dense sampling in different directions. When the scene changes rapidly, this dense sampling process must be completed within a short time to avoid ghosting issues and ensure reconstruction quality. This necessitates tomography for dynamic scenes requiring sampling capabilities far exceeding those of traditional methods.

[0004] Over the past few decades, numerous studies have proposed various algorithms for tomographic acquisition and reconstruction of dynamic scenes. For specific dynamic phenomena, the properties of the observed scene can be utilized to solve the problem. For example, Chen et al. proposed a scan reconstruction algorithm for periodic cardiac motion (Chen Guang-Hong, Theriault-Lauzier Pascal, Tang Jie, Nett Brian, Leng Shuai, Zambelli Joseph, Qi Zhihua, Bevins Nicholas, Raval Amish, Reeder Scott. 2011. Time-resolved interventional cardiac C-arm cone-beamCT: An application of the PICCS algorithm. IEEE transactions on medical imaging. 31, 4, 907-923). However, these methods are limited to the characteristics of certain types of scenes and lack universality. Some studies have attempted to improve sampling speed by reducing the number of point source samples and making strong prior assumptions to calculate reconstructions. However, these assumptions limit the application scenarios of the methods (Zang Guangming, Idoughi Ramzi, Wang Congli, Bennett Anthony, Du Jianguo, Skeen Scott, Roberts William L., Wonka Peter, Heidrich Wolfgang. 2020. TomoFluid: reconstructing dynamic fluid from sparse viewvideos. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 1870-1879). Therefore, there is an urgent need to propose tomographic acquisition and reconstruction algorithms for general dynamic scenes. Summary of the Invention

[0005] The purpose of this invention is to address the problems of insufficient acquisition capabilities and the need for strong prior assumptions in reconstruction of existing tomographic scanning methods in dynamic scenarios, and to propose a method that significantly improves acquisition capabilities and is applicable to general scenarios.

[0006] To achieve the aforementioned objectives, this invention proposes a learning-based tomographic imaging and reconstruction method that measures scene density distribution using illumination multiplexing, comprising the following steps:

[0007] (1) Generate training data: Obtain the parameters of the scanning device, including the spatial position of the light source and the sensor, and all sensor readings when any light source emits light at maximum intensity in an empty scene; use these parameters to generate a CT image of a single light source emitting light through the scene to reach the sensor, as training data.

[0008] (2) Train the neural network based on the data generated in step (1); the characteristics of the neural network are as follows:

[0009] a. Input consists of CT images from all light sources of the scanning device;

[0010] b. The output is the corresponding density field;

[0011] c. The first layer of the neural network is a fully connected linear layer. The parameter matrix of the fully connected linear layer is obtained by the following formula:

[0012] W l =f w (W raw )

[0013] Among them W raw The parameters to be trained; W l The illumination intensity matrix corresponding to the imaging process has a size of k×n. s n s f is the number of light sources in the scanning device, k is the number of samples; w For a mapping, used for W raw Perform a transformation to generate the illumination intensity matrix W. l It can correspond to the possible luminous intensity of the light source;

[0014] d. The second and subsequent layers are non-linear neural networks, and the output of the last layer is the density field reconstruction result; after training, the illumination intensity matrix W of the first linear fully connected layer is... l take out;

[0015] (3) The light source of the scanning device emits light row by row according to the illumination intensity matrix obtained in step (2), illuminating the target scene in sequence, and obtains the measurement value matrix M through the sensor, which has a size of k×n. d n d M represents the number of sensors; M is used as the output of the first linear fully connected layer of the neural network to calculate the reconstructed density field.

[0016] Further, in step (1), the specific method for generating CT images is as follows: randomly place several objects with different densities in the effective area of ​​the scene, and generate CT images based on the selected light model according to the calibrated light source and sensor positions.

[0017] Furthermore, the light ray model is a linear absorption model, with the following formula:

[0018]

[0019] Where I is a matrix composed of CT images from different light sources, with a size of n. s ×n d , element I in I ij Let x be the reading of the j-th sensor when the i-th light source emits light at maximum intensity under a given density field, and let x be the vector of the density field discretized into voxels, with a length equal to the number of voxels n. v K represents the Radon transform expressed in terms of a third-order tensor, ×3 represents the mode-3 product of a tensor and a vector, and ⊙ represents the element-wise multiplication of matrices. Let I be the density field when it is zero everywhere.

[0020] Furthermore, in step (2), the relationship between the linear fully connected layer and the input is as follows:

[0021] M = W l I

[0022] Where I is a matrix composed of CT images from different light sources, with a size of n. s ×n d .

[0023] Furthermore, in step (2), the neural network used for reconstruction can be represented as follows:

[0024] x nn =f recon (D nn )=f recon (f nn (M))

[0025] Where f nn Map M to SinogramD nn Then, the tomographic reconstruction method was used. recon The density field reconstruction result x is obtained nn .

[0026] Furthermore, the tomographic reconstruction method can be implemented using a filtered backprojection method.

[0027] Furthermore, in step (2), the loss function used for training... It can be represented as follows:

[0028]

[0029]

[0030] in Used to evaluate the quality of density field reconstruction Used to give luminous intensity specific properties, g w λ is a function for practically evaluating the properties of luminescence intensity. r With λ p Used to balance the weights between different loss functions.

[0031] Furthermore, the loss function used to evaluate the quality of the density field reconstruction. It can be represented as:

[0032]

[0033] Furthermore, the function g that actually evaluates the properties of luminescence intensity... w The design can be tailored to different scenarios. Two design approaches are given below, but they are not the only ones:

[0034] a. For dynamic scenarios requiring high-speed scanning, let g w (W l )=-∑|W l |, prompting W l The values ​​tend to become binary.

[0035] b. For scenarios requiring low-dose scanning, let g w (W l )=∑|W l |, prompting W l The value tends to be minimized.

[0036] The beneficial effects of this invention are as follows: This invention obtains a tomographic imaging and reconstruction method based on illumination multiplexing through learning. Compared with traditional point-source tomographic methods for dynamic scenes, it can sample the entire light space, significantly improving acquisition efficiency. The reconstruction algorithm does not require strong prior assumptions and is applicable to general scenes. Compared with traditional illumination multiplexing imaging methods, the luminous intensity and reconstruction algorithm used are learned by neural networks, requiring far fewer inputs than traditional methods, resulting in high reconstruction quality. Under the same hardware conditions, the proposed tomographic scanning method can acquire and reconstruct dynamic scenes that traditional methods cannot achieve. Furthermore, this method is not limited to a specific scanning device and has high universality. Attached Figure Description

[0037] Figure 1 A schematic diagram of the learning-based illumination multiplexing imaging reconstruction process;

[0038] Figure 2 Side view of the scanning device used in the embodiment;

[0039] Figure 3Top view and visual layout diagram of the acquisition module of the scanning device used in the embodiment;

[0040] Figure 4 In this embodiment, a CT image is generated using training data from a specific light source;

[0041] Figure 5 A CT image of a specific light source in an empty scene, as shown in the embodiment;

[0042] Figure 6 A schematic diagram of the network structure in the embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] This invention proposes a learning-based tomographic imaging and reconstruction method that measures scene density distribution using illumination multiplexing. The steps are as follows: Figure 1 As shown, a detailed explanation follows:

[0045] (1) Generate training data: Obtain the parameters of the scanning device, including the spatial position of the light source and the sensor, and the readings of all sensors when any light source emits light at maximum intensity in an empty scene; use these parameters to generate the acquisition results of the scanning device as training data.

[0046] Specifically, the scanning device includes at least two light sources and at least one sensor, each sensor being able to receive light from multiple light sources simultaneously, and the intensity of all light sources is adjustable; the target scene is located in part or all of the optical path formed by the light sources and the sensors.

[0047] In one embodiment, the scene is detected using visible light. Figure 2 The side view of the scanning equipment used is shown. Optical fibers are used to transmit the light emitted by the LED array to the acquisition module as a light source. The receiver part of the acquisition module receives the light and also transmits it to the sensor array via optical fibers, where a camera above takes a picture. Figure 3 The diagram shows a top-down view and a visual layout of the acquisition module. In this embodiment, 24 acquisition modules are connected in a ring, numbered #0, #1, ..., #23. Each acquisition module has 10 rows and 16 columns, with odd-numbered columns representing light sources and even-numbered columns representing sensors. The effective acquisition area is 32×128×128 pixels, corresponding to a voxel resolution of 32×128×128. Figure 4 and Figure 5 These are CT images generated using a single light source in a scanning device, in scenes with objects and in empty scenes, respectively.

[0048] Different ray models can be used to generate CT images. In this embodiment, a linear absorption model is used, but it is not limited to this. The formula for the linear absorption model is as follows:

[0049]

[0050] Where I is a matrix composed of CT images from different light sources, with a size of n. s ×n d , element I in I ij Let x be the reading of the j-th sensor when the i-th light source emits light at maximum intensity under a given density field, and let x be the vector of the density field discretized into voxels, with a length equal to the number of voxels n. v K represents the Radon transform expressed in terms of a third-order tensor, ×3 represents the mode-3 product of a tensor and a vector, and ⊙ represents the element-wise multiplication of matrices. I is the density field that is zero everywhere. elements in

[0051] The reading of the j-th sensor when the i-th light source emits light at its maximum intensity in an empty scene.

[0052] (2) Train the neural network based on the training data generated in step (1). The neural network used in this embodiment... Figure 6 The diagram illustrates, but is not limited to, convolutional layers unless otherwise specified. A detailed description of the neural network used is as follows:

[0053] a. Input consists of CT images from all light sources of the scanning device;

[0054] b. The output is the corresponding density field;

[0055] c. The first layer of the neural network is a fully connected linear layer. The parameter matrix of the fully connected linear layer is obtained using the following formula:

[0056] W l =f w (W raw )

[0057] Among them W raw The parameters to be trained; W l The illumination intensity matrix corresponding to the imaging process has a size of k×n. s n s f is the number of light sources in the scanning device, k is the number of samples; w For a mapping, used for W raw Perform a transformation to generate the illumination intensity matrix W. l This corresponds to the possible luminous intensity of the light source; in this embodiment, fw Choosing the Sigmoid function, W raw Transform to [0,1, such that W l The elements have actual physical meaning; the relationship between the linear fully connected layer and the input is as follows:

[0058] M = W l I

[0059] Where M represents the sensor's measurement value matrix.

[0060] d. The second and subsequent layers are nonlinear mapping networks, with the last layer outputting a vector representing the reconstructed density field; after training, the illumination intensity matrix W of the first linear fully connected layer is... l take out;

[0061] Specifically, the neural network used for reconstruction is represented as follows:

[0062] x nn =f recon (D nn )=f recon (f nn (M))

[0063] Where f nn Map M to SinogramD nn Then, the tomographic reconstruction method was used. recon The density field reconstruction result x is obtained nn Tomographic reconstruction methods can use filtered back projection, but are not limited to this.

[0064] In this embodiment, the measurement value matrix is ​​first divided into 24 sub-matrices according to the acquisition module. For the acquisition module with the sequence number G... i The submatrix is ​​shifted left by rows by G. i ×8 pixels, ensuring all sub-matrices share the same parameterization form to leverage the rotation invariance of the scan geometry in this embodiment. The moved measurement matrix is ​​reconstructed by a 24-branch shared-parameter decoding network to obtain the sinogram for each acquisition module. These sinograms are then rearranged by moving the acquisition module index back to its original position to form the reconstructed sinogram. Finally, the density field is reconstructed using a differentiable 3D-FBPNet. It should be noted that the network structure shown here is one embodiment of the invention and not a limitation. The loss function used for training... It can be represented as follows:

[0065]

[0066]

[0067]

[0068] in Used to evaluate the quality of density field reconstruction Used to give luminous intensity specific properties, g w λ is a function for practically evaluating the properties of luminescence intensity. r With λ p Used to balance the weights between different loss functions. λ r With λ p The preferred values ​​are 1.0 and 1e-5.

[0069] To increase scanning speed, higher luminous intensity should be encouraged; therefore, g w Preferred options are:

[0070] g w (W l )=-∑|W l |

[0071] (3) The light source of the scanning device emits light row by row according to the illumination intensity matrix obtained in step (2), illuminating the target scene in sequence, and obtains the measurement value matrix M with size k×n through the sensor. d n d M represents the number of sensors; M is used as the output of the first linear fully connected layer of the neural network to calculate the reconstructed density field.

[0072] Corresponding to the aforementioned embodiments of the learning-based computed tomography imaging and reconstruction method, the present invention also provides embodiments of the learning-based computed tomography imaging and reconstruction apparatus.

[0073] The learning-based computed tomography imaging and reconstruction apparatus provided in this embodiment of the invention includes a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement the learning-based computed tomography imaging and reconstruction method in the above embodiment.

[0074] The embodiments of the learning-based computed tomography imaging and reconstruction device of the present invention can be applied to any device with data processing capabilities, such as a computer. The device embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of the device with data processing capabilities loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, in addition to the processor, memory, network interface, and non-volatile memory, the device with data processing capabilities in the embodiments may also include other hardware depending on its actual functions; these will not be elaborated further.

[0075] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the learning-based computed tomography imaging and reconstruction method described in the above embodiments.

[0076] The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0077] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A learning-based tomographic imaging and reconstruction method, characterized in that, The following steps are taken to measure scene density distribution using illumination multiplexing: (1) Generate training data: Obtain the parameters of the scanning device, including the spatial position of the light source and the sensor, and all sensor readings when any light source emits light at maximum intensity in an empty scene; use these parameters to generate a CT image of a single light source emitting light through the scene to reach the sensor, as training data. (2) Train the neural network based on the data generated in step (1); the characteristics of the neural network are as follows: a. Input consists of CT images from all light sources of the scanning device; b. The output is the corresponding density field; c. The first layer of the neural network is a fully connected linear layer. The parameter matrix of the fully connected linear layer is obtained by the following formula: W l =f w (W raw ) Among them W raw The parameters to be trained; W l The illumination intensity matrix corresponding to the imaging process has a size of k×n. s n s f is the number of light sources in the scanning device, k is the number of samples; w For a mapping, used for W raw Perform a transformation to generate the illumination intensity matrix W. l It can correspond to the possible luminous intensity of the light source; d. The second and subsequent layers are non-linear neural networks, and the output of the last layer is the density field reconstruction result; after training, the illumination intensity matrix W of the first linear fully connected layer is... l take out; (3) The light source of the scanning device emits light row by row according to the illumination intensity matrix obtained in step (2), illuminating the target scene in sequence, and obtains the measurement value matrix M through the sensor, which has a size of k×n. d n d M represents the number of sensors; M is used as the output of the first linear fully connected layer of the neural network to calculate the reconstructed density field.

2. The learning-based tomographic imaging and reconstruction method according to claim 1, characterized in that, In step (1), the specific method for generating CT images is as follows: several objects with different densities are randomly placed in the effective area of ​​the scene, and CT images are generated based on the selected light model according to the calibrated light source and sensor positions.

3. The learning-based tomographic imaging and reconstruction method according to claim 2, characterized in that, The light ray model is a linear absorption model, and the formula is as follows: Where I is a matrix composed of CT images from different light sources, with a size of n. s ×n d , element I in I ij Let x be the reading of the j-th sensor when the i-th light source emits light at maximum intensity under a given density field, and let x be the vector of the density field discretized into voxels, with a length equal to the number of voxels n. v K represents the Radon transform expressed in terms of a third-order tensor, ×3 represents the mode-3 product of a tensor and a vector, and ⊙ represents the element-wise multiplication of matrices. Let I be the density field when it is zero everywhere.

4. The learning-based computed tomography imaging and reconstruction method according to claim 1, characterized in that, In step (2), the relationship between the linear fully connected layer and the input is as follows: M=W l I Where I is a matrix composed of CT images from different light sources, with a size of n. s ×n d .

5. The learning-based tomographic imaging and reconstruction method according to claim 1, characterized in that, In step (2), the neural network used for reconstruction is represented as follows: x nn =f recon (D nn )=f recon (f nn (M)) Where f nn Map M to Sinogram D nn Then, the tomographic reconstruction method was used. recon The density field reconstruction result x is obtained nn .

6. The learning-based tomographic imaging and reconstruction method according to claim 5, characterized in that, The tomographic reconstruction method is implemented using the filtered back projection method.

7. The learning-based tomographic imaging and reconstruction method according to claim 1, characterized in that, In step (2), the loss function used for training It is expressed as follows: in Used to evaluate the quality of density field reconstruction Used to give luminous intensity specific properties, g w λ is a function for practically evaluating the properties of luminescence intensity. r With λ p Used to balance the weights between different loss functions.

8. The learning-based tomographic imaging and reconstruction method according to claim 7, characterized in that, loss function Where x is the vector of the density field discretized into voxels. nn This is the result of density field reconstruction.

9. The learning-based tomographic imaging and reconstruction method according to claim 7, characterized in that, For dynamic scenarios requiring high-speed scanning, let g w (W l )=-∑|W l |, prompting W l The values ​​tend to become binary.

10. The learning-based tomographic imaging and reconstruction method according to claim 7, characterized in that, For scenarios requiring low-dose scanning, let g w (W l )=∑|W l |, prompting W l The value tends to be minimized.

Images