Learning-based computed tomography imaging and reconstruction method

Abstract

The present invention provides a learning-based computed tomography (CT) imaging and reconstruction method, for use in measuring scene density distribution by means of illumination reuse. During imaging, a light source emits light on the basis of the intensity obtained by pre-learning, light from different directions is absorbed and attenuated by a scene and then reaches a sensor, and computational reconstruction is performed on measurement values to obtain density information of the scene. Luminous intensity and a reconstruction algorithm are obtained by means of neural network learning. In the method, a CT imaging process is modeled as a linear fully-connected layer, and a weight corresponds to the luminous intensity of the light source during imaging; the reconstruction algorithm is modeled as a nonlinear neural network, and targeted optimization can be performed on the basis of the characteristics of scan geometry. According to the method, the amount of acquisition data required is small, and the computational reconstruction does not require a strong priori assumption, so that efficient and high-quality CT acquisition and reconstruction is achieved, and thus the method can be applied to three-dimensional reconstruction of a high-speed dynamic scene.

Description

A learning-based tomography imaging and reconstruction method Technical Field

[0001] The present invention relates to the technical field of computed tomography (CT), and in particular to imaging and reconstruction methods. Background Art

[0002] Computed tomography (CT) is a key research area in medical imaging and computer vision. This technology reconstructs the internal structure of an object by measuring the amount of light absorbed by a scene in different directions. This technology has a wide range of applications, including medical imaging, industrial monitoring, aviation security, and cultural heritage preservation.

[0003] In particular, applying tomography to dynamic scenes holds immense scientific and practical value. For example, mechanical inspection and medical diagnosis require 3D reconstruction of high-speed, dynamic scenes. However, extending traditional tomography to dynamic scenes presents a key challenge: high-quality reconstruction often requires dense sampling in different directions. When the scene changes rapidly, dense sampling must be completed within a short timeframe to avoid ghosting and maintain reconstruction quality. This necessitates that tomography for dynamic scenes require sampling capabilities far exceeding those of traditional methods.

[0004] In the past few decades, many studies have proposed different algorithms for tomography acquisition and reconstruction in dynamic scenes. For specific dynamic phenomena, the properties of the observed scene can be used to solve them. For example, Chen et al. proposed a scanning reconstruction algorithm for periodic cardiac motion (Chen Guang-Hong, Theriault-Lauzier Pascal, Tang Jie, Nett Brian, Leng Shuai, Zambelli Joseph, Qi Zhihua, Bevins Nicholas, Raval Amish, Reeder Scott. 2011. Time-resolved interventional cardiac C-arm cone-beam CT: An application of the PICCS algorithm. IEEE transactions on medical imaging. 31, 4, 907-923). This type of method is limited to the characteristics of a certain type of scene and is not universally applicable. Some studies have also attempted to improve sampling speed by reducing the number of point light source samples and making strong prior assumptions to calculate reconstruction. However, these assumptions limit the application scenarios of the methods (Zang Guangming, Idoughi Ramzi, Wang Congli, Bennett Anthony, Du Jianguo, Skeen Scott, Roberts William L., Wonka Peter, Heidrich Wolfgang. 2020. TomoFluid: reconstructing dynamic fluid from sparse view videos. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 1870-1879). Therefore, there is an urgent need to propose a tomography acquisition and reconstruction algorithm for general dynamic scenes.

[0005] Summary of the Invention

[0006] The purpose of the present invention is to propose a method that greatly improves the acquisition capability and is oriented to general scenarios, in order to address the problems that the existing tomography methods have insufficient acquisition capability in dynamic scenes and reconstruction requires strong prior assumptions.

[0007] To achieve the above-mentioned object, the present invention proposes a learning-based tomography imaging and reconstruction method, which measures the scene density distribution in an illumination multiplexing manner, including the following steps:

[0008] (1) Generate training data: Obtain the parameters of the scanning device, including the spatial position of the light source and the sensor, and all sensor readings when any light source emits light at maximum intensity in an empty scene; use these parameters to generate a CT image produced by light emitted by a single light source through the scene and reaching the sensor as training data;

[0009] (2) Train a neural network based on the data generated in step (1); the characteristics of the neural network are as follows:

[0010] a. Input is a CT image of all light sources of the scanning device;

[0011] b. The output is the corresponding density field;

[0012] c. The first layer of the neural network is a linear fully connected layer. The parameter matrix of the linear fully connected layer is obtained by the following formula: W l =f w (W raw )

[0013] Where W raw is the parameter to be trained; W l Corresponding to the illumination intensity matrix during imaging, the size is k×n s , n s is the number of light sources of the scanning device, k is the number of samples; f w is a mapping for W raw Transform so that the generated illumination intensity matrix W l Can correspond to the possible luminous intensity of the light source;

[0014] d. The second and subsequent layers are nonlinear neural networks, and the output of the last layer is the density field reconstruction result. After the training, the illumination intensity matrix W of the first linear fully connected layer is converted to l take out;

[0015] (3) The light source of the scanning device emits light row by row according to the light intensity matrix obtained in step (2), and illuminates the target scene in turn. The sensor obtains the measurement value matrix M, which is k×n in size. d , n d is the number of sensors; M is used as the output of the first linear fully connected layer of the neural network to calculate the reconstructed density field.

[0016] Furthermore, in step (1), the specific method for generating the CT image is: randomly placing a number of objects with different densities in the effective area of ​​the scene, and generating the CT image based on the selected light model according to the calibrated light source and sensor positions.

[0017] Furthermore, the light model is a linear absorption model, and the formula is as follows:

[0018] Where I is a matrix composed of CT images of different light sources, and its size is n s ×n d , element I in I ij is the reading of the jth sensor when the i-th light source emits light at maximum intensity under a given density field, x is the vector after the density field is discretized into voxels, and its length is the number of voxels n v , K is the Radon transform represented by a third-order tensor, ×3 is the mode-3 product between a tensor and a vector, and ⊙ is the element-by-element multiplication between matrices. is I when the density field is zero everywhere.

[0019] Furthermore, in step (2), the relationship between the linear fully connected layer and the input is as follows: M = W l I

[0020] Where I is a matrix composed of CT images of different light sources, and its size is n s ×n d .

[0021] Furthermore, in step (2), the neural network used for reconstruction can be expressed as follows: nn =f recon (D nn )=f recon (f nn (M))

[0022] where f nn Map M to Sinogram D nn , and then use the tomographic reconstruction method f recon Get the density field reconstruction result x nn .

[0023] Furthermore, the tomography reconstruction method may be implemented using a filtered back projection method.

[0024] Furthermore, in step (2), the loss function used for training is It can be expressed as follows:

[0025] in Used to evaluate the quality of density field reconstruction, Used to make the luminous intensity have specific properties, g w is the function that actually evaluates the luminous intensity properties, λ r and λ pUsed to balance the weights between different loss functions.

[0026] Furthermore, the loss function used to evaluate the quality of density field reconstruction is It can be expressed as:

[0027] Furthermore, the function g that actually evaluates the luminous intensity property w The design can be carried out according to different scenarios. The following two design methods are given but not limited to them:

[0028] a. For dynamic scenes that require high-speed scanning, let g w (W l )=-∑|W l |, prompting W l The value tends to be binary.

[0029] b. For scenarios requiring low-dose scanning, let g w (W l )=∑|W l |, prompting W l The value tends to be minimized.

[0030] The beneficial effects of the present invention are as follows: the present invention obtains a tomographic imaging and reconstruction method of light multiplexing through learning. Compared with the traditional point light source tomographic scanning method for dynamic scenes, it can sample in the entire light space, greatly improving the acquisition efficiency, and the reconstruction algorithm does not require strong prior assumptions and is applicable to general scenes. Compared with the traditional light multiplexing imaging method, the luminous intensity and reconstruction algorithm used are learned by the neural network, and the number of inputs required is much lower than the traditional method, and the reconstruction quality is high. Under the same hardware conditions, the proposed tomographic scanning method can collect and reconstruct dynamic scenes that cannot be achieved by traditional methods. In addition, the method is not limited to a specific scanning device and has a high universal application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the learning-based illumination multiplexing imaging reconstruction process;

[0032] Figure 2 is a side view of the scanning device used in the embodiment;

[0033] FIG3 is a top view and a visual layout diagram of the acquisition module of the scanning device used in the embodiment;

[0034] FIG4 is a CT image generated using training data for a specific light source in an embodiment;

[0035] FIG5 is a CT image of a specific light source in an empty scene according to an embodiment;

[0036] FIG6 is a schematic diagram of a network structure in an embodiment. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] This paper proposes a learning-based tomography imaging and reconstruction method to measure scene density distribution in a light multiplexing manner. The steps are shown in Figure 1 and explained in detail as follows:

[0039] (1) Generate training data: Obtain the parameters of the scanning device, including the spatial position of the light source and the sensor, and all sensor readings when any light source emits light at maximum intensity in an empty scene; use these parameters to generate the acquisition results of the scanning device as training data.

[0040] Specifically, the scanning device includes at least two light sources and at least one sensor. Each sensor can simultaneously receive light from multiple light sources, and the intensity of all light sources is adjustable. The target scene is located in part or all of the light path formed by the light source and the sensor.

[0041] In one embodiment, visible light is used to detect the scene. Figure 2 shows a side view of the scanning device used. Optical fibers are used to transmit light from an LED array to an acquisition module as a light source. The light is then received by the acquisition module's receiver, which also transmits it to the sensor array via optical fibers, where a camera above captures the image. Figure 3 shows a top view and visual layout of the acquisition module. In this embodiment, 24 acquisition modules are connected in a ring, numbered #0, #1, ..., #23. Each acquisition module has 10 rows and 16 columns, with odd-numbered columns representing light sources and even-numbered columns representing sensors. The effective acquisition area is 32 mm × 128 mm × 128 mm, corresponding to a voxel resolution of 32 × 128 × 128. Figures 4 and 5 show CT images produced using a single light source in the scanning device with an object and an empty scene, respectively.

[0042] Different light models can be used to generate CT images. In this embodiment, a linear absorption model is used, but not limited thereto. The formula of the linear absorption model is as follows:

[0043] Where I is a matrix composed of CT images of different light sources, and its size is n s ×n d , element I in I ij is the reading of the jth sensor when the i-th light source emits light at maximum intensity under a given density field, x is the vector after the density field is discretized into voxels, and its length is the number of voxels n v , K is the Radon transform represented by a third-order tensor, ×3 is the mode-3 product between a tensor and a vector, and ⊙ is the element-by-element multiplication between matrices. is I when the density field is zero everywhere, Elements in is the reading of the jth sensor when the i-th light source emits light at maximum intensity in an empty scene.

[0044] (2) Train a neural network based on the training data generated in step (1). The neural network used in this embodiment is shown in FIG6 , but is not limited thereto. Black arrows in the figure represent convolutional layers unless otherwise specified. The specific description of the neural network used is as follows:

[0045] a. Input is a CT image of all light sources of the scanning device;

[0046] b. The output is the corresponding density field;

[0047] c. The first layer of the neural network is a linear fully connected layer. The parameter matrix of the linear fully connected layer is obtained by the following formula: W l =f w (W raw )

[0048] Where W raw is the parameter to be trained; W l Corresponding to the illumination intensity matrix during imaging, the size is k×n s , n s is the number of light sources of the scanning device, k is the number of samples; f w is a mapping for W raw Transform so that the generated illumination intensity matrix W l It can correspond to the possible luminous intensity of the light source. In this embodiment, f w Select the Sigmoid function and set W raw Transform to [0,1] so that W l The elements have actual physical meanings; the relationship between the linear fully connected layer and the input is as follows: M = W l I

[0049] Where M represents the sensor measurement matrix.

[0050] d. The second and subsequent layers are nonlinear mapping networks, where the output vector of the last layer is the reconstructed density field. After the training, the illumination intensity matrix W of the first linear fully connected layer is converted to l take out;

[0051] Specifically, the neural network used for reconstruction is expressed as follows: nn =f recon (D nn )=f recon (f nn (M))

[0052] where fnn Map M to Sinogram D nn , and then use the tomographic reconstruction method f recon Get the density field reconstruction result x nn The tomography reconstruction method may use a filtered back projection method, but is not limited thereto.

[0053] In this embodiment, the measurement value matrix is ​​first divided into 24 sub-matrices according to the acquisition module. i Submatrix of G, shift the submatrix to the left by row i ×8 pixels, so that all sub-matrices share the same parameterized form to utilize the rotation invariance of the scanning geometry in the embodiment. The moved measurement value matrix is ​​reconstructed by a decoding network with 24 shared parameters to obtain the sinogram of each acquisition module, and the acquisition module sequence is moved back to its original position and pieced together to form the reconstructed sinogram. Finally, the density field is reconstructed by a differentiable 3D-FBPNet. It should be noted that the network structure shown here is a technical solution of one embodiment of the present invention and not a limitation. The loss function used for training It can be expressed as follows:

[0054] in Used to evaluate the quality of density field reconstruction, Used to make the luminous intensity have specific properties, g w is the function that actually evaluates the luminous intensity properties, λ r and λ p Used to balance the weights between different loss functions. r and λ p Preferred are 1.0 and 1e-5.

[0055] In order to increase the scanning speed, the luminous intensity should be increased, so g w Preferably: g w (W l )=-∑|W l |

[0056] (3) The light source of the scanning device emits light row by row according to the light intensity matrix obtained in step (2), illuminating the target scene in turn, and obtains the measurement value matrix M through the sensor, which is k×n in size. d , n d is the number of sensors; M is used as the output of the first linear fully connected layer of the neural network to calculate the reconstructed density field.

[0057] Corresponding to the aforementioned embodiments of the learning-based tomography imaging and reconstruction method, the present invention also provides embodiments of the learning-based tomography imaging and reconstruction device.

[0058] The learning-based tomography imaging and reconstruction device provided in an embodiment of the present invention includes a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement the learning-based tomography imaging and reconstruction method in the above embodiment.

[0059] The embodiment of the learning-based tomography imaging and reconstruction device of the present invention can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, in addition to the processor, memory, network interface, and non-volatile memory, the any device with data processing capabilities in which the device in the embodiment is located can also include other hardware according to the actual function of the any device with data processing capabilities, which will not be described in detail.

[0060] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the learning-based tomography imaging and reconstruction method in the above embodiment is implemented.

[0061] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0062] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A learning-based tomography imaging and reconstruction method, characterized in that: To measure the scene density distribution by light multiplexing, the steps are as follows: (1) Generate training data: Obtain the parameters of the scanning device, including the spatial position of the light source and the sensor, and all sensor readings when any light source emits light at maximum intensity in an empty scene; use these parameters to generate a CT image produced by light emitted by a single light source through the scene and reaching the sensor as training data; (2) Train a neural network based on the data generated in step (1); the characteristics of the neural network are as follows: a. Input is the CT image of all light sources of the scanning device; b. The output is the corresponding density field; c. The first layer of the neural network is a linear fully connected layer. The parameter matrix of the linear fully connected layer is obtained by the following formula: W l =f w (W raw ) Where W raw is the parameter to be trained; W l Corresponding to the illumination intensity matrix during imaging, the size is k×n s , n s is the number of light sources of the scanning device, k is the number of samples; f w is a mapping for W raw Transform so that the generated illumination intensity matrix W l Can correspond to the possible luminous intensity of the light source; d. The second and subsequent layers are nonlinear neural networks, and the output of the last layer is the density field reconstruction result. After the training, the illumination intensity matrix W of the first linear fully connected layer is converted into l take out; (3) The light source of the scanning device emits light row by row according to the light intensity matrix obtained in step (2), illuminating the target scene in turn, and obtains a measurement value matrix M through the sensor, with a size of k×n d , n d is the number of sensors; M is used as the output of the first linear fully connected layer of the neural network to calculate the reconstructed density field.

2. The learning-based tomography imaging and reconstruction method according to claim 1, characterized in that: In the step (1), the specific method for generating the CT image is: randomly placing a number of objects with different densities in the effective area of ​​the scene, and generating the CT image based on the selected light model according to the calibrated light source and sensor positions.

3. The learning-based tomography imaging and reconstruction method according to claim 2, characterized in that: The light model is a linear absorption model, and the formula is as follows: Where I is a matrix composed of CT images of different light sources, and its size is n s ×n d , element I in I ij is the reading of the jth sensor when the i-th light source emits light at maximum intensity under a given density field, x is the vector after the density field is discretized into voxels, and its length is voxels Quantity v , K is the Radon transform represented by a third-order tensor, ×3 is the mode-3 product between a tensor and a vector, ⊙ is the element-by-element multiplication between matrices, is I when the density field is 0 everywhere.

4. The learning-based tomography imaging and reconstruction method according to claim 1, characterized in that: In step (2), the relationship between the linear fully connected layer and the input is as follows: M = W l I Where I is a matrix composed of CT images of different light sources, and its size is n s ×n d .

5. The learning-based tomography imaging and reconstruction method according to claim 1, characterized in that: In step (2), the neural network used for reconstruction is expressed as follows: nn =f recon (D nn ) = f recon (f nn (M)) where f nn Map M to Sinogram D nn , and then use the tomographic reconstruction method f recon Get the density field reconstruction result x nn .

6. The learning-based tomography imaging and reconstruction method according to claim 5, characterized in that: The tomographic reconstruction method is implemented using a filtered back projection method.

7. The learning-based tomography imaging and reconstruction method according to claim 1, characterized in that: In step (2), the loss function used for training is It is expressed as follows: in Used to evaluate the quality of density field reconstruction, Used to make the luminous intensity have specific properties, g w is the function that actually evaluates the luminous intensity properties, λ r With λ p Used to balance the weights between different loss functions.

8. The learning-based tomography imaging and reconstruction method according to claim 7, characterized in that: Loss Function Where x is the vector after the density field is discretized into voxels, x nn is the density field reconstruction result.

9. The learning-based tomography imaging and reconstruction method according to claim 7, characterized in that: For dynamic scenes that require high-speed scanning, let g w (W l )=-∑|W l |, prompting W l The value tends to be binary.

10. The learning-based tomography imaging and reconstruction method according to claim 7, characterized in that: For scenarios that require low-dose scanning, let g w (W l )=∑|W l |, prompting W l The value tends to be minimized.

Images