A three-dimensional optical reconstruction method based on CT images

Abstract

The application relates to a three-dimensional optical reconstruction method based on a CT image, comprising the following steps: acquiring multi-angle CT projection signals and optical data of an object to be reconstructed by means of a coaxial scanning device; reconstructing the CT projection signals of all angles to obtain CT three-dimensional voxel data and extract CT surface voxel data of the CT three-dimensional voxel data; aligning and registering each pixel point in the optical data with a corresponding pixel point in the CT three-dimensional surface voxel data, and mapping the coordinates of the pixel point in the optical data to a CT image coordinate system of the CT three-dimensional surface voxel data to obtain a first optical data set with three-dimensional space coordinates in the CT image coordinate system under all angles; and splicing all the optical data in the data set to form information after three-dimensional optical reconstruction. The above method can provide accurate three-dimensional space information of an object for three-dimensional optical reconstruction by means of CT scanning, and can generate an accurate and comprehensive optical three-dimensional reconstructed object.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of three-dimensional optical reconstruction, and particularly relates to a three-dimensional optical reconstruction method based on CT images. BACKGROUND

[0002] Three-dimensional reconstruction refers to converting two-dimensional images or projection data into spatial information of a three-dimensional object, and the reconstructed model is convenient for computer display and further processing, and has wide application in many fields such as medicine, biology, engineering and computer vision, and has important significance for studying and analyzing the structure and morphology of an object. With the continuous progress of imaging technology, high-resolution three-dimensional reconstruction has become an important research direction. Researchers improve algorithms, use high-resolution sensors and improve data acquisition methods to achieve more accurate and detailed three-dimensional reconstruction results. In the related art, a three-dimensional reconstruction method based on a depth camera can provide depth information of an object, so that the object can be directly modeled, mainly including a structured light projection method and a TOF (Time off light) method. The structured light projection method projects an image coded according to a certain rule to an object to be measured by using a light source, and the coded pattern is modulated by the shape of the object surface to generate deformation. The structured light with deformation is photographed by a camera, and the depth information of the object to be detected is obtained through the positional relationship between the camera and the projection light source and the degree of structured light deformation. Although this method has high accuracy, it is easily disturbed by ambient light around the object, and the structured light projector must be calibrated in advance. The TOF (Time off light) method calculates the depth distance of the surface of the object to be measured by recording the propagation time of the light beam. The system emission device emits a pulse signal, which is reflected by the object to be measured and received by the detector, and the depth value can be calculated through the time from emission to reception of the light signal and the speed of light. The TOF technology may be limited by noise and precision under long-distance or weak light conditions, and has low resolution, and cannot realize high-precision three-dimensional reconstruction, and thus cannot accurately restore the shape and geometric structure of the object. SUMMARY

[0003] (I) Technical problems to be solved

[0004] In order to solve the above problems of the prior art, the application provides a three-dimensional optical reconstruction method based on CT images.

[0005] (II) Technical solutions

[0006] In order to achieve the above purpose, the main technical scheme adopted by the application includes:

[0007] The application provides a three-dimensional optical reconstruction method based on CT images, which comprises:

[0008] S10, acquiring CT projection signals and optical data of a plurality of angles of the object to be reconstructed by means of a coaxial scanning device; each angle of the plurality of angles is a rotation angle of the rotating coaxial scanning device relative to the stationary object to be reconstructed; the coaxial scanning device is fixed with a CT imaging device and an optical imaging device having an installation angle with the CT imaging device;

[0009] S20, reconstructing the CT projection signals of all angles according to a reconstruction algorithm to obtain CT three-dimensional voxel data of the object to be reconstructed, and extracting surface voxel data of the CT three-dimensional voxel data to obtain CT three-dimensional surface voxel data of the object to be reconstructed;

[0010] S30, aligning and registering each pixel point in the optical data with a corresponding pixel point in the CT three-dimensional surface voxel data, and mapping the coordinates of the pixel points in the aligned and registered optical data to a CT image coordinate system of the CT three-dimensional surface voxel data to obtain a first optical data set having three-dimensional space coordinates in the CT image coordinate system under all angles;

[0011] S40, splicing all optical data in the first optical data set to form three-dimensional optical reconstructed information.

[0012] Optionally, the optical imaging device is a hyperspectral imaging device, and the optical data is hyperspectral data;

[0013] The optical imaging device is a fluorescence imaging device, and the optical data is fluorescence optical data;

[0014] The optical imaging device is an RGB imaging device, and the optical data is RGB optical data;

[0015] The plurality of angles of the CT projection signals and the optical data includes N CT projection signals and N optical data;

[0016] Wherein, the rotation angle of the rotating frame of the coaxial scanning device relative to the object is The number of data generated when the coaxial scanning device rotates 360° relative to the object is

[0017] Optionally, the S20 of reconstructing the CT projection signals of all angles according to the reconstruction algorithm to obtain the CT three-dimensional voxel data of the object to be reconstructed includes:

[0018] Pretreating the CT projection signals of each angle, and reconstructing the pretreated CT projection signals of all angles by using a filtered back projection algorithm to obtain the CT three-dimensional voxel data of the object to be reconstructed.

[0019] Optionally, the filtered back-projection algorithm is used to reconstruct the pre-processed CT projection signals of all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed, including:

[0020] S21, one-dimensional Fourier transform is performed on the N pre-processed CT projection signals respectively to obtain the first projection signals in the frequency domain;

[0021] S22, filtering is performed on the N first projection signals in the frequency domain to obtain the N second projection signals after filtering;

[0022] S23, one-dimensional inverse Fourier transform is performed on the N second projection signals to restore them to the time domain to obtain the N third projection signals after filtering in the time domain;

[0023] S24, back-projection is performed on each third projection signal, the back-projection is to average the projection signals under each angle to each point passing through the object according to the original projection path of each angle, and the back-projection signals of the same point on the object under all angles are accumulated to obtain the ray attenuation coefficient of each point of the object, and the CT three-dimensional voxel data of the object is reconstructed;

[0024] The CT three-dimensional voxel data includes three-dimensional spatial coordinates of each voxel in the reconstructed object CT image and HU values of each voxel position, and the HU values reflect the absorption degree of the object to be reconstructed to X-rays.

[0025] Optionally, the surface voxel data of the object CT three-dimensional voxel data in S20 is extracted to obtain the CT three-dimensional surface voxel data of the object to be reconstructed, and further includes:

[0026] S25, the CT three-dimensional voxel data is optimized to obtain the CT three-dimensional voxel data after optimization;

[0027] S26, the surface information of the CT three-dimensional voxel data after optimization is extracted;

[0028] S27, based on the preset voxel data threshold, the surface information of the CT three-dimensional voxel data is divided into object belonging voxel data and background belonging voxel data;

[0029] S28, based on the object belonging voxel data, the voxel data of the object boundary is obtained in a traversal manner; the HU value of the voxel data of the non-object boundary in the object belonging voxel data is set to 0 to obtain the surface voxel data of the object, i.e. the CT three-dimensional surface voxel data.

[0030] Optionally, S30 includes:

[0031] The coaxial scanning device rotates relative to the object, and the imaging angle of each imaging position is an included angle between the imaging position and the object The installation angle between the CT imaging device and the optical imaging device is θ;

[0032] The coordinate system of each imaging device of the coaxial scanning device is as follows: the coordinate origin is located at the center axis of the rotating frame and at the same height as the optical axis of the imaging device, the coordinate Z axis points to the center of the imaging device from the coordinate origin, the X-ray imaging Z axis points to the X-ray center from the coordinate origin, and the XY plane is perpendicular to the Z axis;

[0033] S31, for each imaging view , the XY plane is selected as the projection plane, orthogonal projection is performed along the negative direction of the Z axis, three-dimensional data (x, y, z, HU) containing the surface voxel of the object is projected into the XY plane under the corresponding angle, each pixel point in the plane is the projection position of the three-dimensional surface voxel data on the projection plane, and a CT two-dimensional projection image (x, y, HU) under the angle is formed; to obtain CT two-dimensional projection images under all imaging views ;

[0034] S32, for the CT two-dimensional projection image under the imaging view and the optical data under the imaging view , feature detection is performed, and significant feature points in the CT two-dimensional projection image and the optical data are obtained;

[0035] S33, a feature descriptor of each significant feature point is obtained, and matching is performed, and matched feature points exceeding a preset threshold are obtained;

[0036] S34, according to the matched feature point pairs, a spatial coordinate conversion mapping of the CT two-dimensional projection image under the imaging view and the optical data under the imaging view is obtained;

[0037] S35, based on the spatial coordinate conversion mapping, the CT two-dimensional projection image under the imaging view and the optical data under the imaging view are registered;

[0038] Based on the modes of the S32 to the S35, all imaging views CT two-dimensional projection images and optical data under all imaging views are aligned and registered.

[0039] Optionally, the S34 includes:

[0040] S341, based on each matched feature point pair, the coordinates of the feature points belonging to each imaging device are divided by the focal length of the imaging device to obtain normalized feature point pair coordinates;

[0041] S342, constructing a linear equation based on the normalized feature point pair coordinates;

[0042] Setting p(x, y) and p'(x', y') are normalized feature point pair coordinates;

[0043] p(x, y) corresponds to the CT two-dimensional projection image, and p'(x', y') corresponds to the optical data;

[0044] Through the linear equation p' T Fp = 0, determine the basis matrix, wherein F is the basis matrix;

[0045] Collecting the linear equations constructed by all feature point pairs, the basis matrix is solved;

[0046] S343, based on the basis matrix, using the intrinsic parameters of the CT imaging device and the optical imaging device to perform triangulation, and mapping the normalized feature point pair coordinates to the three-dimensional points in the world coordinate system;

[0047] S344, using the three-dimensional point coordinates mapped by the normalized feature point pairs in the world coordinate system, obtaining the spatial coordinate conversion mapping, which includes a translation vector and a rotation matrix.

[0048] Optionally, the S34 comprises:

[0049] Converting the pixel position in the optical data at each angle to the coordinate in the coordinate system of the registered CT two-dimensional projection image;

[0050] Establishing the correspondence between the pixels in the optical data and the spatial positions in the CT three-dimensional voxel data, corresponding the pixel information in the optical data at each angle to the spatial positions of the CT three-dimensional surface voxel data, and obtaining the first optical data set with three-dimensional spatial coordinates at all angles.

[0051] Optionally, the S40 comprises:

[0052] Using the S30 method to traverse all adjacent optical data, registering the adjacent optical data, identifying the overlapping regions, splicing all optical data in the first optical data set based on the identified overlapping regions, and forming the three-dimensional optical reconstruction information;

[0053] Optimizing the three-dimensional optical reconstruction information to obtain complete three-dimensional optical reconstruction information.

[0054] In a second aspect, the present application also provides a computing device, comprising a memory and a processor, the memory stores a computer program, the processor executes the computer program in the memory, and performs the steps of the three-dimensional optical reconstruction method based on the CT image according to any one of the first aspect.

[0055] (III) Beneficial Effects

[0056] The three-dimensional optical reconstruction method of the present application does not need pre-calibration in advance, is less affected by ambient light, and improves the accuracy of three-dimensional optical data reconstruction. The CT image has a high spatial resolution and can provide the fine structure of the object and accurate three-dimensional spatial coordinates, which enables the three-dimensional optical reconstruction method based on the CT image to realize high-precision three-dimensional optical reconstruction, accurately restore the shape and geometric structure in the optical image of the object, and is especially suitable for three-dimensional optical reconstruction of objects with a dramatic change in surface, i.e., a dramatic change in depth. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 Fig. 1 is a flowchart of a three-dimensional optical reconstruction method based on a CT image provided by the present application;

[0058] Fig. 2(a) and Fig. 2(b) are schematic diagrams of a coaxial scanning device provided by the present application;

[0059] Figure 3 (a) is a schematic diagram of X-ray imaging when the imaging device rotates through an angle relative to the object;

[0060] Figure 3 (b) is a schematic diagram of CT surface voxel data after the CT three-dimensional voxel data containing the object and the background after reconstruction are segmented to separate the object background and the surface is extracted;

[0061] Figure 3 (c) is a schematic diagram of CT two-dimensional projection images obtained by two-dimensional projection of the CT surface voxel data;

[0062] Figure 3 (d) is a schematic diagram of optical imaging when the imaging device rotates through an angle relative to the object;

[0063] Figure 3 (e) is a schematic diagram of the collected optical image data;

[0064] Figure 3 (f) is a schematic diagram of registration of the CT two-dimensional projection images at the angle and the optical data at the angle. DETAILED DESCRIPTION

[0065] ​​​​For a better understanding of the above technical solutions, the exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application can be more clearly, thoroughly understood and the scope of the present application can be completely conveyed to those skilled in the art.

[0066] The three-dimensional optical reconstruction method of the embodiments of the present application is not the optical reconstruction of human body images in the medical field. The object to be reconstructed in the embodiments of the present application can be a plant or other food plants with a large volume.

[0067] Embodiment one

[0068] As shown in Figures 1 to 3 The present application provides a three-dimensional optical reconstruction method based on CT images. The execution subject of the method is any computing device, which includes the following steps:

[0069] S10, acquiring multi-angle CT projection signals and optical data of an object to be reconstructed by means of a coaxial scanning device; each angle in the multi-angle is a rotation angle of the coaxial scanning device relative to the stationary object to be reconstructed; the coaxial scanning device is fixed with a CT imaging device and an optical imaging device having an installation included angle with the CT imaging device, and the computing device of the embodiment can be electrically connected with the CT imaging device and the optical imaging device. In other embodiments, the processing function of the computing device can be integrated in the CT imaging device or the optical imaging device.

[0070] The optical data in this step can be understood as an optical image, which is described as optical data in this embodiment.

[0071] For example, the optical imaging device is a hyperspectral imaging device, and the optical data is hyperspectral data;

[0072] The optical imaging device is a fluorescence imaging device, and the optical data is fluorescence optical data. The optical imaging device described above can be an RGB imaging device, and the optical data can also be RGB optical data.

[0073] In this embodiment, the multi-angle CT projection signals and optical data include N CT projection signals and N optical data;

[0074] Wherein, the rotation angle of the rotating frame of the coaxial scanning device relative to the object is The number of data generated when the coaxial scanning device rotates 360° relative to the object is N is a natural number greater than or equal to 1. Preferably, N is a natural number greater than or equal to 10. In the present embodiment, N can be 360, i.e. the rotating frame rotates relative to the object through an angle from 0° to 360°, and the data at each angle is recorded in turn as For example, the data number is 360 when the angle is rotated by 1° each time. Different angle precisions result in different CT imaging resolutions.

[0075] S20, reconstructing the CT projection signals of all angles according to a reconstruction algorithm to obtain CT three-dimensional voxel data of the object to be reconstructed, and extracting surface voxel data of the CT three-dimensional voxel data to obtain CT three-dimensional surface voxel data of the object to be reconstructed.

[0076] Generally, the CT projection signals of each angle are preprocessed, the filtered back projection algorithm is used to reconstruct the preprocessed CT projection signals of all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed, and then the surface information of the CT three-dimensional voxel data is extracted to obtain the CT three-dimensional surface voxel data of the object to be reconstructed.

[0077] S30, aligning and registering each pixel point in the optical data and the corresponding pixel point in the CT three-dimensional surface voxel data, and mapping the coordinates of the pixel points in the aligned and registered optical data to the CT image coordinate system of the CT three-dimensional surface voxel data to obtain a first optical data set having three-dimensional space coordinates in the CT image coordinate system under all angles.

[0078] S40, splicing all the optical data in the first optical data set to form three-dimensional optical reconstructed information.

[0079] The method of the present embodiment does not need to be calibrated in advance, is less affected by ambient light, and improves the accuracy of three-dimensional optical data reconstruction. The CT image has a high spatial resolution and can provide the fine structure and accurate three-dimensional space coordinates of the object, which enables the three-dimensional optical reconstruction method based on the CT image to realize high-precision three-dimensional optical reconstruction, accurately restore the shape and geometric structure in the optical image of the object, and is especially suitable for three-dimensional optical reconstruction of objects with a dramatic change in surface, i.e. a dramatic change in depth.

[0080] In order to better understand the processes of steps S20 and S30, the steps S20 and S30 will be described step by step as follows.

[0081] The process of step S20 includes the following sub-steps:

[0082] S21, performing one-dimensional Fourier transform on the N preprocessed CT projection signals respectively to obtain first projection signals in the frequency domain;

[0083] S22, filtering the first projection signals in the N frequency domains to obtain filtered second projection signals in the N frequency domains;

[0084] S23, performing one-dimensional inverse Fourier transform on the N second projection signals to restore them to the time domain to obtain filtered third projection signals in the time domain;

[0085] S24, performing back projection on each third projection signal, wherein the back projection is to average the projection signals at each angle according to the original projection path of each angle to each point passing through the object, and to accumulate the back projection signals at the same point on the object at all angles to obtain a ray attenuation coefficient of each point of the object, thereby reconstructing CT three-dimensional voxel data of the object.

[0086] That is, a CT three-dimensional voxel data obtained by accumulation at all angles.

[0087] The CT three-dimensional voxel data includes three-dimensional spatial coordinates of each voxel in the reconstructed object CT image and a HU (Hounsfield Unit) value at the position of each voxel, wherein the HU value reflects the degree of X-ray absorption of the object to be reconstructed.

[0088] In the field, a data grid point in a three-dimensional image is referred to as a voxel, and a data grid point in a two-dimensional image is referred to as a pixel. The following sub-steps are a detailed description of obtaining CT three-dimensional surface voxel data of the object to be reconstructed by extracting surface voxel data of the CT three-dimensional voxel data of the object in S20.

[0089] S25, performing optimization processing on the CT three-dimensional voxel data to obtain optimized CT three-dimensional voxel data;

[0090] S26, extracting surface information of the CT three-dimensional voxel data from the optimized CT three-dimensional voxel data.

[0091] For example, for the CT voxel data described above, a suitable threshold value that can distinguish between object voxels and background voxels is selected, and the threshold value will be used to divide the voxels into two parts: object voxel data and background voxel data; the selected threshold value is used to divide the voxel data into two regions: one is the object voxel data, and the other is the background voxel data, as described in sub-step S27.

[0092] S27, dividing the CT three-dimensional voxel data into object-associated voxel data and background-associated voxel data based on a pre-set voxel data threshold value;

[0093] S28, obtaining object boundary voxel data in a traversal manner based on the object-associated voxel data;

[0094] That is, traversing all the object's voxel data, for each voxel data, checking whether its adjacent voxel data belongs to the background voxel data, if there is background voxel data in the adjacent voxel data of a certain voxel data, then this voxel data is on the boundary of the object;

[0095] S29, setting the HU value of the non-object boundary voxel data in the object voxel data to 0, obtaining the surface voxel data of the object, that is, the CT three-dimensional surface voxel data.

[0096] For example, the HU value of the voxel data not on the boundary is set to 0, and the HU value of the boundary voxel data is maintained unchanged, so that the three-dimensional voxel data formed only contains the surface voxel data of the object.

[0097] Through the above sub-steps S21 to S29, a process of obtaining the CT three-dimensional voxel data of the object to be reconstructed and obtaining the CT three-dimensional surface voxel data in step S20 is described, and other ways can also be used in other embodiments, which are not limited by the present embodiment.

[0098] In the present embodiment, the coaxial scanning device rotates relative to the object, and the imaging angle of each imaging position is The installation angle between the CT imaging device and the optical imaging device is θ;

[0099] (The angle turned Refers to each imaging position, there are N imaging positions, then there are N N imaging data);

[0100] In the present embodiment, the coordinate system of each imaging device is defined as: the coordinate origin is located at the center axis of the rotating frame and is at the same height as the optical axis of the imaging device, the coordinate Z axis points to the center of the imaging device from the coordinate origin, the X-ray imaging Z axis points to the X-ray center from the coordinate origin, and the XY plane is perpendicular to the Z axis.

[0101] Correspondingly, the process of step S30 includes the following sub-steps:

[0102] S31, for each imaging angle The surface voxel data under, select the XY plane as the projection plane, and perform orthogonal projection along the negative direction of the Z axis, project the three-dimensional data (x, y, z, HU) containing the object surface voxel into the XY plane under the corresponding angle, and each pixel point in the plane is the projection position of the three-dimensional surface voxel data on the projection plane, forming the two-dimensional projection image (x, y, HU) of the CT under the angle. Obtain the CT two-dimensional projection image under all imaging angles .

[0103] S32, for the CT two-dimensional projection image under the imaging angle Under the imaging angle obtaining respective significant feature points in the CT two-dimensional projection image and the optical data;

[0104] S33, obtaining feature descriptors of the respective significant feature points and performing matching to obtain matched feature point pairs exceeding a preset threshold;

[0105] S34, obtaining, according to the matched feature point pairs, a spatial coordinate conversion mapping of the CT two-dimensional projection image under the imaging view angle and the optical data under the imaging view angle ;

[0106] The sub-step S34 further includes the following sub-steps:

[0107] S341, based on each matched feature point pair, dividing the coordinates of the feature points belonging to each imaging device by the focal length of the imaging device to obtain normalized feature point pair coordinates;

[0108] S342, based on the normalized feature point pair coordinates, constructing a linear equation;

[0109] Let p(x, y) and p'(x', y') be the normalized feature point pair coordinates;

[0110] p(x, y) corresponds to the CT two-dimensional projection image, and p'(x', y') corresponds to the optical data;

[0111] determining a fundamental matrix through the linear equation p' T Fp = 0, where F is the fundamental matrix and T is the transpose;

[0112] Collecting the linear equations constructed by all feature point pairs to solve the fundamental matrix;

[0113] S343, based on the fundamental matrix, performing triangulation using the intrinsic parameters of the imaging device to which the CT two-dimensional projection image belongs and the imaging device to which the optical data belongs, and mapping the normalized feature point pair coordinates to three-dimensional points in a world coordinate system;

[0114] S344, using the three-dimensional point coordinates mapped by the normalized feature point pairs in the world coordinate system, obtaining the spatial coordinate conversion mapping, which includes a translation vector and a rotation matrix.

[0115] S35, based on the spatial coordinate conversion mapping, registering the CT two-dimensional projection image under the imaging view angle and the optical data under the imaging view angle ;

[0116] Based on the manner of sub-step S32 to sub-step S35, all imaging view angles Aligning and registering the CT two-dimensional projection images and optical data under all imaging perspectives.

[0117] Converting the pixel positions in the optical data under each angle into coordinates in the coordinate system of the registered CT two-dimensional projection images;

[0118] Establishing the correspondence between the pixels in the optical data and the spatial positions in the CT three-dimensional voxel data, corresponding the pixel information in the optical data under each angle to the spatial positions of the CT three-dimensional surface voxels, and obtaining a first optical data set with three-dimensional spatial coordinates under all angles.

[0119] The above process is applicable not only to hyperspectral images, but also to fluorescence optical data and RGB optical data, etc. The optical imaging data of the coaxial scanning device is configured according to actual needs, and the corresponding optical data is obtained.

[0120] Embodiment Two

[0121] Fig. 2(a), Fig. 2(b), Figure 3 (a) to Figure 3 (f) detail the three-dimensional optical reconstruction method based on CT images of the present embodiment. The optical data of the present embodiment is fluorescence image or other optical image. The method of the present embodiment can include the following steps:

[0122] 201. First, the CT projection signals and fluorescence data under multiple angles of the object to be reconstructed are obtained by means of the coaxial scanning device. The object is relatively stationary during the process of obtaining three-dimensional data. There is a CT projection signal and fluorescence data under each angle; the multiple angles can be the angles at which the imaging device in the coaxial scanning device rotates around the object to be reconstructed.

[0123] The coaxial scanning device of the present embodiment includes a rotating frame, a CT imaging device (i.e. an X-ray source and an X-ray detector) fixed on the rotating frame, and a fluorescence imaging device (i.e. a light source and a camera).

[0124] As shown in Fig. 2(a) and Fig. 2(b) are schematic diagrams of the coaxial scanning device, wherein XYZ is the object coordinate system, and X'Y'Z is the rotating frame coordinate system. The CT imaging device (including the X-ray source and the X-ray detector, the X-ray source facing the X-ray detector, and the object to be reconstructed in between) and the optical imaging device (such as a fluorescence camera, the optical imaging device here is not limited to a fluorescence camera, but can also be other optical imaging devices, and is not limited to one kind, but can also have multiple optical imaging devices at the same time. Here, the fluorescence imaging camera is taken as an example) are installed on the ring-shaped rotating frame, the imaging device centers are consistent in height, the installation angle between the X-ray source and the fluorescence imaging camera is θ, and the object to be reconstructed is located at the center of the ring-shaped rotating frame.

[0125] When imaging, the object is stationary, and the rotating frame rotates the imaging device thereon around the object (i.e., the object coordinate system remains unchanged, and the rotating frame coordinate system rotates around the Z axis), and each time a certain angle (e.g. The smaller the angle, the higher the CT reconstruction accuracy) to complete one imaging. The imaging includes:

[0126] 1) CT imaging at the angle, mainly refers to the X-ray intensity signal received by the ray detector after the X-ray passes through the object and attenuates at the angle, referred to as the CT projection signal at the angle;

[0127] 2) Fluorescence imaging at the angle is optical image data for reconstruction. Rotating a circle completes the acquisition of CT projection signals and optical image data at all angles. Assuming that imaging is performed once every 1° rotation, when the rotating frame with the device rotates and scans 360°, 360 CT projection signals and 360 fluorescence data can be obtained.

[0128] 202, data preprocessing.

[0129] The CT projection signal is preprocessed to reduce noise and enhance image quality.

[0130] The data preprocessing can be realized in existing ways, such as removing artifacts, gamma correction, filtering, and the like. The present embodiment is not limited thereto, and is selected according to actual needs.

[0131] 203, using a reconstruction algorithm to reconstruct the CT projection signals at all angles to obtain CT three-dimensional voxel data of the object to be reconstructed.

[0132] The essence of CT reconstruction is to solve the X-ray attenuation coefficient distribution (i.e., the X-ray attenuation coefficient of different parts of the object. Different substances have different X-ray attenuation, and CT imaging is based on this to non-destructively detect the internal material distribution of the object) inside the object using the CT projection signals obtained in step 201.

[0133] In the present embodiment, a filtered back projection algorithm (Filtered Back Projection, FBP) is used. The reconstruction steps of the FBP algorithm are:

[0134] 1) One-dimensional Fourier transform is performed on the 360 CT projection signals in the time domain to obtain 360 projection signals in the frequency domain;

[0135] 2) Filtering is performed on the 360 CT projection signals in the frequency domain to obtain 360 filtered CT projection signals;

[0136] 3) One-dimensional inverse Fourier transform is performed on the 360 filtered CT projection signals to restore them to the time domain, to obtain 360 filtered CT projection signals in the time domain;

[0137] 4) Back projection is performed on each filtered projection signal; the back projection signals at 360 angles are accumulated to calculate the attenuation of each part of the object, i.e. to reconstruct a three-dimensional voxel data of the object.

[0138] Back projection is to distribute each projection signal at an angle to each point passing through the object according to its original projection path. The three-dimensional voxel data of the object contains three-dimensional spatial coordinates and the attenuation coefficient of the object in the voxel to X-rays.

[0139] All three-dimensional voxels form the three-dimensional spatial morphology of the object, which provides the accurate three-dimensional spatial coordinates of the object for the subsequent three-dimensional optical reconstruction of the object.

[0140] 204, subsequent processing of the three-dimensional voxel data of the object obtained by CT reconstruction, including denoising and contrast enhancement, to obtain CT three-dimensional voxel data with improved quality.

[0141] Denoising is to reduce noise and artifacts in the image to improve the quality of the reconstructed data; contrast enhancement can improve the readability of the reconstructed data.

[0142] 205, extracting surface information of the CT three-dimensional voxel data of the object obtained in step 204.

[0143] Based on the pre-set voxel data threshold, the CT three-dimensional voxel data is divided into object belonging voxel data and background belonging voxel data; based on the object belonging voxel data, the voxel data of the object boundary is obtained by traversal; the HU value of the voxel data of the non-object boundary in the object belonging voxel data is set to 0, and the HU value of the voxel data of the object interface is retained as the original HU value, to obtain the surface voxel data of the object, i.e. the CT three-dimensional surface voxel data.

[0144] 206, registering the CT three-dimensional surface voxel data of the object obtained in step 205 and the fluorescence data obtained in step 201.

[0145] Generally, registration refers to mapping one image to another image in two or more groups of image data obtained at different times or by different imaging devices, so that the points corresponding to the same position in space in the two images are one-to-one.

[0146] Since the CT imaging device and the fluorescence imaging device are both installed on the rotating frame, the angle between them in the same three-dimensional coordinate system (the coordinate system of the rotating frame) is θ, then the CT imaging view angle when the rotating frame rotates through an angle of to the object (as shown in Fig. 2(a)) and rotates through an angle of The imaging view at the time (as shown in Fig. 2(b)) corresponds to the same part of the object.

[0147] The steps of registering the CT three-dimensional surface voxel data reconstructed in step 205 and the fluorescence data are as follows:

[0148] 1) The coordinate system of each imaging device of the coaxial scanning device is as follows: the coordinate origin is located on the central axis of the rotating frame and at the same height as the optical axis of the imaging device, the coordinate Z axis points to the center of the imaging device from the coordinate origin, and the XY plane is perpendicular to the Z axis; for the surface voxel data at each imaging view , the XY plane is selected as the projection plane, and the orthogonal projection is performed along the negative direction of the Z axis, so that the three-dimensional data (x, y, z, HU) containing the surface voxel of the object is projected into the XY plane at the corresponding angle, and each pixel point in the plane is the projection position of the three-dimensional surface voxel data on the projection plane, thereby forming the two-dimensional projection image (x, y, HU) of the CT at the angle. The CT two-dimensional projection images at all imaging views

[0149] 2) The feature detection is performed on the CT two-dimensional projection image at the angle and the optical image at the corresponding angle in a manual or automatic manner, and the significant feature points in the CT two-dimensional projection image and the optical image are obtained.

[0150] For example, the feature detection can be performed in a manual or automatic manner to find the significant feature points (which can be edges, intersections, contours, shapes, structures, etc.) in the image. The manual detection method involves marking the features of interest on the image, and is suitable for the case where specific features need to be selected with high precision. The automatic feature detection utilizes a computer vision library such as OpenCV, and by calling appropriate feature detection algorithms such as Harris corner detection, SIFT, SURF, FAST, ORB, etc., the feature points can be located effectively and quickly.

[0151] 3) The feature descriptors (the feature descriptors are quantity vectors used to represent the regions around the feature points) of the feature points found in the CT two-dimensional projection image at the angle and the optical image at the corresponding angle are calculated, and by the feature descriptors of the feature points of the two images, the feature points in the two images are matched (for example, the nearest neighbor matching is adopted, and each feature point in one image is matched with the closest feature point in the other image). After the matching is completed, the quality of the matching is determined by quantifying the similarity between the descriptors of the feature points of the two images. The similarity measure can be the Euclidean distance between the descriptor vectors, and the smaller the distance, the more similar the feature points, and the better the matching of the feature points.

[0152] Thus, the corresponding salient features between the 2D CT projection image and the corresponding fluorescence image are found. By comparing the distance metric between the feature descriptors, the best matching descriptor pair is selected to achieve feature point matching between the images.

[0153] 4) The spatial coordinate transformation relationship between the two images is calculated by matching feature point pairs; the specific steps are as follows: (1) For each feature point pair, the coordinates of the feature points are normalized by dividing them by the focal length of the imaging device; (2) For the normalized coordinates of each feature point pair, a linear equation is constructed. Assume that p(x,y) and p'(x',y') are the normalized coordinates of any pair of matching points in the CT two-dimensional projection image and the optical image, and the basic matrix is ​​given by the linear equation p' T Fp=0 definition, where F is the basic matrix, combining all angles CT two-dimensional projection images and corresponding angles under The linear equations constructed by all the feature point pairs in the optical image under the CT image are solved to obtain the basic matrix describing the geometric relationship between the two imaging devices (CT imaging detector and optical imaging camera). (3) The intrinsic parameters of the CT imaging detector and optical imaging camera are used to perform triangulation and map the feature point pairs to three-dimensional points in the world coordinate system. (4) The spatial coordinate transformation relationship between the CT two-dimensional projection image and the optical image is calculated using the three-dimensional point coordinates of the feature point pairs in the world coordinate system, including the translation vector and rotation matrix.

[0154] That is, the angle is calculated by matching feature point pairs CT two-dimensional projection images and corresponding angles under The spatial coordinate transformation relationship between the optical images under (such as translation, rotation, scaling, etc.), similarly, traverse all Complete steps 2) through 4) for all imaging angles, and finally register the two images using the calculated spatial coordinate transformation. This ensures pixel alignment between the CT 2D projection image and the optical image at every angle. This means pixel alignment between the CT 2D projection image and the optical data is achieved at all imaging angles.

[0155] 207. Coordinate mapping.

[0156] First, the coordinates of the optical image pixels are mapped into the CT image coordinate system.

[0157] The pixel position in the optical data at each angle is converted into the coordinate in the coordinate system of the registered CT two-dimensional projection image; the corresponding relationship between the pixel in the optical data and the spatial position in the CT three-dimensional voxel data is established, the image pixel information in the optical data at each angle is corresponded to the spatial position of the CT three-dimensional surface voxel, and the optical data set with three-dimensional spatial coordinates at all angles is obtained.

[0158] 208. Since the optical images at adjacent angles have sufficient overlap, the optical data with three-dimensional spatial coordinates at two adjacent angles can be registered by the registration method used in step 205, so as to be spliced and reconstructed into a complete three-dimensional optical image.

[0159] 209. Three-dimensional reconstruction post-processing. The generated three-dimensional optical image is further optimized, and the splicing place can be made more smooth and fluent by Gaussian filtering, so as to realize more real three-dimensional optical reconstruction.

[0160] In actual application, the filtering parameters are fine-tuned according to the image characteristics and requirements to obtain the best effect.

[0161] The method of the embodiment can generate an accurate and comprehensive three-dimensional model by using the high resolution and multi-layer information provided by CT scanning. This provides a powerful tool and resource for visualization, analysis and application.

[0162] The embodiment has the advantage of high resolution, that is, the CT image has high spatial resolution, can capture the tiny details and structures of the object, and can accurately express the shape and characteristics of the object. By inputting the CT image as data, real coordinate information of the three-dimensional space of the object can be provided, the three-dimensional space is positioned for preparation of three-dimensional optical reconstruction of the object, and it is helpful to generate a highly accurate three-dimensional optical model.

[0163] In addition, the embodiment of the application also provides a computing device, comprising a memory and a processor, the memory stores a computer program, the processor executes the computer program in the memory, and executes the steps of the three-dimensional optical reconstruction method based on the CT image according to any of the above embodiments.

[0164] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are contained in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0165] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and the person skilled in the art can modify, modify, replace and modify the above-described embodiments within the scope of the present application.

Claims

1. A three-dimensional optical reconstruction method based on CT images, characterized in that: include: S10, acquiring CT projection signals and optical data of the object to be reconstructed at multiple angles by means of a coaxial scanning device; each of the multiple angles being a rotation angle of the rotating coaxial scanning device relative to the stationary object to be reconstructed; the coaxial scanning device being fixed with a CT imaging device and an optical imaging device having a mounting angle with the CT imaging device; S20. Reconstructing the CT projection signals at all angles according to a reconstruction algorithm to obtain CT three-dimensional voxel data of the object to be reconstructed, and extracting surface voxel data of the CT three-dimensional voxel data of the object to obtain CT three-dimensional surface voxel data of the object to be reconstructed; S30, aligning and registering each pixel point in the optical data with a corresponding pixel point in the CT three-dimensional surface voxel data, and mapping the coordinates of the aligned and registered pixel points in the optical data to a CT image coordinate system of the CT three-dimensional voxel data, to obtain a first optical data set having three-dimensional spatial coordinates in the CT image coordinate system at all angles; S40, splicing all optical data in the first optical data set to form three-dimensional optically reconstructed information; S20 reconstructs the CT projection signals at all angles according to the reconstruction algorithm to obtain CT three-dimensional voxel data of the object to be reconstructed, including: The CT projection signals at each angle are preprocessed, and the filtered back-projection algorithm is used to reconstruct the preprocessed CT projection signals at all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed; The filtered back-projection algorithm is used to reconstruct the pre-processed CT projection signals at all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed, including: S21, performing one-dimensional Fourier transform on each of the N preprocessed CT projection signals to obtain N first projection signals in the frequency domain; S22, filtering the N first projection signals in the frequency domain to obtain N filtered second projection signals; S23, performing a one-dimensional inverse Fourier transform on the N second projection signals to restore them to the time domain, thereby obtaining N filtered third projection signals in the time domain; S24. Back-projecting each third projection signal. Back-projection involves evenly distributing the projection signal at each angle to each point passing through the object along its original projection path. Back-projection signals at the same point on the object at all angles are accumulated to obtain the ray attenuation coefficient for each point on the object, thereby reconstructing 3D CT voxel data of the object. The CT three-dimensional voxel data includes: the three-dimensional spatial coordinates of each voxel in the reconstructed object CT image and the HU value of each voxel position, and the HU value reflects the absorption degree of the object to be reconstructed to X-rays.

2. The method according to claim 1, characterized in that The optical imaging device is a hyperspectral imaging device, and the optical data is hyperspectral data; If the optical imaging device is a fluorescence imaging device, the optical data is fluorescence optical data; If the optical imaging device is an RGB imaging device, the optical data is RGB optical data; The multi-angle CT projection signals and optical data include: N CT projection signals and N optical data; The rotating frame of the coaxial scanning device rotates relative to the object each time by an angle of Then the number of data generated when the coaxial scanning device rotates 360° relative to the object is 3. The method according to claim 1, characterized in that Extracting surface voxel data of the object's CT three-dimensional voxel data in S20 to obtain CT three-dimensional surface voxel data of the object to be reconstructed includes: S25, optimizing the CT three-dimensional voxel data to obtain optimized CT three-dimensional voxel data; S26, extracting surface information of the optimized CT three-dimensional voxel data; S27, dividing the surface information of the CT three-dimensional voxel data into voxel data belonging to the object and voxel data belonging to the background based on a preset voxel data threshold; S28. Based on the voxel data of the object, obtain the voxel data of the object boundary by traversing; set the HU value of the voxel data of the object that is not the object boundary in the voxel data of the object to 0, and obtain the surface voxel data of the object, namely, the CT three-dimensional surface voxel data.

4. The method according to claim 3, characterized in that The S30 includes: The coaxial scanning device rotates relative to the object, and the angle between the imaging angle and the object at each imaging position is The installation angle between the CT imaging device and the optical imaging device is θ; The coordinate system of each imaging device in the coaxial scanning device is as follows: the coordinate origin is located at the central axis of the rotating frame and is at the same height as the optical axis of the imaging device. The coordinate Z axis points from the coordinate origin to the center of each imaging device. The X-ray imaging Z axis points from the coordinate origin to the X-ray center. The XY plane is perpendicular to the Z axis. S31, for each imaging perspective The surface voxel data under the XY plane is selected as the projection plane, and orthogonal projection is performed along the negative direction of the Z axis to project the three-dimensional data (x, y, z, HU) containing the object surface voxels into the XY plane under the corresponding angle. Each pixel point in the plane is the projection position of the three-dimensional surface voxel data on the projection plane, forming a CT two-dimensional projection image (x, y, HU) at the angle; to obtain all imaging angles CT two-dimensional projection images under; S32. Imaging viewing angle CT two-dimensional projection images and imaging viewing angles under Perform feature detection on the optical data to obtain significant feature points in the CT two-dimensional projection image and the optical data; S33, obtaining feature descriptors of the respective significant feature points and performing matching to obtain matching feature point pairs exceeding a preset threshold; S34. Obtain imaging perspective based on matched feature point pairs CT two-dimensional projection images and imaging viewing angles under The spatial coordinate transformation mapping of the optical data under S35, based on the spatial coordinate conversion mapping, the imaging perspective CT two-dimensional projection images and imaging viewing angles under The optical data under the Based on the method from S32 to S35, traverse all imaging angles The CT two-dimensional projection images and optical data at all imaging angles are aligned and registered.

5. The method according to claim 4, characterized in that The S34 includes: S341. Based on each matched feature point pair, divide the coordinates of the feature point belonging to each imaging device by the focal length of the imaging device to obtain normalized feature point pair coordinates; S342. Construct a linear equation based on the normalized feature point pair coordinates; Assume p(x,y) and p'(x',y') are the normalized coordinates of the feature point pair; p(x,y) corresponds to the CT two-dimensional projection image, and p'(x',y') corresponds to the optical data; Through the linear equation p′ T Fp=0, determine the basic matrix, where F is the basic matrix; Collect all the linear equations constructed by the feature point pairs and solve the basic matrix; S343. Based on the fundamental matrix, triangulation is performed using the intrinsic parameters of the CT imaging device and the optical imaging device to map the normalized feature point pair coordinates to three-dimensional points in the world coordinate system; S344. Using the three-dimensional point coordinates of the mapped normalized feature point pairs in the world coordinate system, obtain the spatial coordinate transformation mapping, where the spatial coordinate transformation mapping includes a translation vector and a rotation matrix.

6. The method according to claim 4, characterized in that The S34 includes: Convert the pixel position in the optical data at each angle into the coordinates in the registered CT two-dimensional projection image coordinate system; A correspondence is established between pixels in the optical data and spatial positions in the CT three-dimensional surface voxel data, and the pixel information in the optical data at each angle is mapped to the spatial position of the CT three-dimensional surface voxel data to obtain a first optical data set with three-dimensional spatial coordinates at all angles.

7. The method according to claim 6, characterized in that The S40 includes: Using the S30 method, traversing all adjacent optical data, registering the adjacent optical data, identifying overlapping areas, and splicing all optical data in the first optical data set based on the identified overlapping areas to form three-dimensional optically reconstructed information; The information after three-dimensional optical reconstruction is optimized to obtain complete three-dimensional optical reconstruction information.

8. A computing device, characterized in that include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program in the memory and performs the steps of the three-dimensional optical reconstruction method based on CT images as described in any one of claims 1 to 7.

Images