Hyperspectral three-dimensional collaborative imaging method based on object surface coding
Through the hyperspectral three-dimensional collaborative imaging method based on object surface encoding, using Hadamma matrix coding and inverse dispersion operation, the problem of difficult to take into account in the existing technology of high efficiency and high quality imaging is achieved, and high-precision hyperspectral three-dimensional imaging is achieved. And suitable for miniaturized design.
Patent Information
- Application Number
- CN202410670474.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-28
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing hyperspectral three-dimensional collaborative imaging methods are difficult to achieve high efficiency and high quality simultaneously when combined with imaging. The reconstruction quality depends on the algorithm and has poor migration, and the data fusion accuracy is limited by the acquisition system and algorithm confidence.
Using hyperspectral three-dimensional collaborative imaging method based on object surface encoding, the 4D encoded compressed data of the object is obtained through the hyperspectral three-dimensional collaborative imaging system, and using Hadamma matrix encoding and inverse dispersion operation, the stripping algorithm obtains spectral three-dimensional data, avoiding data fusion and Phase solution to achieve high-precision imaging.
High-precision hyperspectral three-dimensional collaborative imaging is realized, the system structure is simple, the component size is small, suitable for miniaturization design, improve imaging quality and improve time efficiency.
Smart Images

Figure CN118247443B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a hyperspectral three-dimensional collaborative imaging method based on object plane coding, and belongs to the technical field of computer image processing. Background Art
[0002] In the field of hyperspectral imaging, due to the "fingerprint effect" of materials, spectral imaging technology has shown great application prospects in remote sensing, medical treatment, cultural relics protection and other fields. The number of imaging bands has also transitioned from multispectral imaging with dozens of bands to hyperspectral imaging with more imaging bands. In the field of three-dimensional imaging, because it can provide depth information of the scene, three-dimensional reconstruction technology has a wide range of applications in medical imaging, cultural relics restoration, autonomous driving and other fields.
[0003] Therefore, given the advantages of hyperspectral imaging and three-dimensional imaging, hyperspectral three-dimensional collaborative imaging is particularly important. However, due to the increase in imaging dimensions and the increase in data volume, the difficulty of solving the problem has also increased. How to achieve the combination of two imaging methods through a single system has become a difficult point in the research. Among the current hyperspectral three-dimensional collaborative imaging methods, there are mainly two. One is based on compressed sensing and deep learning, which can achieve transient imaging, but because its reconstruction is a highly pathological process, the reconstruction quality is highly dependent on the algorithm and has poor transferability, so the reconstruction quality is not ideal. The other is based on the idea of fusion, which mainly collects data from two dimensions and then fuses the data before displaying it. The fusion accuracy is subject to the matching degree of the acquisition system and the confidence of the fusion algorithm. At the same time, the time spent on data acquisition is also relatively large, and the imaging quality is also average, which cannot take into account both universality and efficiency.
[0004] Therefore, a hyperspectral three-dimensional collaborative imaging method based on object plane coding is needed to solve the above problems. Summary of the invention
[0005] Purpose of the invention: In view of the problems existing in the prior art, the present invention provides a hyperspectral three-dimensional collaborative imaging method based on object surface coding.
[0006] A hyperspectral three-dimensional collaborative imaging method based on object plane coding adopts a hyperspectral three-dimensional collaborative imaging system, wherein the hyperspectral three-dimensional collaborative imaging system comprises a projection device, a dispersion device and an image acquisition device, wherein the projection device and the image acquisition device are synchronously triggered, and both the projection device and the image acquisition device are facing an object, and the dispersion device is arranged between the projection device and the object, and when calibrating the system, the object is a calibration device;
[0007] The following steps are involved:
[0008] Step 1: The hyperspectral three-dimensional collaborative imaging system respectively obtains the coded compressed data of the calibration device at different positions;
[0009] Step 2: Decode the collected coded compressed data of the calibration device to obtain the dispersion slope K of the system and the linear mapping of the true depth in the pixel space;
[0010] Step 3: Use the hyperspectral three-dimensional collaborative imaging system to obtain 4D coded compressed data of the object, and perform inverse dispersion operation according to the dispersion slope K of the system to obtain 4D data of the object;
[0011] Step 4: Use a stripping algorithm to strip the 4D data of the object to obtain the spectral three-dimensional data of the object.
[0012] Furthermore, the calibration device is a standard flat panel, the projection device is a DLP fringe projector, the image acquisition device is a grayscale camera, and the dispersion device is an Amici dispersion prism.
[0013] Furthermore, it also includes a guide rail, and the calibration device is arranged on the guide rail.
[0014] Furthermore, the guide rail is a stepping stage.
[0015] Furthermore, in step 1, the encoding process of the encoded compressed data adopts Hadamard matrix encoding, and in step 2, the encoded compressed data of the collected calibration device is decoded by inverting the Hadamard matrix.
[0016] Furthermore, in step 2, the collected coded compressed data of the calibration device is decoded to obtain the system's dispersion slope K and the linear mapping of the true depth in the pixel space. Specifically, the system's dispersion slope K is obtained by linear correction of the decoded data at a certain position, and the linear mapping of the true depth in the pixel space is obtained by linear fitting of the decoded data in the pixel space corresponding to different positions.
[0017] Furthermore, in step 3, the 4D coded compressed data of the object is obtained by using the hyperspectral three-dimensional collaborative imaging system, and the 4D coded compressed data is represented by the following formula:
[0018] ,
[0019] In the formula, represents the dispersion slope of the dispersion device, represents the spectral dimension, represents the dispersion operation, Indicates a delete operation. Represents an insert operation. Indicated in The coded information deleted with depth in Indicated in The zero vector embedded with depth in , represents the n-th order Hadamard matrix, Indicates the y-axis coordinate information of the spectral dimension corresponding to the i-th row, represents the spectral depth information, express Copied layer, Represents Hadamard.
[0020] Furthermore, in step three, the 4D coded compressed data of the object is obtained by using the hyperspectral three-dimensional collaborative imaging system, and the inverse dispersion operation is performed according to the dispersion slope K of the system to obtain the 4D data of the object. Specifically, the 4D coded compressed data of the object is obtained by using the hyperspectral three-dimensional collaborative imaging system, the 4D coded compressed data of the object is decoded to obtain the undispersed spectral three-dimensional data, and then the inverse dispersion operation is performed according to the dispersion slope K of the system to obtain the 4D data of the object.
[0021] Furthermore, decoding the 4D coded compressed data of the object to obtain the undispersed spectral three-dimensional data is specifically performed by inverting the Hadamard matrix of the collected compressed coded data to obtain the undispersed spectral three-dimensional data.
[0022] Furthermore, in step 4, the 4D data of the object is stripped by a stripping algorithm to obtain the spectral three-dimensional data of the object, including the following steps:
[0023] 41. Preprocess the 4D data by setting a threshold to obtain 4D data without noise interference;
[0024] 42. The position coordinates of all the center points in a certain spectral dimension are obtained by selecting the center point, and the grayscale values of all the recorded points are extracted according to the coordinates to restore the three-dimensional morphology of the corresponding scene in the spectral dimension. The data of the remaining spectral dimensions only need to be extracted in the 4D data according to the coordinates of the recorded points to obtain the spectral three-dimensional data of the object.
[0025] Beneficial effects: The method for hyperspectral three-dimensional collaborative imaging based on object plane coding of the present invention does not require data fusion and phase solution in the process of solving spectral three-dimensional data, and can achieve high-precision hyperspectral three-dimensional collaborative imaging. The hyperspectral three-dimensional collaborative imaging system used has a simple structure, and the size of the components is small, which is suitable for miniaturization design. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is a schematic diagram of the process of the hyperspectral three-dimensional collaborative imaging method based on object plane coding of the present invention;
[0027] Figure 2 It is a schematic diagram of the object plane coding hyperspectral three-dimensional imaging system of the present invention;
[0028] Figure 3This is the process of obtaining the system dispersion slope K in the present invention;
[0029] Figure 4 is the linear fitting result of the real depth in the pixel space in the present invention;
[0030] Figure 5 This is a schematic diagram of the simulation of the 4D data encoding, compression and acquisition process in the present invention;
[0031] Figure 6 It is a schematic diagram of the simulation process of 4D coded compressed data decoding in the present invention;
[0032] Figure 7 It is a three-dimensional information source diagram of the hyperspectral three-dimensional imaging system;
[0033] Figure 8 Schematic diagram of the three-dimensional doll imaging results of the hyperspectral three-dimensional imaging system. DETAILED DESCRIPTION
[0034] The preferred embodiments of the present invention will be described below in conjunction with the accompanying drawings to more clearly and completely illustrate the technical solutions of the present invention.
[0035] See also Figure 1 and Figure 2 As shown, the hyperspectral three-dimensional collaborative imaging method based on object plane coding of the present invention adopts a hyperspectral three-dimensional collaborative imaging system, including a projection device, a dispersion device and an image acquisition device. The projection device is a DLP stripe projector, the dispersion device is a dispersion prism, and the image acquisition device is a grayscale camera. The calibration device is a standard flat plate, and the movement of the standard flat plate in different positions is achieved by a stepping displacement stage. Among them, the projector adopts DLP6500 with a synchronous trigger function, and the frame rate can reach 250FPS in normal mode. The dispersion prism uses an Amisi dispersion prism with linear dispersion capability. The camera uses Basler-ACA1920-155um, the pixel is 5.86um, and the maximum frame rate is 160Hz. The flatness range of the standard flat plate is 0.01-0.03mm. The stepping displacement stage uses a QRXQ CNC synchronous belt slide module. The displacement accuracy can reach 0.01mm.
[0036] The hyperspectral three-dimensional collaborative imaging method based on object plane coding of the present invention comprises the following steps:
[0037] Step 1: Based on the hyperspectral three-dimensional collaborative imaging system, the encoded compressed data of the calibration device at different positions are obtained respectively.
[0038] The encoding process of the present invention adopts Hadamard matrix encoding. The Hadamard encoding matrix is burned into the projection sequence of the projector according to the Hadamard sequence, and the encoding compression data of the standard flat plate is collected by using the synchronous trigger mode of the projector and the camera. The standard flat plate is moved within a range of 100 mm by a stepping stage, and the coordinate value before each movement is recorded, and twenty sets of encoding compression data at different positions are collected.
[0039] Step 2: Based on the principles of spectral shift invariance and multiplexing encoding and decoding, the encoded compressed data of the collected calibration device is decoded, and a quantitative relationship between the modulation degree and the three-dimensional morphology is constructed to obtain the system's dispersion slope K and the linear mapping of the true depth in the pixel space.
[0040] like Figure 3 As shown in FIG. 1 , the multiplexing encoding and decoding principle described in step 2, wherein the decoding process uses the Hadamard matrix inversion. The decoded spectral data of a single row in the standard plate is as follows: Figure 3 As shown in part a, the data of one spectral dimension is extracted from the decoded data as follows Figure 3 As shown in part b, the tilted spectral data is then corrected back using a circular shift algorithm. Figure 3 As shown in part c, the slope value is set to meet the requirement that 60% of the points are in the same column of spectral dimensions, and the optimal dispersion slope of the system is found. Figure 3 As shown in part d.
[0041] like Figure 4 As shown in Table 1, the quantitative relationship between the modulation degree and the three-dimensional morphology described in step 2 can be obtained by decoding the coded compressed data of 20 groups at different positions and repeating the spectral data correction steps to obtain the depth value of the pixel space corresponding to the same point at different positions. Figure 4 From the results shown, the system mean square error is approximately 0.25 mm.
[0042] Table 1 Depth values of the same point in different positions in the pixel space
[0043]
[0044] Step 3: Obtain 4D coded compressed data of the object based on the hyperspectral three-dimensional collaborative imaging system, where the coded information will produce two changes corresponding to the system dispersion slope K according to the actual depth. During the decoding process, the inverse dispersion operation is performed according to the system dispersion slope K to restore the 4D data of the object.
[0045] like Figure 5 As shown, the collection process of the 4D coded compressed data in step 3 is simulated as follows:
[0046] The spectral depth information of a single row of an object is expressed as , the n-order Hadamard matrix H is written as ,in Without considering the modulation of depth z on the encoding process, The encoding process can be expressed as:
[0047] (1)
[0048] in Indicates the y-axis coordinate information of the spectral dimension corresponding to the i-th row, represents the spectral depth information, express Copied layer, represents the Hadamard product. After adding the depth modulation z, The encoded information in will produce two changes corresponding to the system dispersion slope K according to the actual depth. One is the shadow caused by the depth, which we call insertion Operation, one is due to the occlusion caused by depth, which we call deletion operate.
[0049] (2)
[0050] (3)
[0051] in, Indicated in The zero vector embedded with depth in , Indicated in The encoded information is deleted with depth. After adding spectral modulation, The encoded information in the dispersion prism will be linearly dispersed according to the dispersion characteristics, which we call dispersion operate.
[0052] (4)
[0053] in, Indicates the dispersion slope of the dispersion device, the spectral dimension According to the dispersion slope After completing the deep spectrum modulation, the image acquisition device Perform compression acquisition to obtain the final compressed sampling data .
[0054] (5)
[0055] like Figure 6As shown, the coded information in step 3 generates two changes corresponding to the system dispersion slope K according to the real depth, one is shadow and the other is occlusion. In the decoding process, these two changes will respectively cause the system dispersion slope K to shift backward ( Figure 6 Right Arrow Path) and Forward ( Figure 6 Left-pointing arrow path), and then record the shape information of the object.
[0056] like Figure 7 As shown, the decoding of the 4D coded compressed data in step 3 only requires inverting the Hadamard matrix of the collected compressed coded data to obtain the undispersed spectral three-dimensional data, and finally the inverse dispersion operation is performed through the dispersion slope K of the system to restore the 4D data of the object.
[0057] Step 4: Use the stripping algorithm to strip the 4D data and finally obtain the spectral three-dimensional data of the object.
[0058] The stripping algorithm described in step 4 first pre-processes the 4D data by setting a threshold to obtain 4D data without noise interference, and then obtains the position coordinates of all center points in a certain spectral dimension by selecting the center point. The grayscale values of all recorded points are extracted according to the coordinates to restore the three-dimensional morphology of the real scene corresponding to the spectral dimension. The data of the remaining spectral dimensions only need to be extracted in the 4D data according to the coordinates of the recorded points to obtain all the spectral three-dimensional data of the object.
[0059] Finally, if Figure 8 As shown in the figure, we selected a colored doll as the actual measurement scene of the system. By reconstructing the collected data, we can achieve high-spectral three-dimensional imaging with a spectral resolution of nearly 3nm, nearly 120 imaging bands, and a spatial accuracy of 0.25mm. 40 bands were selected for display, and it can be seen that the reconstruction accuracy is high and the spectral details are obvious. The three bands of 450nm, 550nm, and 650nm were selected for RGB pseudo-color synthesis and displayed from three perspectives. It can be seen that compared with the two-dimensional color image, it has a higher consistency.
Claims
1. A hyperspectral three-dimensional collaborative imaging method based on object surface coding, characterized in that: A hyperspectral three-dimensional collaborative imaging system is used, which includes a projection device, a dispersion device and an image acquisition device. The projection device and the image acquisition device are synchronously triggered, and both the projection device and the image acquisition device are facing an object. The dispersion device is arranged between the projection device and the object. When calibrating the system, the object is a calibration device. The following steps are involved: Step 1: The hyperspectral three-dimensional collaborative imaging system respectively obtains the coded compressed data of the calibration device at different positions; Step 2: Decode the collected coded compressed data of the calibration device to obtain the dispersion slope K of the system and the linear mapping of the true depth in the pixel space; Step 3: Use the hyperspectral three-dimensional collaborative imaging system to obtain 4D coded compressed data of the object, and perform inverse dispersion operation according to the dispersion slope K of the system to obtain 4D data of the object; In step 3, the 4D coded compressed data of the object is obtained by using the hyperspectral three-dimensional collaborative imaging system. The 4D coded compressed data is represented by the following formula: , In the formula, represents the dispersion slope of the dispersion device, represents the spectral dimension, represents the dispersion operation, Indicates a delete operation. Represents an insert operation. Indicated in The coded information deleted with depth in Indicated in The zero vector embedded with depth in , represents the n-th order Hadamard matrix, Indicates the y-axis coordinate information of the spectral dimension corresponding to the i-th row, represents the spectral depth information, express Copied layer, Represents Hadamard; Step 4: Use a stripping algorithm to strip the 4D data of the object to obtain the spectral three-dimensional data of the object.
2. The hyperspectral three-dimensional collaborative imaging method based on object plane coding according to claim 1, characterized in that: The calibration device is a standard flat panel, the projection device is a DLP fringe projector, the image acquisition device is a grayscale camera, and the dispersion device is an Amici dispersion prism.
3. The hyperspectral three-dimensional collaborative imaging method based on object plane coding according to claim 1, characterized in that: It also includes a guide rail, and the calibration device is arranged on the guide rail.
4. The hyperspectral three-dimensional collaborative imaging method based on object plane coding according to claim 3, characterized in that: The guide rail is a stepping platform.
5. The hyperspectral three-dimensional collaborative imaging method based on object plane coding according to claim 1, characterized in that: In step 1, the encoding process of the encoded compressed data adopts Hadamard matrix encoding, and in step 2, the encoded compressed data of the collected calibration device is decoded by inverting the Hadamard matrix.
6. The hyperspectral three-dimensional collaborative imaging method based on object plane coding according to claim 1, characterized in that: In step 2, the collected coded compressed data of the calibration device is decoded to obtain the system's dispersion slope K and the linear mapping of the true depth in the pixel space. Specifically, the system's dispersion slope K is obtained by linear correction of the decoded data at a certain position, and the linear mapping of the true depth in the pixel space is obtained by linear fitting of the decoded data in the pixel space corresponding to different positions.
7. The hyperspectral three-dimensional collaborative imaging method based on object plane coding according to claim 1, characterized in that: In step three, the 4D coded compressed data of the object is obtained by using the hyperspectral three-dimensional collaborative imaging system, and the inverse dispersion operation is performed according to the dispersion slope K of the system to obtain the 4D data of the object. Specifically, the 4D coded compressed data of the object is obtained by using the hyperspectral three-dimensional collaborative imaging system, the 4D coded compressed data of the object is decoded to obtain the undispersed spectral three-dimensional data, and then the inverse dispersion operation is performed according to the dispersion slope K of the system to obtain the 4D data of the object.
8. The hyperspectral three-dimensional collaborative imaging method based on object plane coding according to claim 7, characterized in that: The 4D coded compressed data of the object is decoded to obtain the 3D spectral data without dispersion. Specifically, the Hadamard matrix is inverted on the collected compressed coded data to obtain the 3D spectral data without dispersion.
9. The hyperspectral three-dimensional collaborative imaging method based on object plane coding according to claim 1, characterized in that: In step 4, the 4D data of the object is stripped by a stripping algorithm to obtain the spectral three-dimensional data of the object, including the following steps:
41. Preprocess the 4D data by setting a threshold to obtain 4D data without noise interference; 42. The position coordinates of all the center points in a certain spectral dimension are obtained by selecting the center point, and the grayscale values of all the recorded points are extracted according to the coordinates to restore the three-dimensional morphology of the corresponding scene in the spectral dimension. The data of the remaining spectral dimensions only need to be extracted in the 4D data according to the coordinates of the recorded points to obtain the spectral three-dimensional data of the object.
