4d multi-frame spectral three-dimensional imaging method based on aperture coding and digital image correlation

Abstract

The application discloses a 4D multi-frame spectral three-dimensional imaging method based on aperture coding and digital image correlation, adopts a 4D imaging system, and tightly couples and cooperatively images two different mechanisms of spectral imaging and three-dimensional imaging. In the system, a mask in an aperture coding spectral imaging module is removed, a target object is actively illuminated with an aperture coding pattern, and then an image after spatial light intensity modulation is obtained, so that the effect of the mask is achieved. The application combines active coded aperture snapshot spectral imaging with digital image correlation (DIC), creatively combines a digital speckle image used for 3D imaging and a random code required for spectral imaging, and can synchronously perform spectral reconstruction and three-dimensional reconstruction, so that the system speed is greatly improved. Since the active CASSI scheme is adopted to realize spectral imaging, and no mechanical motion structure and mask are contained, the device structure is stable and fast sampling can be achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to a 4D multi-frame spectral three-dimensional imaging method based on aperture coding and digital image correlation, belonging to the field of optical measurement technology. BACKGROUND

[0002] The spectrum of a point in a scene is represented by the distribution of its electromagnetic radiation over a certain wavelength range. In traditional digital imaging devices, the spectrum is measured with three-channel red, green, blue (RGB) sensors, which are designed to match the three-viewpoint color measurement in the human visual system. However, the three-channel representation cannot capture the complex details of the natural scene spectrum, which come from the diversity and complexity of the illumination and reflection spectrum in the real world. Since the properties of various materials and objects can be inferred from detailed spectra, a collection system for accurate spectral measurement can be an effective tool for scientific research and engineering applications.

[0003] With the maturation of under-sampling spectral imaging systems and the significant improvement of back-end computing power, people began to hope that the available spectral data could have higher dimensions and accuracy. With the progress of optical metrology methods in three-dimensional (3D) shape measurement in recent years, high-spectral imaging systems are no longer satisfied with spectral analysis of two-dimensional (2D) image planes. Optical methods based on triangulation have been used for three-dimensional shape measurement, combined with spectral measurement means to obtain four-dimensional (4D) data of the target. The three-dimensional imaging solutions used in these works can be divided into passive and active. Passive includes stereo vision (Stereo vision) and structure from motion (SfM, Structure from Motion) and so on. While active includes structured light (SL, Structured Light) and laser triangulation (Laser triangulation).

[0004] The implementation of the above scheme depends on the combination of spectral imaging modules and three-dimensional imaging modules, and the system structure becomes complex. In turn, it causes low degree of coupling between modules, difficulty in registration, high cost, and other drawbacks. In addition, most of the spectral imaging modules in the above system contain mechanical motion structures, which have a negative impact on the stability of the system and limit the sampling rate. There are also filter-based spectral imaging modules, such as acousto-optic tunable filters (AOTF, Acousto-optic Tunable Filter) or Fabry-Pérot interference (FPI, Fabry-Pérot Interference) filters, which simplify the system structure, but also bring high prices.

[0005] Therefore, a new 4D multi-frame spectral three-dimensional imaging method is needed to solve the above problems. SUMMARY

[0006] The present application aims to provide a 4D multi-frame spectral three-dimensional imaging method based on aperture coding and digital image correlation to solve the problems raised in the background.

[0007] A 4D multi-frame spectral three-dimensional imaging method based on aperture coding and digital image correlation, using a 4D imaging system, the 4D imaging system comprising a projector, an aperture coding spectral imaging module and a second vision module,

[0008] The projector, aperture coding spectral imaging module and second vision module are all facing the target object;

[0009] The aperture coding spectral imaging module comprises a collimating objective lens L1, an Amici dispersion prism L2, a first focusing lens L3 and a first camera, which are sequentially arranged in front of the target object;

[0010] When the Amici dispersion prism L2 is removed from the aperture coding spectral imaging module, the aperture coding spectral imaging module becomes a first vision module;

[0011] The second vision module comprises a second focusing lens L3 and a second camera, which are sequentially arranged in front of the target object;

[0012] The first vision module and the second vision module constitute a binocular stereo vision module;

[0013] Comprising the following steps:

[0014] Step 1: The projector projects an aperture coding pattern and a digital speckle pattern onto the surface of the target object, and uses the aperture coding spectral imaging module to collect a random coding image, and uses the binocular stereo vision module to collect a digital speckle image;

[0015] Step 2: The random coding image collected in step 1 is reconstructed by TwIST algorithm to obtain a spectral reconstruction result; the digital speckle image collected in step 1 is measured by DIC method to obtain a three-dimensional point cloud;

[0016] Step 3: The spectral reconstruction result of step 2 and the three-dimensional point cloud are fused to obtain a 4D multi-frame spectral three-dimensional imaging.

[0017] Further, the measurement value c on the detector of the first camera of step 1 is represented by the following formula:

[0018] c = Hf

[0019] Where H is a linear operator and f is the spectral source density.

[0020] Further, the light projected by the projector onto the object is white.

[0021] Further, the projector in step one projects the same aperture coding pattern to the target object with red light and blue light in sequence.

[0022] Further, the offset distance Θ of the random coding image in step two is obtained by the following formula:

[0023] Θ = Q1(λ -2 ) 2 + Q2(λ -2 ) + Q3

[0024] Where Q1, Q2 and Q3 are weight coefficients of wavelength position distribution in the camera coordinates of the first camera.

[0025] Further, the distortion state P of the random coding image in step two is represented by the following formula:

[0026] P = Norm(histeq(P b1 / P b2 ) + histeq(P r1 / P r2 ))

[0027] Where histeq(·) is a histogram equalization function, Norm(·) is a normalization function, P b1 and P r1 are the actual shape of the random coding pattern, P b2 and P r2 are the negative template of the random coding pattern.

[0028] Further, the spectral reconstruction result of step two is fused with the three-dimensional point cloud in step three, including the following steps:

[0029] Step 31, the peak value images of the red, green and blue three peaks in the spectral reconstruction result of step two are taken as RGB channels, and color blending is performed to obtain a two-dimensional color image;

[0030] Step 32, the three-dimensional space points of the three-dimensional point cloud in step two are corresponded to the point coordinates of the two-dimensional image to obtain three-dimensional space point coordinates;

[0031] Step 33, the coordinates of the two-dimensional color image in step 31 are corresponded to the three-dimensional space point coordinates to obtain 4D multi-frame spectral three-dimensional imaging.

[0032] Further, the binocular three-dimensional measurement in step two is performed by the DIC method, and the reference subset and the target subset in the DIC method are matched by using the ZNCC coefficient:

[0033] C ZNCC (P)=1-0.5×C ZNSSD (P)

[0034]

[0035] In the formula, wherein, f(x, y) is the gray intensity at the (x, y) coordinate position of the original picture, g(x', y') is the gray intensity at the (x', y') coordinate position of the deformed picture, is the average gray intensity of the original picture, is the average gray intensity of the target picture, P is the directional vector relative to the displacement mapping function, and N is the total number of effective object pixels in the subset.

[0036] Further, the aperture coding pattern adopts multi-frame coding. The multi-frame coding can effectively improve the spectral recovery quality, and then improve the quality of three-dimensional reconstruction.

[0037] Beneficial effects: the 4D multi-frame spectral three-dimensional imaging method based on aperture coding and digital image correlation of the application combines aperture coding and digital image correlation, creatively combines the digital speckle image for 3D imaging and the random coding image required for spectral imaging, and three-dimensional reconstruction and spectral reconstruction can be performed synchronously, so that the system speed is greatly improved. The aperture coding spectral imaging module is used to realize spectral imaging, does not contain mechanical motion structure and mask plate, can realize stable device structure and fast sampling, has simple structure, and is stable and easy to maintain. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 Fig. 1 is a structural schematic diagram of a 4D spectral imaging system;

[0039] Figure 2 Fig. 2 is a schematic diagram of a conventional digital image correlation three-dimensional measurement imaging system;

[0040] Figure 3 Fig. 3 is a standard ball three-dimensional reconstruction result diagram;

[0041] Figure 4 Fig. 4 is a comparison diagram of FPP and DIC three-dimensional reconstruction results;

[0042] Figure 5 Fig. 5 is a system spectral measurement verification diagram using a standard color doll;

[0043] Figure 6 Fig. 6 is a color doll 4D atlas measurement result diagram.

[0044] Figure 7 This is a graph showing the 4D spectral measurement results of a mango. Detailed Implementation

[0045] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0046] A. 4D Spectral Imaging System

[0047] like Figure 1 As shown, the projector (DLP6500) projects a series of templates onto the surface of the observed target. The spectral imaging module obtains the dispersive image of the target after illumination modulation by the projector. L1 is a collimating objective lens; light is collimated by L1 before reaching the Amici dispersion prism L2. The light is dispersed into a spectral range of 420-660 nm by the Amici dispersion prism L2, and then focused onto the camera (Hikvision CE120-10UM) by the focusing lens L3. The desired dispersive image is projected by the projector (DLP6500) and acquired by two cameras (Hikvision CE120-10UM) for 3D reconstruction and spectral reconstruction models.

[0048] Collimating objective L1, Amish prism L2, focusing lens L3, and camera form a maskless quasi-monochromatic speckle coded snapshot imaging spectrometer (SD-CASSI) system. Structured light modulates the spatial information of all wavelengths in the detected target with a coded pattern. This produces an image of multiple coded scenes at wavelength-dependent locations on the detector array plane. The spatial intensity pattern on this plane contains a coded mixture of spatial and spectral information about the scene. Images are acquired simultaneously by the left and right cameras, and the acquired digital speckle patterns are used for 3D reconstruction. The entire process is fast, can be performed synchronously, and achieves high-precision, high-speed 4D imaging.

[0049] B. Mathematical Model of System Operation

[0050] This system is mainly divided into two modules: aperture-coded spectral imaging module and binocular stereo vision module.

[0051] The spectral intensity entering the instrument under uniform illumination (without structured light coding) can be expressed as f0(x, y; λ). Let the transfer function of the coded pattern be T(x, y), then the spectral intensity entering the aperture is:

[0052] f1(x,y;λ)=f0(x,y;λ)T(x,y) (1)

[0053] After propagation through the dispersive element, the spectral density of the detector plane is:

[0054] f2(x, y; λ) = ∫∫δ(x' - [x + Θ(λ - λc)]) δ(y' - y) x f1(x', y'; λ) dx' dy'

[0055] = f0(x + Θ(λ - λ c ), y; λ) T(x + Θ(λ - λ c ), y) (2)

[0056] Unlike previous analyses, the dispersion shift of different wavelengths is not approximated as linear due to the use of an Ames prism in this scheme. In the above equation, Θ represents the dispersion shift of the dispersive element, which is linearly positively correlated with the dispersion angle θ at different λ c under near-field conditions. The δ(·) function describes the propagation through the single magnification imaging optics and the dispersive element with linear dispersion α and central wavelength λ c The detector array is not wavelength sensitive, measuring the intensity of the incident light rather than the spectral density. Therefore, the continuous image on the detector array can be represented as:

[0057] c(x, y) = ∫f0(x + Θ(λ - λ c ), y; λ) T(x + Θ(λ - λ c ), y) dλ (3)

[0058] It is worth noting that this image is a sum over the wavelength dimension of a data cube that has been mask-modulated and post-cropped. The detector pixelizes the spectral density with pixel size Δ, where each pixel position (n, m) at the image c corresponds to a physical position (x, y). Then, the captured image c(m, n) can be described as:

[0059]

[0060] Let each element of the coded aperture be the same size as the detector pixel Δ, then the mask function T(x, y) can be represented as a discrete Boolean function of a two-dimensional square pinhole array t(m', n'):

[0061]

[0062] Therefore, we have:

[0063]

[0064] Expressing the discrete form of the spectral source density f0(x, y; λ) as f ijk , the aperture coding function T(x, y) as t ijThe measurements of the detector can be written in matrix form as:

[0065]

[0066] The above equation can also be described as a matrix-vector equation:

[0067] c = Hf (8)

[0068] where H is a linear operator representing the forward model of the system. The matrix H is constructed by registering the mask pattern of the target wavelength and the dispersion coefficient. The matrix H projects the voxels of the three-dimensional sample and shear information f to the pixels of the detector array c. By minimizing ||c = Hf||2 2 f is estimated as the hyperspectral image of the object. Here the TwIST is used as the reconstruction algorithm.

[0069] Figure 2 A typical DIC three-dimensional measurement imaging system is shown. The difference from Figure 1 is that the collimating lens and the dispersion element required for spectral imaging are removed. In the operation of the system, the projector projects a structured pattern (usually a digital speckle pattern) onto the object surface; the double camera records the image of the digital speckle pattern, and the corresponding point relationship is obtained through stereo matching; and the three-dimensional coordinates are calculated through the camera calibration parameters.

[0070] In order to track the same point in I left and I right , a reference subset centered on the point of interest is extracted from I left . Then, in each iteration, a target subset centered on the initial guess is extracted from I right , transformed and compared to the reference subset. Once the transformed target subset that best matches the reference subset is found, the corresponding points between I left and I right are determined. The criterion we use to determine the best match between the reference and target subsets is the modified zero-mean normalized sum of squared differences (ZNSSD) criterion, which is insensitive to potential scale and shift variations in the subset intensities. The ZNSSD coefficient can be expressed as:

[0071]

[0072] where f(x, y) is the gray intensity at the (x, y) coordinate position of the original picture, and g(x', y') is the gray intensity at the (x', y') coordinate position of the deformed picture. is the average gray intensity of the original picture, is the average gray intensity of the target picture. P is the directional vector relative to the displacement mapping function. N is the total number of valid object pixels in the subset.

[0073] The relationship between the ZNSSD coefficient and the zero-mean normalized correlation coefficient (ZNCC) is shown in the following equation:

[0074] C ZNCC (P) = 1 - 0.5 x C ZNSSD (P) (10)

[0075] Since the range of the ZNCC coefficient is -1 to 1, the larger the number, the higher the similarity between the target subset and the reference subset, so it is relatively simple to prove the similarity.

[0076] In the stereo vision method, when a point in the three-dimensional space is captured by two cameras at the same time, the three-dimensional data of the point is obtained by using the two-dimensional coordinates of the point mapped in the coordinate systems of the two different cameras and combining the device calibration parameters. After stereo matching, for a point (X, Y, Z) on the object to be measured, the points in the two-dimensional pixel coordinate system mapped to the left and right cameras are (u l , v l ) and (u r , v r ). Through the above process, the corresponding points on the object to be measured can be found in the two-dimensional coordinates of the imaging devices. Subsequently, using the parameters for device calibration in the previous section, the three-dimensional coordinates of the corresponding points can be obtained.

[0077]

[0078]

[0079] where M1 is the product of the internal and external reference matrices of the left camera, and M2 is the product of the internal and external reference matrices of the right camera. Four linear equations with respect to the three-dimensional coordinates (X, Y, Z) can be obtained by relating the two equations

[0080]

[0081] The above equation contains four equations and three unknowns, so the equation set can be solved to obtain the optimal solution using the least squares method.

[0082] C, code mode extraction and geometric accuracy verification

[0083] Unlike imaging spectrometers using spatial light modulators such as DMD, the aperture encoding pattern required by active CASSI is distorted by the modulation of the shape change of the object surface, and is no longer a regular pattern. After dispersion through the Amici prism, spatial aliasing occurs, which causes the specific shape of the aperture encoding pattern to be unable to be directly taken. For example, Figure 3The method to solve this problem in this system takes advantage of the way the projector's white light is composed: narrow bandwidth red and blue light and longer bandwidth green light. The projector projects the same aperture-encoding pattern with red and blue light in succession. Because the dispersion length of the Amici prism is very short, the dispersed white light with bandwidths over 200 nm only occupies about 20 pixels. Therefore, the dispersion length of the red and blue light with bandwidths of only 20 nm is only between 2-3 pixels. After reducing the camera exposure time, the dispersion length can be reduced to within one pixel, and then the actual shape of the aperture-encoding pattern, i.e., P b1 , P r1 , can be obtained. On the other hand, the system also captures the "negative template" of the "0" and "1" reversed, i.e., P b2 , P a . Dividing the positive template by the negative template can effectively reduce the gray level unevenness caused by color differences or shadows. Using this method, the state of the encoding template after distortion P can be extracted, and the start and end positions of the dispersion can be accurately obtained, which facilitates the selection of ROI. The following is the process of obtaining P:

[0084] P = Norm(histeq(P b1 / P b2 )+histeq(P r1 / P r2 )) (14)

[0085] Where histeq(·) is the histogram equalization function, and Norm(·) is the normalization function.

[0086] In terms of three-dimensional accuracy, a standard ceramic ball with a diameter of 50.798 mm was used in the experiment to verify the three-dimensional accuracy of the system. The final three-dimensional reconstruction error of the digital speckle, RMSE, was 0.0912 mm. In order to verify the accuracy and effect, we used fringe projection profilometry (FPP), a structured light three-dimensional reconstruction method with relatively high accuracy, as a reference for comparison. The final three-dimensional reconstruction error of FPP, RMSE, was 0.0857. In summary, in terms of error, FPP is relatively more accurate compared to the two methods, FPP and digital speckle.

[0087] To this end, we used a white plastic doll to perform three-dimensional reconstruction using FPP and digital scattering methods, respectively, and compared the details under the same number of point clouds. From the details in sections a and b of Figure 4 , it can be seen that the detail recovery effect of FPP is relatively better than that of digital scattering, but FPP uses 12-step phase-shifted 8th-order Gray code projection, which is much slower than the single-frame projection of digital scattering. In general, digital speckle is slightly less accurate than FPP, but has more advantages in speed.

[0088] D, spectral calibration

[0089] For a single independent wedge prism, the deflection angle θ can be simply expressed as:

[0090] θ = (n - 1) a (16)

[0091] This shows that the deflection angle is only related to the apex angle a and the refractive index n when the light is normally or nearly normally incident, and δ is positively related to n.

[0092] The description of normal dispersion is given by the empirical formula derived by Augustin-Louis Cauchy (1789-1857) in 1836:

[0093]

[0094] where P1, P2 and P3 are three coefficients called Cauchy constants.

[0095] Combining the above equations, the relationship between the deflection angle θ and the wavelength λ can be expressed as:

[0096]

[0097] where P1' = (P1-1) a, P2' = aP2, P3' = aP3. The Amici prism used in this chapter can be regarded as the gluing of three wedge prisms, so its deflection angle can still be expressed in the form of the above equation. Under the near-field condition, the image shift distance Θ and the deflection angle θ are linearly related. Therefore, there is:

[0098] Θ = Q1(λ -2 ) 2 + Q2(λ -2 ) + Q3 (19)

[0099] where Q1, Q2 and Q3 are the weight coefficients of the wavelength position distribution in the camera coordinates.

[0100] For a CMOS array with equal distance of image elements, the number of image elements n p corresponding to the discretized shift distance Θ.

[0101] A color doll shown in Figure 5 was used to calibrate the wavelength and check the quality of spectral recovery. Figure 5 Part b of FIG. 6 shows the fitting curves of different wavelengths and their positions. In the wavelength calibration of this experiment, the spectral images of the color doll under 420-660 nm were collected in turn using the filter (Thorlabs, 10 nm), and the positions np of the images under each wavelength band were recorded. Figure 5The spectral curves of several color regions in the color doll are listed in the middle part c, wherein the standard curve is taken from the data cube shot by the filter set. After curve fitting and down-sampling, the average SAM of each color region in the standard color plate reaches 4.2.

[0102] E, system actual target observation result

[0103] The system used in the experiment is a maskless quasi-dispersive aperture coded snapshot imaging spectrometer (SD-CASSI) system and a binocular stereo vision system formed by a collimating objective lens, a focusing lens, an Ames prism, a DLP6500 projector with a resolution of 1920*1080 and two Hikvision CE120-10UM cameras with a resolution of 4000*3036.

[0104] The experiment first performs 4D imaging on the previous color doll. A color doll similar to a cat has rubber clay stuck on the upper part of the region, such as Figure 6 Part a in the middle. The color standard of the color spectrum image comes from CIE 1931.

[0105] After the imaging system provided by the application is detected, the spectral information of different regions is clearly presented, such as Figure 6 Part b in the middle.

[0106] As shown in part a in the middle, in the experiment, we choose mango as the experimental object, paint some regions with glue to simulate damaged regions, and hope to distinguish the damaged regions through the spectrum. Figure 7 Part b in the middle is the 4D imaging result of the mango, and the single-band image is taken out to distinguish the regions with glue and the regions without glue. Figure 7 Part c in the middle is to compare the spectral curve of the glue region with the spectral curve of the region without glue, and it can be obviously seen that the spectral curve of the glue region is different from the spectral curve of the region without glue in different wave bands. Figure 7

[0107] In summary, the application provides a 4D multi-frame spectral three-dimensional imaging system based on aperture coding and digital image correlation, and realizes four-dimensional spectral imaging. The system combines CASSI and DIC, adopts an active projection mode, projects the projection patterns required by the two respectively, and synchronously performs three-dimensional reconstruction and spectral reconstruction. The spectral characteristics of the white LED projector light source are used to determine the dispersion length. The correlation calculation method is used to solve the distorted dispersion "mask" caused by the target surface topography modulation.

[0108] ​The above detailed description merely describes the preferred embodiment of the application, and is not intended for restricting the protection scope of the application. Without departing from the design concept and spirit of the application, various modifications, replacements and improvements of the technical solutions of the application made by those skilled in the art according to the description and drawings provided by the application should all belong to the protection scope of the application.

Claims

1. A 4D multi-frame spectral three-dimensional imaging method based on aperture coding and digital image correlation, characterized in that, A 4D imaging system is adopted, which comprises a projector, an aperture coding spectral imaging module and a second vision module, The projector, the aperture coding spectral imaging module and the second vision module are all opposite to the target object; The aperture coding spectral imaging module comprises a collimating objective lens L1, an Amici dispersion prism L2, a first focusing lens L3 and a first camera, which are sequentially arranged in front of the target object; When the Amici dispersion prism L2 is removed from the aperture coding spectral imaging module, the aperture coding spectral imaging module becomes a first vision module; The second vision module comprises a second focusing lens L3 and a second camera, which are sequentially arranged in front of the target object; The first vision module and the second vision module constitute a binocular stereo vision module; The method comprises the following steps: Step 1: The projector projects an aperture coding pattern and a digital speckle pattern onto the surface of the target object, a random coding image is acquired by using the aperture coding spectral imaging module, and a digital speckle image is acquired by using the binocular stereo vision module; Step 2: The random coding image acquired in step 1 is subjected to spectral reconstruction by using a TwIST reconstruction algorithm to obtain a spectral reconstruction result, and the digital speckle image acquired in step 1 is subjected to binocular vision three-dimensional measurement by using a DIC method to obtain a three-dimensional point cloud; Step 3: The spectral reconstruction result of step 2 is fused with the three-dimensional point cloud to obtain 4D multi-frame spectral three-dimensional imaging.

2. The 4D multi-frame spectral three-dimensional imaging method based on aperture coding and digital image correlation of claim 1, wherein, The measurement value c on the detector of the first camera in step 1 is represented by the following formula: c = Hf In the formula, H is a linear operator, and f is a spectral source density.

3. The 4D multi-frame spectral 3D imaging method based on aperture coding and digital image correlation of claim 1, wherein, The light projected by the projector onto the target object is white.

4. The 4D multi-frame spectral 3D imaging method based on aperture coding and digital image correlation of claim 1, wherein, In step 1, the projector projects the same aperture coding pattern to the target object by using red light and blue light in sequence.

5. The 4D multi-frame spectral 3D imaging method based on aperture coding and digital image correlation of claim 1, wherein, In step 2, the offset distance Θ of the random coding image is obtained by using the following formula: Θ = Q1(λ -2 ) 2 + Q2(λ -2 ) + Q3 In the formula, Q1, Q2 and Q3 are weight coefficients of wavelength position distribution in the camera coordinates of the first camera.

6. The 4D multi-frame spectral 3D imaging method based on aperture coding and digital image correlation of claim 1, wherein, In step 2, the distortion state P of the random coding image is represented by the following formula: P = Norm(histeq(P b1 / P b2 )+histeq(P r1 / P r2 )) where histeq(·) is a histogram equalization function, Norm(·) is a normalization function, P b1 and P r1 is the actual shape of the random coding pattern, P b2 and P r2 is the negative template of the random coding pattern.

7. The 4D multi-frame spectral 3D imaging method based on aperture coding and digital image correlation of claim 1, wherein, In step 3, the spectral reconstruction result of step 2 is fused with the three-dimensional point cloud, which comprises the following steps: Step 31: The peak images of red, green and blue three peaks in the spectral reconstruction result of step 2 are taken as RGB channels to perform fusion toning to obtain a two-dimensional color image; Step 32: The three-dimensional space points of the three-dimensional point cloud in step 2 are corresponded to the point coordinates of the two-dimensional image to obtain three-dimensional space point coordinates; Step 33: The coordinates of the two-dimensional color image in step 31 are corresponded to the three-dimensional space point coordinates to obtain 4D multi-frame spectral three-dimensional imaging.

8. The 4D multi-frame spectral 3D imaging method based on aperture coding and digital image correlation of claim 1, wherein, In step 2, the reference subset and the target subset in the DIC method are matched by using a ZNCC coefficient in the DIC method for binocular vision three-dimensional measurement: C ZNCC (P) = 1 - 0.5 x C ZNSSD (P) wherein, f(x,y) is the gray intensity at the coordinate position (x,y) of the original picture, g(x',y') is the gray intensity at the coordinate position (x',y') of the deformed picture, is the average gray intensity of the original picture, is the average gray intensity of the target picture, P is the directional vector relative to the displacement mapping function, and N is the total number of valid object pixels in the subset.

9. The 4D multi-frame spectral 3D imaging method based on aperture coding and digital image correlation of claim 1, wherein, The aperture coding pattern adopts multi-frame coding.

Images