A time-domain compressed three-dimensional contour dynamic imaging device
By introducing time-domain compressed three-dimensional contour dynamic imaging (CPSP) technology in phase shift fringe projection contour (PSP), combining compression perception and deep learning, the problem of high-speed 3D imaging frame rate limitation is solved, and efficient dynamic imaging is achieved, suitable for a variety of application scenarios.
Patent Information
- Application Number
- CN202510185858.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-20
- Publication Date
- 2025-06-10
- Estimated Expiration
- 2045-02-20
AI Technical Summary
Existing phase shifted stripe projection profile (PSP) is limited by the camera frame rate when shooting high-speed 3D scenes, making it difficult to achieve efficient dynamic imaging.
The time-domain compressed three-dimensional contour dynamic imaging (CPSP) device is used, and the coding aperture compression time-domain imaging (CACTI) technology in compression perception (CS) theory is combined with deep learning-based image reconstruction, breaking through the camera frame rate limit and improving imaging speed.
On the basis of retaining the 3D contour reconstruction capability, the imaging speed is improved and the shooting of dynamic scenes is achieved, with a frame rate of 11 frames per second, which is suitable for research such as industrial observation, robot intelligence and biological imaging.
Smart Images

Figure CN119665863B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of high-speed fringe projection profilometry, and particularly to a time-domain compressed three-dimensional contour dynamic imaging device (CPSP). Background Art
[0002] Fringe projection profilometry (FPP) has become one of the most widely used optical measurement techniques in high-precision three-dimensional (3D) surface measurement. By projecting a structured fringe pattern onto the target surface and recording its deformation, FPP effectively captures the surface contour of the object to be measured in a non-contact manner. To improve the measurement speed of FPP when shooting high-speed 3D scenes, various speed-up methods have been proposed, mainly divided into improving the hardware speed and improving the algorithm speed. In terms of improving the hardware speed, by introducing a high-speed digital light projector (DLP), a high-speed camera, using defocused binary fringes instead of sinusoidal octal fringes, etc., the hardware steps of fringe projection and image acquisition are optimized, thereby improving the measurement speed; in terms of improving the algorithm speed, by developing new methods such as Fourier fringe profilometry (FTP), phase-shift fringe projection profilometry (PSP), etc., or improving the performance of 3D contour reconstruction methods such as introducing a time-phase unwrapping algorithm, the algorithm performance is improved on the premise of ensuring the reconstruction quality, thereby improving the measurement speed. In recent years, phase-shift fringe projection profilometry (PSP) has gradually become the mainstream application solution for high-precision 3D measurement.
[0003] Compressed sensing (CS) is an emerging signal acquisition and reconstruction method that can reconstruct signals at a sampling rate much lower than that required by traditional sampling theory. Its basic idea is to utilize the sparsity or compressibility of the signal, and through non-linear mapping and optimization algorithms during the acquisition process, recover the complete signal from a small amount of sampled data. In the field of optical measurement, compressed sensing can improve the acquisition efficiency by reducing the amount of sampled data, and greatly reduce the required storage space and computational burden while ensuring the reconstruction quality, so it can be applied to the speed-up method of phase-shift fringe projection profilometry (PSP). In recent years, compressed sensing has shown its powerful potential in high-precision measurement tasks such as optical imaging, medical imaging, and radar imaging, and has gradually become one of the mainstream technologies for efficient acquisition and rapid reconstruction. Summary of the Invention
[0004] The present invention relates to a time-domain compressed three-dimensional contour dynamic imaging (CPSP) device, which can break through the camera frame rate limit while retaining the 3D contour reconstruction ability of phase-shift fringe projection profilometry (PSP), and further improve the imaging speed. CPSP combines the coded aperture compressed time-domain imaging (CACTI) technology in compressed sensing (CS) theory with deep learning-based image reconstruction. The CACTI technology improves the imaging speed by reconstructing multiple images from a single compressed image, and the deep learning-based image reconstruction uses algorithms to achieve 3D contour reconstruction without reducing the imaging speed. The present invention has innovations in both hardware and algorithms. The CPSP device can capture dynamic scenes with a frame rate of 11 frames per second, providing an ideal tool for research such as industrial observation, robot intelligence, and biological imaging.
[0005] The specific technical solution for achieving the object of the present invention is as follows:
[0006] A time-domain compressed three-dimensional contour dynamic imaging device, characterized in that the device includes a fringe projection system, a light collection system, a high-speed encoder, a data acquisition system, and a computer;
[0007] The fringe projection system includes a fringe projection device and a sample to be measured; the fringe projection device can project sinusoidal gray-scale fringes distributed in the horizontal direction along the optical path transmission direction onto the surface of the sample to be measured. The fringes are modulated by the height of the sample to be measured and then deformed, and then enter the light collection system; the sample to be measured is a dynamic moving object or a stationary object;
[0008] The light collection system includes a camera lens, a first lens, a second lens, and a first aperture;
[0009] Since the field of view of the sample to be measured is larger than the area of the lens, a camera lens with a relatively high magnification is used. Its focal length is 50 mm and the aperture value is 1.4; the camera lens focuses the deformed fringes on the front focal plane of the first lens.
[0010] Both the first lens and the second lens are achromatic doublet lenses with a diameter of 25.4 mm, which can focus multi-color light of different wavelengths passing through the lens to the same position;
[0011] The focal length of the first lens is 250 mm, the focal length of the second lens is 150 mm, the rear focal plane of the first lens is the front focal plane of the second lens, that is, the distance between the two lenses is 400 mm, so that the first lens and the second lens form a 4f optical system with a reduction ratio of 60%; the high-speed encoder is located on the rear focal plane of this 4f optical system (that is, 150 mm away from lens 2), ensuring the efficiency of transmitted light and the quality of the image focused on the high-speed encoder;
[0012] The first aperture is located behind the second lens, and its function is to filter the diffraction orders of the 4f optical system composed of the first lens and the second lens by controlling the aperture size, thereby improving the quality of the image focused on the high-speed encoder;
[0013] The high-speed encoder mentioned above is a digital micromirror device (DMD) with a micromirror array of 1920×1080. The side length of each micromirror is 7.56 micrometers, and it can encode the image on its surface; the light beam enters the data acquisition system after being reflected by the digital micromirror device (DMD);
[0014] The object under test, all components in the light collection system, and the optical center of the high-speed encoder are located on a straight line, namely the optical path, and the focal planes of these components are perpendicular to this optical path;
[0015] The data acquisition system includes a mirror, a second aperture, a third lens, a fourth lens, a third aperture, and a camera;
[0016] The mirror is a highly reflective silver-coated mirror with a diameter of 12.7 mm, which has a high reflectivity for light of different wavelengths and is used to guide the light transmitted by the high-speed encoder into the data acquisition system;
[0017] The second aperture is located behind the mirror, and its function is to filter the diffraction orders of the light beam after reflection by controlling the aperture size, thereby improving the quality of the image passing through the third lens;
[0018] Both the third lens and the fourth lens are achromatic doublet lenses with a diameter of 12.7 mm. The focal length of the third lens is 250 mm, and the focal length of the fourth lens is 200 mm. The rear focal plane of the third lens is the front focal plane of the fourth lens, that is, the distance between the two lenses is 450 mm, so that the third lens and the fourth lens form a 4f optical system with a reduction ratio of 80%; the spatial position of the front focal plane of the third lens coincides with the image of the high-speed encoder formed by the mirror; the optical center of the camera is located on the rear focal plane of the fourth lens;
[0019] The third aperture is located behind the fourth lens, and its function is to filter the diffraction orders of the 4f optical system composed of the third lens and the fourth lens by controlling the aperture size, thereby improving the quality of the image focused on the camera;
[0020] The optical centers of all components in the data acquisition system are located on a straight line, namely the optical path, and the focal planes of these components are perpendicular to this optical path;
[0021] Since the reduction ratio of the 4f optical system composed of the first lens and the second lens is 60%, and the reduction ratio of the 4f optical system composed of the third lens and the fourth lens is 80%, the image entering the camera is reduced by 48% compared to the object under test;
[0022] The entire process from the light being emitted by the fringe projection device until it is saved by the camera can be called the compressive sampling process;
[0023] The described computer is respectively connected to the fringe projection device, the high-speed encoder, and the camera, and outputs control signals with a period of 1∶1∶3, so that the exposure times of the three devices are 30ms∶30ms∶90ms respectively; after obtaining the image saved by the camera, the computer runs the three-dimensional surface reconstruction algorithm;
[0024] At this time, the optical centers of all the optical elements of the time-domain compressed three-dimensional contour dynamic imaging device are located on the same horizontal plane.
[0025] The described compressive sampling process and the three-dimensional surface reconstruction algorithm can be completed by mathematical derivation, which includes a combined phase unwrapping technique and a deep learning-based image reconstruction algorithm, namely the CPSP algorithm. The specific data processing process is as follows:
[0026] Let the intensity L of the sine fringes projected by the fringe projection device p be distributed as:
[0027] , (1)
[0028] where (x, y) represents the two-dimensional plane coordinate system, A P (x, y) represents the basic intensity distribution, B P (x, y) represents the maximum intensity amplitude of the sine variation, φ P (x, y) represents the wrapped phase corresponding to the height of the sample to be measured; N is an integer of at least 3, and the natural number variable n ≤ N represents the current phase shift step number; for each phase shift step, the projector projects one fringe, and a total of N fringes are projected; after the projected fringes are modulated by the height and deformed, the intensity L of the deformed fringes d is distributed as:
[0029] , (2)
[0030] where α(x, y) represents the surface reflectivity of the sample to be measured, and β(x, y) represents the ambient light noise;
[0031] The N deformed fringes L d are then encoded by the random binary aperture c(x, y, n) on the high-speed encoder, and then sequentially superimposed into a compressed image m(x, y) by the camera within a single exposure time:
[0032] , (3)
[0033] Let m and d be the matrix forms of the compressed image m(x, y) and the deformed fringe L d respectively, then equation (3) is simplified to the matrix form:
[0034] , (4)
[0035] where C is the spatial encoding operator, T is the image superposition operator of the camera, and A = CT is the total operator representing the data acquisition process of formulas (1) to (3);
[0036] From the compressed image m, the deformed fringe containing the height information of the item to be measured needs to be reconstructed; based on the maximum likelihood estimation, the mathematical model of this reconstruction process can be modeled as a numerical optimization problem:
[0037] , (5)
[0038] where Θ is a sparse transformation operator, ||•|| 1 represents the L1 norm;
[0039] To solve this problem, an image reconstruction method based on the alternating direction multiplier method to construct an augmented Lagrangian equation is used to transform it into three iterative equations, namely three sub-problems, and they are gradually solved through constrained alternating iteration; an image denoising variable v, a Lagrange multiplier u, and a Lagrange regularization factor ρ are introduced, and the reconstructed deformed fringe d [i.e., the matrix form of L r (x, y, n)] is calculated after the iteration:
[0040] , (6)
[0041] where I represents the identity matrix, k is the number of iteration steps, A T is the matrix transpose of the operator A, Denosier Θ is the denoising step under the sparse prior; when the reconstructed deformed fringe L r (x, y, n) is solved, the wrapped phase corresponding to the height of the sample to be measured can be extracted:
[0042] , (7)
[0043] Applying the spatial phase unwrapping algorithm, the absolute phase is calculated from the wrapped phase, and this absolute phase corresponds one-to-one to the height distribution of the sample to be measured, thereby visualizing the three-dimensional contour of the sample to be measured.
[0044] The beneficial effects of the present invention are as follows: based on the original Phase-Shifting Projection Profilometry (PSP), it combines the Coding Aperture Compressive Temporal Imaging (CACTI) technology in Compressive Sensing (CS) theory with deep learning-based image reconstruction. The CACTI technology improves the imaging speed by reconstructing multiple images from a single compressed image, while the deep learning-based image reconstruction uses algorithms to achieve 3D contour reconstruction without reducing the imaging speed. Compared with the original PSP device that can only capture static scenes, the CPSP device improves the imaging speed and can thus capture dynamic scenes, with a frame rate of 11 frames per second, providing an ideal tool for research such as industrial observation, robot intelligence, and biological imaging. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 is a schematic structural diagram of the present invention;
[0046] Figure 2 is a flowchart of the working process of the present invention;
[0047] Figure 3 is a schematic diagram of the reconstruction results for two static objects to be measured in Embodiment 1 of the present invention; (a) For the ammonite model, the compressed image captured by the camera, and the wrapped phase calculated based on the reconstructed image; (b) The original contour of the ammonite model, and the 3D contour reconstructed by this device; (c) The relative error between the reconstruction result and the original object, expressed as a percentage; (d) For the David model, the compressed image captured by the camera, and the wrapped phase calculated based on the reconstructed image; (e) The original contour of the David model, and the 3D contour reconstructed by this device; (f) The relative error between the reconstruction result and the original object, expressed as a percentage;
[0048] Figure 4 is a schematic diagram of the reconstruction results for three dynamic objects to be measured in Embodiment 2 of the present invention; (a) A chick model moving from right to left; (b) A small fish model rotating clockwise; (c) An air film model with a single fluctuation; (d) The change in the translation distance of the object to be measured over time in Figure (a); (e) The change in the rotation angle over time in Figure (b); (f) The change in the fluctuation height over time in Figure (c).
[0049] In the figure: 100 - fringe projection system; 200 - light collection system; 300 - high-speed encoder; 400 - data acquisition system; 500 - computer; 101 - fringe projection device; 102 - sample to be measured; 201 - camera lens; 202 - first lens; 203 - second lens; 204 - first aperture; 401 - mirror; 402 - second aperture; 403 - third lens; 404 - fourth lens; 405 - third aperture; 406 - camera. DETAILED DESCRIPTION OF THE INVENTION
[0050] The present invention will be described in detail below in conjunction with the accompanying drawings and embodiments.
[0051] Referring to Figure 1 , the present invention includes a fringe projection system 100, a light collection system 200, a high-speed encoder 300, a data acquisition system 400, and a computer 500. The fringe projection system 100 includes a fringe projection device 101 and a sample to be measured 102; the light collection system 200 includes a camera lens 201, a first lens 202, a second lens 203, and a first aperture 204; the high-speed encoder 300 includes a digital micromirror device DMD; the data acquisition system 400 includes a mirror 401, a second aperture 402, a third lens 403, a fourth lens 404, a third aperture 405, and a camera 406; the computer 500 is respectively connected to the fringe projection device 101, the high-speed encoder 300, and the camera 406.
[0052] Referring to Figure 1 , Figure 2 , after the system is set up, three structured light eight-bit fringe images with a resolution of 1920*1080 are loaded into the fringe projection device 101. Their intensity distributions are sinusoidal along the x-axis, with a period of 20 pixels. The sinusoidal phase differences between the three images are 2π / 3, and the duration of each fringe is 30 ms. The fringe projection device projects this structured light fringe onto the sample to be measured 102, which is deformed due to the height modulation of the object to be measured, and then the structured light is reflected into the light collection system 200. The deformed fringe image is immediately collected by the camera lens 201 and passes through a 4f system with a reduction ratio of 60% composed of the first lens 202 and the second lens 203 along the optical path. After passing through the first aperture 204 to filter out the diffraction orders, it is finally clearly focused on the digital micromirror device DMD of the high-speed encoder 300.
[0053] Referring to Figure 1 , Figure 2 , the random binary apertures on the digital micromirror device DMD generate three random coding patterns, and their transformation is synchronously controlled by the computer 500. Therefore, the duration of each code is also 30 ms. Each code is sequentially applied to the deformed fringe to obtain three coded fringes. The coded fringes enter the data acquisition system 400 according to the optical path.
[0054] Referring to Figure 1 , Figure 2, The light on the digital micromirror device DMD is received and reflected by the mirror 401, filtered by the second aperture 402 for diffraction orders, and then enters the 4f optical system composed of the third lens 403 and the fourth lens 404 with a reduction magnification of 80%, enhancing the imaging clarity. Subsequently, after being filtered by the third aperture 405 for diffraction orders, it is focused on the camera 406 and stored by it. The camera is also controlled by the computer 500, but the period of its signal is three times that of the previous two, so the exposure times of the three devices are 30ms: 30ms: 90ms respectively. Therefore, the image recorded by the camera is the superposition of three encoded fringe images generated by the high-speed encoder 300 within its single exposure time, that is, a compressed image.
[0055] Refer to Figure 2 , Figure 3 , After the camera records the compressed image, the computer 500 needs to reconstruct the three-dimensional contour of the dynamic object to be measured based on this. In the present invention, the alternating direction method of multipliers based on deep learning is used to solve, and iterative calculations are carried out by balancing the measurement constraint and the sparsity constraint. After the numerical initialization in the iterative framework, a forward estimation based on maximum likelihood estimation is adopted, combined with the input compressed image, and the loss function is constructed and calculated and then generalized projection is performed on it. The denoiser restores the image details and preserves the feature information by removing clutter, noise, etc. from the image. Here, the denoiser DRUNET trained by deep learning is used. The discriminator is used to identify whether the image quality reaches the preset index. If it meets the standard, the image is output. If it does not meet the standard, the variable is updated and enters the next round of iterative calculations. Until the iteration ends, three deformed fringe images can be reconstructed from one input fringe image, and this result should be exactly the same as the three original deformed fringes projected on the object to be measured in the ideal state. According to the reconstructed result, the wrapped phase can be calculated according to Equation (8), the absolute phase corresponding to the height distribution of the object to be measured can be calculated by using the spatial unwrapping algorithm, and then the 3D contour can be obtained by using the phase-height mapping. Finally, the three-dimensional motion contour of the sample to be measured is visualized.
[0056] Embodiment 1
[0057] In this embodiment, the sample observed by the CPSP device is designed as two static objects, and the original three-dimensional contour data is given. The purpose of this embodiment is to calculate the reconstruction and reduction ability of the CPSP device for the surface of complex objects to be measured.
[0058] Refer to Figure 1 , The device described in this embodiment includes a fringe projection system 100, a light collection system 200, a high-speed encoder 300, a data acquisition system 400, and a computer 500.
[0059] The fringe projection device 101 projects a sinusoidal structured light with a wavelength of 532 nm, and its sinusoidal period is 20 pixels.
[0060] The samples to be measured are static ammonite gypsum models and David gypsum models with complex structures. The change in their surface height causes the sine fringes to deform, which are then collected by the camera lens 201.
[0061] The first lens 202 and the second lens 203 are both achromatic doublet lenses with a diameter of 25.4 mm, and the third lens 403 and the fourth lens 404 are both achromatic doublet lenses with a diameter of 12.7 mm. Their function is to focus polychromatic light of different wavelengths passing through the lenses to the same position. Among them, the focal lengths of the first lens 202 and the second lens 203 are 250 mm and 150 mm respectively. The rear focal plane of the first lens is the front focal plane of the second lens (i.e., the distance between the two lenses is 400 mm), so that the first lens and the second lens form a 4f optical system with a reduction ratio of 60%. Similarly, the focal length of the third lens 403 is 250 mm, and the focal length of the fourth lens 404 is 200 mm, forming a 4f optical system with a reduction ratio of 80%. Therefore, the reduction ratio of the image entering the camera 406 compared to the sample to be measured is 48%.
[0062] Refer to Figure 1 、 Figure 2 The digital micromirror device DMD of the high-speed encoder 300 has a micromirror array of 1920×1080, and the side length of each micromirror is 7.56 microns. Its function is to load the encoded pattern and perform a Hardmard product with the scene when reflecting the light beam, so as to encode the scene.
[0063] The mirror 401 is a highly reflective silver-coated mirror with a diameter of 12.7 mm, which has a high reflectivity for light of different wavelengths and is used to guide the light transmitted by the high-speed encoder 300 into the data acquisition system 400.
[0064] The functions of the first aperture 204, the second aperture 402, and the third aperture 405 are to adjust the aperture size, filter the diffraction orders of the light beam passing through them, and enhance the imaging quality. Among them, the first aperture 204 and the third aperture 405 are used to filter the diffraction orders formed by the 4f optical system in front of them, and the second aperture 402 is used to filter the diffraction orders caused by the encoding of the digital micromirror device DMD and the turning of the mirror.
[0065] The function of the camera 406 is to receive the accumulation of multiple encoded fringe images by the digital micromirror device DMD during the exposure time and record them as compressed encoded images. The exposure time is set to 90 milliseconds.
[0066] The function of the computer 500 is to provide a synchronous trigger signal. After receiving the signal, the fringe projection device 101, the high-speed encoder 300, and the camera 406 start working simultaneously, and the exposure time is set to 30 ms∶30 ms∶90 ms.
[0067] Refer to Figure 3 , in this embodiment, it is a schematic diagram of the reconstruction results for two static objects to be measured. Figures (a)-(c) are data of the ammonite model, and Figures (d)-(f) are data of the David model. For the ammonite model in Figure (a), the compressed images captured by the camera and the wrapped phase calculated based on the reconstructed fringes are given. In Figure (b), the original contour of the ammonite model is compared with the 3D contour reconstructed by this device. In Figure (c), the relative error between the reconstruction result and the original object, expressed as a percentage, is generally within ±5%. In Figure (d), the compressed images captured by the camera and the wrapped phase calculated based on the reconstructed fringes are given. In Figure (e), the original contour of the model is compared with the 3D contour reconstructed by this device. In Figure (f), the relative error between the reconstruction result and the original object, expressed as a percentage, is generally within ±2%. The numerical difference in the error mainly comes from the complexity of the object to be measured. The more complex it is, the greater the error, but it still has a high fidelity.
[0068] Embodiment 2
[0069] Refer to Figure 4 , in this embodiment, it is a schematic diagram of the reconstruction results for three dynamic objects to be measured. The samples observed by the CPSP device are designed as three dynamic objects, namely (a) a chick model moving from right to left, (b) a small fish model rotating clockwise, and (c) an air film model with a single fluctuation. The purpose of this embodiment is to calculate the reconstruction and restoration ability of the CPSP device for the surface of dynamic objects to be measured.
[0070] The setup of the device details is the same as in Embodiment 1, only replacing the object to be measured 102.
[0071] Refer to Figure 4 , (d) shows the change in the translation distance of the chick model in space over time. The time interval between every two counting points is 0.9 s. In (a), the spatial positions of the chick model at six different moments are selected to describe the translation movement from right to left. According to the reconstruction data of the CPSP device, the speed of this translation movement is approximately 0.5324 mm / s.
[0072] Refer to Figure 4 , (e) shows the change in the rotation angle of the small fish model in space over time. The time interval between every two counting points is 1.8 s. In (b), the spatial positions of the small fish model at six different moments are selected to describe the clockwise rotation movement. According to the reconstruction data of the CPSP device, the angular velocity of this rotation movement is approximately 2.4353° / s.
[0073] Refer to Figure 4, (f) shows the change in the fluctuation height of the gas film model in space over time, and the time interval between every two counting points is 0.45 s. (c) The spatial positions of the gas film model at six different moments are selected from it, thus describing the fluctuation motion that rises from bottom to top and then recovers. The 3D contour surface of the motion and the velocity parameters, etc. are consistent with the original data, so the CPSP device still has a high fidelity for dynamic samples.
[0074] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.
Claims
1. A time domain compressed three-dimensional contour dynamic imaging device, characterized in that: The device comprises a fringe projection system (100), a light collection system (200), a high-speed encoder (300), a data acquisition system (400) and a computer (500); The fringe projection system (100) comprises a fringe projection device (101) and a sample to be tested (102); the fringe projection device (101) can project sinusoidal grayscale fringes distributed in a horizontal direction along the transmission direction of an optical path onto the surface of the sample to be tested (102); the fringes are modulated by the height of the sample to be tested (102) to produce deformation, and then enter the light collection system (200); wherein the sample to be tested (102) is a dynamic moving object or a stationary object; The light collection system (200) comprises a camera lens (201), a first lens (202), a second lens (203) and a first aperture (204); A camera lens (201) is used, the focal length of which is 50 mm and the aperture value is 1.4; The camera lens focuses the deformed fringes on the surface of the sample to be measured onto the front focal plane of the first lens (202); The first lens (202) and the second lens (203) are achromatic doublet lenses with a diameter of 25.4 mm, which can focus polychromatic light of different wavelengths passing through the lenses to the same position; The focal length of the first lens (202) is 250 mm, the focal length of the second lens (203) is 150 mm, the back focal plane of the first lens is the front focal plane of the second lens, that is, the distance between the two lenses is 400 mm, and the first lens and the second lens form a 4f optical system with a reduction ratio of 60%; The first aperture (204) is located behind the second lens (203) and has a function of filtering the diffraction order of the 4f optical system composed of the first lens and the second lens by controlling the size of the aperture, thereby improving the quality of the image focused on the high-speed encoder (300); The high-speed encoder (300) is a digital micromirror device (DMD), which has a 1920×1080 micromirror array, each micromirror has a side length of 7.56 μm, and can encode the image on its surface; The light beam enters the data acquisition system (400) after being reflected by the digital micromirror device (DMD); The optical centers of the sample to be tested (102), all the components in the light collection system (200), and the high-speed encoder (300) are all located on a straight line, i.e., an optical path, and the focal planes of these components are perpendicular to the optical path; The data acquisition system (400) comprises a reflector (401), a second aperture (402), a third lens (403), a fourth lens (404), a third aperture (405) and a camera (406); The reflector (401) is a high-reflectivity silver-coated mirror with a diameter of 12.7 mm, and is used to guide the light transmitted by the high-speed encoder (300) into the data acquisition system (400); The second aperture (402) is located behind the reflector (401) and filters the diffraction order of the light beam after being reflected by controlling the size of the aperture, thereby improving the quality of the image passing through the third lens (403); The third lens (403) and the fourth lens (404) are achromatic doublet lenses with a diameter of 12.7 mm. The focal length of the third lens (403) is 250 mm, the focal length of the fourth lens (404) is 200 mm, the back focal plane of the third lens is the front focal plane of the fourth lens, the distance between the two lenses is 450 mm, and the third lens and the fourth lens form a 4f optical system with a reduction ratio of 80%. The spatial position of the front focal plane of the third lens coincides with the mirror image generated by the high-speed encoder (300) on the reflector (401); the optical center of the camera (406) is located on the back focal plane of the fourth lens; The third aperture (405) is located behind the fourth lens (404) and filters the diffraction order of the 4f optical system composed of the third lens and the fourth lens by controlling the size of the aperture, thereby improving the quality of the image focused on the camera (406); The optical centers of all elements in the data acquisition system (400) are located on a straight line, i.e., an optical path, and the focal planes of these elements are perpendicular to the optical path; The 4f optical system composed of the first lens and the second lens has a reduction ratio of 60%, the 4f optical system composed of the third lens and the fourth lens has a reduction ratio of 80%, and the image entering the camera (406) has a reduction ratio of 48% compared to the sample to be tested (102); The entire process from the light being emitted from the fringe projection device (101) to entering the camera (406) for storage is a compressed sampling process; The computer (500) is connected to the fringe projection device (101), the high-speed encoder (300) and the camera (406) respectively, and outputs a control signal with a cycle of 1:1:3, so that the exposure time of the three devices is 30ms:30ms:90ms respectively; after acquiring the image stored in the camera (406), the computer (500) runs a three-dimensional surface reconstruction algorithm; The optical centers of all optical elements are located on the same horizontal plane; where: The compressed sampling process and the three-dimensional surface reconstruction algorithm are completed by mathematical derivation, which includes the combination of phase unwrapping and the image reconstruction algorithm based on deep learning, namely the CPSP algorithm. The specific data processing process is as follows: Assume that the intensity L of the sinusoidal fringe projected by the fringe projection device (101) is p The distribution is: Where (x, y) represents a two-dimensional plane coordinate system, A P (x,y) represents the basic intensity distribution, B P (x,y) represents the maximum intensity amplitude of the sinusoidal change, represents the wrapped phase corresponding to the height of the sample to be tested (102); N is an integer of at least 3, representing the total number of phase shift steps, and the natural number variable n≤N represents the current number of phase shift steps; for each phase shift step, the projector projects a stripe, and a total of N stripes are projected; after the projected stripes are deformed by height modulation, the intensity of the deformed stripes L d The distribution is: Where α(x,y) represents the surface reflectivity of the sample to be tested, and β(x,y) represents the ambient light noise; N pieces of deformed stripes L d It is then encoded by the random binary aperture c(x,y,n) on the high-speed encoder and then stacked into a compressed image m(x,y) by the camera within a single exposure time: Let m and d be the compressed image m(x,y) and the deformed stripe L respectively. d In matrix form, equation (3) is simplified to matrix form: m=CTd=Ad, (4) Where C is the spatial encoding operator, T is the camera's image superposition operator, and A=CT is the total operator representing the data acquisition process of equations (1) to (3); From the compressed image m, the deformed stripes containing the height information of the object to be measured need to be reconstructed; Based on maximum likelihood estimation, the mathematical model of this reconstruction process can be modeled as a numerical optimization problem: Here Θ is a sparse transformation operator, and ||·||1 represents the L1 norm; In order to solve this problem, an image reconstruction method based on the alternating direction multiplier method is used to construct the augmented Lagrangian equation, which is transformed into three iterative equations, i.e., three sub-problems, which are gradually solved through constrained alternating iterations. The image denoising variable v, the Lagrangian multiplier u and the Lagrangian regularization factor ρ are introduced, and the reconstructed deformed stripes d [i.e., L r (x,y,n)]: Where I represents the identity matrix, k is the number of iterations, and A T is the matrix transpose of operator A, Denosier Θ It is a denoising step under sparse prior; when the reconstructed deformed stripes L are obtained r After (x, y, n), the wrapped phase corresponding to the height of the sample to be measured can be extracted: The spatial phase unwrapping algorithm is applied to calculate the absolute phase from the wrapped phase. This absolute phase corresponds one-to-one with the height distribution of the sample to be tested, thereby visualizing the three-dimensional contour of the sample to be tested.
Citation Information
Patent Citations
Three-dimensional imaging system and method based on arc binary coding phase shift stripe projection
CN112712585A
Compressed video imaging system and method
CN116017151A