Time domain compression three-dimensional contour dynamic imaging device

By introducing time-domain compressed three-dimensional contour dynamic imaging (CPSP) technology in phase shift fringe projection contour (PSP), combining compression perception and deep learning, the problem of high-speed 3D imaging frame rate limitation is solved, and efficient dynamic imaging is achieved, suitable for a variety of application scenarios.

CN119665863AActive Publication Date: 2025-03-21EAST CHINA NORMAL UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202510185858.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-02-20
Publication Date
2025-03-21
Estimated Expiration
2045-02-20

AI Technical Summary

Technical Problem

Existing phase shifted stripe projection profile (PSP) is limited by the camera frame rate when shooting high-speed 3D scenes, making it difficult to achieve efficient dynamic imaging.

Method used

The time-domain compressed three-dimensional contour dynamic imaging (CPSP) device is used, and the coding aperture compression time-domain imaging (CACTI) technology in compression perception (CS) theory is combined with deep learning-based image reconstruction, breaking through the camera frame rate limit and improving imaging speed.

Benefits of technology

On the basis of retaining the 3D contour reconstruction capability, the imaging speed is improved and the shooting of dynamic scenes is achieved, with a frame rate of 11 frames per second, which is suitable for research such as industrial observation, robot intelligence and biological imaging.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119665863A_ABST
    Figure CN119665863A_ABST
Patent Text Reader

Abstract

The invention discloses a time domain compression three-dimensional contour dynamic imaging device, the frame rate of which reaches 11 frames per second, and the time domain compression three-dimensional contour dynamic imaging device is composed of a fringe projection system, an illumination collection system, a high-speed encoder, a data acquisition system and a computer. When a three-dimensional dynamic scene is shot, the fringe projection system projects a plurality of steps of phase-shift sine structured light to a dynamic sample, the illumination collection system images the dynamic scene to the high-speed encoder and applies a compression sampling operation, and the data acquisition system shoots and stores the compressed image. And the computer uniformly provides synchronous signals for the fringe projection system, the high-speed encoder and the data acquisition system, so that the systems work synchronously, and a reconstruction algorithm is operated after a compressed image is obtained. Compared with a traditional three-dimensional contour imaging device, the three-dimensional contour imaging device can shoot dynamic scenes, the imaging speed is increased, and an ideal tool is provided for research of industrial observation, robot intelligence, biological imaging and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of high-speed fringe projection profilometry, in particular to a time-domain compressed three-dimensional profile dynamic imaging device (CPSP). Background Art

[0002] Fringe projection profilometry (FPP) has become one of the most widely used optical measurement techniques in high-precision three-dimensional (3D) surface measurement. By projecting a structured fringe pattern onto the target surface and recording its deformation, FPP effectively captures the surface profile of the object to be measured in a non-contact manner. In order to improve the measurement speed of FPP when shooting high-speed 3D scenes, various speed-up methods have been proposed, which are mainly divided into improving hardware speed and improving algorithm speed. In terms of improving hardware speed, the hardware steps of fringe projection and image acquisition are optimized by introducing high-speed digital light projectors (DLP), high-speed cameras, and using defocused binary fringes instead of sinusoidal octet fringes, thereby improving the measurement speed; in terms of improving algorithm speed, by developing new methods such as Fourier fringe profilometry (FTP), phase-shifted fringe projection profilometry (PSP), etc., or improving the performance of 3D profile reconstruction methods such as introducing time phase unwrapping algorithms, the algorithm performance is improved while ensuring the reconstruction quality, thereby improving the measurement speed. In recent years, phase-shifted fringe projection profilometry (PSP) has gradually become a mainstream application solution for high-precision 3D measurement.

[0003] Compressed sensing (CS) is an emerging signal acquisition and reconstruction method that can reconstruct signals at a sampling rate far lower than that required by traditional sampling theory. The basic idea is to use the sparsity or compressibility of the signal to recover the complete signal from a small amount of sampled data through nonlinear mapping and optimization algorithms during the acquisition process. In the field of optical measurement, compressed sensing can improve acquisition efficiency by reducing the amount of sampled data, and greatly reduce the required storage space and computational burden while ensuring the quality of reconstruction. Therefore, it can be applied to the speed-up method of phase-shifted fringe projection profilometry (PSP). In recent years, compressed sensing has shown its great potential in high-precision measurement tasks such as optical imaging, medical imaging, and radar imaging, and has gradually become one of the mainstream technologies for efficient acquisition and fast reconstruction. Summary of the invention

[0004] The present invention is a time-domain compressed three-dimensional contour dynamic imaging (CPSP) device, which can break through the camera frame rate limit on the basis of retaining the 3D contour reconstruction capability of phase-shifted fringe projection profilometry (PSP), and further improve the imaging speed. CPSP combines the coded aperture compressed time-domain imaging (CACTI) technology in the compressed sensing (CS) theory with image reconstruction based on deep learning, wherein the CACTI technology improves the imaging speed by reconstructing multiple images from a compressed image, while the image reconstruction based on deep learning uses an algorithm to achieve 3D contour reconstruction without reducing the imaging speed. The present invention is innovative in both hardware and algorithm. The CPSP device can capture dynamic scenes with a frame rate of 11 frames per second, providing an ideal tool for research on industrial observation, robot intelligence, biological imaging, etc.

[0005] The specific technical solution for achieving the purpose of the present invention is:

[0006] A time domain compressed three-dimensional contour dynamic imaging device, characterized in that the device includes a fringe projection system, an illumination collection system, a high-speed encoder, a data acquisition system and a computer;

[0007] The fringe projection system comprises a fringe projection device and a sample to be tested; the fringe projection device can project sinusoidal grayscale fringes distributed in the horizontal direction along the light path transmission direction onto the surface of the sample to be tested, and the fringes are modulated by the height of the sample to be tested so as to be deformed and then enter the light collection system; wherein the sample to be tested is a dynamic moving object or a stationary object;

[0008] The light collection system comprises a camera lens, a first lens, a second lens and a first aperture;

[0009] Since the field of view of the sample to be tested is larger than the lens area, a camera lens with a higher zoom ratio is used, with a focal length of 50 mm and an aperture value of 1.4; the camera lens focuses the deformed fringes on the surface of the sample to be tested onto the front focal plane of the first lens;

[0010] The first lens and the second lens are both achromatic doublet lenses with a diameter of 25.4 mm, which can focus polychromatic light of different wavelengths passing through the lenses to the same position;

[0011] The focal length of the first lens is 250mm, the focal length of the second lens is 150mm, the back focal plane of the first lens is the front focal plane of the second lens, that is, the distance between the two lenses is 400mm, so that the first lens and the second lens form a 4f optical system with a reduction ratio of 60%; the high-speed encoder is located on the back focal plane of this 4f optical system (that is, the distance from lens 2 is 150mm), which ensures the efficiency of light transmission and the quality of the image focused on the high-speed encoder;

[0012] The first aperture is located behind the second lens. Its function is to filter the diffraction order of the 4f optical system composed of the first lens and the second lens by controlling the size of the aperture, thereby improving the quality of the image focused on the high-speed encoder;

[0013] The high-speed encoder is a digital micromirror device (DMD) having a 1920×1080 micromirror array, each micromirror having a side length of 7.56 microns, capable of encoding images on its surface; the light beam enters the data acquisition system after being reflected by the digital micromirror device (DMD);

[0014] The optical centers of the sample to be tested, all components in the light collection system and the high-speed encoder are located on a straight line, i.e., the optical path, and the focal planes of these components are perpendicular to the optical path;

[0015] The data acquisition system includes a reflector, a second aperture, a third lens, a fourth lens, a third aperture and a camera;

[0016] The reflector is a high-reflectivity silver-coated mirror with a diameter of 12.7 mm. It has a high reflectivity for light of different wavelengths and is used to guide the light transmitted by the high-speed encoder into the data acquisition system.

[0017] The second aperture is located behind the reflector. Its function is to filter the diffraction order of the reflected light beam by controlling the size of the aperture, thereby improving the quality of the image passing through the third lens.

[0018] The third lens and the fourth lens are both achromatic doublet lenses with a diameter of 12.7mm. The focal length of the third lens is 250mm, and the focal length of the fourth lens is 200mm. The back focal plane of the third lens is the front focal plane of the fourth lens, that is, the distance between the two lenses is 450mm, so that the third lens and the fourth lens form a 4f optical system with a reduction ratio of 80%; the spatial position of the front focal plane of the third lens coincides with the mirror image generated by the high-speed encoder on the reflector; the optical center of the camera is located on the back focal plane of the fourth lens;

[0019] The third aperture is located behind the fourth lens. Its function is to filter the diffraction order of the 4f optical system composed of the third lens and the fourth lens by controlling the size of the aperture, thereby improving the quality of the image focused on the camera.

[0020] The optical centers of all components in the data acquisition system are located on a straight line, i.e., an optical path, and the focal planes of these components are perpendicular to this optical path;

[0021] Since the 4f optical system composed of the first lens and the second lens has a reduction ratio of 60%, and the 4f optical system composed of the third lens and the fourth lens has a reduction ratio of 80%, the image entering the camera has a reduction ratio of 48% compared to the sample to be tested;

[0022] The whole process from the light being emitted from the fringe projection device to the light being stored in the camera can be called a compressed sampling process;

[0023] The computer is connected to the fringe projection device, the high-speed encoder and the camera respectively, and outputs a control signal with a period of 1:1:3, so that the exposure time of the three devices is 30ms:30ms:90ms respectively; after acquiring the image saved by the camera, the computer runs a three-dimensional surface reconstruction algorithm;

[0024] The optical centers of all optical elements of the time-domain compressed three-dimensional profile dynamic imaging device are located on the same horizontal plane.

[0025] The compressed sampling process and the three-dimensional surface reconstruction algorithm can be completed by mathematical deduction, which includes a CPSP algorithm that combines phase unwrapping technology and an image reconstruction algorithm based on deep learning. The specific data processing process is as follows:

[0026] Assume that the intensity L of the sinusoidal fringe projected by the fringe projection device is p The distribution is:

[0027] , (1)

[0028] Where (x, y) represents a two-dimensional plane coordinate system, A P (x, y) represents the basic intensity distribution, B P (x, y) represents the maximum intensity amplitude of the sinusoidal change, φ P (x, y) represents the wrapping phase corresponding to the height of the sample to be tested; N is an integer of at least 3, representing the total number of phase shift steps, and the natural number variable n≤N represents the current number of phase shift steps; for each phase shift step, the projector projects a fringe, and a total of N fringe are projected; after the projected fringe is deformed by height modulation, the intensity of the deformed fringe L d The distribution is:

[0029] , (2)

[0030] Where α(x, y) represents the surface reflectivity of the sample to be tested, and β(x, y) represents the ambient light noise;

[0031] N pieces of deformed stripes L d It is then encoded by the random binary aperture c(x, y, n) on the high-speed encoder and then stacked into a compressed image m(x, y) by the camera within a single exposure time:

[0032] , (3)

[0033] Let m and d be the compressed image m(x, y) and the deformed stripe L respectively. d In matrix form, equation (3) is simplified to matrix form:

[0034] , (4)

[0035] Where C is the spatial encoding operator, T is the camera's image stacking operator, and A=CT is the total operator representing the data acquisition process of formulas (1) to (3);

[0036] From the compressed image m, the deformed stripes containing the height information of the object to be measured need to be reconstructed; based on the maximum likelihood estimation, the mathematical model of this reconstruction process can be modeled as a numerical optimization problem:

[0037] , (5)

[0038] Here Θ is a sparse transformation operator, ||•|| 1 represents the L1 norm;

[0039] In order to solve this problem, an image reconstruction method based on the alternating direction multiplier method is used to construct the augmented Lagrangian equation to transform it into three iterative equations, i.e., three sub-problems, which are gradually solved through constrained alternating iterations. The image denoising variable v, the Lagrangian multiplier u and the Lagrangian regularization factor ρ are introduced, and the reconstructed deformed stripes d [i.e., L r (x, y, n)]:

[0040] , (6)

[0041] Where I represents the identity matrix, k is the number of iterations, and A T is the matrix transpose of operator A, Denosier Θ It is a denoising step under sparse prior; when the reconstructed deformed stripes L are obtained r (x, y, n), the wrapped phase corresponding to the height of the sample to be measured can be extracted:

[0042] , (7)

[0043] The spatial phase unwrapping algorithm is applied to calculate the absolute phase from the wrapped phase. This absolute phase corresponds one-to-one with the height distribution of the sample to be tested, thereby visualizing the three-dimensional contour of the sample to be tested.

[0044] The beneficial effect of the present invention is that it combines the coded aperture compressed time-domain imaging (CACTI) technology in the compressed sensing (CS) theory with image reconstruction based on deep learning on the basis of the original phase-shift fringe projection profilometry (PSP), wherein the CACTI technology improves the imaging speed by reconstructing multiple images from a compressed image, while the image reconstruction based on deep learning uses an algorithm to achieve 3D contour reconstruction without reducing the imaging speed. Compared with the original PSP device that can only shoot static scenes, the CPSP device improves the imaging speed and can shoot dynamic scenes, with a frame rate of 11 frames per second, providing an ideal tool for research on industrial observation, robot intelligence, biological imaging, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the structure of the present invention;

[0046] Figure 2 It is a work flow chart of the present invention;

[0047] Figure 3 1 is a schematic diagram of the reconstruction results of two static objects to be tested in Example 1 of the present invention; (a) for the ammonite model, the compressed image captured by the camera and the wrapped phase calculated based on the reconstructed image; (b) the original contour of the ammonite model and the 3D contour reconstructed by the device; (c) the relative error between the reconstruction result and the original object, expressed in percentage; (d) for the David model, the compressed image captured by the camera and the wrapped phase calculated based on the reconstructed image; (e) the original contour of the David model and the 3D contour reconstructed by the device; (f) the relative error between the reconstruction result and the original object, expressed in percentage;

[0048] Figure 4 Schematic diagram of reconstruction results for three dynamic test objects in Example 2 of the present invention; (a) a chicken model translating from right to left; (b) a fish model rotating clockwise; (c) an air film model of a single fluctuation; (d) is the variation of the translation distance of the test object in (a) over time; (e) is the variation of the rotation angle in (b) over time; (f) is the variation of the fluctuation height in (c) over time.

[0049] In the figure: 100-fringe projection system; 200-light collection system; 300-high-speed encoder; 400-data acquisition system; 500-computer; 101-fringe projection device; 102-sample to be tested; 201-camera lens; 202-first lens; 203-second lens; 204-first aperture; 401-reflector; 402-second aperture; 403-third lens; 404-fourth lens; 405-third aperture; 406-camera. DETAILED DESCRIPTION

[0050] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0051] See also Figure 1 The present invention includes a fringe projection system 100, an illumination collection system 200, a high-speed encoder 300, a data acquisition system 400 and a computer 500. The fringe projection system 100 includes a fringe projection device 101 and a sample to be measured 102; the illumination collection system 200 includes a camera lens 201, a first lens 202, a second lens 203 and a first aperture 204; the high-speed encoder 300 includes a digital micromirror device DMD; the data acquisition system 400 includes a reflector 401, a second aperture 402, a third lens 403, a fourth lens 404, a third aperture 405 and a camera 406; the computer 500 is connected to the fringe projection device 101, the high-speed encoder 300 and the camera 406 respectively.

[0052] See also Figure 1 , Figure 2 After the system is built, three structured light eight-bit fringe images with a resolution of 1920*1080 are loaded into the fringe projection device 101. The intensity distribution is sinusoidal on the x-axis with a period of 20 pixels. The sinusoidal phase difference of the three images is 2π / 3, and the duration of each fringe is 30ms. The fringe projection device projects this structured light fringe onto the sample to be tested 102, which is deformed due to the height modulation of the object to be tested, and then the structured light is reflected into the light collection system 200. The deformed fringe image is then collected by the camera lens 201, and passes through the 4f system with a reduction ratio of 60% composed of the first lens 202 and the second lens 203 along the optical path. After filtering out the diffraction order through the first aperture 204, it is finally clearly focused on the digital micromirror device DMD of the high-speed encoder 300.

[0053] See also Figure 1 , Figure 2 The random binary aperture on the digital micromirror device DMD generates three random code patterns, and the transformation is synchronously controlled by the computer 500, so the duration of each code is also 30ms. Each code is applied to the deformed stripes in turn to obtain three coded stripes. The coded stripes enter the data acquisition system 400 according to the optical path.

[0054] See also Figure 1 , Figure 2, the light on the digital micromirror device DMD is received and reflected by the reflector 401, and after being filtered by the second aperture 402, the light enters the 4f optical system with a reduction ratio of 80% composed of the third lens 403 and the fourth lens 404, which enhances the clarity of the image. Then, after being filtered by the third aperture 405, the light is focused on the camera 406 and stored by it. The camera is also controlled by the computer 500, but the period of its signal is three times that of the previous two, so the exposure time of the three devices is 30ms: 30ms: 90ms respectively. Therefore, the image recorded by the camera is the superposition of the three encoded fringe images generated by the high-speed encoder 300 within its single exposure time, that is, a compressed image.

[0055] See also Figure 2 , Figure 3 After the camera records the compressed image, the computer 500 needs to reconstruct the three-dimensional contour of the dynamic object to be measured based on this. In the present invention, the alternating direction multiplier method based on deep learning is used to solve it, and iterative calculation is performed by balancing the measurement constraint and the sparsity constraint. After the numerical initialization of the iterative framework, the forward estimation based on the maximum likelihood estimation is adopted. Combined with the input compressed image, the loss function is constructed and calculated, and then the generalized projection is performed. The denoiser removes clutter, noise and other operations on the image to restore the image details and save the feature information. Here, the denoiser DRUNET trained by deep learning is used. The discriminator is used to identify whether the image quality meets the preset index. If it meets the standard, the image is output. If it does not meet the standard, the variable is updated and the next round of iterative calculation is entered. Until the iteration ends, three deformed stripe images can be reconstructed from the input stripe image. This result should be completely consistent with the three original deformed stripes projected on the object to be measured in an ideal state. According to the reconstructed result, the wrapped phase can be calculated according to formula (8), and the absolute phase corresponding to the height distribution of the object to be measured is calculated by the spatial unwrapping algorithm, and then the 3D contour is obtained by phase-height mapping. Finally, the three-dimensional motion profile of the sample to be tested is visualized.

[0056] Example 1

[0057] In this embodiment, the samples observed by the CPSP device are designed as two static objects, and the original three-dimensional contour data are given. This embodiment aims to calculate the reconstruction and restoration capability of the CPSP device for the surface of a complex object to be measured.

[0058] See also Figure 1 The device described in this embodiment includes a fringe projection system 100, a light collection system 200, a high-speed encoder 300, a data acquisition system 400 and a computer 500.

[0059] The fringe projection device 101 projects sinusoidal structured light with a wavelength of 532 nm, and a sinusoidal period of the light is 20 pixels.

[0060] The samples to be tested are static ammonite plaster models and David plaster models with complex structures. Changes in the surface height cause the sinusoidal fringes to deform, which are then collected by the camera lens 201 .

[0061] The first lens 202 and the second lens 203 are both achromatic doublet lenses with a diameter of 25.4 mm, and the third lens 403 and the fourth lens 404 are both achromatic doublet lenses with a diameter of 12.7 mm, and their function is to focus polychromatic light of different wavelengths passing through the lenses to the same position. The focal lengths of the first lens 202 and the second lens 203 are 250 mm and 150 mm respectively, and the back focal plane of the first lens is the front focal plane of the second lens (i.e., the distance between the two lenses is 400 mm), so that the first lens and the second lens form a 4f optical system with a reduction ratio of 60%; similarly, the focal length of the third lens 403 is 250 mm, and the focal length of the fourth lens 404 is 200 mm, forming a 4f optical system with a reduction ratio of 80%. Therefore, the image entering the camera 406 has a reduction ratio of 48% compared to the sample to be tested.

[0062] See also Figure 1 , Figure 2 The digital micromirror device (DMD) of the high-speed encoder 300 has a 1920×1080 micromirror array, and the side length of each micromirror is 7.56 microns. Its function is to load the coding pattern and perform Hardmard product with the scene when reflecting the light beam, thereby encoding the scene.

[0063] The reflector 401 is a high-reflectivity silver-coated mirror with a diameter of 12.7 mm. It has a high reflectivity for light of different wavelengths and is used to guide the light transmitted by the high-speed encoder 300 into the data acquisition system 400.

[0064] The first aperture 204, the second aperture 402, and the third aperture 405 are used to adjust the aperture size, filter the diffraction order of the light beam passing through, and enhance the imaging quality. The first aperture 204 and the third aperture 405 are used to filter the diffraction order formed by the 4f optical system in front of them, and the second aperture 402 is used to filter the diffraction order caused by the digital micromirror device DMD encoding and the mirror steering.

[0065] The function of the camera 406 is to receive the accumulation of multiple DMD-encoded fringe images within the exposure time and record them as compressed encoded images. The exposure time is set to 90 milliseconds.

[0066] The function of the computer 500 is to provide a synchronous trigger signal. After receiving the signal, the fringe projection device 101, the high-speed encoder 300 and the camera 406 start working at the same time, and the exposure time is set to 30ms:30ms:90ms.

[0067] See also Figure 3 , is a schematic diagram of the reconstruction results for two static objects to be tested in this embodiment. Figures (a)-(c) are data of the ammonite model, and Figures (d)-(f) are data of the David model. Figure (a) shows the compressed image captured by the camera and the wrapped phase calculated based on the reconstructed stripes for the ammonite model. Figure (b) compares the original contour of the ammonite model with the 3D contour reconstructed by the device. Figure (c) The relative error between the reconstruction result and the original object, expressed as a percentage, is generally ±5%. Figure (d) shows the compressed image captured by the camera and the wrapped phase calculated based on the reconstructed stripes. Figure (e) compares the original contour of the model with the 3D contour reconstructed by the device. Figure (f) The relative error between the reconstruction result and the original object, expressed as a percentage, is generally ±2%. The numerical difference in the error mainly comes from the complexity of the object to be tested. The more complex it is, the greater the error, but it still has a higher fidelity.

[0068] Example 2

[0069] See also Figure 4 , is a schematic diagram of the reconstruction results of three dynamic test objects in this embodiment. The samples observed by the CPSP device are designed as three dynamic objects, namely (a) a chicken model that translates from right to left, (b) a fish model that rotates clockwise, and (c) a single-fluctuation air film model. This embodiment aims to calculate the reconstruction and restoration ability of the CPSP device for the surface of dynamic test objects.

[0070] The construction of the equipment details is the same as that of Example 1, except that the sample 102 to be tested is replaced.

[0071] See also Figure 4 , (d) is the change of the chick model's translation distance in space over time, and the time interval between each two counting points is 0.9s. (a) selects the spatial position of the chick model at six different moments to describe the translational motion from right to left. According to the reconstructed data of the CPSP device, the speed of this translational motion is about 0.5324mm / s.

[0072] See also Figure 4 , (e) is the change of the rotation angle of the fish model in space over time, and the time interval between each two counting points is 1.8s. (b) selects the spatial position of the fish model at six different moments to describe the clockwise rotation movement. According to the reconstructed data of the CPSP device, the angular velocity of this rotation movement is about 2.4353° / s.

[0073] See also Figure 4, (f) is the change of the height of the air film model in space with time, and the time interval between each two counting points is 0.45s. (c) The spatial position of the air film model at six different moments was selected to describe the fluctuation motion from bottom to top and then recover. The 3D contour surface and velocity parameters of the motion are consistent with the original data, so the CPSP device still has high fidelity for dynamic samples.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A time domain compressed three-dimensional contour dynamic imaging device, characterized in that: The device comprises a fringe projection system (100), a light collection system (200), a high-speed encoder (300), a data acquisition system (400) and a computer (500); The fringe projection system (100) comprises a fringe projection device (101) and a sample to be tested (102); the fringe projection device (101) can project sinusoidal grayscale fringes distributed in a horizontal direction along the transmission direction of an optical path onto the surface of the sample to be tested (102); the fringes are modulated by the height of the sample to be tested (102) to produce deformation, and then enter the light collection system (200); wherein the sample to be tested (102) is a dynamically moving object or a stationary object; The light collection system (200) comprises a camera lens (201), a first lens (202), a second lens (203) and a first aperture (204); A camera lens (201) is used, the focal length of which is 50 mm and the aperture value is 1.4; the camera lens focuses the deformed fringes on the surface of the sample to be measured onto the front focal plane of the first lens (202); The first lens (202) and the second lens (203) are achromatic doublet lenses with a diameter of 25.4 mm, which can focus polychromatic light of different wavelengths passing through the lenses to the same position; The focal length of the first lens (202) is 250 mm, the focal length of the second lens (203) is 150 mm, the back focal plane of the first lens is the front focal plane of the second lens, that is, the distance between the two lenses is 400 mm, and the first lens and the second lens form a 4f optical system with a reduction ratio of 60%; The first aperture (204) is located behind the second lens (203) and functions to filter the diffraction order of the 4f optical system composed of the first lens and the second lens by controlling the size of the aperture, thereby improving the quality of the image focused on the high-speed encoder (300); The high-speed encoder (300) is a digital micromirror device (DMD), which has a 1920×1080 micromirror array, each micromirror has a side length of 7.56 micrometers, and can encode images on its surface; The light beam enters the data acquisition system (400) after being reflected by the digital micromirror device (DMD); The optical centers of the sample to be tested (102), all components in the light collection system (200), and the high-speed encoder (300) are all located on a straight line, i.e., an optical path, and the focal planes of these components are perpendicular to the optical path; The data acquisition system (400) comprises a reflector (401), a second aperture (402), a third lens (403), a fourth lens (404), a third aperture (405) and a camera (406); The reflector (401) is a high-reflectivity silver-coated mirror with a diameter of 12.7 mm, and is used to guide the light transmitted by the high-speed encoder (300) into the data acquisition system (400); The second aperture (402) is located behind the reflector (401) and filters the diffraction order of the light beam after being reflected by controlling the size of the aperture, thereby improving the quality of the image passing through the third lens (403); The third lens (403) and the fourth lens (404) are achromatic doublet lenses with a diameter of 12.7 mm. The focal length of the third lens (403) is 250 mm, the focal length of the fourth lens (404) is 200 mm, the back focal plane of the third lens is the front focal plane of the fourth lens, that is, the distance between the two lenses is 450 mm, and the third lens and the fourth lens form a 4f optical system with a reduction ratio of 80%; the spatial position of the front focal plane of the third lens coincides with the mirror image generated by the high-speed encoder (300) on the reflector (401); the optical center of the camera (406) is located on the back focal plane of the fourth lens; The third aperture (405) is located behind the fourth lens (404) and filters the diffraction order of the 4f optical system composed of the third lens and the fourth lens by controlling the size of the aperture, thereby improving the quality of the image focused on the camera (406); The optical centers of all elements in the data acquisition system (400) are located on a straight line, i.e., an optical path, and the focal planes of these elements are perpendicular to the optical path; The 4f optical system composed of the first lens and the second lens has a reduction ratio of 60%, the 4f optical system composed of the third lens and the fourth lens has a reduction ratio of 80%, and the image entering the camera (406) has a reduction ratio of 48% compared to the sample to be tested (102); The entire process from the light being emitted from the fringe projection device (101) to entering the camera (406) for storage is a compressed sampling process; The computer (500) is connected to the fringe projection device (101), the high-speed encoder (300) and the camera (406) respectively, and outputs a control signal with a period of 1:1:3, so that the exposure times of the three devices are 30ms:30ms:90ms respectively; after acquiring the image stored in the camera (406), the computer (500) runs a three-dimensional surface reconstruction algorithm; The optical centers of all optical elements are located on the same horizontal plane; where: The compressed sampling process and the three-dimensional surface reconstruction algorithm are completed by mathematical derivation, which includes the combination of phase unwrapping and the image reconstruction algorithm based on deep learning, namely the CPSP algorithm. The specific data processing process is as follows: Assume that the intensity L of the sinusoidal fringe projected by the fringe projection device (101) is p The distribution is: ,(1) Where (x, y) represents a two-dimensional plane coordinate system, A P (x, y) represents the basic intensity distribution, B P (x, y) represents the maximum intensity amplitude of the sinusoidal change, φ P (x, y) represents the wrapped phase corresponding to the height of the sample to be tested (102); N is an integer of at least 3, representing the total number of phase shift steps, and the natural number variable n≤N represents the current number of phase shift steps; for each phase shift step, the projector projects a fringe, and a total of N fringe are projected; after the projected fringe is deformed by height modulation, the intensity of the deformed fringe L d The distribution is: ,(2) Where α(x, y) represents the surface reflectivity of the sample to be tested, and β(x, y) represents the ambient light noise; N pieces of deformed stripes L d It is then encoded by the random binary aperture c(x, y, n) on the high-speed encoder and then stacked into a compressed image m(x, y) by the camera within a single exposure time: ,(3) Let m and d be the compressed image m(x, y) and the deformed stripe L respectively. d In matrix form, equation (3) is simplified to matrix form: ,(4) Where C is the spatial encoding operator, T is the camera's image stacking operator, and A=CT is the total operator representing the data acquisition process of formulas (1) to (3); From the compressed image m, the deformed stripes containing the height information of the object to be measured need to be reconstructed; Based on maximum likelihood estimation, the mathematical model of this reconstruction process can be modeled as a numerical optimization problem: ,(5) Here Θ is a sparse transformation operator, ||•||1 represents the L1 norm; In order to solve this problem, an image reconstruction method based on the alternating direction multiplier method is used to construct the augmented Lagrangian equation, which is transformed into three iterative equations, i.e., three sub-problems, which are gradually solved through constrained alternating iterations. The image denoising variable v, the Lagrangian multiplier u and the Lagrangian regularization factor ρ are introduced, and the reconstructed deformed stripes d [i.e., L r (x, y, n)]: ,(6) Where I represents the identity matrix, k is the number of iterations, and A T is the matrix transpose of operator A, Denosier Θ It is a denoising step under sparse prior; when the reconstructed deformed stripes L are obtained r (x, y, n), the wrapped phase corresponding to the height of the sample to be measured can be extracted: , (7) The spatial phase unwrapping algorithm is applied to calculate the absolute phase from the wrapped phase. This absolute phase corresponds one-to-one with the height distribution of the sample to be tested, thereby visualizing the three-dimensional contour of the sample to be tested.

Citation Information

Patent Citations

  • Three-dimensional imaging system and method based on arc binary coding phase shift stripe projection

    CN112712585A

  • Compressed video imaging system and method

    CN116017151A

  • Method for obtaining single-exposure four-dimensional compression imaging and imaging system

    CN116366993A

  • Spectral three-dimensional imaging method and system based on fringe projection and application

    CN117589086A

  • Compressed sensing hyperspectral coding structured light imaging system and method

    CN118190162A