MicroLED-based structured light 3D camera and phase optimization method

Abstract

The invention discloses a MicroLED-based structured light 3D camera and a phase optimization method, and belongs to the technical field of computer vision and optical measurement. The camera is composed of a three-dimensional imaging unit and a control unit, and the three-dimensional imaging unit integrates a micro projector based on a MicroLED and a monocular industrial camera, so that a flexible extension structure can be adopted to extend into narrow areas such as pipelines and precision parts; the control unit is responsible for image projection, image acquisition control and three-dimensional reconstruction; according to the phase optimization method, an initial phase is obtained by combining a multi-frequency heterodyne method with a four-step phase shift method, then a gray model is constructed to solve parameters such as a background item and a modulation item pixel by pixel, iterative correction is performed on pixels exceeding a dynamic range, and meanwhile, a gray and phase dual convergence criterion is set to complete phase optimization. Miniaturization, low power consumption and high real-time performance are achieved by means of the MicroLED, phase calculation precision and robustness are improved through iterative optimization, and a scheme is provided for precision part detection, medical endoscopic imaging and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and optical measurement, in particular to a MicroLED-based structured light 3D camera and a phase optimization method. BACKGROUND

[0002] With the development of industrial manufacturing precision and medical imaging minimally invasive, three-dimensional reconstruction technology has been widely concerned in fields such as precision part surface detection, internal wall defect detection, microstructure measurement and medical image assistance. Among them, structured light three-dimensional imaging technology has become one of the mainstream methods of three-dimensional reconstruction due to its fast measurement speed, high resolution and non-contact advantages.

[0003] Traditional structured light systems are usually composed of a projection module (such as DLP or LCoS), a camera and related optical elements. The projection module generates a periodic fringe pattern projected onto the object surface, and the distorted fringe is captured by the camera, and the three-dimensional topography of the object is obtained by phase unwrapping. This method has achieved good results in conventional scenarios, but still has the following problems: 1) Large system volume, not conducive to miniaturization and portability: DLP projector requires complex optical path and optical engine, overall volume is large; LCoS is based on liquid crystal modulation, light efficiency is low, external strong light source is needed; These solutions are difficult to apply to detection in narrow spaces such as pipes, engine cylinders, and internal parts of precision parts.

[0004] 2) Limited phase unwrapping accuracy: Although the traditional multi-frequency heterodyne method can solve the phase ambiguity problem, in high-contrast surfaces and complex reflection environments, the phase error is often amplified; When there are pixel saturation, noise interference and uneven illumination in the captured image, the directly unwrapped phase will deviate, affecting the final three-dimensional reconstruction accuracy.

[0005] 3) System energy consumption and real-time problem: Industrial and medical applications have high requirements for real-time performance, but traditional light sources have slow response speed and high energy consumption, which is not conducive to high frame rate acquisition.

[0006] In recent years, MicroLED (Micro Light Emitting Diode) as a new type of display and projection technology, with its high brightness, low power consumption, miniaturization, arrayability, fast response and other characteristics, has gradually become an ideal choice to replace traditional light sources. MicroLED can be made into a micro probe type projection module, integrated with a monocular camera, thereby realizing a small structured light three-dimensional imaging system. This scheme is particularly suitable for part internal wall detection, microelectronic device detection and medical endoscopic imaging in narrow space scenarios.

[0007] However, even if MicroLED is introduced, the problem of miniaturization is solved, and the phase accuracy and robustness in three-dimensional reconstruction are still bottlenecks. The existing method lacks an effective iterative optimization mechanism to handle errors caused by saturated pixels and noise. SUMMARY

[0008] The purpose of the present application is to provide a MicroLED-based structured light 3D camera and a phase optimization method. Through the miniaturization advantage of the MicroLED light source, it is beneficial to realize the exploration into narrow space, and through the iterative-based phase optimization algorithm, the phase calculation accuracy is improved, and high-quality three-dimensional reconstruction is realized.

[0009] To achieve the above purpose, the present application provides a MicroLED-based structured light 3D camera, which comprises a three-dimensional imaging unit and a control unit, the three-dimensional imaging unit is connected with the control unit through a cable; the three-dimensional imaging unit comprises a micro projector and a monocular industrial camera; the micro projector comprises a MicroLED-based light source and a lens; The control unit comprises a picture projection and picture acquisition control unit and a calculation processing unit, the control unit is used for controlling the micro projector to project a stripe pattern, controlling the monocular industrial camera to synchronously acquire images and performing three-dimensional reconstruction calculation, and the control unit comprises a PC and an industrial computer.

[0010] The MicroLED-based structured light 3D camera, the present application further provides a phase optimization method, comprising the following steps: S1, hardware system building: a micro projector is constructed through a MicroLED-based light source and a lens, the micro projector cooperates with a monocular industrial camera to construct a three-dimensional imaging unit, and the three-dimensional imaging unit is connected with a control unit through a cable; S2, initial phase calculation: a wrapped phase with pixel-level accuracy and a global phase for unwrapping are acquired by using a multi-frequency heterodyne method, and an initial phase distribution is obtained; S3, three-dimensional reconstruction phase optimization iteration: S31, based on the initial phase, a gray model formula is constructed: ; Wherein I is the acquired gray value, A is a background item, B is a modulation item, is a phase, is a phase shift; A and B are calculated pixel by pixel, and are iteratively updated; S32, gray value judgment: if the calculated gray value is between 20 and 255, iteration correction is not needed, otherwise iteration correction is needed; S33, iteration and setting of stop condition: when the difference between the gray value and the phase value before and after iteration is less than the corresponding threshold value ε, iteration is stopped, the final optimized phase distribution is obtained, and three-dimensional reconstruction is completed.

[0011] Preferably, S2 specifically includes the following steps: S21, pattern projection: project three groups of sinusoidal fringe images with different frequencies by using the multi-frequency heterodyne method, including high frequency, medium frequency and low frequency; for each group of frequencies, use the four-step phase shift method to collect four images by adjusting the phase shift; S22, initial phase solution: after collecting multiple fringe images by monocular industrial camera, calculate the wrapped phase of each group of frequencies by four-step phase shift method; use multi-frequency heterodyne principle to unwrap the wrapped phase of different frequencies, eliminate 2Π cycle slip ambiguity, and obtain accurate and unambiguous initial phase distribution.

[0012] Preferably, in S31, the gray value I is solved as follows: Expand the formula in S31: ; Where, , ; For different phase shift angles i image, construct a linear equation group: ; Solve A, B1 and B2 by least square method, and get the modulation from .

[0013] Preferably, in S32, the specific process is as follows: Substitute A, B, into the gray value model: ; Correction judgment: if falls outside the effective dynamic range of monocular industrial camera, i.e. > 255 or < 20, it is determined that the pixel is saturated or underexposed, and iterative correction is needed. Gray update: if correction is needed, update the gray value of the image ; this step estimates the real brightness of the pixel according to the physical model, so that the gray recovery is more in line with the actual imaging characteristics, and reduces the phase distortion caused by saturated pixels.

[0014] Preferably, in S33, the specific content includes the following: S331, phase update: based on the updated gray value I k in S32, perform multi-frequency heterodyne operation again to calculate the new phase value ; S332, loop iteration: repeat steps S31 to S33 to solve I k+1 , ; S333, double convergence criterion: set iteration stop condition, when the following two conditions are met at the same time, judge convergence and terminate iteration: Gray convergence: ; Phase convergence: ; Wherein , The preset threshold value is adaptively set according to the sensor noise level and the required reconstruction accuracy; output the updated phase 优 .

[0015] Therefore, the application adopts the above-mentioned MicroLED-based structured light 3D camera and phase optimization method, which has the following beneficial effects: 1) Introducing MicroLED as the light source of the micro projection system, relying on the characteristics of miniaturization and arrayability, a micro probe type projection module can be made, which directly projects through the lens, saving the light source processing part of the conventional projection, and the system can be designed as a probe or a flexible extension structure, realizing miniaturization; Relying on the volume advantage, it can enter narrow space, providing new solutions for part inner wall detection, microelectronic device detection, medical endoscopic imaging and other fields, widening the application scenarios; At the same time, MicroLED has the characteristics of high brightness, low power consumption and fast response, which meets the needs of high frame rate acquisition and high real-time in industrial and medical fields; 2) The initial phase is obtained by the multi-frequency heterodyne method, and then combined with the joint iterative optimization of the brightness term A, the modulation degree B and the phase The adaptive gray scale correction of saturated and underexposed pixels is realized, the real brightness of the pixels is estimated according to the physical model, the gray scale recovery is more in line with the actual imaging characteristics, and the phase distortion caused by saturated pixels is fundamentally reduced; 3) The double convergence criterion of gray scale and phase is adopted to ensure the stability of the iteration process and the reliability of the results. Through multiple iterations, the errors caused by noise, nonlinear response and uneven exposure are effectively suppressed, and the continuity and accuracy of the phase distribution are improved.

[0016] The technical solutions of the application will be further described in detail below with the help of drawings and examples. DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flow chart of the phase optimization method of the application; Figure 2 is a structural schematic diagram of the MicroLED-based structured light 3D camera of the application.

[0018] REFERENCE NUMERALS 1, monocular industrial camera; 2, image projection and acquisition control unit; 3, calculation processing unit; 4, control unit; 5, MicroLED-based light source; 6, micro projector; 7, three-dimensional imaging unit; 8, lens. DETAILED DESCRIPTION

[0019] The technical solutions of the present application are further described below by means of the accompanying drawings and examples.

[0020] Unless otherwise defined, the technical terms or scientific terms used in the present application shall be understood as the usual meaning understood by a person with ordinary skill in the art to which the present application belongs. The terms "first", "second" and the like used in the present application do not represent any order, number or importance, but are only used to distinguish different components. The terms "include" or "contain" and the like mean that the elements or objects appearing before the terms cover the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connected" or "connected" and the like are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "up", "down", "left", "right" and the like are only used to represent relative positional relationships, and when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0021] Example 1 The present application provides a MicroLED-based structured light 3D camera, the structural diagram is as shown in Figure 2 , which comprises a three-dimensional imaging unit 7 and a control unit 4, the three-dimensional imaging unit 7 is connected with the control unit 4 through a cable; the three-dimensional imaging unit 7 comprises a micro projector 6 and a monocular industrial camera 1; the micro projector 6 comprises a MicroLED-based light source 5 and a lens 8; the three-dimensional imaging unit 7 as a whole can be designed as a flexible extension structure so as to extend into narrow areas including pipes and inner walls which cannot be reached by traditional systems.

[0022] The control unit 4 comprises an image projection and acquisition control unit 2 and a calculation processing unit 3, the control unit 4 is used for controlling the micro projector 6 to project a stripe pattern, controlling the monocular industrial camera 1 to synchronously acquire images and performing three-dimensional reconstruction calculation, and the control unit 4 comprises a PC and an industrial computer.

[0023] The MicroLED-based structured light 3D camera proposes a phase optimization method, the flow is as shown in Figure 1 , comprising the following steps: S1, hardware system building: constructing a micro projector 6 through a MicroLED-based light source 5 and a lens 8, constructing a three-dimensional imaging unit 7 by matching the micro projector 6 with a monocular industrial camera 1, and connecting the three-dimensional imaging unit 7 with a control unit 4 through a cable; S2, calculating initial phase: obtaining wrapped phase with pixel-level precision and global phase for unwrapping by using multi-frequency heterodyne method, to obtain initial phase distribution; specifically including the following steps: S21, pattern projection: projecting three groups of sinusoidal fringe images with different frequencies by using multi-frequency heterodyne method, including high frequency, medium frequency and low frequency; for each group of frequencies, four-step phase shift method is adopted, and four images are collected by adjusting the phase shift; S22, initial phase solving: after the monocular industrial camera 1 collects a plurality of fringe images, the wrapped phase of each group of frequencies is calculated by using four-step phase shift method; the wrapped phases of different frequencies are unwrapped by using multi-frequency heterodyne principle, and the 2Π cycle jump ambiguity is eliminated, so as to obtain accurate and unambiguous initial phase distribution.

[0024] S3, three-dimensional reconstruction phase optimization iteration: S31, based on the initial phase, a gray model formula is constructed: ; Where I is the collected gray value, A is the background term, B is the modulation term, is the phase, is the phase shift; A and B are calculated pixel by pixel and iteratively updated; The solving method of the collected gray value I is as follows: The formula in S31 is expanded as follows: ; Where, , ; For different phase shift angles i , the linear equation set is constructed: ; A, B1 and B2 are solved by least square method, and the modulation degree is obtained from .

[0025] S32, gray value judgment: if the calculated gray value is between 20 and 255, no iteration correction is needed, otherwise iteration correction is needed; the specific process is as follows: Substitute A, B, into the gray value model: ; Correction judgment: if falls outside the effective dynamic range of the monocular industrial camera, that is, I calc > 255 or I calc <20, it is determined that the pixel is saturated or underexposed, and iteration correction is needed; Gray update: if correction is needed, update the gray value I calcThe step estimates the real brightness of the pixel according to a physical model, so that the gray scale recovery is more in line with the actual imaging characteristics, and the phase distortion caused by the saturated pixel is reduced.

[0026] S33, iteration and setting of a stop condition: when the difference between the gray scale value and the phase value before and after iteration is less than the corresponding threshold value epsilon, iteration is stopped, and the final optimized phase distribution is obtained to complete three-dimensional reconstruction. Specifically, the following contents are included: S331, phase update: based on the updated gray scale value I k , the multi-frequency heterodyne operation is performed again, and a new phase value is calculated ; S332, loop iteration: steps S31 to S33 are repeatedly executed to solve I k+1 , ; S333, double convergence criterion: set the iteration stop condition, and when the following two conditions are met, determine convergence and terminate iteration: gray scale convergence: ; phase convergence: ; wherein , is a preset threshold value, and the threshold value is adaptively set according to the sensor noise level and the required reconstruction accuracy; and the updated phase 优 .

[0027] Therefore, the application adopts the above-mentioned structure light 3D camera based on MicroLED and the phase optimization method, introduces MicroLED as the light source of the micro projection system, is easy to realize miniaturization, can enter narrow space, provides a new solution for part inner wall detection, microelectronic device detection, medical endoscopic imaging and other fields, and widens the application scenarios; the initial phase is obtained through the multi-frequency heterodyne method, and then combined with the joint iterative optimization of the brightness term A, the modulation degree B and the phase , adaptive gray scale correction of saturated and underexposed pixels is realized; through multiple iterations, the double convergence criterion of gray scale and phase is adopted, errors caused by noise, nonlinear response and uneven exposure are effectively suppressed, and the continuity and accuracy of the phase distribution are improved.

[0028] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the application and not to limit them, although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the application can still be modified or replaced by equivalents, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the application.

Claims

1. A structured light 3D camera based on MicroLEDs, characterized in that, The application relates to a three-dimensional imaging system, comprising a three-dimensional imaging unit and a control unit, wherein the three-dimensional imaging unit is connected with the control unit through a cable; the three-dimensional imaging unit comprises a micro projector and a monocular industrial camera; the micro projector comprises a MicroLED-based light source and a lens; the control unit comprises a pattern projection and image acquisition control unit and a computing processing unit, and is used for controlling the micro projector to project a fringe pattern, controlling the monocular industrial camera to synchronously acquire images, and performing three-dimensional reconstruction calculation; the control unit comprises a PC and an industrial computer.

2. The phase optimization method of the structure light 3D camera based on Micro LED according to claim 1, wherein, The application further relates to a three-dimensional imaging method, comprising the following steps: S1, hardware system construction: a micro projector is constructed through a MicroLED-based light source and a lens, the micro projector is matched with a monocular industrial camera to construct a three-dimensional imaging unit, and the three-dimensional imaging unit is connected with a control unit through a cable; S2, initial phase calculation: a wrapped phase with pixel-level precision and a global phase used for unwrapping are acquired by using a multi-frequency heterodyne method, so that an initial phase distribution is obtained; S3, three-dimensional reconstruction phase optimization iteration: S31, based on the initial phase, a gray model formula is constructed: ； where I is the collected grayscale value, A is the background term, B is the modulation term, is the phase, is the phase shift; A, B are calculated pixel by pixel and updated iteratively; S32, gray value judgment: if the calculated gray value is between 20 and 255, iteration correction is not needed, otherwise, iteration correction is needed; S33, iteration and setting of a stop condition: when the difference between the gray value and the phase value before and after iteration is less than a corresponding threshold value epsilon, iteration is stopped, a final optimized phase distribution is obtained, and three-dimensional reconstruction is completed.

3. The phase optimization method of claim 2, wherein, In S2, the following steps are further included: S21, pattern projection: three groups of sinusoidal fringe images with different frequencies, including high frequency, medium frequency and low frequency, are projected by using a multi-frequency heterodyne method; for each group of frequencies, a four-step phase shift method is adopted, and four images are acquired by adjusting the phase shift; S22, initial phase solving: after a plurality of fringe images are acquired by the monocular industrial camera, the wrapped phase of each group of frequencies is calculated by using a four-step phase shift method; the wrapped phases of different frequencies are unwrapped by using a multi-frequency heterodyne principle, 2Pi cycle jump ambiguity is eliminated, and an accurate and non-fuzzy initial phase distribution is obtained.

4. The phase optimization method of claim 2, wherein, In S31, the solving method of the acquired gray value I is as follows: The formula in S31 is expanded as follows: ； wherein , ; For different phase shift angles i of the image, a system of linear equations is constructed: ； A, B1, B2 are solved by least square method, and the modulation factor is obtained from the modulation factor is obtained from 5. The phase optimization method of claim 4, wherein, In S32, the specific process is as follows: A, B, Substitute the gray value model: ; Correction decision: if falling outside the effective dynamic range of the monocular industrial camera, i.e. I calc > 255 or I calc < 20, the pixel is determined to be saturated or underexposed and needs to be corrected iteratively; Gray level update: if correction is needed, update the gray level I of the image calc ; this step estimates the real brightness of the pixel according to a physical model, making the gray level recovery more consistent with the real imaging characteristics and reducing the phase distortion caused by saturated pixels.

6. The phase optimization method of claim 5, wherein, In S33, the following contents are further included: S331, phase update: based on the updated gray value I in S32 k Again, the multi-frequency heterodyne operation is performed to calculate the new phase value ; S332, loop iteration: repeat execution of steps S31 to S33, solve I k+1 , ; S333, double convergence criterion: when the following two conditions are met simultaneously, it is determined that convergence is achieved and iteration is terminated: Gray scale convergence: ; Phase convergence: ; wherein , is a preset threshold value, the threshold value being adaptively set according to a sensor noise level and a required reconstruction accuracy; and outputting the updated phase 优 .

Images