Micro-led-based structured light 3D camera and phase optimization method

Abstract

The application discloses a MicroLED-based structured light 3D camera and a phase optimization method, and belongs to the technical field of computer vision and optical measurement. The camera is composed of a three-dimensional imaging unit and a control unit. The three-dimensional imaging unit integrates a MicroLED-based miniature projector and a monocular industrial camera, which is conducive to exploring narrow areas such as pipelines and precision parts by using a flexible extension structure. The control unit is responsible for projection control, image acquisition control and three-dimensional reconstruction. The phase optimization method is to obtain an initial phase by a multi-frequency heterodyne method combined with a four-step phase shift method, then construct a gray scale model to solve parameters such as background items and modulation items pixel by pixel, and iteratively correct pixels exceeding the dynamic range. Meanwhile, a gray scale and phase double convergence criterion is set to complete phase optimization. The application realizes miniaturization, low power consumption and high real-time performance with the aid of MicroLED, improves phase calculation accuracy and robustness through iterative optimization, and provides a scheme for precision part detection and medical endoscopic imaging.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and optical measurement technology, and in particular to a structured light 3D camera based on MicroLED and a phase optimization method. Background Technology

[0002] With the advancement of precision in industrial manufacturing and minimally invasive medical imaging, 3D reconstruction technology has gained widespread attention in fields such as precision part surface inspection, internal wall defect detection, microstructure measurement, and medical imaging assistance. Among these, structured light 3D imaging technology has become one of the mainstream methods for 3D reconstruction due to its advantages such as fast measurement speed, high resolution, and non-contact operation.

[0003] Traditional structured light systems typically consist of a projection module (such as DLP or LCoS), a camera, and related optical components. The projection module generates a periodic fringe pattern and projects it onto the object's surface. The camera captures the distorted fringes, and phase calculation is then used to obtain the object's three-dimensional shape. This method has achieved good results in typical scenarios, but it still has the following problems: 1) The system is large in size, which is not conducive to miniaturization and portability: DLP projectors require complex optical paths and optical engines, resulting in a large overall size; LCoS is based on liquid crystal modulation, which has low light efficiency and requires an external strong light source; these solutions are difficult to apply to the detection of narrow spaces such as pipes, engine cylinders, and the inside of precision parts.

[0004] 2) Limited accuracy of phase calculation: Although the traditional multi-frequency heterodyne method can solve the phase ambiguity problem, it often leads to phase error amplification on high-contrast surfaces and in complex reflection environments. When there is pixel saturation, noise interference and uneven illumination in the acquired image, the phase obtained by direct calculation will be deviated, affecting the final accuracy of 3D reconstruction.

[0005] 3) System energy consumption and real-time performance issues: Industrial and medical applications have high requirements for real-time performance, but traditional light sources have slow response speeds and high energy consumption, which are not conducive to high frame rate acquisition.

[0006] In recent years, MicroLED (micro light-emitting diode) has emerged as a novel display and projection technology, gradually becoming an ideal alternative to traditional light sources due to its high brightness, low power consumption, miniaturization, arrayability, and fast response. MicroLEDs can be fabricated into miniature probe-type projection modules, integrated with a monocular camera, thereby realizing a miniaturized structured light 3D imaging system. This approach is particularly suitable for confined space scenarios such as internal wall inspection of parts, microelectronic device inspection, and medical endoscopic imaging.

[0007] However, even with the introduction of MicroLEDs and the resolution of miniaturization issues, phase accuracy and robustness in 3D reconstruction remain bottlenecks. Existing methods lack effective iterative optimization mechanisms to handle errors caused by saturated pixels and noise. Summary of the Invention

[0008] The purpose of this invention is to provide a structured light 3D camera based on MicroLED and a phase optimization method. By leveraging the miniaturization advantage of MicroLED light sources, it is possible to penetrate narrow spaces. At the same time, the phase calculation accuracy is improved through an iterative phase optimization algorithm, thereby achieving high-quality 3D reconstruction.

[0009] To achieve the above objectives, the present invention provides a structured light 3D camera based on MicroLED, comprising a three-dimensional imaging unit and a control unit, wherein the three-dimensional imaging unit and the control unit are connected by a cable; the three-dimensional imaging unit includes a micro projector and a monocular industrial camera; the micro projector includes a MicroLED-based light source and a lens; The control unit includes a projection and acquisition control unit and a computing and processing unit. The control unit is used to control the micro projector to project stripe patterns, control the monocular industrial camera to synchronously acquire images, and perform three-dimensional reconstruction calculations. The control unit includes a PC and an industrial computer.

[0010] Based on a MicroLED-based structured light 3D camera, this invention also provides a phase optimization method, including the following steps: S1. Hardware system setup: A micro projector is built using a MicroLED-based light source and lens. The micro projector works with a monocular industrial camera to build a 3D imaging unit, and the 3D imaging unit is connected to the control unit via cables. S2. Calculate the initial phase: Use the multi-frequency heterodyne method to obtain the wrapping phase with pixel-level precision and the global phase for unwrapping, and obtain the initial phase distribution; S3, Three-dimensional reconstruction phase optimization iteration: S31. Based on the initial phase, construct the grayscale model formula: ; Where I represents the acquired grayscale value, A represents the background term, and B represents the modulation term. For phase, For phase shifting; calculate A and B pixel by pixel and update iteratively; S32. Gray value judgment: If the calculated gray value is between 20 and 255, no iterative correction is needed; otherwise, iterative correction is needed. S33. Iterate and set stopping conditions: When the difference between the gray value and the phase value before and after the iteration is less than the corresponding threshold ε, stop the iteration and obtain the final optimized phase distribution to complete the three-dimensional reconstruction.

[0011] Preferably, S2 specifically includes the following steps: S21. Pattern projection: Three sets of sinusoidal fringe images of different frequencies are projected using the multi-frequency heterodyne method, including high frequency, medium frequency and low frequency; for each set of frequencies, a four-step phase shift method is used to acquire four images by adjusting the phase shift. S22. Initial phase solution: After acquiring multiple stripe images by a monocular industrial camera, the wrapped phase of each frequency group is calculated by the four-step phase shift method; the wrapped phase of different frequencies is unwrapped by using the multi-frequency heterodyne principle to eliminate 2π cycle jump ambiguity and obtain an accurate and ambiguity-free initial phase distribution.

[0012] Preferably, in S31, the method for calculating the collected grayscale value I is as follows: Expanding the formula in S31, we get: ; in, , ; For different phase shift angles i From the image, construct a system of linear equations: ; A, B1, and B2 are obtained by solving using the least squares method, and then... The adjustment system was obtained.

[0013] Preferably, in S32, the specific process is as follows: Place A, B, Substitute into the grayscale model: ; Correction judgment: If Falling outside the effective dynamic range of a monocular industrial camera, i.e. >255 or If the value is less than 20, the pixel is determined to be either saturated or underexposed, and iterative correction is required. Grayscale update: If correction is needed, update the grayscale values ​​of the image. This step estimates the true brightness of pixels based on a physical model, making grayscale recovery more consistent with actual imaging characteristics and reducing phase distortion caused by saturated pixels.

[0014] Preferably, S33 specifically includes the following: S331, Phase Update: Based on the updated grayscale value I in S32 k Then, perform multi-frequency heterodyne calculation again to obtain new phase values. ; S332, Iterative Loop: Repeat steps S31 to S33 to solve I. k+1 , ; S333, Dual Convergence Criterion: Set an iteration stopping condition. When both of the following conditions are met simultaneously, convergence is determined and the iteration is terminated: Grayscale convergence: ; Phase convergence: ; in , The preset threshold is set adaptively based on the sensor noise level and the required reconstruction accuracy; the updated phase is output. 优 .

[0015] Therefore, the present invention employs the above-mentioned structured light 3D camera and phase optimization method based on MicroLED, and has the following beneficial effects: 1) Introducing MicroLEDs as the light source for micro-projection systems, their miniaturization and arrayability allow for the fabrication of micro probe-type projection modules. Projection occurs directly through a lens, eliminating the need for light source processing in conventional projection systems. The system can be designed as a probe-shaped or flexible extension structure, achieving miniaturization. Leveraging its size advantage, it can enter confined spaces, providing novel solutions for fields such as internal component inspection, microelectronic device inspection, and medical endoscopic imaging, thus broadening application scenarios. Simultaneously, MicroLEDs possess high brightness, low power consumption, and fast response characteristics, meeting the demands of high frame rate acquisition and high real-time performance in industrial and medical fields. 2) Obtain the initial phase using the multi-frequency heterodyne method, and then combine it with the luminance term A, modulation index B, and phase. The joint iterative optimization enables adaptive grayscale correction for saturated and underexposed pixels, and can estimate the true brightness of pixels based on the physical model, making grayscale recovery more consistent with actual imaging characteristics, and fundamentally reducing phase distortion caused by saturated pixels. 3) A dual convergence criterion of grayscale and phase is adopted to ensure the stability of the iterative process and the reliability of the results. Through multiple iterations, errors caused by noise, nonlinear response, and uneven exposure are effectively suppressed, improving the continuity and accuracy of the phase distribution.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] Figure 1 This is a flowchart of the phase optimization method of the present invention; Figure 2 This is a schematic diagram of the structure of the MicroLED-based structured light 3D camera of this invention.

[0018] Figure Labels 1. Monocular industrial camera; 2. Image acquisition and control unit; 3. Computation and processing unit; 4. Control unit; 5. MicroLED-based light source; 6. Miniature projector; 7. 3D imaging unit; 8. Lens. Detailed Implementation

[0019] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0021] Example 1 This invention provides a structured light 3D camera based on MicroLED, the structure of which is shown in the figure below. Figure 2 As shown, it includes a 3D imaging unit 7 and a control unit 4, which are connected by a cable. The 3D imaging unit 7 includes a micro projector 6 and a monocular industrial camera 1. The micro projector 6 includes a MicroLED-based light source 5 and a lens 8. The 3D imaging unit 7 as a whole can be designed as a flexible extension structure to reach narrow areas, including pipes and inner walls, that are inaccessible to traditional systems.

[0022] The control unit 4 includes a projection and acquisition control unit 2 and a computing and processing unit 3. The control unit 4 is used to control the micro projector 6 to project stripe patterns, control the monocular industrial camera 1 to synchronously acquire images, and perform three-dimensional reconstruction calculations. The control unit 4 includes a PC and an industrial computer.

[0023] A phase optimization method is proposed for a structured light 3D camera based on MicroLED, the process of which is as follows: Figure 1 As shown, it includes the following steps: S1. Hardware system setup: A micro projector 6 is constructed using a MicroLED-based light source 5 and lens 8. The micro projector 6, together with a monocular industrial camera 1, forms a three-dimensional imaging unit 7, which is then connected to the control unit 4 via a cable. S2. Calculate the initial phase: Use the multi-frequency heterodyne method to obtain the wrapping phase with pixel-level precision and the global phase for unwrapping, to obtain the initial phase distribution; specifically including the following steps: S21. Pattern projection: Three sets of sinusoidal fringe images of different frequencies are projected using the multi-frequency heterodyne method, including high frequency, medium frequency and low frequency; for each set of frequencies, a four-step phase shift method is used to acquire four images by adjusting the phase shift. S22. Initial phase solution: After acquiring multiple stripe images by the monocular industrial camera 1, the wrapped phase of each frequency group is calculated by the four-step phase shift method; the wrapped phase of different frequencies is unwrapped by using the multi-frequency heterodyne principle to eliminate 2π cycle slip ambiguity and obtain an accurate and ambiguity-free initial phase distribution.

[0024] S3, Three-dimensional reconstruction phase optimization iteration: S31. Based on the initial phase, construct the grayscale model formula: ; Where I represents the acquired grayscale value, A represents the background term, and B represents the modulation term. For phase, For phase shifting; calculate A and B pixel by pixel and update iteratively; The method for calculating the collected grayscale value I is as follows: Expanding the formula in S31, we get: ; in, , ; For different phase shift angles i From the image, construct a system of linear equations: ; A, B1, and B2 are obtained by solving using the least squares method, and then... The adjustment system was obtained.

[0025] S32. Grayscale value judgment: If the calculated grayscale value is between 20 and 255, no iterative correction is needed; otherwise, iterative correction is required. The specific process is as follows: Place A, B, Substitute into the grayscale model: ; Correction judgment: If Falling outside the effective dynamic range of a monocular industrial camera, i.e., I calc >255 or I calc If the value is less than 20, the pixel is determined to be either saturated or underexposed, and iterative correction is required. Grayscale update: If correction is needed, update the grayscale value I of the image. calcThis step estimates the true brightness of pixels based on a physical model, making grayscale recovery more consistent with actual imaging characteristics and reducing phase distortion caused by saturated pixels.

[0026] S33. Iterate and set a stopping condition: When the difference between the grayscale value and the phase value before and after the iteration is less than the corresponding threshold ε, stop the iteration to obtain the final optimized phase distribution and complete the 3D reconstruction. Specifically, this includes the following: S331, Phase Update: Based on the updated grayscale value I in S32 k Then, perform multi-frequency heterodyne calculation again to obtain new phase values. ; S332, Iterative Loop: Repeat steps S31 to S33 to solve I. k+1 , ; S333, Dual Convergence Criterion: Set an iteration stopping condition. When both of the following conditions are met simultaneously, convergence is determined and the iteration is terminated: Grayscale convergence: ; Phase convergence: ; in , The preset threshold is set adaptively based on the sensor noise level and the required reconstruction accuracy; the updated phase is output. 优 .

[0027] Therefore, this invention employs the aforementioned MicroLED-based structured light 3D camera and phase optimization method, introducing MicroLED as the light source for a micro-projection system. This facilitates miniaturization, enabling it to operate in confined spaces and providing novel solutions for fields such as internal wall inspection of parts, microelectronic device inspection, and medical endoscopic imaging, thus broadening application scenarios. The initial phase is obtained through a multi-frequency heterodyne method, and then combined with the brightness term A, modulation index B, and phase... The joint iterative optimization achieves adaptive grayscale correction for saturated and underexposed pixels; by adopting dual convergence criteria of grayscale and phase, and through multiple rounds of iteration, it effectively suppresses errors caused by noise, nonlinear response and uneven exposure, and improves the continuity and accuracy of phase distribution.

[0028] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A phase optimization method for a structured light 3D camera based on MicroLED, characterized in that, Includes the following steps: S1. Hardware system setup: A micro projector is built using a MicroLED-based light source and lens. The micro projector works with a monocular industrial camera to build a 3D imaging unit, and the 3D imaging unit is connected to the control unit via cables. S2. Calculate the initial phase: Use the multi-frequency heterodyne method to obtain the wrapping phase with pixel-level precision and the global phase for unwrapping, and obtain the initial phase distribution; S3, Three-dimensional reconstruction phase optimization iteration: S31. Based on the initial phase, construct the grayscale model formula: ； Where I represents the acquired grayscale value, A represents the background term, and B represents the modulation term. For phase, For phase shifting; calculate A and B pixel by pixel and update iteratively; S32. Gray value judgment: If the calculated gray value is between 20 and 255, no iterative correction is needed; otherwise, iterative correction is needed. The specific process is as follows: Place A, B, Substitute into the grayscale model: ; Correction judgment: If Falling outside the effective dynamic range of a monocular industrial camera, i.e., I calc >255 or I calc If the value is less than 20, the pixel is determined to be either saturated or underexposed, and iterative correction is required. Grayscale update: If correction is needed, update the grayscale value I of the image. calc This step estimates the true brightness of pixels based on a physical model, making grayscale recovery more consistent with actual imaging characteristics and reducing phase distortion caused by saturated pixels. S33. Iterate and set a stopping condition: When the difference between the grayscale value and the phase value before and after the iteration is less than the corresponding threshold ε, stop the iteration, obtain the final optimized phase distribution, and complete the 3D reconstruction; specifically Includes the following: S331, Phase Update: Based on the updated grayscale value I in S32 k Then, perform multi-frequency heterodyne calculation again to obtain new phase values. ; S332, Iterative Loop: Repeat steps S31 to S33 to solve I. k+1 , ; S333, Dual Convergence Criterion: Set an iteration stopping condition. When both of the following conditions are met simultaneously, convergence is determined and the iteration is terminated: Grayscale convergence: ; Phase convergence: ; in , The preset threshold is set adaptively based on the sensor noise level and the required reconstruction accuracy; the updated phase is output. 优 .

2. The phase optimization method according to claim 1, characterized in that, S2 specifically includes the following steps: S21. Pattern projection: Three sets of sinusoidal fringe images of different frequencies are projected using the multi-frequency heterodyne method, including high frequency, medium frequency and low frequency; for each set of frequencies, a four-step phase shift method is used to acquire four images by adjusting the phase shift. S22. Initial phase solution: After acquiring multiple stripe images by a monocular industrial camera, the wrapped phase of each frequency group is calculated by a four-step phase shifting method; the wrapped phase of different frequencies is unwrapped using the multi-frequency heterodyne principle to eliminate 2π cycle slip ambiguity and obtain an accurate and ambiguity-free initial phase distribution.

3. A MicroLED-based structured light 3D camera that implements the phase optimization method according to any one of claims 1-2, characterized in that: It includes a 3D imaging unit and a control unit, the 3D imaging unit and the control unit being connected by a cable; the 3D imaging unit includes a micro projector and a monocular industrial camera; the micro projector includes a MicroLED-based light source and a lens; The control unit includes a projection and acquisition control unit and a computing and processing unit. The control unit is used to control the micro projector to project stripe patterns, control the monocular industrial camera to synchronously acquire images, and perform three-dimensional reconstruction calculations. The control unit includes a PC and an industrial computer.

Images