Structure light three-dimensional reconstruction system and method based on fold-diffraction hybrid imaging

Abstract

The application belongs to the technical field of structured light, and relates to a structured light three-dimensional reconstruction system and method based on a fold-diffraction hybrid imaging. The system comprises a structured light projection module, a fold-diffraction hybrid imaging module, a target object and a data processing module. The structured light projection module, the fold-diffraction hybrid imaging module and the target object are arranged at intervals. The structured light projection module is connected with the data processing module to emit structured light and produce diffuse reflection on the surface of the target object. The fold-diffraction hybrid imaging module comprises filter plates, Fresnel zone plates, liquid lenses, solid lenses and near-infrared cameras arranged at intervals. The filter plates receive the diffuse reflected light and transmit the light to the near-infrared cameras in turn. The liquid lenses are connected with the data processing module to switch the structured light detection mode by controlling the input voltage of the liquid lenses. The data processing module is connected with the near-infrared cameras to collect multi-line structured light images and realize three-dimensional reconstruction. The application can switch the structured light detection mode.

Description

Technical Field

[0001] This application relates to the field of structured light technology, and in particular to a structured light three-dimensional reconstruction system and method based on refractive indexing and diffraction hybrid imaging. Background Technology

[0002] Structured light is a type of active optical 3D measurement method that can perform visual inspections in low-light (or even no-light) and texture-lacking environments, achieving high accuracy and stability. Structured light vision systems can serve as the "eyes" of unmanned system platforms and are expected to be applied in complex environments such as urban underground utility tunnels, buildings, and caves to guide platforms in performing tasks such as reconnaissance, bomb disposal, and inspection.

[0003] Currently, some structured light 3D cameras have been launched on the market, including Kinect, Intel RealSense and other series of products. They usually use wide-angle lenses with fixed focal lengths, which reduces the difficulty of system calibration to some extent. However, due to the single field of view, the system cannot reasonably adjust the imaging field of view according to actual needs and target size and position. This may cause some detailed information of the target to be incomplete during the measurement process, affecting the accuracy and range of the structured light system's three-dimensional measurement.

[0004] If a zoom lens is used, traditional optical zoom systems typically consist of multiple solid-state lenses and servo motors, resulting in a large system size and complex structure that cannot meet the design requirements of lightweight and miniaturized structured light vision systems.

[0005] In addition, industrial-grade structured light cameras are mainly developed based on DLP micromirror chipset technology. Although they can acquire high-precision three-dimensional position data, their measurement depth range is limited, usually within 1m.

[0006] In conclusion, existing technologies each have their own drawbacks and cannot be combined to address all of them. Summary of the Invention

[0007] Therefore, it is necessary to provide a structured light 3D reconstruction system and method based on refractive indexing and diffraction hybrid imaging to address the above-mentioned technical problems, which can switch structured light detection modes.

[0008] A structured light 3D reconstruction system based on refracto-diffracto hybrid imaging includes: a structured light projection module, a refracto-diffracto hybrid imaging module, a target object, and a data processing module; the structured light projection module, the refracto-diffracto hybrid imaging module, and the target object are spaced out.

[0009] The structured light projection module is connected to the data processing module to emit structured light and generate diffuse reflection on the surface of the target object.

[0010] The refractive index-diffraction hybrid imaging module includes a filter, a Fresnel zone plate, a liquid lens, a solid lens, and a near-infrared camera arranged at intervals. The filter receives diffusely reflected light and sequentially transmits it to the Fresnel zone plate, the liquid lens, the solid lens, and the near-infrared camera. The liquid lens is connected to the data processing module so that the structured light detection mode can be switched by controlling the input voltage of the liquid lens.

[0011] The data processing module is connected to the near-infrared camera to acquire multi-line structured light images and achieve three-dimensional reconstruction.

[0012] In one embodiment, switching the structured light detection mode is achieved by controlling the input voltage of the liquid lens, including:

[0013] Calculate the optical power of the Fresnel zone plate as Φ b At that time, the first correspondence between the input voltage and optical power of the liquid lens is as follows:

[0014] d1d2d3Φ b Φ L (u)Φ s -(d1d2+d1d3)Φ b Φ L (u)-d1d3Φ b Φ s -d2d3Φ b Φ s -d2d3Φ L (u)Φ s +(d1+d2+d3)Φ b +(d2+d3)Φ L (u)+d3Φ s -1 = 0

[0015] In the formula, d1 is the distance between the Fresnel zone plate and the liquid lens, d2 is the distance between the liquid lens and the solid lens, d3 is the distance between the solid lens and the near-infrared camera, and Φ b Φ is the optical power of the Fresnel zone plate. L (u) is the mapping function of the optical power of the liquid lens with respect to the input voltage, and Φs is the optical power of the solid lens;

[0016] Based on the first correspondence, u is solved to obtain the input voltage of the liquid lens in the wide-angle measurement mode.

[0017] In one embodiment, switching the structured light detection mode by controlling the input voltage of the liquid lens further includes:

[0018] The optical power of the Fresnel zone plate is calculated as -Φ b The second correspondence between the input voltage and optical power of the liquid lens:

[0019] d1d2d3Φ b Φ L (u)Φ s -(d1d2+d1d3)Φ b Φ L (u)-d1d3Φ b Φ s -d2d3Φ b Φ s +d2d3Φ L (u)Φ s +(d1+d2+d3)Φ b -(d2+d3)Φ L (u)-d3Φ s +1 = 0

[0020] Based on the second correspondence, u is solved to obtain the input voltage of the liquid lens in the telephoto measurement mode.

[0021] In one embodiment, the optical power of the Fresnel zone plate is ±Φ b Calculated by the following formula:

[0022]

[0023] In the formula, λ0 is the center wavelength of the filter, and ρ1 is the radius of the first half-wave zone of the Fresnel zone plate.

[0024] In one embodiment, the structured light projection module includes: a line laser, a micro MEMS galvanometer, a laser controller, and a galvanometer driving circuit board;

[0025] The line laser is connected to the data processing module via the laser controller, so as to emit structured light under the first command sent by the data processing module to the laser controller;

[0026] The micro MEMS galvanometer is connected to the data processing module via the galvanometer driving circuit board, so as to reflect the structured light under the second command sent by the data processing module to the galvanometer driving circuit board.

[0027] In one embodiment, the first instruction includes: the on / off state of the laser, the PWM frequency, the duty cycle, and the trigger parameters;

[0028] The second instruction includes: galvanometer scanning angle, oscillation frequency, and open / closed state.

[0029] In one embodiment, the data processing module coordinates the control of the PWM frequency of the line laser and the oscillation frequency of the micro-MEMS galvanometer to satisfy the following formula in order to form a stable multi-line structured light image.

[0030] f L =Nf G

[0031] In the formula, f L f is the PWM frequency of the line laser. G denoted as the oscillation frequency of the micro-MEMS galvanometer, and N as the number of light stripes to be projected.

[0032] The structured light 3D reconstruction method based on refraction-diffraction hybrid imaging, employing the aforementioned structured light 3D reconstruction system based on refraction-diffraction hybrid imaging, includes:

[0033] Select the structured light detection mode based on the distance to the target;

[0034] Based on the structured light detection mode, the input voltage of the liquid lens is calculated, and the liquid lens is controlled with the input voltage to obtain a multi-line structured light image;

[0035] Based on the multi-line structured light image, the pixel points corresponding to the center of the structured light stripes are extracted, and combined with the calibration parameters in the structured light detection mode, three-dimensional point cloud data is generated.

[0036] Based on the aforementioned 3D point cloud data, 3D reconstruction is performed.

[0037] In one embodiment, selecting a structured light detection mode based on the distance to the target includes:

[0038] Near-infrared images are acquired, and the distance to targets in the near-infrared images is estimated based on the image edge detection algorithm. Target regions are marked with rectangular boxes, and the ratio of the height of the rectangular box in the near-infrared image to the overall image height is used as the judgment ratio.

[0039] If the ratio is greater than the preset threshold, select the wide-angle measurement mode for structured light detection.

[0040] If the ratio is less than or equal to the preset threshold, select the long-focus measurement mode for structured light detection.

[0041] In one embodiment, performing 3D reconstruction based on the 3D point cloud data includes:

[0042] The three-dimensional point cloud data is filtered, and the three-dimensional reconstruction result of the target object is obtained based on the Delaunay triangulation method.

[0043] The aforementioned structured light 3D reconstruction system and method based on refractive index-diffraction hybrid imaging incorporates a liquid-solid composite optics refractive index-diffraction hybrid imaging module. By adjusting the input voltage of the liquid lens, the system can switch between structured light detection modes (including wide-angle and telephoto measurement modes), allowing for flexible switching between wide and narrow fields of view according to actual needs. This enhances the stereo perception range of unmanned platform vision systems in environments with insufficient lighting (or even no light) and lack of texture, overcoming the limitations of traditional structured light cameras such as single field of view and limited measurable range. It is expected to improve the 3D detection capabilities of unmanned systems in complex environments such as underground spaces and caves. Furthermore, the system of this application is small in size and simple in structure, meeting the design requirements of lightweight and miniaturization. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of a structured light 3D reconstruction system based on refractive indexing and diffraction hybrid imaging in one embodiment;

[0045] Figure 2 This is a schematic diagram of the refractive index-diffraction hybrid imaging module of a structured light 3D reconstruction system based on refractive index-diffraction hybrid imaging in one embodiment.

[0046] Figure 3 This is a flowchart illustrating a structured light 3D reconstruction method based on refracto-diffraction hybrid imaging in one embodiment.

[0047] Figure 4 This is a schematic diagram of the architecture of a structured light 3D reconstruction method based on refracto-diffraction hybrid imaging in one embodiment.

[0048] Figure label:

[0049] 1. Line laser, 2. Miniature MEMS galvanometer, 3. Laser controller, 4. Galvanometer drive circuit board, 5. Filter, 6. Fresnel zone plate, 7. Liquid lens, 8. Solid lens, 9. Near-infrared camera, 10. Liquid lens drive circuit board, 11. Housing, 12. Data processing module, 13. Target object. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0051] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0052] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. In the description of this application, "multiple sets" means at least two sets, such as two sets, three sets, etc., unless otherwise explicitly specified.

[0053] In this application, unless otherwise expressly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two elements or the interaction between two elements, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0054] Furthermore, the technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.

[0055] This application provides a structured light 3D reconstruction system based on refractive indexing and diffraction hybrid imaging, such as Figure 1 and Figure 2 As shown, it includes: a structured light projection module, a refraction-diffraction hybrid imaging module, a target object, and a data processing module.

[0056] The connection between the components is as follows: the structured light projection module, the refractive index and diffraction hybrid imaging module, and the target object are spaced apart, and both the structured light projection module and the refractive index and diffraction hybrid imaging module are connected to the data processing module.

[0057] The structured light projection module emits structured light and generates diffuse reflection on the surface of the target object. The structured light projection module includes: a line laser 1, a miniature MEMS galvanometer 2, a laser controller 3, and a galvanometer driver circuit board 4.

[0058] The refractive index-diffraction hybrid imaging module is used to receive diffusely reflected light and switch between different structured light detection modes under different input voltages of the liquid lens. The refractive index-diffraction hybrid imaging module includes: a filter 5 spaced apart, a Fresnel zone plate 6, a liquid lens 7, a solid lens 8, a near-infrared camera 9, and a liquid lens driving circuit board 10.

[0059] Target object 13 is used to receive structured light and generate diffuse reflection.

[0060] The data processing module 12 is used to issue serial communication commands (including: first command, second command, and third command), control the line laser, the micro-MEMS galvanometer, and the liquid lens, and acquire multi-line structured light images from the near-infrared camera. Specifically: it is connected to the line laser via a laser controller to control the structured light emitted by the line laser; it is connected to the micro-MEMS galvanometer via a galvanometer driver circuit board to control the structured light reflected by the micro-MEMS galvanometer; it is connected to the liquid lens via a liquid lens driver circuit board to switch the structured light detection mode by controlling the input voltage of the liquid lens; and it is connected to the near-infrared camera to acquire multi-line structured light images for 3D reconstruction. Specifically, the data processing module can be a host computer.

[0061] A line laser 1 is connected to a data processing module via a laser controller. Upon receiving a first command from the data processing module to the laser controller, it emits structured light (i.e., laser light). The first command includes the laser's on / off state, PWM frequency, duty cycle, and trigger parameters. Preferably, the center wavelength of the line laser is λ0, satisfying 780nm < λ0 < 900nm. The brightness adjustment method is PWM control to avoid detection and meet the detection requirements.

[0062] The miniature MEMS galvanometer 2 is connected to the data processing module via a galvanometer driving circuit board. It reflects the structured light upon receiving a second command from the data processing module to the galvanometer driving circuit board. This second command includes the galvanometer scanning angle, oscillation frequency, and on / off state. Preferably, the miniature MEMS galvanometer operates in a single-axis scanning mode.

[0063] Laser controller 3 receives the first instruction from the data processing module and regulates the line laser, specifically by regulating the on / off state of the laser (i.e., structured light), PWM frequency, duty cycle, and trigger parameters (controlling the laser triggering time to be consistent with the oscillation time of the galvanometer).

[0064] The galvanometer driver circuit board 4 receives the second instruction from the data processing module and regulates the micro MEMS galvanometer, specifically controlling the galvanometer scanning angle, oscillation frequency, and opening / closing state, and together with the laser controller, ensures that the laser triggering time is consistent with the galvanometer oscillation time.

[0065] Filter 5 receives diffusely reflected light and then filters it before passing it to the Fresnel zone plate. It should be noted that the filter wavelength is the same as that of the line laser, and the filter is a narrowband filter with a center wavelength λ0.

[0066] Fresnel zone plate 6 receives light from the filter and transmits it to the liquid lens. Specifically, the odd-numbered half-wave zones of the Fresnel zone plate are transparent, while the even-numbered half-wave zones are opaque, utilizing the characteristics of different optical powers to adjust different fields of view.

[0067] The liquid lens 7 receives light from the Fresnel zone plate, changes its optical power value by adjusting the input voltage, and then transmits the changed optical power to the solid lens. It is mainly used for imaging field transformation.

[0068] Solid lens 8 receives light from liquid lens and transmits it to near-infrared camera after correction of optical power and aberrations in the correction system.

[0069] The near-infrared camera 9 receives light from the solid lens and transmits the multi-line structured light image to the data processing module.

[0070] The liquid lens drive circuit board 10 receives the third instruction from the data processing module and regulates the input voltage of the liquid lens.

[0071] It should also be noted that the line laser 1, the micro MEMS galvanometer 2, the laser controller 3, the galvanometer driving circuit board 4, the filter 5, the Fresnel zone plate 6, the liquid lens 7, the solid lens 8, the near-infrared camera 9, and the liquid lens driving circuit board 10 are all mounted on the housing 11, forming a whole.

[0072] In this embodiment, the structured light emitted by the line laser is reflected onto the target object by a miniature MEMS galvanometer, causing diffuse reflection on the target object and then transmitted to the filter of the refractive-diffraction hybrid imaging module. When the optical power of the Fresnel zone plate is different, the input voltage of the liquid lens needs to be adjusted to keep the position of the image plane A stable.

[0073] By controlling the input voltage of the liquid lens in the refractive-diffraction hybrid imaging module, the switching of structured light detection modes (including wide-angle measurement mode and telephoto measurement mode) is achieved. Specifically:

[0074] (1) The optical power of the liquid lens under different input voltages was obtained by measuring with a focal length meter; based on this, the mapping function Φ of the optical power of the liquid lens with respect to the input voltage was obtained by cubic curve fitting based on the least squares method. L (u).

[0075] (2) The Fresnel zone plate has two conjugate principal focal lengths, corresponding to optical powers of ±Φ. b Calculated by the following formula:

[0076]

[0077] In the formula, λ0 is the center wavelength of the filter, and ρ1 is the radius of the first half-wave zone of the Fresnel zone plate.

[0078] (3) The image plane offset equation is obtained by the Gaussian bracket method. Based on the image plane stability condition, the optical power ±Φ of the Fresnel zone plate is obtained. b The relationship between the input voltages of the corresponding liquid lens. Specifically:

[0079] Calculate the optical power of the Fresnel zone plate as Φ b At that time, the first correspondence between the input voltage and optical power of the liquid lens is as follows:

[0080] d1d2d3Φ b Φ L (u)Φ s -(d1d2+d1d3)Φ b Φ L (u)-d1d3Φ b Φ s -d2d3Φ b Φ s -d2d3Φ L (u)Φ s +(d1+d2+d3)Φ b +(d2+d3)Φ L (u)+d3Φ s -1 = 0

[0081] In the formula, d1 is the distance between the Fresnel zone plate and the liquid lens, d2 is the distance between the liquid lens and the solid lens, d3 is the distance between the solid lens and the near-infrared camera, and Φ b Φ is the optical power of a Fresnel zone plate. L (u) is the mapping function of the optical power of the liquid lens with respect to the input voltage, and Φs is the optical power of the solid lens;

[0082] Based on the first correspondence, the optical power of the liquid lens is obtained, and based on Φ L (u) Solve for u to obtain the solution u1, which is the input voltage of the liquid lens in the wide-angle measurement mode. That is, when the input voltage of the liquid lens is adjusted to u1, the system switches to the wide-angle measurement mode.

[0083] The optical power of the Fresnel zone plate is calculated as -Φ b The second correspondence between the input voltage and optical power of the liquid lens:

[0084] d1d2d3Φ b Φ L (u)Φ s -(d1d2+d1d3)Φb Φ L (u)-d1d3Φ b Φ s -d2d3Φ b Φ s +d2d3Φ L (u)Φ s +(d1+d2+d3)Φ b -(d2+d3)Φ L (u)-d3Φ s +1 = 0

[0085] Based on the second correspondence, the optical power of the liquid lens is obtained, and based on Φ L (u) Solve for u to obtain the solution u2, which is the input voltage of the liquid lens in the long focal length measurement mode. That is, when the input voltage of the liquid lens is adjusted to u2, the system switches to the long focal length measurement mode.

[0086] Therefore, different fields of view in structured light imaging can be switched by adjusting the input voltage of a single liquid lens.

[0087] The data processing module, in conjunction with the laser controller, controls the PWM frequency of the line laser and the oscillation frequency of the micro-MEMS galvanometer to satisfy the following formula, so that the near-infrared camera can form a stable multi-line structured light image;

[0088] f L =Nf G

[0089] In the formula, f L f is the PWM frequency of the line laser. G denoted as the oscillation frequency of the micro-MEMS galvanometer, and N as the number of light stripes to be projected.

[0090] It should be noted that, within a certain scanning range, the number of projected light stripes is adjusted by the PWM frequency of the line laser, and the line width of the light stripes is adjusted by the duty cycle parameter of the line laser.

[0091] It should also be noted that controlling the exposure time and exposure moment of the near-infrared camera can capture multi-line structured light images modulated by the target.

[0092] The above-mentioned structured light 3D reconstruction system based on refracto-diffraction hybrid imaging was designed with... Liquid-solid composite opticsThe refractive index-diffraction hybrid imaging module, by adjusting the input voltage of the liquid lens, can switch between structured light detection modes (including wide-angle measurement mode and telephoto measurement mode). Given a certain level of 3D measurement accuracy, a wide field of view (corresponding to wide-angle mode) allows for measuring targets at close ranges and covers a large field of view, while a narrow field of view (corresponding to telephoto mode) allows for measuring targets at long distances and covers a small field of view. Therefore, compared to a single-field-of-view structured light camera, this application expands both the range of measurable target distances and the size of the field of view. With a fixed range of measurable target distances or the size of the field of view, it improves measurement accuracy, enabling flexible switching between wide and narrow fields of view according to actual needs. This allows for reasonable adjustment of the imaging field of view, enhancing the stereo perception range of the unmanned platform's vision system in environments with insufficient lighting (or even no light) and lack of texture. It also improves the accuracy, depth, and range of 3D measurements by the structured light system, obtaining high-precision 3D position data. This overcomes the problems of traditional structured light cameras, such as a single field of view and limited measurable range, and enhances the 3D detection capabilities of unmanned systems in complex environments such as underground spaces and caves. Furthermore, this application achieves rapid switching of the detection field of view by adjusting the voltage of a single liquid lens, without any moving components in the system. In addition, the system of this application is small in size and simple in structure, meeting the design requirements of lightweight and miniaturization.

[0093] This application also provides a structured light 3D reconstruction method based on refractive indexing and diffraction hybrid imaging, employing a structured light 3D reconstruction system based on refractive indexing and diffraction hybrid imaging, such as... Figure 3 As shown, in one embodiment, the following steps are included:

[0094] Step 302: Select the structured light detection mode based on the distance to the target.

[0095] Specifically:

[0096] Near-infrared images (i.e., images captured by a near-infrared camera and formed after passing through a narrowband filter) are acquired. Based on an image edge detection algorithm (such as the Canny edge detection algorithm), the target distance in the near-infrared image is estimated. The target area is marked with a rectangular box. The ratio of the height of the rectangular box in the near-infrared image to the overall image height of the near-infrared image is used as the judgment ratio. This ratio is obtained from the ratio of the corresponding number of pixels. As the target distance decreases, the judgment ratio increases accordingly.

[0097] If the ratio is greater than the preset threshold β, the structured light detection mode is selected as the wide-angle measurement mode, corresponding to the liquid lens input voltage u1, and the smaller value f of the line laser PWM frequency is selected. L1 As an adjustment factor; if the determination ratio is less than or equal to the preset threshold β, the structured light detection mode is selected as the long focal length measurement mode, corresponding to the liquid lens input voltage u2, and the maximum value f of the line laser PWM frequency is selected.L2 As a regulating factor.

[0098] Step 304: Calculate the input voltage of the liquid lens according to the structured light detection mode, and control the liquid lens with the input voltage to obtain a multi-line structured light image.

[0099] Step 306: Based on the multi-line structured light image, extract the pixel points corresponding to the center of the structured light stripes, and combine them with the calibration parameters in the structured light detection mode to generate three-dimensional point cloud data.

[0100] Specifically:

[0101] Based on multi-line structured light images captured by a near-infrared camera, the Steger algorithm is used to extract the image coordinates of the pixels corresponding to the center of the structured light stripes, and combined with the calibration parameters in the structured light detection mode, three-dimensional point cloud data is generated.

[0102] It should be noted that the calibration parameters are pre-calibrated, including the calibration of the camera's intrinsic and extrinsic parameters and the calibration of the scanning light plane. Specifically, the camera's intrinsic and extrinsic parameters are calibrated using Zhang's calibration method in both telephoto and wide-angle modes, and the results are recorded. The scanning light plane is calibrated as follows: 1) Multiple images of a chessboard containing structured light patterns (at different locations) are taken using the camera, and the images are saved; 2) The Steger algorithm is used to obtain the pixel points corresponding to the centers of the line structured light stripes in the images. Combined with the camera's intrinsic and extrinsic parameters, the spatial coordinates corresponding to the light plane sampling points are obtained. Based on the spatial coordinates corresponding to the sampling points, the coefficients of the light plane equation are fitted; 3) The light plane equations corresponding to two sets of light stripes are obtained by calibration using the above method. These equations can be used to calculate the spatial orientation of the rotation axis of the line structured light scan. Based on the spatial orientation of the rotation axis, the instantaneous rotation angle (obtained directly through the rotating mirror device), and a certain light plane equation, all line structured light plane equations can be obtained.

[0103] Step 308: Perform 3D reconstruction based on the 3D point cloud data.

[0104] Specifically:

[0105] A statistical filter is used to filter the 3D point cloud data, and triangulation is performed based on the Delaunay triangulation method to obtain the 3D reconstruction result of the target object.

[0106] The above-mentioned structured light 3D reconstruction method based on refracto-diffraction hybrid imaging, such as Figure 4As shown, by adjusting the input voltage of the liquid lens, the structured light detection mode (including wide-angle measurement mode and telephoto measurement mode) can be switched, thereby enabling flexible switching between wide and narrow fields of view according to actual needs. This improves the stereo perception range of the unmanned platform vision system in environments with insufficient lighting (or even no light) and lack of texture, and overcomes the problems of single field of view and limited measurable range of traditional structured light cameras. It is expected to improve the three-dimensional detection capability of unmanned systems in complex environments such as underground spaces and caves.

[0107] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A structured light 3D reconstruction system based on refraction-diffraction hybrid imaging, characterized in that, include: Structured light projection module, refractive index-diffraction hybrid imaging module, target object and data processing module; Structured light projection module, refraction-diffraction hybrid imaging module, and target object spacing settings; The structured light projection module is connected to the data processing module to emit structured light and generate diffuse reflection on the surface of the target object. The refractive index-diffraction hybrid imaging module includes: a filter, a Fresnel zone plate, a liquid lens, a solid lens, and a near-infrared camera arranged at intervals; the filter receives diffusely reflected light and sequentially transmits it to the Fresnel zone plate, the liquid lens, the solid lens, and the near-infrared camera; the liquid lens is connected to a data processing module to switch the structured light detection mode by controlling the input voltage of the liquid lens; The data processing module is connected to the near-infrared camera to acquire multi-line structured light images and realize three-dimensional reconstruction. By controlling the input voltage of the liquid lens, the structured light detection mode can be switched, including: Calculate the optical power of the Fresnel zone plate as Φ b At that time, the first correspondence between the input voltage and optical power of the liquid lens is as follows: In the formula, d 1 represents the distance between the Fresnel zone plate and the liquid lens. d 2 represents the distance between the liquid lens and the solid lens. d 3 represents the distance between the solid-state lens and the near-infrared camera, Φ b Φ is the optical power of the Fresnel zone plate. L ( u ) is the mapping function of the optical power of the liquid lens with respect to the input voltage, Φ s The optical power of a solid-state lens; Based on the first correspondence, u The input voltage of the liquid lens in the wide-angle measurement mode is obtained by solving the problem. Switching between structured light detection modes is achieved by controlling the input voltage of the liquid lens, and the method also includes: Calculate the optical power of the Fresnel zone plate as follows: - Φ b The second correspondence between the input voltage and optical power of the liquid lens: According to the second correspondence, u The input voltage of the liquid lens in the telephoto measurement mode is obtained by solving the problem. The optical power of the Fresnel zone plate is ±Φ b Calculated by the following formula: In the formula, λ 0 is the center wavelength of the filter. ρ 1 represents the radius of the first half-wave zone of the Fresnel zone plate; The data processing module coordinates the control of the PWM frequency of the line laser and the oscillation frequency of the micro MEMS galvanometer to satisfy the following formula, so as to form a stable multi-line structured light image. In the formula, f L The PWM frequency of the line laser. f G The oscillation frequency of the micro-MEMS galvanometer. N The number of light stripes to be projected.

2. The structured light 3D reconstruction system based on refractive indexing and diffraction hybrid imaging according to claim 1, characterized in that, The structured light projection module includes: a line laser, a micro MEMS galvanometer, a laser controller, and a galvanometer driving circuit board; The line laser is connected to the data processing module via the laser controller, so as to emit structured light under the first command sent by the data processing module to the laser controller; The micro MEMS galvanometer is connected to the data processing module via the galvanometer driving circuit board, so as to reflect the structured light under the second command sent by the data processing module to the galvanometer driving circuit board.

3. The structured light 3D reconstruction system based on refractive indexing and diffraction hybrid imaging according to claim 2, characterized in that, The first instruction includes: the laser's on / off state, PWM frequency, duty cycle, and trigger parameters; The second instruction includes: galvanometer scanning angle, oscillation frequency, and open / closed state.

4. A structured light 3D reconstruction method based on refraction-diffraction hybrid imaging, characterized in that, The structured light 3D reconstruction system based on refraction-diffraction hybrid imaging as described in any one of claims 1 to 3 includes: Select the structured light detection mode based on the distance to the target; Based on the structured light detection mode, the input voltage of the liquid lens is calculated, and the liquid lens is controlled with the input voltage to obtain a multi-line structured light image; Based on the multi-line structured light image, the pixel points corresponding to the center of the structured light stripes are extracted, and combined with the calibration parameters in the structured light detection mode, three-dimensional point cloud data is generated. Based on the aforementioned 3D point cloud data, 3D reconstruction is performed.

5. The structured light three-dimensional reconstruction method based on refractive indexing and diffraction hybrid imaging according to claim 4, characterized in that, Select the structured light detection mode based on the distance to the target, including: Near-infrared images are acquired, and the distance to targets in the near-infrared images is estimated based on the image edge detection algorithm. Target regions are marked with rectangular boxes, and the ratio of the height of the rectangular box in the near-infrared image to the overall image height is used as the judgment ratio. If the ratio is greater than the preset threshold, select the wide-angle measurement mode for structured light detection. If the ratio is less than or equal to the preset threshold, select the long-focus measurement mode for structured light detection.

6. The structured light three-dimensional reconstruction method based on refractive indexing and diffraction hybrid imaging according to claim 5, characterized in that, Based on the aforementioned 3D point cloud data, 3D reconstruction is performed, including: The three-dimensional point cloud data is filtered, and the three-dimensional reconstruction result of the target object is obtained based on the Delaunay triangulation method.

Images