Three-dimensional reconstruction method and system for strong ambient light filtering and depolarization error correction

Abstract

The present disclosure relates to a three-dimensional reconstruction method and system for filtering out strong ambient light and correcting depolarization errors. The method comprises: obtaining ambient polarized light and polarized structured light reflection components, calculating the Stokes vector distributions of the ambient polarized light and polarized structured light reflection components, obtaining a polarization image of the ambient light and a reflection image of the polarized structured light, determining a polarization threshold based on the ambient light polarization image, and extracting the polarization of the projection area based on the structured light reflection polarization image; establishing an ambient light separation model based on the principle of light reflection depolarization, generating an abnormal pixel mask based on the polarization threshold and the ambient light separation model; establishing an error tracing model based on the ambient light separation model, performing component decomposition and error correction on the polarized structured light phase data in the abnormal pixel mask area; and reconstructing a three-dimensional point cloud using the corrected phase data and fusing it with images captured by a polarization camera. The disclosed method can achieve precise three-dimensional measurement of polarized structured light under strong ambient light interference.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision, and in particular to a three-dimensional reconstruction method and system for filtering out strong ambient light and correcting depolarization errors. Background Art

[0002] Structured light 3D measurement, due to its accuracy and efficiency, has gained traction in a variety of applications, including industrial automation, cultural relic digitization, and autonomous vehicles. In practical applications, due to the limited dynamic range of image sensors, the signal-to-noise ratio (SNR) of captured images can be affected by ambient light interference. This effect is particularly pronounced in scenes with strong ambient light. For example, in outdoor environments, the ambient light brightness can be 2-5 orders of magnitude greater than the projected structured light. The interference energy is significantly greater than the projected light source energy, resulting in an excessively low SNR that renders decoding and reconstruction impossible using currently used structured light 3D reconstruction equipment. Therefore, solving the problem of structured light 3D reconstruction in strong ambient light is of great significance to computer vision.

[0003] To address the issue of strong ambient light interference, traditional research generally uses or combines the following methods: optimizing the projection light source, optimizing the encoding strategy, and image post-processing methods. For example, methods include combining narrow spectral bandwidth lasers, optical bandpass filters, and polarization filters; methods that use color information to mark the light plane and automatically adjust the projection pattern based on scene characteristics to improve the signal-to-noise ratio; deep learning-based ambient light mitigation methods that model interference as ambient light-induced phase error (ALPE) to extract high-precision phase information from single edge images affected by strong light; and methods such as line structured light and polarization encoding. Consequently, traditional methods suffer from low signal-to-noise ratio and depolarization errors in 3D reconstruction under strong ambient light. Summary of the Invention

[0004] In order to solve the problems of low signal-to-noise ratio and depolarization error in traditional 3D reconstruction under strong ambient light, the present disclosure proposes a 3D reconstruction method that filters out strong ambient light and corrects depolarization error to solve the above problems.

[0005] According to one aspect of the present disclosure, a three-dimensional reconstruction method for filtering out strong ambient light and correcting depolarization errors is provided, comprising:

[0006] S10, obtaining ambient polarized light and polarized structured light reflection components, calculating the Stokes vector distributions of the ambient polarized light and polarized structured light reflection components respectively, obtaining a polarization image of the ambient light and a reflection image of the polarized structured light, determining a polarization degree threshold based on the polarization image of the ambient light, and extracting the polarization degree of the projection area based on the structured light reflection polarization image;

[0007] S20, establishing an ambient light separation model based on the principle of light reflection depolarization, and generating an abnormal pixel mask according to the polarization degree threshold and the ambient light separation model;

[0008] S30, establishing an error tracing model based on the ambient light separation model, and performing component decomposition and error correction on the polarization structured light phase data of the abnormal pixel mask area according to the error tracing model;

[0009] S40, reconstructing a three-dimensional point cloud by fusing the corrected phase data with the image captured by the polarization camera.

[0010] Preferably, according to the acquired polarization image of ambient light and the polarization image of structured light reflection, the polarization image of ambient light and the polarization image of structured light reflection are decomposed into polarized light intensities perpendicular to the incident plane and parallel to the incident plane, expressed as:

[0011] ,

[0012] ,

[0013] Where, is the total light intensity received by the camera when the reflected light passes through a single-layer linear polarizer, is the unpolarized ambient light intensity, is the horizontal polarization component of the ambient light, is the vertical polarization component of the ambient light, is the vertical polarization component of the polarized structured light, is the horizontal polarization component of the polarized structured light, is the polarization angle of the external linear polarizer, is the polarization angle of the built-in linear polarizer of the polarization camera.

[0014] Preferably, an ambient light separation model is established based on the reflective depolarization principle of light, including:

[0015] Performing polarization state decomposition on the total light intensity using a Stokes vector;

[0016] Establishing an incoherent superposition model of the ambient polarized light and the reflected components of the polarized structured light through the Stokes vector;

[0017] Based on the relationship between the polarized light intensities perpendicular to the incident plane and parallel to the incident plane, the polarization components are separated by Mueller matrix calculation.

[0018] Preferably, the separation of polarization components is achieved by Mueller matrix operation, including:

[0019] Establish the Stokes vector transformation relationship between incident light and reflected light;

[0020] Express the Mueller matrix as a function of the surface reflectance properties;

[0021] The amplitude reflection coefficient of the polarized light intensity perpendicular to the incident plane and the polarized light intensity parallel to the incident plane is determined experimentally;

[0022] The polarization state of the incident light is solved based on the measured Stokes vector of the reflected light.

[0023] Preferably, the measured Stokes vector of the reflected light includes: an image captured by a polarization camera after the ambient light is reflected, diffracted, and scattered by the object surface, expressed as:

[0024] ,

[0025] ,

[0026] ,

[0027] ,

[0028] ,

[0029] Where, represents the unpolarized light component from the ambient light, represents the Stokes vector of the actual image acquired by the polarization camera, is the Stokes vector of ambient light reflection, is the Stokes vector of the vertical polarization component of the ambient light, is the Stokes vector of the horizontal polarization component of the ambient light, is the Stokes vector of the vertically polarized light after the structured light is reflected, is the Stokes vector of the horizontally polarized light after the structured light is reflected, DOLP is the degree of polarization, and are the maximum and minimum light intensities passing through the polarizer at orthogonal azimuths, is the mask threshold of the polarized structured light emitted by the projector, is the Stokes vector from the projected light component in the pixel affected by the misseparation, is the projection mask matrix, is the Stokes vector of the actual ambient light component in the pixel.

[0030] Preferably, error correction is performed on the phase data of the polarization structured light, which is expressed as:

[0031] .

[0032] Preferably, reconstructing a three-dimensional point cloud by fusing the corrected phase data with an image captured by a polarization camera comprises:

[0033] Obtain images captured by the polarization camera at 0°, 45°, and 90°, and calculate the Stokes vector of ambient light reflection, the Stokes vector of horizontally polarized light, and the Stokes vector of vertically polarized light;

[0034] The corrected phase data is fused with the normal part image of the mask area;

[0035] Phase decoding is performed using a twelve-step phase shift decoding method, and the three-dimensional point cloud is reconstructed using a stereo matching algorithm.

[0036] According to one aspect of the present disclosure, a three-dimensional reconstruction system for filtering out strong ambient light and correcting depolarization errors is provided, comprising:

[0037] A module for acquiring a polarized image of ambient light and a reflected image of polarized structured light acquires the ambient polarized light and the reflected components of the polarized structured light, calculates the Stokes vector distributions of the ambient polarized light and the reflected components of the polarized structured light, obtains the polarized image of the ambient light and the reflected image of the polarized structured light, determines a polarization threshold based on the polarized image of the ambient light, and extracts the polarization degree of the projection area based on the polarized image of the structured light reflection;

[0038] The abnormal pixel mask generation module establishes an ambient light separation model based on the principle of light reflection depolarization, and generates an abnormal pixel mask according to the polarization threshold and the ambient light separation model;

[0039] The error correction module establishes an error tracing model based on the ambient light separation model. Based on the error tracing model, it performs component decomposition and error correction on the polarization structured light phase data of the abnormal pixel mask area.

[0040] The 3D point cloud reconstruction module reconstructs the 3D point cloud by fusing the corrected phase data with the image captured by the polarization camera.

[0041] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to: execute the above-mentioned three-dimensional reconstruction method for filtering out strong ambient light and correcting depolarization errors.

[0042] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the three-dimensional reconstruction method for filtering strong ambient light and correcting depolarization errors is implemented.

[0043] Compared with the prior art, the beneficial effects of the present disclosure are:

[0044] 1) This paper constructs a short-focal-length projection measurement system with a double-layer linear polarizer. It obtains polarization images of ambient light and structured light reflection polarization images, and establishes an ambient light separation model based on the principle of light reflection depolarization. It identifies abnormal pixels based on the DOLP threshold projection mask, and uses the Stokes vector to establish an error tracing model to correct abnormal pixels affected by the depolarization effect and causing phase errors.

[0045] 2) The present disclosure obtains a reconstructed image by fusing the corrected pixels and ultimately obtains a complete three-dimensional point cloud.

[0046] 3) The present disclosure can achieve precise three-dimensional measurement of polarized structured light under strong ambient light interference by separating ambient light and solving the phase error problem caused by depolarization effect.

[0047] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0048] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0050] Figure 1 A flow chart of a three-dimensional reconstruction method for filtering out strong ambient light and correcting depolarization errors is shown;

[0051] Figure 2 A structural block diagram of a three-dimensional reconstruction system for filtering out strong ambient light and correcting depolarization errors in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0052] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0053] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0054] The term "and / or" herein simply describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent the existence of three situations: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0055] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0056] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] Example 1

[0058] Based on the above ideas, the present invention proposes a three-dimensional reconstruction method for filtering out strong ambient light and correcting depolarization errors. Figure 1 A flow chart of a 3D reconstruction method for filtering out strong ambient light and correcting for depolarization errors is shown. The method comprises:

[0059] S10, obtaining ambient polarized light and polarized structured light reflection components, calculating the Stokes vector distributions of the ambient polarized light and polarized structured light reflection components respectively, obtaining a polarization image of the ambient light and a reflection image of the polarized structured light, determining a polarization degree threshold based on the polarization image of the ambient light, and extracting the polarization degree of the projection area based on the structured light reflection polarization image;

[0060] S20, establishing an ambient light separation model based on the principle of light reflection depolarization, and generating an abnormal pixel mask according to the polarization degree threshold and the ambient light separation model;

[0061] S30, establishing an error tracing model based on the ambient light separation model, and performing component decomposition and error correction on the polarization structured light phase data of the abnormal pixel mask area according to the error tracing model;

[0062] S40, reconstructing a three-dimensional point cloud by fusing the corrected phase data with the image captured by the polarization camera.

[0063] The present disclosure provides a 3D reconstruction method for filtering out strong ambient light and correcting depolarization errors, which specifically includes the following steps:

[0064] S10. Obtain ambient polarized light and polarized structured light reflection components, calculate the Stokes vector distributions of the ambient polarized light and polarized structured light reflection components respectively, obtain a polarization image of the ambient light and a reflection image of the polarized structured light, determine a polarization degree threshold based on the polarization image of the ambient light, and extract the polarization degree of the projection area based on the structured light reflection polarization image.

[0065] In this embodiment, a double-layer linear polarizer short-focus projection measurement system is constructed. The double-layer linear polarizer short-focus projection measurement system is an optical measurement system consisting of a linear polarizer, a polarization camera, and an LCD projector. The projector can project polarization state images at four angles, namely, polarization state images of 0°, 45°, 90°, and 135°.

[0066] Polarization state images are primarily acquired using a camera with a focal plane segmentation area array sensor (such as the IMX250MYR). The polarization sensor in a polarization camera works by adding a polarizer above the photodiode of a conventional CMOS image sensor. Four polarizers at different angles (90°, 45°, 135°, and 0°) are placed on individual pixels, with each group of four pixels serving as a calculation unit. The correlation between polarizers of different orientations enables the calculation of the degree and direction of polarization. By performing relevant operations directly on the image acquired by the polarization camera, steps such as segmentation, reassembly, and recalculation are omitted, reducing system complexity and improving the overall process speed.

[0067] Furthermore, according to the acquired polarization image of ambient light and the polarization image of structured light reflection, the polarization image of ambient light and the polarization image of structured light reflection are decomposed into polarized light intensities perpendicular to the incident plane and parallel to the incident plane, which are expressed as:

[0068] ,

[0069] ,

[0070] Where, is the total light intensity received by the camera when the reflected light passes through a single-layer linear polarizer, is the unpolarized ambient light intensity, is the horizontal polarization component of the ambient light, is the vertical polarization component of the ambient light, is the vertical polarization component of the polarized structured light, is the horizontal polarization component of the polarized structured light, is the polarization angle of the external linear polarizer, is the polarization angle of the built-in linear polarizer of the polarization camera.

[0071] S20. Establish an ambient light separation model based on the principle of light reflection depolarization, and generate an abnormal pixel mask according to the polarization degree threshold and the ambient light separation model.

[0072] In this embodiment, the polarization camera, linear polarizer, and projector optical center are placed at almost the same level, with the linear polarizer close to the polarization camera. The two are placed nearly parallel and their relative positions are fixed. Turn on the projector light source, project an image with a polarization angle of 0°, adjust the angle of the linear polarizer in front of the polarization camera, use the polarization camera to receive the reflected image, observe the light intensity distribution on the 0° image, and rotate the external linear polarizer so that the camera captures the entire object while not overexposing the entire captured image. Record the polarization angle of the linear polarizer. Use the SLR camera calibration method to calibrate the measurement system so that the system's measurement error is within 0.2 pixels. Turn off the projector light source and receive an image illuminated only by ambient light.

[0073] Furthermore, an ambient light separation model is established based on the principle of reflected depolarization of light, including: using the Stokes vector to decompose the polarization state of the total light intensity; establishing an incoherent superposition model of the ambient polarized light and the reflected component of the polarized structured light through the Stokes vector; based on the relationship between the polarized light intensity perpendicular to the incident plane and parallel to the incident plane, the polarization component separation is achieved through Mueller matrix operation.

[0074] The superposition of incoherent light is described using the Stokes vector, which is expressed as:

[0075] ,

[0076] ,

[0077] Where, is the amplitude of the s component in the light vector, is the amplitude of the p component, is the phase difference, Indicates the total irradiance (light intensity) of the light beam, Represents the intensity of the linearly polarized light component in the 0° direction, Represents the intensity of the linearly polarized light component in the 45° direction, represents the intensity of the circularly polarized light component in the beam, is the total polarization intensity.

[0078] Furthermore, the polarization components are separated through Mueller matrix operations, including: establishing a Stokes vector transformation relationship between the incident light and the reflected light; expressing the Mueller matrix as a function of the surface reflection characteristics; experimentally measuring the amplitude reflection coefficient of the polarized light intensity perpendicular to the incident plane and the polarized light intensity parallel to the incident plane; and solving the polarization state of the incident light based on the measured Stokes vector of the reflected light.

[0079] In this embodiment, the Mueller matrix is ​​used to describe the reflection characteristics of the object, which is expressed as:

[0080] ,

[0081] ,

[0082] ,

[0083] ,

[0084] ,

[0085] ,

[0086] ,

[0087] Where, is the Stokes vector of the incident light, is the Stokes vector of the outgoing light, , is the incident angle of the light, is the angle of refraction of the light, is the introduced reflection rotation matrix.

[0088] Furthermore, under ambient light, a calibrated system is used to sequentially capture polarized stripe images modulated by the object, and the captured images with four polarization angles are segmented. The ambient light components in the image are separated and the phase error is corrected according to the aforementioned method, and then fused with the pixels of the normal phase part to obtain an image for reconstruction after separating the ambient light.

[0089] The measured Stokes vector of reflected light includes the image captured by the polarization camera after the ambient light is reflected, diffracted, and scattered by the object surface, and is expressed as:

[0090] ,

[0091] ,

[0092] ,

[0093] ,

[0094] ,

[0095] Where, represents the unpolarized light component from the ambient light, represents the Stokes vector of the actual image acquired by the polarization camera, is the Stokes vector of ambient light reflection, is the Stokes vector of the vertical polarization component of the ambient light, is the Stokes vector of the horizontal polarization component of the ambient light, is the Stokes vector of the vertically polarized light after the structured light is reflected, is the Stokes vector of the horizontally polarized light after the structured light is reflected, DOLP is the degree of polarization, and are the maximum and minimum light intensities passing through the polarizer at orthogonal azimuths, is the mask threshold of the polarized structured light emitted by the projector, is the Stokes vector from the projected light component in the pixel affected by the misseparation, is the projection mask matrix, is the Stokes vector of the actual ambient light component in the pixel.

[0096] S30. Establish an error tracing model based on the ambient light separation model, and perform component decomposition and error correction on the polarization structured light phase data of the abnormal pixel mask area according to the error tracing model.

[0097] In this embodiment, the DOLP threshold projection mask is used to identify abnormal pixels, and the Stokes vector is used to establish an error tracing model to correct abnormal pixels affected by the depolarization effect and causing phase errors. Error correction is performed on the phase data of the polarized structured light, which is expressed as:

[0098] ,

[0099] The polarization camera is used to simultaneously capture images at 0°, 45°, and 90°, and the Stokes vector of the ambient light reflection, the Stokes vector of the horizontally polarized light, and the Stokes vector of the vertically polarized light are calculated. The corrected phase error partial image is fused with the normal partial image of the mask area to obtain the corrected 3D reconstructed image.

[0100] S40, reconstructing a three-dimensional point cloud by fusing the corrected phase data with the image captured by the polarization camera.

[0101] In this embodiment, the corrected phase data is fused with the image captured by the polarization camera to reconstruct a three-dimensional point cloud, including: obtaining images captured by the polarization camera at 0°, 45°, and 90°, calculating the Stokes vector of ambient light reflection, the Stokes vector of horizontally polarized light, and the Stokes vector of vertically polarized light; fusing the corrected phase data with the image of the normal part of the mask area; using a twelve-step phase shift decoding method to perform phase decoding, and reconstructing the three-dimensional point cloud through a stereo matching algorithm.

[0102] The disclosed embodiment utilizes an adjustable external linear polarizer to proportionally reduce the image light intensity to an analyzable range, and after determining the parameters, calibrates the system and collects polarized structured light images. Furthermore, the DOLP threshold projection mask system is used to accurately identify pixel points with phase errors caused by the depolarization effect. By taking advantage of the polarization camera's ability to simultaneously receive four polarization angle images, the established light component tracing equation is used to quickly correct the pixel points that produce errors, and then reintegrate them with the normal parts to obtain an image for three-dimensional point cloud information recovery, and obtain a complete point cloud. The filtering of strong ambient light and the precise correction of phase errors are achieved, and the measurement speed, measurement accuracy, and robustness all meet the strict requirements of accurate three-dimensional reconstruction.

[0103] Example 2

[0104] As another aspect of the embodiment of the present disclosure, a three-dimensional reconstruction system 100 for filtering out strong ambient light and correcting depolarization errors is also provided. Figure 2 Shown, including:

[0105] A polarization image of ambient light and a reflection image of polarized structured light are acquired by a module 1, which acquires the ambient polarized light and the reflection components of the polarized structured light, calculates the Stokes vector distributions of the ambient polarized light and the reflection components of the polarized structured light, respectively, obtains the polarization image of the ambient light and the reflection image of the polarized structured light, determines a polarization degree threshold based on the polarization image of the ambient light, and extracts the polarization degree of the projection area based on the polarization image of the structured light reflection;

[0106] Abnormal pixel mask generation module 2, which establishes an ambient light separation model based on the principle of light reflection depolarization, and generates an abnormal pixel mask according to the polarization threshold and the ambient light separation model;

[0107] Error correction module 3 establishes an error tracing model based on the ambient light separation model, and performs component decomposition and error correction on the polarization structured light phase data of the abnormal pixel mask area according to the error tracing model;

[0108] The 3D point cloud reconstruction module 4 reconstructs the 3D point cloud by fusing the corrected phase data with the image captured by the polarization camera.

[0109] In the absence of any contradiction, the above modules in the system of the embodiment of the present disclosure can implement any implementation of the above method.

[0110] Based on the description of the above embodiments, it can be seen that the embodiments of the present disclosure can achieve the following technical effects:

[0111] 1) This paper constructs a short-focal-length projection measurement system with a double-layer linear polarizer. It obtains polarization images of ambient light and structured light reflection polarization images, and establishes an ambient light separation model based on the principle of light reflection depolarization. It identifies abnormal pixels based on the DOLP threshold projection mask, and uses the Stokes vector to establish an error tracing model to correct abnormal pixels affected by the depolarization effect and causing phase errors.

[0112] 2) The present disclosure obtains a reconstructed image by fusing the corrected pixels and ultimately obtains a complete three-dimensional point cloud.

[0113] 3) The present disclosure can achieve precise three-dimensional measurement of polarized structured light under strong ambient light interference by separating ambient light and solving the phase error problem caused by depolarization effect.

[0114] The present disclosure also provides an electronic device comprising: a processor; and a memory for storing instructions executable by the processor. The processor is configured to implement the aforementioned 3D reconstruction method for filtering strong ambient light and correcting depolarization errors. The electronic device can be provided as a terminal, server, or other device.

[0115] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon. When executed by a processor, the computer program instructions implement the aforementioned 3D reconstruction method for filtering strong ambient light and correcting depolarization errors. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0116] Those skilled in the art will understand that in the above-mentioned three-dimensional reconstruction method and system for strong ambient light filtering and depolarization error correction in the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0117] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0118] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technical improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A three-dimensional reconstruction method for filtering out strong ambient light and correcting depolarization errors, characterized in that: The steps include: S10, obtaining ambient polarized light and polarized structured light reflection components, calculating the Stokes vector distributions of the ambient polarized light and polarized structured light reflection components respectively, obtaining a polarization image of the ambient light and a reflection image of the polarized structured light, determining a polarization degree threshold based on the polarization image of the ambient light, and extracting the polarization degree of the projection area based on the structured light reflection polarization image; S20, establishing an ambient light separation model based on the principle of light reflection depolarization, and generating an abnormal pixel mask according to the polarization degree threshold and the ambient light separation model; An ambient light separation model is established based on the principle of light reflection depolarization, including: Polarization decomposition of the total light intensity using Stokes vectors; Establishing an incoherent superposition model of the ambient polarized light and the reflected components of the polarized structured light through the Stokes vector; Based on the relationship between the polarized light intensities perpendicular to the incident plane and parallel to the incident plane, the polarization components are separated by Mueller matrix operation. Polarization component separation is achieved through Mueller matrix operations, including: Establish the Stokes vector transformation relationship between incident light and reflected light; Express the Mueller matrix as a function of the surface reflectance properties; The amplitude reflection coefficient of the polarized light intensity perpendicular to the incident plane and the polarized light intensity parallel to the incident plane is determined experimentally; The polarization state of the incident light is solved based on the measured Stokes vector of the reflected light; The measured Stokes vector of reflected light includes the image captured by the polarization camera after the ambient light is reflected, diffracted, and scattered by the object surface, and is expressed as: ， ， ， ， ， Where, represents the unpolarized light component from the ambient light, represents the Stokes vector of the actual image acquired by the polarization camera, is the Stokes vector of ambient light reflection, is the Stokes vector of the vertical polarization component of the ambient light, is the Stokes vector of the horizontal polarization component of the ambient light, is the Stokes vector of the vertically polarized light after the structured light is reflected, is the Stokes vector of the horizontally polarized light after the structured light is reflected, DOLP is the degree of polarization, and are the maximum and minimum light intensities passing through the polarizer at orthogonal azimuths, is the mask threshold of the polarized structured light emitted by the projector, is the Stokes vector from the projected light component in the pixel affected by the misseparation, is the projection mask matrix, is the Stokes vector of the actual ambient light component in the pixel; S30, establishing an error tracing model based on the ambient light separation model, and performing component decomposition and error correction on the polarization structured light phase data of the abnormal pixel mask area according to the error tracing model; S40, reconstructing a three-dimensional point cloud by fusing the corrected phase data with the image captured by the polarization camera.

2. The method according to claim 1, characterized in that According to the acquired polarization image of ambient light and the polarization image of structured light reflection, the polarization image of ambient light and the polarization image of structured light reflection are decomposed into polarized light intensities perpendicular to the incident plane and parallel to the incident plane, which are expressed as: ， ， Where, is the total light intensity received by the camera when the reflected light passes through a single-layer linear polarizer, is the unpolarized ambient light intensity, is the horizontal polarization component of the ambient light, is the vertical polarization component of the ambient light, is the vertical polarization component of the polarized structured light, is the horizontal polarization component of the polarized structured light, is the polarization angle of the external linear polarizer, is the polarization angle of the built-in linear polarizer of the polarization camera.

3. The method according to claim 1, characterized in that The error correction of the phase data of the polarization structured light is expressed as: 。 4. The method according to any one of claims 1 or 3, characterized in that The corrected phase data is fused with the image captured by the polarization camera to reconstruct a 3D point cloud, including: Obtain images captured by the polarization camera at 0°, 45°, and 90°, and calculate the Stokes vector of ambient light reflection, the Stokes vector of horizontally polarized light, and the Stokes vector of vertically polarized light; The corrected phase data is fused with the normal part image of the mask area; Phase decoding is performed using a twelve-step phase shift decoding method, and the three-dimensional point cloud is reconstructed using a stereo matching algorithm.

5. A 3D reconstruction system with strong ambient light filtering and depolarization error correction, characterized by: include: A module for acquiring a polarized image of ambient light and a reflected image of polarized structured light acquires the ambient polarized light and the reflected components of the polarized structured light, calculates the Stokes vector distributions of the ambient polarized light and the reflected components of the polarized structured light, obtains the polarized image of the ambient light and the reflected image of the polarized structured light, determines a polarization threshold based on the polarized image of the ambient light, and extracts the polarization degree of the projection area based on the polarized image of the structured light reflection; The abnormal pixel mask generation module establishes an ambient light separation model based on the principle of light reflection depolarization, and generates an abnormal pixel mask according to the polarization threshold and the ambient light separation model; An ambient light separation model is established based on the principle of light reflection depolarization, including: Polarization decomposition of the total light intensity using Stokes vectors; Establishing an incoherent superposition model of the ambient polarized light and the reflected components of the polarized structured light through the Stokes vector; Based on the relationship between the polarized light intensities perpendicular to the incident plane and parallel to the incident plane, the polarization components are separated by Mueller matrix operation. Polarization component separation is achieved through Mueller matrix operations, including: Establish the Stokes vector transformation relationship between incident light and reflected light; Express the Mueller matrix as a function of the surface reflectance properties; The amplitude reflection coefficient of the polarized light intensity perpendicular to the incident plane and the polarized light intensity parallel to the incident plane is determined experimentally; The polarization state of the incident light is solved based on the measured Stokes vector of the reflected light; The measured Stokes vector of reflected light includes the image captured by the polarization camera after the ambient light is reflected, diffracted, and scattered by the object surface, and is expressed as: ， ， ， ， ， Where, represents the unpolarized light component from the ambient light, represents the Stokes vector of the actual image acquired by the polarization camera, is the Stokes vector of ambient light reflection, is the Stokes vector of the vertical polarization component of the ambient light, is the Stokes vector of the horizontal polarization component of the ambient light, is the Stokes vector of the vertically polarized light after the structured light is reflected, is the Stokes vector of the horizontally polarized light after the structured light is reflected, DOLP is the degree of polarization, and are the maximum and minimum light intensities passing through the polarizer at orthogonal azimuths, is the mask threshold of the polarized structured light emitted by the projector, is the Stokes vector from the projected light component in the pixel affected by the misseparation, is the projection mask matrix, is the Stokes vector of the actual ambient light component in the pixel; The error correction module establishes an error tracing model based on the ambient light separation model. Based on the error tracing model, it performs component decomposition and error correction on the polarization structured light phase data of the abnormal pixel mask area. The 3D point cloud reconstruction module reconstructs the 3D point cloud by fusing the corrected phase data with the image captured by the polarization camera.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the three-dimensional reconstruction method for filtering out strong ambient light and correcting depolarization errors according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the three-dimensional reconstruction method for filtering out strong ambient light and correcting depolarization errors as claimed in any one of claims 1 to 4 is implemented.

Images