A high-dynamic adaptive weight phase vector fusion three-dimensional reconstruction method

Abstract

This invention discloses a high-dynamic adaptive weighted phase vector fusion 3D reconstruction method, belonging to the field of structured light 3D measurement technology. Addressing the multi-frequency heterodyne fringe misalignment and abrupt changes caused by wave crest truncation on highly reflective object surfaces, the method first acquires a set of phase-shifted fringe images under different projection intensities; extracts vector components and local photometric features from individual pixels; constructs a model based on adaptive quality assessment, fusing physical signal-to-noise ratio, Gaussian brightness prior, and a saturation soft penalty mechanism to calculate the fitting weight for each intensity level; uses these weights to perform weighted vector accumulation on the vector components of each projection intensity level to obtain a fused total vector that suppresses overexposure and random noise; finally, the wrapping phase is calculated. This invention acquires a small set of images with different projection intensities, effectively eliminating the phase abrupt changes that occur in previous high-dynamic image phase edge fusion methods, achieving high-precision, complete 3D reconstruction with high reflectivity in high dynamic range scenes.

Description

Technical Field

[0001] This invention belongs to the field of machine vision and optical 3D measurement technology, specifically relating to a variable projection intensity adaptive phase vector fusion 3D reconstruction method for complex surfaces with high reflectivity and low reflectivity. Background Technology

[0002] 3D reconstruction technology is an important research direction in the fields of computer vision and optical measurement. Due to its ultra-high measurement accuracy and non-contact characteristics, it has extremely wide applications in areas such as industrial robot guidance, surface defect detection, and high-precision welding interaction. Although surface structured light 3D reconstruction technology has achieved great success in key technologies and industrialization, some challenges remain to be solved when facing complex industrial environments.

[0003] In traditional structured light 3D measurement, a projector projects coded fringes containing phase information onto an object. The camera captures this pattern information and calculates the wrapped phase. Then, using multi-frequency heterodyne and other unfolding algorithms, the absolute phase is obtained. This is further aided by binocular epipolar correction and stereo matching to recover dense parallax, and finally, the object's 3D spatial coordinates are reconstructed using a binocular calibration model. This entire process is highly dependent on the quality of the fringe image captured by the camera. However, when the object is specularly reflective, or when the specular reflective areas do not conform to Lambert's law of reflection (such as reflective aluminum plates or stainless steel), local overexposure often occurs, leading to a loss of phase information. Furthermore, when the object is matte black, black rubber, or other dark-colored objects, local underexposure can also occur, resulting in incorrect phase information. All of these factors contribute to unsatisfactory 3D reconstruction results.

[0004] Currently, various methods exist for measuring complex reflective scenes, with High Dynamic Range (HDR) technology being the most effective and cost-efficient method without increasing hardware costs. HDR primarily achieves this by adjusting camera exposure time and projected light intensity. However, traditional HDR structured light technology still has significant shortcomings. First, to cover the full dynamic range, it often requires acquiring a large number of images at different exposure levels, which is cumbersome and inefficient. Second, traditional HDR fusion algorithms typically use fixed threshold mask stitching or simple modulation weighting for phase fusion. This method is highly susceptible to severe edge step errors at the boundaries between effective regions spanning different brightness levels due to minute phase discontinuities. Especially when overexposure distortion is incorrectly introduced, it can cause multi-frequency heterodyne unpacking algorithms to fail at transition edges. Miscalculation of the stripe order will ultimately manifest as a continuous periodic wave fault or a large area of ​​point cloud voids in the three-dimensional point cloud. Summary of the Invention

[0005] The purpose of this invention is to address the problems in existing high dynamic range structured light 3D measurement technology, such as phase jumps, point cloud edge errors, and holes caused by high reflectivity and low reflectivity leading to suboptimal signal waves during multi-frequency heterodyne unpacking. This invention provides an adaptive phase vector fusion 3D reconstruction method that resists high reflectivity. This method, through adaptive weighted vector fitting in the complex plane, can completely eliminate fusion discontinuities and achieve high-precision, complete reconstruction of complex workpieces. The specific implementation includes the following steps:

[0006] Step 1: Project K sets of phase-shifted fringe patterns with different exposure intensities onto the object under test using a projection device (where... ( ), capturing striped patterns through a camera.

[0007] Step 2: For each pixel in the image set, extract the four-step phase-shift gray value sequence under the same projection brightness level, and extract the vector components and modulation degree used to characterize the phase and quality features of the pixel.

[0008] Let the four-step phase-shift grayscale value of a pixel at the k-th level of projected brightness be... , , , ,

[0009] The vector component includes a sine component. With cosine component :

[0010]

[0011]

[0012] The modulation level of this pixel at this projection brightness level The calculation formula is:

[0013]

[0014] The photometric features include average gray value. and maximum grayscale value ,satisfy:

[0015]

[0016]

[0017] Step 3: Based on the photometric characteristics, construct an adaptive quality evaluation model based on multidimensional photometric priors, and calculate the comprehensive fitting weight ω corresponding to each projection brightness level. k Due to physical signal-to-noise ratio Gaussian brightness weight Saturation soft penalty weight The decision is made jointly, and the formula is as follows:

[0018]

[0019] Utilizing the adjustment system Local average gray value The physical signal-to-noise ratio evaluation model is constructed by the ratio of the square roots of the two. :

[0020]

[0021] This is a minimal constant used to prevent the denominator from being zero.

[0022] To suppress the nonlinear response of the camera sensor in extremely dark and extremely bright regions, the median value of the sensor's optimal linear response range is used. To achieve the desired result, construct a Gaussian distribution function. :

[0023]

[0024] in, A preset standard deviation parameter is used to control brightness.

[0025] Setting the nonlinear distortion grayscale threshold Set the maximum grayscale value of the camera. ,

[0026] when , ;when The sine wave is determined to undergo peak truncation distortion after a threshold, therefore a linear sloping penalty is applied to the weight of this gear position. :

[0027]

[0028] in It is a preset minimum nonzero constant.

[0029] Step 4: In the complex plane, the vector components of all projection brightness levels are weighted and accumulated using the comprehensive fitting weights to obtain the fused total vector that suppresses overexposure and random noise.

[0030] Weighted summation fitting is performed on the orthogonal vectors of all image sets under all K-level projection brightness.

[0031]

[0032]

[0033] Step 5: Use the arctangent function to calculate the final fused wrap phase of a single pixel. :

[0034]

[0035] and the calculation results Mapped to Within a continuous interval, after obtaining the absolute phase, subsequent processing such as stereo correction and stereo matching (common steps in this field) is performed to finally generate high-precision 3D point cloud data of the measured object. Attached Figure Description

[0036] Figure 1 Overall flowchart.

[0037] Figure 2 A flowchart of the method of the present invention.

[0038] Figure 3 Point cloud maps obtained for different projection intensities.

[0039] Figure 4 The fusion results are compared: the left side represents the proposed method, and the right side represents the traditional method.

[0040] Figure 5 Comparison of workpieces with bevel welds of different fusion strengths.

[0041] Figure 6 The method of this invention is compared with the traditional phase fusion method. The left side shows the method of this invention, and the right side shows the traditional method.

[0042] Figure 7 Different projected strengths of groove welds

[0043] Figure 8 The left image shows this method, and the right image shows the traditional method. Detailed Implementation

[0044] This invention provides a phase vector fusion 3D reconstruction method with highly dynamic adaptive weights, the overall process of which is as follows: Figure 1 , Figure 2 The specific implementation process of the solution is as follows:

[0045] Step 1: Acquire a multi-frequency phase-shift fringe image set. Control the projection device to project K sets of images onto the object under test. Phase-shifted stripe patterns with varying projection intensities. Traditional HDR 3D measurement typically uses varying camera exposure time to acquire high dynamic range images. However, this significantly prolongs the overall system acquisition time, easily introducing motion blur from environmental vibrations or subtle object movements. To overcome this drawback, this embodiment globally locks the camera exposure time and actively controls the projector's output brightness level to acquire image sets with different brightness levels.

[0046] Step 2: Extract orthogonal vector components and local photometric features. For each pixel in the image set, extract the four-step phase-shifted grayscale value sequence under the same projection brightness level. From the principles of physical optics, it is known that in actual industrial settings, the theoretical model for the four-step phase-shifted grayscale values ​​received by a pixel under the Kth projection brightness level satisfies:

[0047]

[0048] Where A represents the superposition of ambient background light intensity and fundamental reflected light, and B represents the intensity of the fringe modulation. The true phase to be solved. The phase shift amount (in the four phase shift steps, 0, ...) is... , , Considering the actual light-sensing process of industrial cameras, in order to eliminate the interference of the unknown quantity A at the algorithm level, this embodiment uses a computer to perform the following calculation to extract the orthogonal vector component (sine component N) used to characterize the phase of the pixel. k With cosine component D k ):

[0049]

[0050]

[0051] The modulation level of this pixel at this projection brightness level The calculation formula is: Simultaneously, the photometric features of the pixel at that setting are extracted, including the local average gray value. and local maximum gray value .

[0052] Step 3: Construct an adaptive quality evaluation model. Due to the complex surface reflectivity of workpieces in actual industrial settings, relying solely on modulation depth cannot accurately reflect the physical quality of the signal. This embodiment calculates the comprehensive fitting weights corresponding to each projection brightness level from three physical dimensions. The formula is The specific prior model is constructed as follows:

[0053] (1) Physical signal-to-noise ratio weighting This embodiment uses the ratio of modulation density to the square root of the average gray level to construct an evaluation model: ,in For a very small constant (e.g., 1×10⁻⁶) -5 This feature can adaptively enhance the dominance of high-contrast signals in dark areas during fusion.

[0054] (2) Gaussian brightness prior weights To suppress nonlinear response distortion of the camera sensor in extremely dark and extremely bright regions, this embodiment uses the median of the camera's optimal linear response range. (For an 8-bit camera, i.e., a camera with a grayscale range of 0-255, a value of 128 is preferred.) To determine the desired value, construct a Gaussian distribution function: ,in An empirical standard deviation parameter for controlling the luminance attenuation bandwidth.

[0055] (3) Saturation soft penalty weight This is the core fallback mechanism for anti-high reflectivity in this invention. A nonlinear distortion grayscale threshold is set. (In this embodiment, 230 is preferred), and the maximum grayscale extreme value of the camera. (Usually 255). When When this is the case, it indicates that the fringe peaks are complete. ;when At this point, the sinusoidal ripple begins to exhibit peak truncation distortion. If used directly, the calculated vector direction will be skewed, leading to unpacking misalignment. Therefore, this embodiment applies a linear slippage penalty to the weight of this gear:

[0056] .in This is a preset, minimal non-zero constant (e.g., 0.1). When localized overexposure occurs across the entire range, this method will not produce dead black holes like traditional methods, but will instead utilize the preserved minimal constant. Provide a safety net.

[0057] Step four: Perform weighted vector accumulation in the complex plane. In this embodiment, the comprehensive fitting weights obtained in step three are used in the two-dimensional complex plane. Weighted summation fitting of orthogonal vectors under all K-level projected brightness:

[0058]

[0059]

[0060] Step 5, calculate the wrapping phase. The final fused wrapping phase of each pixel is calculated using the arctangent function. and the calculation results Mapped to Within a continuous interval, after obtaining the absolute phase, subsequent processing such as stereo correction and stereo matching (common steps in this field) is performed to finally generate high-precision 3D point cloud data of the measured object.

[0061] To verify the effectiveness of the proposed method, a high dynamic range structured light 3D measurement system is also provided. This system consists of a DLP projector (VB-6 optical engine) and two high-resolution monochrome industrial cameras. The spatial resolutions of the projector and cameras are 1280×720 and 1440×1080, respectively, and both cameras are equipped with 12mm focal length lenses. The generated three-frequency fringe patterns have frequencies of 140, 134, and 129. Combined with Radxa's RK3588S embedded core board, all peripherals are controlled through the core board.

[0062] First, collect image sets with different projection intensities, and then process the image sets as follows: Figure 3 as well as Figure 6 Three-dimensional reconstruction was performed, and point cloud effects were obtained under different projection intensities, as shown below. Figure 4 and Figure 7 It is evident that normal reconstruction is not ideal for steel workpieces. The method of this invention, using 2 to 4 images with different projection intensities, can perform complete, high-precision, and robust 3D measurements of most reflective workpieces encountered in industrial settings, even in complex reflective scenarios.

Claims

1. A phase vector fusion 3D reconstruction method with highly dynamic adaptive weights, the specific steps of which are as follows: Step 1, project K sets of phase-shift fringe patterns with different projection intensities to the measured object by a projection device (wherein ), and control the camera to synchronously shoot at a fixed exposure time to obtain a multi-frequency phase-shift fringe image set of the measured object under different projection intensities; Step 2: For each pixel in the image set, extract the four-step phase-shift gray value sequence under the same projection brightness level, and calculate the vector components and modulation degree used to characterize the phase and quality features of the pixel. Step 3, constructing an adaptive quality evaluation model based on multi-dimensional photometric prior according to the photometric characteristics, and calculating the comprehensive fitting weight ω corresponding to each projection brightness level k ; Step 4: In the complex plane, the vector components of all projection brightness levels are weighted and accumulated using the comprehensive fitting weights to obtain the fused total vector that suppresses overexposure and random noise. Step 5: Calculate the wrapping phase of the pixel based on the fused total vector, and generate a 3D point cloud of the object under test by combining multi-frequency heterodyne unwrapping and stereo matching algorithms.

2. The highly dynamic adaptive weighted phase vector fusion 3D reconstruction method according to claim 1, characterized in that, In step 2, suppose the four-step phase-shift gray scale values of a certain pixel under the kth projection luminance are , , , , the vector components include a sine component and a cosine component : a modulation degree of the pixel at the projection brightness level The calculation formula is: The photometric features include an average gray value and a maximum gray value satisfying:

3. The high-dynamic adaptive-weight phase vector fusion three-dimensional reconstruction method according to claim 2, wherein the adaptive quality evaluation model in step 3 is fitted with a weight ω k determined by a physical signal-to-noise ratio , a Gaussian brightness weight , and a saturation soft penalty weight , and the formula is:

4. The phase vector fusion 3D reconstruction method with high dynamic adaptive weights as described in claim 3, its physical signal-to-noise ratio... The calculation formula is as follows: Utilizing the adjustment system Local average gray value The ratio of the square roots of these terms forms the physical signal-to-noise ratio evaluation model: This represents a minimal constant used to prevent the denominator from being zero.

5. The phase vector fusion 3D reconstruction method with high dynamic adaptive weights according to claim 3, wherein the Gaussian brightness prior weights The calculation method is as follows: To suppress the nonlinear response of the camera sensor in extremely dark and extremely bright regions, the median value of the sensor's optimal linear response range is used. To achieve the desired outcome, construct the Gaussian distribution function: in, A preset standard deviation parameter is used to control brightness.

6. The phase vector fusion 3D reconstruction method with highly dynamic adaptive weights according to claim 3, wherein the saturation soft-penalty weights The calculation method is as follows: Set nonlinear distortion grayscale threshold Set the maximum grayscale value of the camera. , when , ;when The sine wave is determined to undergo peak truncation distortion after a threshold, therefore a linear sloping penalty is applied to the weight of this gear position. in It is a preset minimum nonzero constant.

7. A highly dynamic adaptive weighted phase vector fusion 3D reconstruction method according to any one of claims 2 to 6, characterized in that, In steps 4 and 5, the specific process of performing weighted vector accumulation and solving the wrapped phase in the complex plane is as follows: Weighted summation fitting is performed on the orthogonal vectors of all image sets under all K-level projection brightness. The final fused wrap phase of a single pixel is calculated using the arctangent function. : and the calculation results Mapped to Within a continuous interval.

Images