High dynamic range three-dimensional reconstruction method based on adaptive reverse multiple exposure technology

Abstract

The invention provides a self-adaptive reverse multiple exposure phase fusion technology, and relates to the field of high-dynamic-range three-dimensional reconstruction. The method includes the following steps that firstly, a saturated pixel mask matrix is obtained by judging pixels of a high-reflection area, then uniform grey-scale maps are selected step by step through a gradient adjustment method to obtain the optimal local projection intensity, and local coding and decoding are conducted on the high-reflection area through the optimal projection intensity pattern. And replacing an invalid phase value in the high-SNR image with a local phase value under the optimal projection intensity through an edge matching and phase fusion technology, and obtaining high-quality complete three-dimensional imaging information through coordinate mapping after a complete phase diagram is obtained. According to the method, the optimal projection intensity is adaptively determined through a small number of patterns, meanwhile, the high SNR of imaging patterns is guaranteed, and high-precision complete three-dimensional imaging is carried out on an HDR object. Experimental results show that in a traditional method, a large number of point clouds are lost in a saturated pixel area, the imaging integrity and the measurement precision cannot achieve the normal imaging effect, and the algorithm of the invention shows high integrity and high point cloud density in a dark area and a high-reflection area.

Description

Technical Field

[0001] The present invention relates to a high dynamic range three-dimensional reconstruction method based on adaptive reverse multiple exposure technology Background Art

[0002] Three-dimensional reconstruction technology is a key research area in computer vision and optical metrology, with widespread applications in industrial inspection, medical imaging, cultural heritage preservation, and robotic navigation. Structured light-based 3D reconstruction methods, due to their high accuracy, speed, and non-contact nature, are widely used for high-precision surface topography measurement. However, in high dynamic range (HDR) scenarios, this technology still faces numerous challenges, primarily overexposure of highly reflective surfaces, underexposure of low-reflectivity areas, and reconstruction errors caused by phase information loss.

[0003] In traditional structured light 3D measurement systems, a projector projects a stripe light pattern onto the target object, and a camera captures the deformation information of the pattern and reconstructs the object's surface through phase calculation and triangulation principles. However, when the surface of the object being measured contains highly reflective areas (such as metal, glass, shiny plastic, etc.) and low-reflective areas (such as black rubber, dark fabric, etc.), the reflective characteristics of the projected light can lead to a decrease in measurement accuracy:

[0004] Highly reflective areas (overexposure): When the reflectivity of an object's surface is high, the light intensity received by the camera may exceed its dynamic range (for example, exceeding 255 grayscale values in an 8-bit depth camera), resulting in overexposure of local areas. This in turn causes loss of fringe information and makes it impossible to correctly decode phase information.

[0005] Low-reflective areas (underexposure problem): When the reflectivity of the object surface is low, the light intensity received by the camera is weak, and the signal-to-noise ratio (SNR) is reduced, resulting in a decrease in the accuracy of phase information calculation or even complete loss.

[0006] Both of these problems will directly affect the integrity and accuracy of 3D reconstruction, making it difficult for the measurement system to obtain high-quality point cloud data when the object surface has mixed areas of bright and dark colors.

[0007] To address the measurement issues caused by high-reflection and low-reflection areas, existing methods mainly include the following:

[0008] 1. Multiple Exposure Method

[0009] The Multiple Exposure method is the most common HDR measurement technique. Its core idea is to project multiple fringe images with different exposure intensities and synthesize the optimal phase information in subsequent calculations. The specific process is as follows:

[0010] 1-1) Projecting multiple sets of stripe patterns of different brightness (such as strong exposure, medium exposure, and weak exposure) in sequence;

[0011] 1-2) Select the most suitable fringe image for phase decoding through image processing algorithm;

[0012] 1-3) Combine multiple images with different exposures to generate HDR phase information.

[0013] However, the multiple exposure method has the following disadvantages:

[0014] A large number of projected patterns leads to low measurement efficiency: Traditional multiple exposure methods require projecting 5 to 10 groups of stripe patterns with different brightness. Each group of patterns needs to be collected and processed separately, which significantly increases the measurement time and is not suitable for real-time 3D measurement scenarios.

[0015] Sensitive to lighting changes and affected by ambient light: The multiple exposure method requires continuous capture of multiple images under the same measurement environment. If the ambient light changes, it will lead to inconsistencies between the different exposure images, affecting the quality of the final 3D reconstruction.

[0016] 2Adaptive projection intensity adjustment

[0017] To reduce the number of projection patterns in multiple exposures, some studies have proposed an adaptive projection intensity adjustment method. The core idea of this method is:

[0018] First, detect the reflectivity of the object surface (by analyzing the grayscale value of the projected all-white image);

[0019] Dynamically adjust the projection light intensity based on the reflectivity of different areas, that is, reduce the projection intensity in highly reflective areas and increase it in low-reflective areas, so as to optimize the measurement effect as much as possible in a single exposure.

[0020] Although this method can reduce the number of required projection patterns, it still has the following shortcomings:

[0021] Region boundaries are difficult to accurately define: Adaptive projection is usually calculated based on global image analysis, which makes it difficult to accurately locate the boundaries of overexposed and underexposed areas, resulting in measurement errors in some areas.

[0022] High computational complexity: Real-time adaptive adjustment requires complex mathematical models and optimization algorithms, which increases the computational burden and is not conducive to the practical application of high-speed 3D measurement systems.

[0023] 2.3 Neutral Density Filter

[0024] This method reduces the light intensity in the bright areas by installing filters in the highly reflective areas, thus avoiding overexposure of the camera.

[0025] However, this method has the following problems:

[0026] Additional hardware costs: Variable optical components need to be added to the measurement system, increasing system cost and complexity.

[0027] Unable to adapt to dynamic scenes: This method is effective for fixed scenes, but when the object or lighting environment changes, the filter adjustment is difficult to complete in real time, so it is not suitable for real-time measurement. Summary of the Invention

[0028] To address the problems of low efficiency of existing multiple exposure methods, complex calculations of adaptive methods, and high hardware costs of filter methods, this paper proposes an efficient, accurate, and suitable HDR three-dimensional reconstruction method for complex scenes. This method can effectively solve the measurement errors of high-reflection and low-reflection areas and is suitable for multiple fields such as industrial inspection and medical imaging.

[0029] The technical solution of the present invention to solve the above problem is: a camera calibration method for low-precision planar targets, comprising the following steps:

[0030] Step 1: Identify highly reflective areas;

[0031] Step 2: Adaptive projection intensity adjustment;

[0032] Step 3: Phase demodulation;

[0033] Step 4: Phase fusion;

[0034] Step 5: 3D point cloud reconstruction;

[0035] In the high dynamic range 3D reconstruction method using the adaptive reverse multiple exposure technology, the specific process of step 1 is as follows:

[0036] 1-1) Use a full white image with a projection grayscale value of 255 for projection.

[0037] 1-2) After the camera captures the reflected image, a preset threshold of 248 is used to generate a binary mask image m(x,y). The calculation formula is as follows:

[0038]

[0039] Among them, I(x,y) represents the pixel grayscale value collected by the camera. A pixel with a value of 0 represents a high-reflection area, and a pixel with a value of 1 represents a non-high-reflection area.

[0040] In the high dynamic range 3D reconstruction method using the adaptive reverse multiple exposure technology, the specific process of step 2 is as follows:

[0041] 2-1) In a highly reflective area, if there is a pixel that satisfies I(x,y)≥248, the recursive formula is executed:

[0042]

[0043] That is, the projection grayscale is reduced by 20% each time until all pixels in the high-reflection area satisfy I(x,y)<248;

[0044] 2-2) If there are no saturated pixels in the initial projection intensity, the following recursive formula is used to gradually increase the projection intensity to optimize the signal-to-noise ratio in the low-reflection area:

[0045]

[0046] The above recursive process is performed for a maximum of 5 rounds (i.e. n≤5). Within this range, a suitable Make all pixels in the high-reflective area meet the following conditions: Stop adjustment when the following conditions are met:

[0047] I c (x,y)≤248

[0048] In the high dynamic range 3D reconstruction method using the adaptive reverse multiple exposure technology, the specific process of step 3 is as follows:

[0049] 3-1) Normal projection intensity and optimal low projection intensity The absolute phase calculation of the two groups of projection intensity fringes is performed separately. The phase processing of the fringe patterns with two different projection intensities is performed to obtain two phase maps, which are recorded as high projection intensity phase maps φ. H (x,y) and low-projection intensity phase map φ L (x,y)

[0050] In the high dynamic range 3D reconstruction method using the adaptive reverse multiple exposure technology, the specific process of step 4 is as follows:

[0051] 4-1) Extract the effective phase value in the low-projection intensity phase image and calculate the local phase φ' in the high-reflection area L (x,y):

[0052] φ' L (x,y)=m(x,y)·φ L (x,y)

[0053] 4-2) Calculate the reverse mask m * (x,y):

[0054] m * (x,y)=1-m(x,y)

[0055] 4-3) Extracting the high-projection effective phase φ' H (x,y):

[0056] φ' H (x,y)=m * (x,y)·φ H (x,y)

[0057] 4-4) Pixel-by-pixel fusion φ' L (x,y) and φ' H (x,y) to get the complete phase map:

[0058] φ(x,y)=φ' L (x,y)+φ' H (x,y)

[0059] In the high dynamic range 3D reconstruction method using the adaptive reverse multiple exposure technology, the specific process of step 5 is as follows:

[0060] 5-1) Using the stripe structured light system, calculate the absolute phase φ in the horizontal and vertical directions respectively x (u,v) and φ y (u,v);

[0061] 5-2) The mapping relationship between the projector coordinates and the camera coordinates is established through linear interpolation. The calculation formula is as follows:

[0062]

[0063] Among them, φ x (u,v) and φ y (u, v) are the absolute phase values of the camera coordinates (u, v) in the horizontal and vertical directions, f is the number of sinusoidal cycles of the fringe pattern, H p and W p It is the row and column resolution of the projector device.

[0064] 5-3) Combining the principle of triangulation, the depth information of the object surface is calculated based on the projector's and camera's perspectives to reconstruct a complete high-precision 3D point cloud.

[0065] The beneficial effects of the present invention are:

[0066] The projection pattern is reduced by more than 50%, improving measurement efficiency: Compared with the multiple exposure method, this method only requires 2 to 3 sets of projections, which reduces the acquisition time and makes it suitable for high-speed measurement scenarios.

[0067] Adaptive projection intensity adjustment to improve measurement accuracy in highly reflective areas: Adopting gradient adjustment method, high-precision area-level projection optimization is achieved to ensure reliable measurement results.

[0068] Pixel-by-pixel phase fusion ensures measurement integrity: By fusing the phase information of high and low projection intensities, it solves the problem of overexposure in highly reflective areas while improving the measurement signal-to-noise ratio in low-reflective areas.

[0069] Compatible with existing systems, no additional hardware required: This method can be seamlessly integrated into existing 3D measurement equipment, requiring no additional optical adjustments, reducing implementation costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a binary mask image of the valid area of the example of the present invention

[0071] Figure 2 Flowchart of the gradient adjustment method for determining the optimal low projection intensity of the present invention

[0072] Figure 3 The overall flow chart of the present invention is

[0073] Figure 4 The experimental object diagram of the present invention

[0074] Figure 5 This is a flowchart of the phase fusion of different projection intensities of the present invention

[0075] Figure 6 Grayscale images of the experimental object of the present invention at different projection intensities

[0076] Figure 7 Comparison diagram of the three-dimensional imaging algorithm of the present invention and the traditional method DETAILED DESCRIPTION

[0077] The present invention will be further described below with reference to the accompanying drawings and examples.

[0078] Select objects as keys and coins Figure 4 The key is made of black plastic and metal, with both dark areas and highly reflective areas; the coin is a classic smooth metal with a more complex surface and more imaging details.

[0079] Step 1: Identification of highly reflective areas;

[0080] The specific process of step one is:

[0081] First, project a full white image (I p =255) Get the high-reflection area mask M

[0082] 1-1) Use a full white image with a projection grayscale value of 255 for projection.

[0083] 1-2) After the camera captures the reflected image, it uses the preset threshold 248 to generate a binary mask image m(x,y).

[0084] The calculation formula is as follows:

[0085]

[0086] Among them, I(x,y) represents the pixel grayscale value collected by the camera. A pixel with a value of 0 represents a high-reflection area, and a pixel with a value of 1 represents a non-high-reflection area.

[0087] Step 2: Adaptive projection intensity adjustment;

[0088] The specific process of step 2 is:

[0089] See Figure 2 Flowchart of the gradient adjustment method for optimal low projection intensity. The recursive process is performed for a maximum of 5 rounds (i.e., n≤5). Within this range, a suitable Make all pixels in the high-reflection area meet I c (x,y)≤248, get the best low-projection intensity image

[0090] Step 3: Phase demodulation;

[0091] The specific process of step three is:

[0092] Normal projection intensity and optimal low projection intensity The absolute phase calculation of the two groups of projection intensity fringes is performed separately. The phase processing of the fringe patterns with two different projection intensities is performed to obtain two phase maps, which are recorded as high projection intensity phase maps φ. H (x,y) and low-projection intensity phase map φ L (x,y)

[0093] Step 4: Phase fusion;

[0094] The specific process of step 4 is:

[0095] The local phase image is obtained and fused with the global phase image to obtain a complete phase map. Figure 5 As shown in FIG, different local phase information can be obtained by using two fringe patterns with different projection intensities and the corresponding mask patterns, and finally the global phase information can be obtained.

[0096] 4-1) Extract the effective phase value in the low-projection intensity phase image and calculate the local phase φ' in the high-reflection area L (x,y):

[0097] φ' L (x,y)=m(x,y)·φ L (x,y)

[0098] 4-2) Calculate the reverse mask m*(x,y):

[0099] m * (x,y)=1-m(x,y)

[0100] 4-3) Extracting the high-projection effective phase φ' H (x,y):

[0101] φ' H (x,y)=m * (x,y)·φ H (x,y)

[0102] 4-4) Pixel-by-pixel fusion φ' L (x,y) and φ' H (x,y) to get the complete phase map:

[0103] φ(x,y)=φ' L (x,y)+φ' H (x,y)

[0104] Step 5: 3D point cloud reconstruction;

[0105] The specific process of step five is:

[0106] The three-dimensional imaging of the experimental object is performed by combining the phase-shifted complementary Gray code stripe pattern with the adaptive reverse multiple exposure phase fusion technology, and compared with the imaging effect of the traditional phase-shifted complementary Gray code method. The grayscale images obtained under different projection intensities are shown in Figure 6. The final three-dimensional point cloud imaging effect is as follows Figure 7 shown.

Claims

1. A high dynamic range 3D reconstruction method based on adaptive reverse multiple exposure technology, characterized in that: The following steps are involved: Step 1: Identify high-reflective areas. Project a white image onto the object to be tested, capture the reflected image with a camera, analyze the pixel grayscale values, and use a preset threshold of 248 to generate a binary mask m(x, y) of the high-reflective area. Areas with pixel grayscale values greater than or equal to 248 are marked as high-reflective areas, and the rest are marked as non-high-reflective areas. Step 2: Adaptive projection intensity adjustment: According to the binary mask image, determine whether there are saturated pixels. If so, gradually reduce the projection grayscale value. If not, gradually increase the projection grayscale value. The maximum number of adjustment rounds does not exceed 5 rounds. Finally, the optimal low projection intensity is obtained to meet the requirement of no saturated pixels in the high-reflective area. Step 3: Phase demodulation: Use the optimal low projection intensity and high projection intensity to project the structured light stripe pattern respectively, collect the reflected image, and calculate the corresponding high projection intensity phase map φ H (x, y) and low-projection intensity phase map φ L (x, y); Step 4: Phase fusion, using the binary mask image m(x,y) and the reverse mask m*(x,y)=1-m(x,y), extract φ H (x, y) and φ L (x, y) respectively, and the complete phase map φ(x, y) = φ′ is calculated using a pixel-by-pixel fusion method. L (x, y) + φ′ H (x, y), where φ′ H (x, y) = m*(x, y)·φ H (x, y), φ′ L (x, y) = m(x, y)·φ L (x, y); Step 5: 3D point cloud reconstruction: Based on the complete phase map, the three-dimensional coordinate points on the object surface are calculated using the triangulation principle through the linear interpolation mapping method of the camera coordinates and the projector coordinates, and a high-precision three-dimensional point cloud is generated.

2. The high-reflective area recognition according to claim 1, characterized in that: The specific process of step one is as follows: 1-1) Use a full white image with a projection grayscale value of 255 for projection. 1-2) After the camera captures the reflected image, a preset threshold of 248 is used to generate a binary mask image m(x,y). The calculation formula is as follows: Among them, I(x,y) represents the pixel grayscale value collected by the camera. A pixel with a value of 0 represents a high-reflection area, and a pixel with a value of 1 represents a non-high-reflection area.

3. The adaptive projection intensity adjustment according to claim 1, characterized in that: The specific process of step 2 is as follows: 2-1) In a highly reflective area, if there is a pixel that satisfies I(x,y)≥248, the recursive formula is executed: That is, the projection grayscale is reduced by 20% each time until all pixels in the high-reflection area satisfy I(x,y)<248; 2-2) If there are no saturated pixels in the initial projection intensity, the following recursive formula is used to gradually increase the projection intensity to optimize the signal-to-noise ratio in the low-reflection area: The above recursive process is performed for a maximum of 5 rounds (i.e. n≤5). Within this range, a suitable Make all pixels in the high-reflective area meet the following conditions: Stop adjustment when the following conditions are met: I c (x,y)≤248。 4. The phase demodulation step according to claim 1, characterized in that: The specific process of step three is as follows: 3-1) Normal projection intensity and optimal low projection intensity The absolute phase calculation of the two groups of projection intensity fringes is performed separately. The phase processing of the fringe patterns with two different projection intensities is performed to obtain two phase maps, which are recorded as high projection intensity phase maps φ. H (x, y) and low-projection intensity phase map φ L (x, y).

5. The phase fusion step according to claim 1, characterized in that: The specific process of step 4 is as follows: 4-1) Extract the effective phase value in the low-projection intensity phase image and calculate the local phase φ′ in the high-reflection area L (x, y): φ′ L (x,y)=m(x,y)·φ L (x,y) 4-2) Calculate the reverse mask m*(x,y): m * (x,y)=1-m(x,y) 4-3) Extracting the high-projection effective phase φ′ H (x, y): φ′ H (x,y)=m * (x,y)·φ H (x,y) 4-4) Pixel-by-pixel fusion φ′ L (x, y) and φ′ H (x, y) to obtain the complete phase map: φ(x,y)=φ′ L (x,y)+φ′ H (x, y).

6. The three-dimensional point cloud reconstruction according to claim 1, characterized in that: The specific process of step five is as follows: 5-1) Using the stripe structured light system, calculate the absolute phase φ in the horizontal and vertical directions respectively x (u, v) and φ y (u, v); 5-2) The mapping relationship between the projector coordinates and the camera coordinates is established through linear interpolation. The calculation formula is as follows: Among them, φ x (u, v) and φ y (u, v) are the absolute phase values of the camera coordinates (u, v) in the horizontal and vertical directions, f is the number of sinusoidal cycles of the fringe pattern, H p and W p It is the row and column resolution of the projector device. 5-3) Combining the principle of triangulation, the depth information of the object surface is calculated based on the projector's and camera's perspectives to reconstruct a complete high-precision 3D point cloud.

Images