Self-adaptive multi-exposure fusion three-dimensional measurement method for high dynamic range scene
By employing an adaptive multi-exposure fusion strategy and depth-constrained phase calculation, the problems of local information loss and low measurement efficiency in traditional optical 3D measurement methods under high dynamic range scenarios are solved, achieving high-precision and efficient 3D topography reconstruction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-02
- Publication Date
- 2026-03-13
AI Technical Summary
When faced with high dynamic range scenes, existing optical 3D measurement methods cannot take into account both saturated pixels and invalid dark areas using traditional single exposure settings, resulting in the loss of local 3D information. Furthermore, existing multi-exposure fusion strategies have excessively long total exposure times, affecting measurement efficiency and requiring manual control.
An adaptive multi-exposure fusion strategy is adopted. By using temporal intensity overexposure detection and noise-modulation model, combined with depth-constrained reference plane phase calculation, high-quality stripe areas are adaptively preserved. The total exposure time is shortened by utilizing the temporal superposition principle, thus achieving fully automatic three-dimensional measurement.
It achieves high-precision and high-efficiency fully automatic 3D topography reconstruction, effectively suppresses noise, and completes comprehensive measurement of high dynamic range scenes.
Smart Images
Figure CN121655424A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of optical measurement technology, specifically an adaptive multi-exposure fusion three-dimensional measurement method for high dynamic range scenarios. Background Technology
[0002] Optical 3D measurement faces the challenge of obtaining complete, high-quality 3D data from complex surfaces that simultaneously contain areas of intense brightness and extremely low reflectivity. This challenge is often referred to as the "high-contrast surface measurement" problem. The current common approach to solving this problem is to introduce high dynamic range (HDR) technology, which expands the measurable reflectivity range to ensure the reliability of the entire field of data.
[0003] Fringe projection profilometry (FPP), a mainstream method for optical 3D measurement, achieves 3D measurement by projecting regular phase-shifted fringes and analyzing their phase deformation, and has been widely used in fields such as cultural relic preservation and industrial inspection. However, in high dynamic range scenes, the single exposure setting of traditional FPP cannot simultaneously account for saturated pixels and invalid dark areas, resulting in the loss of local 3D information.
[0004] To overcome this limitation, existing research generally employs a "multi-exposure fusion" strategy: exposure time and fusion area are determined based on experience, pixel grayscale, or phase error data, and then multiple stripe images are fused into a single HDR image, thereby achieving high dynamic range 3D measurement. However, this method has an excessively long total exposure time, severely impacting measurement efficiency. Its performance is limited when measuring complex surfaces with significantly different reflectivity characteristics, preventing comprehensive measurements. Furthermore, it requires manual control of the exposure time, making fully automated measurement impossible. Summary of the Invention
[0005] The purpose of this invention is to propose an adaptive multi-exposure fusion 3D measurement method for high dynamic range scenes, so as to achieve high-precision and high-efficiency fully automatic 3D measurement of high dynamic range scenes.
[0006] The technical solution to achieve the objective of this invention is: an adaptive multi-exposure fusion 3D measurement method for high dynamic range scenes, comprising the following steps:
[0007] Step 1: Collect high-frequency phase shift fringe patterns at different exposure times, and find the initial exposure time that ensures that the effective measurement areas in all phase shift patterns are not overexposed through time-domain light intensity overexposure detection.
[0008] Step 2: Using the high-frequency stripe pattern of the initial exposure time as the starting image for multi-exposure fusion, the stripe region whose phase quality meets the set threshold is adaptively retained according to the noise-modulation model based on the modulation condition, and the next exposure time of multi-exposure fusion is estimated by introducing the temporal superposition principle.
[0009] Step 3: Calculate the effective measurement area of the current fused stripe image using the reference plane phase based on depth constraints, and continue multi-exposure fusion until all effective measurement areas meet the modulation conditions, thus completing the three-dimensional measurement of the entire high dynamic range scene.
[0010] Compared with the prior art, the significant advantages of this invention are:
[0011] This invention employs an adaptive multi-exposure fusion strategy to acquire high signal-to-noise ratio fringe patterns. By utilizing temporal overlay technology, it synthesizes multiple exposed images into a result of equivalent quality to a single long-exposure image, significantly reducing the total exposure time while effectively suppressing noise compared to existing technologies. Furthermore, it introduces depth-constrained reference plane phase estimation to accurately pinpoint the effective measurement area of the fused fringe, achieving more comprehensive and robust measurements. Ultimately, high-precision, high-efficiency, and fully automated 3D topography reconstruction is achieved in high dynamic range scenarios.
[0012] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description
[0013] Figure 1 This is a flowchart illustrating an adaptive multi-exposure fusion 3D measurement method for high dynamic range scenarios.
[0014] Figure 2 The flowchart shows the measurement process for high-reflectivity and low-reflectivity standard spheres using this method. (a) High-frequency fringe patterns under different exposures; (b) Modulation diagram of multi-exposure fusion; (c) High-frequency absolute phase after multi-exposure fusion; (d) Effective measurement area determined by the difference between the multi-exposure fusion phase and the reference phase; (e) Depth map; (f) 3D reconstruction results of the standard sphere. Detailed Implementation
[0015] An adaptive multi-exposure fusion 3D measurement method for high dynamic range scenes includes the following steps:
[0016] Step 1: Collect high-frequency phase shift fringe patterns at different exposure times, and find the initial exposure time that ensures that the effective measurement areas in all phase shift patterns are not overexposed through time-domain light intensity overexposure detection.
[0017] Step 1.1: Collect a set of data at arbitrary exposure times. High-frequency fringe pattern with phase shift ;
[0018] Step 1.2: Obtain the saturation mask image by detecting whether the same pixel in all phase-shifted high-frequency stripe patterns is overexposed. :
[0019]
[0020] in, This is the grayscale threshold. for Step phase-shifted image, These are the pixel coordinates;
[0021] Step 1.3: Apply saturation mask image Perform a four-neighbor connectivity operation and find the largest connected region if the number of pixels in that region is less than 1. Then proceed to step 1.4 and mark the exposure time at this point as [time value]. Otherwise, re-acquire a set with the exposure time halved. High-frequency fringe pattern with phase shift And return to step 1.2;
[0022] Step 1.4: Reduce the grayscale threshold One step The saturated mask image is then recalculated using the formula from step 1.2. And find the largest connected region. If the number of pixels in the largest connected region of this region is greater than 1, then the maximum connected region is found. If yes, proceed to step 1.5; otherwise, repeat step 1.4.
[0023] Step 1.5: Calculate the initial exposure time for multi-exposure fusion. :
[0024]
[0025] in This is the saturation grayscale value. The region with the highest light intensity in the largest connected region The average brightness of each pixel.
[0026] Step 2: Using the high-frequency stripe pattern of the initial exposure time as the starting image for multi-exposure fusion, the stripe region with high phase quality is adaptively retained according to the modulation condition through the "noise-modulation" model, and the next exposure time of multi-exposure fusion is estimated by introducing the temporal superposition principle.
[0027] Step 2.1: Acquire initial exposure time Next group High-frequency fringe pattern with phase shift and it is a multi-exposure fused image. The initial value. Let the current multi-exposure fusion order be denoted as . .
[0028] Step 2.2: Calculate the multi-exposure fusion image using the least squares method. Background light intensity With modulation amplitude :
[0029]
[0030]
[0031] Step 2.3: Introduce the "noise-modulation" model, that is, through the modulation threshold. Select regions with high phase quality stripe patterns and generate a mask image accordingly. This is used for preserving subsequent high-quality fringe images:
[0032]
[0033] in, The upper limit of the target phase error, This indicates the camera's maximum Gaussian noise.
[0034] Step 2.4: Applying the principle of temporal superposition, multiple short-exposure images are fused into an equivalent long-exposure image, and the next exposure time of the multi-exposure fusion is estimated accordingly. :
[0035]
[0036] in for The closest to the modulation threshold The set of pixels (number of pixels) ), The theoretical maximum value of the grayscale of the striped image on this set can be determined by the background light intensity. With adjustment system The average of the sums is estimated.
[0037] Step 3: Calculate the effective measurement area of the current fused stripe image using the reference plane phase based on depth constraints, and continue multi-exposure fusion until all effective measurement areas meet the modulation conditions, thus completing the three-dimensional measurement of the entire high dynamic range scene.
[0038] Step 3.1: Apply a quantitative model of depth-phase difference, i.e., based on the actual measured depth range. Calculate the corresponding phase difference range :
[0039]
[0040] in The height of the point to be measured relative to the reference plane is generally selected as any plane that is at the farthest distance from the point being measured. The vertical distance from the projection-camera system to the reference plane; The difference between the reference plane phase and the high-frequency absolute phase at the same pixel is obtained by acquiring high-frequency phase shift fringes on the reference plane. The physical period of the projected fringes on the reference plane; The baseline distance between the projector and the camera;
[0041] Step 3.2: Process the multi-exposure fused image The standard phase-shifting algorithm is used to calculate the high-frequency absolute phase, and its position is verified. Within the interval, to extract the effective measurement area. Subsequently, selection was made within this area. For the set of pixels with a modulation index that is inversely proportional, calculate the mean modulation index. (Number of pixels is) And determine whether the next set of acquisitions is the last set of high-frequency phase shift fringe patterns according to the following formula:
[0042]
[0043] If the criteria are met, it is determined to be the last exposure, and the exposure time is recalculated. If so, proceed to step 3.4; otherwise, proceed to step 3.3.
[0044]
[0045] This strategy allows the tested area to be exposed in the final exposure. The modulation of all pixels is just above the threshold, which ensures the phase quality of the multi-exposure fused image and compresses the total exposure time to the theoretical minimum by means of the temporal superposition principle.
[0046] Step 3.3: The exposure time for data acquisition is... Phase shift number is Image The images from this exposure were then merged into a multi-exposure fusion image. Above, that is, synthesizing a new fused image. :
[0047]
[0048] Then Substitute the multi-exposure fusion image into step 2.2 And continue to optimize, at this point the fusion order Add 1.
[0049] Step 3.4: The exposure time for data acquisition is... Phase shift number is Image Complete using the formula in step 3.3. The final fusion is performed, and then the high-frequency absolute phase is solved by the standard phase-shifting algorithm on the fused image. The depth information is calculated according to the calibration parameters to complete the entire high dynamic range three-dimensional measurement.
[0050] To verify the effectiveness of the proposed method, a high dynamic range three-dimensional measurement experimental system was constructed. The core of the system consists of a DLP projector (DLP-4500CV) and a high-resolution monochrome industrial camera (Basler acA2440-75 μm), and the system's intrinsic and extrinsic parameters were pre-calibrated. A high-reflectivity standard sphere and a low-reflectivity standard sphere were simultaneously placed within the measurement field of view approximately 1 m from the projection-camera system. The initial grayscale threshold was set to 255. This method primarily acquires high-frequency (128-cycle) fringe images with 8-step phase shifts, supplemented by 3-step phase-shift fringe images with frequencies of 1, 4, 16, and 64, used for calculating the high-frequency absolute phase using the phase-shift method.
[0051] Figure 2 The system demonstrates the entire process of high- and low-reflectivity standard spheres from initial acquisition to final 3D reconstruction: Figure 2 (a) The high-frequency stripe pattern under each exposure is given. After time-domain overexposure detection, the high-frequency stripe pattern under the first exposure just makes the high-reflection area not overexposed. Figure 2 (b) is the modulation graph after multi-exposure fusion. It can be seen that the modulation of the entire standard sphere is higher than the threshold (50), which meets the requirements of high-precision measurement. Figure 2 (c) Displaying the high-frequency absolute phase after fusion; Figure 2 (d) Further, the effective measurement area (standard sphere) is accurately extracted by the difference between this phase and the phase of the reference plane, and the measurement completeness is determined by the consistency of the global modulation index. Figure 2 (e) Depth map calculated using the absolute phase and camera calibration parameters described above; Figure 2 (f) Presents the final three-dimensional reconstruction results, showing that within the same field of view, both the highly reflective white sphere and the low-reflective black sphere can achieve complete and high-precision three-dimensional measurement.
Claims
1. An adaptive multi-exposure fusion 3D measurement method for high dynamic range scenes, characterized in that, Includes the following steps: Step 1: Collect high-frequency phase shift fringe patterns at different exposure times, and find the initial exposure time that ensures that the effective measurement areas in all phase shift patterns are not overexposed through time-domain light intensity overexposure detection. Step 2: Using the high-frequency stripe pattern of the initial exposure time as the starting image for multi-exposure fusion, the stripe region whose phase quality meets the set threshold is adaptively retained according to the noise-modulation model based on the modulation condition, and the next exposure time of multi-exposure fusion is estimated by introducing the temporal superposition principle. Step 3: Calculate the effective measurement area of the current fused stripe image using the reference plane phase based on depth constraints, and continue multi-exposure fusion until all effective measurement areas meet the modulation conditions, thus completing the three-dimensional measurement of the entire high dynamic range scene.
2. The adaptive multi-exposure fusion 3D measurement method for high dynamic range scenes according to claim 1, characterized in that, High-frequency phase-shift fringe patterns were collected at different exposure times. Using time-domain overexposure detection, the initial exposure time was determined to ensure that the effective measurement areas in all phase-shift patterns were not overexposed. The specific method is as follows: Step 1.1: Collect a set of data at arbitrary exposure times. High-frequency fringe pattern with phase shift ; Step 1.2: Obtain the saturation mask image by detecting whether the same pixel in all phase-shifted high-frequency stripe patterns is overexposed. : ； in, This is the grayscale threshold. for Step phase-shifted image, These are the pixel coordinates; Step 1.3: Apply saturation mask image Perform a four-neighbor connectivity operation and find the largest connected region. If the number of pixels in the largest connected region is less than 1, then the maximum connected region is found. Then proceed to step 1.4 and mark the exposure time at this point as [time value]. Otherwise, re-acquire a set with the exposure time halved. High-frequency fringe pattern with phase shift And return to step 1.2; Step 1.4: Reduce the grayscale threshold One step Recalculate the mask image and find the maximum connected component again. If the number of pixels in the maximum connected component is greater than... If yes, proceed to step 1.5; otherwise, repeat step 1.
4. Step 1.5: Calculate the initial exposure time for multi-exposure fusion. : in This is the saturation grayscale value. The region with the highest light intensity in the largest connected region The average brightness of each pixel.
3. The adaptive multi-exposure fusion 3D measurement method for high dynamic range scenes according to claim 2, characterized in that, Using the high-frequency fringe pattern of the initial exposure time as the starting image for multi-exposure fusion, a noise-modulation model is used to adaptively retain fringe regions with high phase quality based on modulation conditions. Furthermore, the temporal superposition principle is introduced to estimate the next exposure time for multi-exposure fusion. The specific method is as follows: Step 2.1: Acquire initial exposure time Next group High-frequency fringe pattern with phase shift and as a multi-exposure fusion image The initial value is denoted as the current multi-exposure fusion order. ; Step 2.2: Calculate the multi-exposure fusion image using the least squares method. Background light intensity With modulation amplitude : ； ； Step 2.3: Introduce the noise-modulation model, that is, through the modulation threshold. Select regions with high phase quality stripe patterns and generate a mask image accordingly. : in, The upper limit of the target phase error, This indicates the camera's maximum Gaussian noise. Step 2.4: Applying the principle of temporal superposition, multiple short-exposure images are fused into an equivalent long-exposure image, and the next exposure time of the multi-exposure fusion is estimated accordingly. : ； in, To adjust the system The closest to the modulation threshold The set of pixels, This represents the theoretical maximum gray level of the striped image within the pixel set.
4. The adaptive multi-exposure fusion 3D measurement method for high dynamic range scenes according to claim 3, characterized in that, The effective measurement area of the current fused fringe image is calculated using a depth-constrained reference plane phase, and multi-exposure fusion continues until all effective measurement areas meet the modulation conditions, thus completing the 3D measurement of the entire high dynamic range scene. The specific method is as follows: Step 3.1: Apply a quantitative model of depth-phase difference, i.e., based on the actual measured depth range. Calculate the corresponding phase difference range : ； in, The height of the point to be measured relative to the reference plane; The vertical distance from the projection-camera system to the reference plane; It is the difference between the reference plane phase and the high-frequency absolute phase at the same pixel location; The physical period of the projected fringes on the reference plane; The baseline distance between the projector and the camera; Step 3.2: Process the multi-exposure fused image The standard phase-shifting algorithm is used to calculate the high-frequency absolute phase, and its position is verified. Within the interval, to extract the effective measurement area. , Subsequently, multi-exposure fused images were selected within the effective measurement area. The set of pixels with the reciprocal modulation intensity is used to calculate the average modulation intensity. It then determines whether the next set of acquisitions is the last set of high-frequency phase-shift fringe patterns; if so, it recalculates the next exposure time. If yes, proceed to step 3.4; otherwise, proceed to step 3.
3. Step 3.3: The exposure time for data acquisition is... Phase shift number is Image The images exposed in this exposure will be merged into a multi-exposure fusion image. Above, that is, synthesizing a new fused image. And bring it into the multi-exposure fusion image in step 2.
2. Continue optimization; at this point, the fusion order... Add 1; Step 3.4: The exposure time for data acquisition is... Phase shift number is Image The image from this exposure is fused onto a multi-exposure fused image. Then, the high-frequency absolute phase is calculated using a standard phase-shifting algorithm on the fused image, and the depth information is calculated based on the calibration parameters to complete the entire high dynamic range three-dimensional measurement.
5. The adaptive multi-exposure fusion 3D measurement method for high dynamic range scenes according to claim 4, characterized in that, The next set of samples is determined to be the last set of high-frequency phase-shift fringe patterns when the following conditions are met: 。 6. The adaptive multi-exposure fusion three-dimensional measurement method for high dynamic range scenes according to claim 4, characterized in that, The specific formula for recalculating the exposure time is: 。 7. The adaptive multi-exposure fusion 3D measurement method for high dynamic range scenes according to claim 4, characterized in that, The images from this exposure will be merged into a multi-exposure fusion image. The specific formula is as follows: ； in, A mask image generated for stripe regions where the phase quality does not reach the modulation threshold. For mask image Invert.