An adaptive high-dynamic fusion method based on structured light stripe block

Abstract

The application discloses a self-adaptive high-dynamic fusion method based on structure light stripe block, which comprises the following steps: S1, obtaining an automatic exposure time, projecting a four-step phase shift grating graph using the exposure time and calculating, calculating the median of the projection light intensity of left and right cameras; S2, calculating a ratio value, calculating each reference modulation value point and corresponding each reference exposure time; S3, comparing the number of the reference exposure time, if the number is greater than 1, continuing to execute the following steps, if the number is equal to 1, using the automatic exposure to project and calculate a complete grating group and directly outputting to the last step; S4, re-projecting using each reference exposure time and calculating each value; S5, performing a fusion operation on different modulation, background light and phase results; S6, calculating a reconstruction result and outputting. The application can realize automatic fusion, overcome a wide dynamic range and accurate imaging, overcome the phenomenon of inconsistent color of a map, vacancy of an overexposed area and the like.

Description

Technical Field

[0001] This invention belongs to the field of structured light fusion technology, and particularly relates to an adaptive high dynamic fusion method based on structured light stripe segmentation. Background Technology

[0002] In industrial inspection, structured light-based 3D measurement technology offers advantages such as non-contact operation, high measurement accuracy, and fast response speed, leading to its widespread application across various industries. However, some common challenges in industrial inspection include the large dynamic range of the measured object's surface. This range includes areas with specular reflections and darker areas with low reflectivity. Typically, brighter reflective areas require shorter exposure times for reconstruction, while darker reflective areas require longer exposure times. Therefore, using the same exposure time for the same scene makes it difficult to reconstruct the entire object within the measurement range.

[0003] Currently, to address the high dynamic range (HDR) issue, some researchers have proposed statistically analyzing the distribution of target brightness, using 3-5 segments to count troughs, with each segment considered as a reflectance. The advantage of this method is that it avoids overexposure and achieves true HDR. However, its disadvantages include difficulty in automated implementation and inability to handle specular reflections. Another approach involves statistically analyzing the distribution of modulation intensity, using several segments based on ratios, with each segment considered as a reflectance. When the modulation intensity exceeds a certain value, the phase accuracy meets requirements. By statistically analyzing the target brightness values ​​at all 66 points within the modulation intensity range, if the exposure time at that point is optimal, the modulation intensity area above that value will also be optimal. The advantage of this method is its suitability for automated implementation, but its disadvantages include overexposure due to specular reflections, resulting in partial HDR loss, and a lack of understanding of the global modulation intensity distribution. Therefore, an adaptive HDR optical measurement method for the measured scene remains relatively scarce. Summary of the Invention

[0004] The purpose of this invention is to provide an adaptive high dynamic range fusion method based on structured light stripe segmentation, so as to solve the problem that existing high dynamic range fusion does not achieve automaticity and that missing regions will occur during automatic high dynamic range fusion.

[0005] Based on the above reasons, this invention discloses an adaptive high dynamic range fusion method based on structured light stripe segmentation, comprising the following steps:

[0006] S1. Obtain the appropriate automatic exposure time for the current scene. Use exposure time Project a four-step phase-shifting grating pattern and solve it, then calculate the median of the projected light intensity for the left and right cameras respectively;

[0007] Formula for projection onto camera:

[0008]

[0009] In the formula, s represents the sensor sensitivity; t represents the camera's exposure time; Indicates the reflectivity of an object; This represents ambient light reflected from an object. This refers to ambient light that directly enters the camera;

[0010] because and The ambient light is much less than the light intensity of the actively projected stripes, so the above equation can be simplified to:

[0011]

[0012] The optimal exposure time and ideal background light value obtained are:

[0013]

[0014] In the formula, This indicates the current projected light intensity value. Indicates the exposure time under the current circumstances;

[0015] Regardless of the exposure time and the intensity of the light captured. It won't change; simply changing the exposure time will achieve the optimal target brightness.

[0016]

[0017]

[0018] in This represents the ideal projected light intensity, with the optimal light intensity value set to 254. Given the exposure time of the current frame, the optimal exposure time for the current frame can be calculated using the above formula. .

[0019] Based on the four-step phase shift, the projected light intensity of the left and right cameras is obtained by superimposing the four fringe patterns:

[0020]

[0021]

[0022] in , Indicates the modulation level of the left and right cameras. , Indicates the background light of the left and right cameras. , Indicates the projected light intensity of the left and right cameras;

[0023] Find the median of the projected light intensity , :

[0024]

[0025]

[0026] in, This indicates an operation to take the median value of the projected light intensity. , This represents the median value of the projected light intensity from the left and right cameras;

[0027] Using the median value to represent the current projected light intensity is more representative, indicating that most light intensities greater than the median value can meet the requirements of the current exposure time. The next step is to find several exposure times that allow all pixels to meet the above conditions and obtain the best results.

[0028] S2. Calculate the ratio values ​​of the left and right cameras respectively, then average the ratio values ​​of the left and right cameras, and obtain the overall equipment change ratio. Calculate the boundary values ​​of different modulation and exposure time corresponding to different ratios.

[0029] S21. Calculate the rate of change of exposure time. :

[0030]

[0031] In the formula, the ideal optimal light intensity value is set to 254;

[0032] S22. Determine if the condition is met. If the conditions are not met, then no changes will be made.

[0033] If satisfied, then ;

[0034] S23. Average the change ratios of the left and right cameras:

[0035]

[0036] in, This represents the rate of change in the exposure time of the right camera. This represents the rate of change in the exposure time of the left camera. This represents the overall change rate of the equipment.

[0037] S24. Calculate the baseline modulation value and the baseline exposure time:

[0038]

[0039] Initial value setting of the modulation system , It is the exposure time obtained in step S1; until the calculation reaches the... Second-rate Then stop calculating the two boundary values.

[0040] S3. Compare the number of the above-mentioned baseline exposure times. ,if If the value is greater than 1, continue execution; if the value is equal to 1, then use automatic exposure. Perform projection and calculation of a complete set of gratings, and output the result directly to the final step;

[0041] S31. If the number of boundary values ​​is equal to 1, then the automatic exposure algorithm is used directly to obtain a single automatic exposure time for the projection and acquisition of a complete set of gratings, and then the calculation is performed.

[0042] S32. If the number of boundary values ​​is greater than 1, then project several sets of stripe gratings with different exposure times to project stripes, and then calculate several sets of modulation, background light, and unfolded phase results.

[0043] S4. Project the grating pattern obtained from the above-calculated exposure time group again and solve it. This method uses a grating pattern with Gray code fringes and sinusoidal fringes phase shift for projection. Gray code has a great suppressive effect on the stage error that occurs when the image light intensity is overexposed, and can effectively improve the accuracy of different solution phases.

[0044] S5. For raster images under different exposure times, fuse them according to the modulation threshold and output the fused result.

[0045]

[0046] in, This indicates the number of times calculated in the previous step. , , This represents the unfolded phase, background light, and modulation obtained in step S2; Indicates satisfaction The region on the image at that time; , , In step S4, the first The calculated unfolded phase, background light, and modulation degree; Indicates the first The secondary image satisfies The region on the image at that time; , , In step S4, the first The calculated unfolded phase, background light, and modulation degree; Indicates the first The secondary image satisfies The region on the image at that time; , , This indicates the unfolded phase, background light, and modulation after fusion; Represents the region of the image.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] This invention presents an adaptive high dynamic range fusion method based on structured light stripe segmentation. This method enables automatic fusion, overcomes a wide dynamic range, and achieves accurate imaging. It also overcomes issues such as inconsistent texture colors and empty areas in overexposed regions. Compared to existing technologies, its use of the median value to represent the current projected light intensity is more representative, indicating that most light intensities greater than the median value are sufficient for the current exposure time, ensuring that all pixels meet the above requirements and achieving optimal results. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the process structure of the present invention.

[0050] Figure 2 This is a schematic diagram of the process structure of the ratio value acquisition method of the present invention. Detailed Implementation

[0051] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0052] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," "front end," "rear end," "both ends," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0053] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] like Figure 1 and Figure 2 As shown, this embodiment discloses an adaptive high dynamic range fusion method based on structured light stripe segmentation, including the following steps:

[0056] S1. Obtain automatic exposure time Use exposure time Project a four-step phase-shifting grating pattern and solve it, then calculate the median of the projected light intensity for the left and right cameras respectively;

[0057] According to the formula for projection onto the camera:

[0058]

[0059] In the formula, s represents the sensor sensitivity; t represents the camera's exposure time; Indicates the reflectivity of an object; This represents ambient light reflected from an object. This refers to ambient light that directly enters the camera;

[0060] because and The ambient light is much less than the light intensity of the actively projected stripes, so the above equation can be simplified to:

[0061]

[0062] The optimal exposure time and ideal background light value obtained are:

[0063]

[0064] In the formula, This indicates the current projected light intensity value. Indicates the exposure time under the current circumstances;

[0065] Regardless of the exposure time and the intensity of the light captured. It won't change; simply changing the exposure time will achieve the optimal target brightness.

[0066]

[0067]

[0068] in This represents the ideal projected light intensity, with the optimal light intensity value set to 254. Given the exposure time of the current frame, the optimal exposure time for the current frame can be calculated using the above formula. ;

[0069] Based on the four-step phase shift, the projected light intensity of the left and right cameras is obtained by superimposing the four fringe patterns:

[0070]

[0071]

[0072] in , Indicates the modulation level of the left and right cameras. , Indicates the background light of the left and right cameras. , Indicates the projected light intensity of the left and right cameras;

[0073] Find the median of the projected light intensity , :

[0074]

[0075]

[0076] in, This indicates an operation to take the median value of the projected light intensity. , This represents the median value of the projected light intensity from the left and right cameras.

[0077] S2. Calculate the ratio value of the entire equipment, calculate the reference modulation value points, and the corresponding reference exposure time;

[0078] S21. Obtain the initial ratio value:

[0079]

[0080] S22. Determine if the condition is met. If the conditions are not met, then no changes will be made.

[0081] If satisfied, then ;

[0082] S23. Average the change ratios of the left and right cameras:

[0083]

[0084] in, This represents the rate of change in the exposure time of the right camera. This represents the rate of change in the exposure time of the left camera. This represents the overall change rate of the equipment.

[0085] S24. Calculate the baseline modulation value and the baseline exposure time:

[0086]

[0087] The initial baseline value of the modulation system is set to , It is the exposure time obtained in step S1; until the calculation reaches the... Second-rate Then stop calculating the two boundary values.

[0088] S3. Compare the number of the above-mentioned baseline exposure times. ,if If the value is greater than 1, continue execution; if the value is equal to 1, then use automatic exposure. Perform projection and calculation of a complete set of gratings, and output directly to the final step; number of reference exposure times. This directly represents the basic situation of the surface reflectivity of the object being measured. If the value is greater than 1, the projection of each reference exposure time stripe is performed and calculated, and then the results of the calculations of different reference stripes are fused; if the value is equal to 1, then automatic exposure is used. Perform projection and calculation of a complete set of gratings, and output the result directly to the final step.

[0089] S4. Reproject using the above-mentioned reference exposure times and solve for each value;

[0090] S5. Perform fusion operations on different modulation levels, background light, and phase results;

[0091] The solution method in step S5:

[0092]

[0093] in, This indicates the number of times calculated in the previous step. , , This represents the unfolded phase, background light, and modulation degree calculated in step S1; Indicates satisfaction The region on the image at that time; , , In step S4, the first The calculated unfolded phase, background light, and modulation degree; Indicates the first The secondary image satisfies The region on the image at that time; , , In step S4, the first The calculated unfolded phase, background light, and modulation degree; Indicates the first The secondary image satisfies The region on the image at that time; , , This indicates the unfolded phase, background light, and modulation after fusion; Represents the region of the image.

[0094] S6. Calculate and output the reconstruction results.

[0095] The high dynamic range fusion method based on structured light stripe segmentation of the present invention uses the median to replace the current projected light intensity, which can achieve automatic fusion and overcome a wide dynamic range and accurate imaging. It can also overcome defects such as inconsistent texture colors and empty areas in overexposed regions.

[0096] The above descriptions are merely embodiments of the present invention, and common knowledge regarding specific structures and characteristics of the solutions is not described in detail here. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present invention is defined by the appended claims rather than the foregoing description. Therefore, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. An adaptive high dynamic fusion method based on structured light fringe patching, characterized in that, The method comprises the following steps: S1, obtain auto exposure time , using exposure time Projecting four-step phase-shift grating pattern and solving, respectively calculating the median of the projected light intensity of left and right cameras; including: According to the formula of projection to shooting: where s represents the sensor sensitivity; t represents the exposure time of the camera; represents the reflectivity of the object; represents the ambient light reflected by the object; represents the ambient light directly entering the camera; Due to With ambient light is much smaller than the light intensity of the active projection fringe, the above formula can be simplified as: The optimal exposure time and the ideal background light value are obtained: wherein denotes the current projection light intensity value, denotes the exposure time in the current case; No matter how long the exposure time and the light intensity, The target brightness can be optimized by changing the exposure time unilaterally. wherein represents the ideal projected light intensity, the ideal optimal light intensity value is set to 254; is the exposure time of the current frame, then the optimal exposure time suitable for the current frame can be calculated according to the above formula ; According to four-step phase shift, four fringe patterns are superimposed to obtain the projection light intensity of left and right cameras: wherein , denotes the modulation of the left camera, , denotes the background light of the left camera, , denotes the projection light intensity of the left camera; Obtaining the median of the projected light intensity , : wherein, denotes the median operation on the projected light intensity, , i.e. the median value of the projected light intensity of the left and right cameras; S2, calculate the ratio value of the entire device, calculate each reference exposure time value, and calculate the corresponding each reference exposure time value; S3, compare the number of the above reference exposure time if greater than 1 continue to execute, if equal to 1, then use automatic exposure projection and solution of a complete set of grating are carried out and directly output to the last step; S4, use the obtained each reference exposure time to perform re-projection, and solve to obtain each value; S5, perform fusion operation on different exposure, background light and phase results; S6, calculate the reconstruction result and output.

2. The adaptive high dynamic fusion method based on structured light fringe patching according to claim 1, characterized in that, The step S2 comprises: S21, obtain the initial ratio value; S22, determining whether the condition is met , if not, no change If satisfied, then ; S23, average the change ratio of the left camera and the right camera: wherein, is a change ratio of the exposure time of the right camera, is a change ratio of the exposure time of the left camera, is a change ratio of the overall device; S24, obtain each reference exposure time value and each reference exposure time value: The modulation degree initial reference value is set to , is the exposure time obtained in step S1; the calculation of the two boundary values is stopped until the th time.

3. The adaptive high dynamic fusion method based on structured light fringe patching according to claim 2, characterized in that, In the step S3: Number of reference exposure times directly represents the surface reflectivity of the measured object, if greater than 1, the projection of each reference exposure time fringe is performed and resolved, and then the results of the resolution of the different reference fringes are fused; if equal to 1, then the automatic exposure the projection and resolution of a complete set of gratings are performed and directly output to the final step.

4. The adaptive high dynamic fusion method based on structured light fringe patching according to claim 3, characterized in that, The solving method in the step S5: wherein, represents the number of times calculated in the previous step, , , represents the unwrapped phase, the background light and the modulation calculated in step S1; represents the region on the image when is satisfied; , , represents the unwrapped phase, the background light and the modulation calculated in step S4 for the time; represents the region on the image when the time image satisfies ; , , represents the unwrapped phase, the background light and the modulation calculated in step S4 for the time; represents the region on the image when the time image satisfies ; , , represents the unwrapped phase, the background light and the modulation after fusion; represents the region of the image.

Images