Exposure value selection method for three-dimensional structured light measurement of surfaces with different reflectivity

Abstract

The present invention discloses an exposure value selection method for three-dimensional structured light measurement of surfaces with different reflectivity, which can avoid regional overexposure and improve the accuracy of single optimal exposure coverage pixel measurement. The method comprises: image acquisition, obtaining a pre-test image, judging whether there is an over-exposed area in the pre-test image, if not, stopping, if yes, selecting multiple exposures, the selection of multiple exposures comprising: S1, initialization of exposure, S2, selecting multiple exposures by using a pixel filter, S3, determining the next exposure value by optimization, and repeating steps S1 to S3 until the ratio of qualified pixels in the pre-test image is greater than or equal to 95%.

Description

Technical Field

[0001] The invention relates to the technical field of machine vision, and in particular to an exposure value selection method for three-dimensional structured light measurement of surfaces with different reflectivity. Background Art

[0002] Structured light-based three-dimensional (SL3D) measurement technology is widely used in object tracking, vision-based robot control and quality inspection due to its high precision and full-scene characteristics. In SL3D measurement, a sequential phase-encoded pattern is projected onto the surface of the target object, and then the camera in the 3D sensor is used to collect the deformed fringe pattern carrying depth information, and then image decoding and triangulation are used to reconstruct the three-dimensional point cloud of the target object.

[0003] At present, SL3D measurement technology is often used to measure the surface of objects with large reflectivity changes. When measuring such objects, the areas with high surface reflectivity are saturated, while the areas with low surface reflectivity are affected by underexposure, such as Figure 1 (a). Both saturation and underexposure will cause information loss in the reconstructed 3D point cloud, such as Figure 1 (b). Although objects can be sprayed with a thin layer of white powder to produce diffuse reflection, the spraying process is time-consuming and will change the surface properties of the object. High dynamic range (HDR) technology can avoid the change of object surface properties, but it requires measuring the object under multiple exposures and combining the features of different reflectivity through image fusion, such as Figure 1 (c, d) It can be seen that the choice of exposure value is very important in HDR measurement.

[0004] The selection of exposure value mainly includes qualitative selection and quantitative selection. Among them, qualitative selection is mainly manual selection based on visualization effect, but qualitative prediction is very time-consuming and easily affected by subjective judgment. Therefore, quantitative selection of camera exposure value is often used in HDR-based SL3D measurement method, but this quantitative selection of exposure value requires the execution of a pre-analysis program to calculate the reflectivity of the target surface, which requires the collection of images with different projection intensities, and then the surface reflectivity is calculated using the pixel intensity of the collected image and the projection intensity. The pixel intensity increases linearly with the increase of projection intensity, and the increase rate is considered to be the surface reflectivity of the pixel position, but it is difficult to pre-analyze each object with different surface reflectivity, so this method is not applicable to batch industrial production.

[0005] Currently, a variety of metric-based exposure selection methods have been proposed, which can automatically adjust the exposure value according to the evaluation of the captured image under dynamic lighting conditions without the need for prior analysis. However, in these methods, the captured image needs to be evaluated according to a pre-set image quality indicator, and then the exposure is repeatedly adjusted until the indicator reaches a maximum value. The purpose is to select an optimal exposure so that the target of interest can be effectively identified, such as lane or sign recognition for autonomous driving. The goal of HDR-based SL3D metrology is to select some exposure values ​​and reconstruct a point cloud to accurately measure the shape and size of the object. The two have different goals, so image quality indicators and a single optimal exposure selection strategy are not suitable for HDR-based SL3D metrology.

[0006] In addition, since most vision algorithms extract features from gradient-rich or entropy-rich regions, the exposure control metric is based on maximizing the gradient or entropy and evaluating the phase encoding information of SL images under low-light conditions through the intensity modulation metric. This metric assumes that the phase encoding information increases monotonically with the intensity modulation, which refers to the amplitude of the encoding pattern in the SL image. However, when the intensity modulation reaches a certain threshold, the phase encoding information reaches a plateau at the maximum value. In order to measure objects with different reflectivities, a larger exposure range and longer exposure time are required, and longer exposure time will lead to overexposure. Therefore, a new metric needs to be established to evaluate the phase encoding information in HDR-based SL3D measurement. In addition, in existing exposure control methods, the objects to be detected (such as lanes or traffic signs) usually have only a small range of surface reflectivity variation; therefore, features can be extracted in a single optimal exposure. In contrast, the goal of industrial metrology is to measure industrial parts with a wide range of surface reflectivity variations, which requires a new strategy to determine multiple exposures and be able to measure pixels that cannot be covered by a single exposure. Summary of the invention

[0007] In view of the defects existing in the prior art, the present invention provides an exposure value selection method for three-dimensional structured light measurement of surfaces with different reflectivity, which can avoid regional overexposure and improve the accuracy of single optimal exposure coverage pixel measurement.

[0008] To achieve the above object, the present invention adopts the following technical solution:

[0009] A method for selecting exposure values ​​for three-dimensional structured light measurement of surfaces with different reflectivity, characterized in that the method comprises: image acquisition to obtain a pre-test image;

[0010] Determine whether the collected pre-test image contains an overexposed area, if not, stop, if yes, select multiple exposures;

[0011] The selection of multiple exposures includes: S1, initialization of exposure; S2, selecting multiple exposures using a pixel filter; S3, determining a next exposure value by optimization until the ratio of qualified pixels in the pre-test image is greater than or equal to 95%.

[0012] Its further characteristic is that

[0013] Before image acquisition, use the default exposure value t default , collecting a frame of image as the pre-test image;

[0014] Further, it is determined whether the collected pre-test image contains an overexposed area, specifically: if the ratio of the number of pixels in the overexposed area to the total number of pixels in the image is ≥ T1, it indicates overexposure, and multiple exposure selection is entered;

[0015] Further, the multiple exposure selection steps include:

[0016] S1. Exposure value initialization includes: gradually adjusting the exposure value t default , so that the exposure value t default Reduce, wherein the initial exposure value after reduction is t1;

[0017] Further, S2, using a pixel filter to select multiple exposures, the multiple exposures refer to reduced exposure values, and the reduced initial exposure values ​​include: t1, t2, ..., t k+1 , where k≥1 collects the exposure values ​​t1, t2, ..., t k+1 The corresponding sequence images under each condition;

[0018] Further, S3, determining the next exposure value by optimization, including: S31 when the kth exposure selection, the exposure value is t k+1 At this time, 12 frames of phase-encoded image sequence I are collected k ;

[0019] S32, through the four-step phase shift method and the phase encoding image sequence I k Calculate the intensity modulation map B k ;

[0020] S33, traversing the intensity modulation graph B k , generate a binary mask image M k , the binary mask image M k is greater than or equal to the threshold T new The number of pixels is taken as the qualified pixel number;

[0021] S34, determine whether the ratio of the number of qualified pixels is ≥ T2, if so, stop iteration, and set the exposure values ​​to t1, t2, ..., t kThe sequence of images collected at the same time are fused to achieve point cloud reconstruction. If not, proceed to S4 to S5;

[0022] S4, the binary mask image M k and the phase-encoded image sequence I k Multiply the corresponding elements of to get 12 frames of filtered updated images

[0023] S5, the updated image The pixels in are used to calculate the image quality index At the same time, the Newton optimization method is used to determine the next exposure value t k+1 , repeat steps S2 to S3 until the ratio of qualified pixels in the pre-test image is greater than or equal to 95%.

[0024] Further, the T1 is 5%;

[0025] Furthermore, the threshold T new The value of is 9.

[0026] Further, in step S5,

[0027]

[0028]

[0029]

[0030] Where L and H are the length and width of the image, respectively. The activation function Ψ(A(x, y))∈[0, 1) controls the weight of the image. Θ(B(x, y))∈[0, 1) is the modulation of B(x, y). λ∈(0, 1] is the drop coefficient used to control the convergence speed. k is an integer.

[0031] Furthermore, the λ=0.3.

[0032] The above structure of the present invention can achieve the following beneficial effects: the present application also proposes a multiple exposure selection strategy to measure those pixels that cannot be covered by a single optimal exposure, which realizes the selection of multiple exposure values ​​by repeatedly executing steps S1 to S3 until most pixels (i.e., the ratio of qualified pixels in the pre-test image is greater than or equal to 95%) are calculated, thereby determining the optimal exposure value. The exposure value selection method is simple. Compared with the HDR method of 15 exposures, the method of the present application only requires 3 to 4 exposures on average, and can obtain similar surface coverage and measurement accuracy, thereby saving exposure selection time and improving measurement efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Included: (a) is an image of an industrial component (automobile stamping part) with a large variation in surface reflectivity;

[0034] Figure 1 (b) is the 3D reconstruction of the image in (a);

[0035] Figure 1 (c) is the effect of high dynamic range (HDR) combined with different reflectivity features of images taken at multiple exposures;

[0036] Figure 1 (d) is the image taken with appropriate exposure in (c) and the point cloud effect (inset) after the image in (c) is reconstructed using the HDR method.

[0037] Figure 2 This is a flow chart of the method for determining the optimal exposure value that is currently commonly used.

[0038] Figure 3 This is a flow chart of the multiple exposure value selection method proposed by the present invention.

[0039] Figure 4 The renderings of industrial parts under different exposure conditions determined by the currently commonly used method for determining the optimal exposure value and the example diagram of the reconstructed point cloud are shown in the figure. Figure 4 (a)(b)(c)(d) are the renderings of industrial parts taken under different exposure conditions. Figure 4 (e) is an example of a point cloud after the images taken at the four exposure values ​​are fused and reconstructed.

[0040] Figure 5 The images of industrial parts (automobile stamping parts) taken under different exposure values ​​calculated by the method of the present invention and the point cloud examples after point cloud reconstruction are shown, wherein: Figure 5 (a), (b), (c), and (d) are effect diagrams of industrial parts taken at different exposure values ​​obtained by the method of the present invention. Figure 5 (e) is an example of a point cloud after the images taken under the four exposure values ​​are fused and reconstructed.

[0041] Figure 6 This is a comparative example diagram of the method of the present invention and the HDR method, wherein: Figure 6 (a) is the image of the industrial component to be photographed. Figure 6 (b) is an example of an industrial component point cloud after image fusion and reconstruction using the HDR method with 15 exposures. Figure 6 (c) is an example of a point cloud of an industrial component obtained by image fusion and reconstruction under the conditions of four exposure values ​​calculated by the method of the present invention. DETAILED DESCRIPTION

[0042] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0043] Figure 2 A commonly used method for determining the optimal exposure value is given. The method includes image acquisition: K1, when the exposure value is t k When collecting sequence images I k ; K2, determine whether the quality of the sequence image is qualified, if so, K3, then directly reconstruct the 3D point cloud, if not, K4, then calculate the exposure value t by maximizing the Q value k+1 , return to step K1, and when the exposure value is t k+1 When collecting sequence images I k+1 , determine whether the image quality is qualified, if not, repeat steps K4 to K2 until the image quality is qualified, where the Q value of K4 is calculated as follows:

[0044] The above method is used to Figure 1 The image shown in (a) is exposed and processed to obtain Figure 1 (b) shows the 3D point cloud reconstruction. Figure 1 (b) It can be seen that the corresponding Figure 1 Both overexposed and underexposed areas in (a) suffer from information loss, which affects the accuracy of subsequent pixel measurements.

[0045] In view of the above problems, the present invention provides a specific embodiment of a method for selecting an exposure value for three-dimensional structured light measurement of surfaces with different reflectivity, referring to Figure 3 , the method comprising:

[0046] Step 1: Image acquisition, obtain the pre-test image. The specific acquisition method is as follows: Before measurement, use the default exposure value t of the camera. default , collect a frame of image as a pre-test image.

[0047] Step 2: determine whether the collected pre-test image contains an overexposed area. If not, stop. If yes, select multiple exposures. Specifically: if the ratio of the number of pixels in the overexposed area to the total number of pixels in the image is ≥5%, it indicates overexposure, and enter step 3 to select multiple exposures.

[0048] Step 3: Multiple exposure selection includes: S1: initialization of exposure, including: gradually adjusting the exposure value t default , so that the exposure value t default Reduce, wherein the initial exposure value after reduction is t1;

[0049] S2. Select multiple exposures using a pixel filter. The multiple exposures refer to reduced exposure values. The reduced initial exposure values ​​include: t1, t2, ..., t k+1 , where k ≥ 1, collect exposure values ​​t1, t2, ... t k+1 The corresponding sequence images under each condition;

[0050] S3, determining the next exposure value by optimization; repeating steps S1 to S3 until the ratio of qualified pixels in the pre-test image is greater than or equal to 95%, specifically including: S31 at the kth exposure selection, the exposure value is t k+1 At this time, 12 frames of phase-encoded image sequence I are collected k ;

[0051] S32, through the four-step phase shift method and phase encoding image sequence I k Calculate the intensity modulation map B k ;

[0052] S33, traversal intensity modulation graph B k , generate a binary mask image M k , the binary mask image M k is greater than or equal to the threshold T new The number of pixels is taken as the qualified pixel number. In this embodiment, T new The experience value is 9;

[0053] S34, determine whether the ratio of qualified pixels is ≥ 95%, if so, stop iteration and set the exposure values ​​to t1, t2, ..., t k The sequential images collected at the same time are fused to achieve point cloud reconstruction. If not, proceed to S4 to S5;

[0054] S4, binary mask image M k and phase coded image sequence I k Multiply the corresponding elements of to get 12 frames of filtered updated images

[0055] S5. Update image The pixels in are used to calculate the image quality index At the same time, the Newton optimization method is used to determine the next exposure value t k+1 In step S5,

[0056]

[0057]

[0058]

[0059] Where L and H are the length and width of the image, respectively. The activation function Ψ(A(x, y))∈[0, 1) controls the weight of the image. Θ(B(x, y))∈[0, 1) is the modulation of B(x, y). λ∈(0, 1] is the drop coefficient, λ=0.3, which is used to control the convergence speed. Repeat the above steps S2 to S3 until the ratio of qualified pixels in the pre-test image is greater than or equal to 95%.

[0060] This application proposes a new image quality index Q new , in which an activation function Ψ(A(x, y))∈[0, 1) is introduced to redistribute the weight of intensity modulation. When the intensity modulation reaches a certain threshold, its weight is suppressed to avoid overexposure of the area. The present application also proposes a multiple exposure selection strategy to measure pixels that cannot be covered by a single optimal exposure, in which the proposed image quality index and a pixel filtering step S33 are used to calculate the next optimal exposure value, and the candidate pixels to be measured are located through step S4. When located through step S4, because the binary mask image is multiplied, the multiplied image is either 0 or 1. At this time, the value of 1 is the candidate pixel. This process is repeated until most pixels (that is, the ratio of qualified pixels in the pre-test image is greater than or equal to 95%) have been calculated.

[0061] Compared with the single optimal exposure method, the present application method improves the surface coverage, reduces the loss of detail features, and reduces the measurement error, such as Figure 4 , Figure 5 As shown, Figure 4 The effect pictures of industrial parts taken under different exposure conditions determined by the currently commonly used method for determining the optimal exposure value and the example pictures of the reconstructed point cloud are shown in the figure. Figure 4 (a)(b)(c)(d) are images of industrial parts with exposure times of 32ms, 41ms, 49ms, and 58ms, respectively. Figure 4 (e) is an example of a point cloud after the images captured at the four exposure times are fused and reconstructed. Figure 4(e) It can be seen that the 3D reconstruction image retains more detailed features of the industrial parts, but some features are still missing in the over-exposed and under-exposed areas, which affects the subsequent measurement accuracy.

[0062] Figure 5 The images of industrial parts taken under different exposure conditions calculated by the method of the present invention and the point cloud examples after point cloud reconstruction are shown in FIG. Figure 5 (a)(b)(c)(d) are images of industrial parts taken with exposure times of 16ms, 29ms, 52ms, and 80ms, respectively. Figure 5 (e) is an example of a point cloud after the images captured at the four exposure times are fused and reconstructed. Figure 5 (e) It can be seen that the 3D reconstruction image retains the detailed features of the industrial parts relatively completely, which is consistent with Figure 4 From the comparison of the point cloud images shown in (e), it can be seen that after adopting the method of the present application, the detailed features of the high-reflective areas of the industrial parts are retained, and the images taken under the optimal exposure value conditions determined by the multiple exposure value selection method of the present application can obtain higher surface coverage and measurement accuracy.

[0063] Compared with the 15-exposure HDR method, the present application method Figure 6 , Figure 6 This is a comparison example diagram of the method of the present invention and the HDR method, wherein: Figure 6 (a) is the physical image of the workpiece to be photographed. Figure 6 (b) is an example of point cloud reconstruction of industrial parts obtained using the optimal exposure value conditions calculated by the HDR method. Figure 6 (c) is an example of point cloud reconstruction of an industrial component obtained by using the optimal exposure value condition calculated by the method of the present invention, Figure 6 (b) with Figure 6 (c) By comparison, it can be seen that the method of the present application and the HDR method can achieve similar surface coverage and measurement accuracy, but the method of the present application only requires an average of 3 to 4 exposures to obtain the optimal exposure value, while the HDR method requires 15 exposures, thereby saving exposure selection time and improving measurement efficiency and accuracy.

[0064] The above are only preferred embodiments of the present application, and the present invention is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present invention should be considered to be included in the protection scope of the present invention.

Claims

1. A method for selecting exposure values ​​for three-dimensional structured light measurement of surfaces with different reflectivity, characterized in that: The method comprises: image acquisition, obtaining a pre-test image; using a default exposure value t default , collecting a frame of image as the pre-test image; Determine whether there is an over-exposure area in the pre-test image, if not, stop, if yes, select multiple exposures; The multiple exposure selection includes: S1, initializing exposure; gradually adjusting the exposure value t default , so that the exposure value t default Reduce, wherein the initial exposure value after reduction is t1; S2. Select multiple exposures using a pixel filter, where the multiple exposures refer to reduced exposure values, and the reduced initial exposure values ​​include: t1, t2, ..., t k+1 , where k≥1; collecting the exposure values ​​t1, t2, ..., t k+1 The corresponding sequence images under each condition; S3, determining the next exposure value by optimization until the ratio of qualified pixels in the pre-test image is greater than or equal to 95%; specifically comprising: S31, when the kth exposure selection is performed, the exposure value is t k+1 At this time, 12 frames of phase-encoded image sequence I are collected k ; S32, through the four-step phase shift method and the phase encoding image sequence I k Calculate the modulation intensity map B k ; S33, traversing the modulation intensity graph B k , generate a binary mask image M k , the binary mask image M k is greater than or equal to the threshold T new The number of pixels is taken as the qualified pixel number; S34, judging whether the ratio of the number of qualified pixels is ≥ T2, if so, stopping the iteration, and setting the exposure values ​​to t1, t2, ..., t k If the image sequence is not acquired, the image fusion is performed to realize point cloud reconstruction; if not, S4 is executed; S4, the binary mask image M k and the phase-encoded image sequence I k Multiply the corresponding elements of to get 12 frames of filtered updated images S5, the updated image The pixels in are used to calculate the image quality index At the same time, the Newton optimization method is used to determine the next exposure value t k+1 , repeat steps S2 to S3 until the ratio of qualified pixels in the pre-test image is greater than or equal to 95%.

2. The exposure value selection method for three-dimensional structured light measurement of surfaces with different reflectivity according to claim 1, characterized in that: Determining whether there is an overexposed area in the pre-test image specifically includes: if the ratio of the number of pixels in the overexposed area to the total number of pixels in the image is ≥ T1, it indicates overexposure, and multiple exposure selection is entered.

3. According to the method for selecting exposure values ​​for three-dimensional structured light measurement of surfaces with different reflectivity according to claim 2, in step S5, the image quality index The calculation method is: in, L, H are the length and width of the image respectively, and the activation function Ψ(A(x,y))∈[0,1) controls the weight of the image. Yes modulation, To update the image λ∈(0,1] is the descent coefficient, which is used to control the speed of convergence.

4. The exposure value selection method for three-dimensional structured light measurement of surfaces with different reflectivity according to claim 3, wherein T1 is 5%, and the threshold T new The empirical value of is 9, and λ=0.3.

Images