High dynamic three-dimensional measurement method based on adaptive distribution of overexposed connected domain intensity

Abstract

The application discloses a kind of high dynamic three-dimensional measurement method based on overexposure connected domain projection intensity adaptive distribution, generates maximum input gray chart and sinusoidal stripe chart, projection to the surface of measured object and synchronous acquisition;According to threshold, overexposure area is identified from the maximum input gray chart collected, and the boundary of overexposure area is extracted and tracked;Combining the parameters obtained by pre-calibration and the sinusoidal stripe collected, the object is preliminarily three-dimensionally reconstructed;The coordinates of boundary are mapped to projector pixel coordinate system and tracked, and are automatically connected to form closed boundary;Binary mask is generated from closed boundary, and the maximum input gray level in closed boundary is reduced to regenerate maximum input gray chart and sinusoidal stripe;Maximum input gray chart and sinusoidal stripe are projected onto the object again and collected, until the maximum input gray chart collected has no overexposure area, and the complete three-dimensional of the object is reconstructed using the corresponding sinusoidal stripe.The application can realize three-dimensional measurement of high dynamic range object.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical measurement, in particular to a high dynamic three-dimensional measurement method based on adaptive distribution of overexposure connected domain intensity. BACKGROUND

[0002] With the development of digital technology, three-dimensional measurement technology has more and more extensive application requirements in various fields. Among them, optical three-dimensional measurement methods are various, which can be roughly divided into multi (double) stereo vision, time-of-flight method, structured light projection method, etc. The shape of the object surface can be flexibly and efficiently perceived without physical contact. Among them, the structured light fringe projection method is considered to be one of the most promising technologies. It projects a pre-designed sinusoidal fringe pattern onto the object through a computer-controlled projector, and a camera captures the corresponding deformed fringe and transmits it to the computer. Then the computer extracts the phase information from the captured fringe image, and finally combines the parameters obtained by pre-calibrating the system to realize the measurement of the real three-dimensional object.

[0003] The fringe projection technology has been applied in many aspects, but when measuring high-reflectivity areas of the object, overexposure phenomenon is easy to occur. Although the aperture or exposure time can be changed to avoid image saturation, the intensity modulation of low-reflectivity areas will be reduced, resulting in poor measurement accuracy. In the current high dynamic three-dimensional measurement method, the technology based on multiple exposures changes the exposure time multiple times, collects deformed fringe images of the same scene multiple times, and fuses to generate a high dynamic range composite fringe image for three-dimensional reconstruction. However, this method requires the collection of a large number of images, resulting in a long overall measurement time and complicated operation. The three-dimensional measurement of high-reflectivity areas is still a challenging problem. SUMMARY

[0004] The purpose of the present application is to provide a high dynamic three-dimensional measurement method based on adaptive distribution of overexposure connected domain intensity, which can realize three-dimensional measurement of high dynamic range objects without changing the camera aperture and exposure time.

[0005] In order to achieve the above purpose, the present application provides the following technical scheme: a high dynamic three-dimensional measurement method based on adaptive distribution of overexposure connected domain intensity, the specific steps are as follows:

[0006] Step 1: using a computer to generate a maximum input grayscale image and a four-step phase shift sinusoidal fringe pattern, using a monocular fringe projection system to project the maximum input grayscale image and the four-step phase shift sinusoidal fringe pattern onto the surface of the measured object and synchronously collecting the sinusoidal fringe pattern and the maximum input grayscale image;

[0007] Step 2: identifying the connected overexposure area from the collected maximum input grayscale image, extracting the boundary of the overexposure area in the camera pixel coordinate system and tracking;

[0008] Step 3: Combine the parameters obtained from system calibration and the continuous phase obtained from the captured sinusoidal fringe to perform preliminary three-dimensional reconstruction of the object;

[0009] Step 4: Map the closed boundary coordinates in the camera pixel coordinate system to the projector pixel coordinate system, track the coordinates of the closed boundary in the projector pixel coordinate system, and automatically connect to form a closed boundary;

[0010] Step 5: Generate a binary mask from the closed boundary, reduce the maximum input grayscale level inside the closed boundary, and regenerate the maximum input grayscale graph and the intensity-adjusted sinusoidal fringe;

[0011] Step 6: Project the maximum input grayscale graph and the sinusoidal fringe onto the object again and capture, repeat steps 2-5 until there is no overexposure area in the captured image of the projected maximum input grayscale graph, and use the continuous phase obtained from the sinusoidal fringe to reconstruct the complete three-dimensional of the object.

[0012] Preferably, the computer-generated four-step phase-shift sinusoidal grating fringe in step 1 is The optical expression is as follows:

[0013]

[0014] Where n = 0, 1, 2, 3 is the phase shift index, (u p ,v p ) is the projector pixel coordinate, M migl is the maximum input grayscale graph, a p represents the average intensity, b p represents the amplitude, is the frequency of the projected sinusoidal fringe.

[0015] Preferably, the maximum input grayscale graph captured in step 2 is represented as I c (u c ,v c ), (u c ,v c ) is the camera pixel coordinate, according to a given threshold thr, greater than or equal to the threshold is marked as overexposure, and the overexposed area is marked by a binary image ,

[0016]

[0017] The binary image is subjected to connected component analysis, each connected component in the image is found and marked, and smaller connected components are removed, the Canny operator is used to extract the boundary of the remaining saturated pixel area, and the mask space of the boundary is obtained

[0018] Preferably, the continuous phase map obtained in step 3 is Φ(u c ,v c ), and the horizontal position of the continuous phase map in the pixel coordinate system of the projector is calculated according to the rule of phase distribution as follows:

[0019]

[0020] U is the resolution in the u direction of the projection space, and T is the period of the stripe in the projection space.

[0021] Combined with the conversion relationship of the horizontal position of the continuous phase map in the pixel coordinate system of the projector, the coordinates (X c ,Y c ,Z c ) in the camera coordinate system to the pixel coordinates (u p ,v p ) under the projector, and the conversion relationship of the coordinates (X c ,Y c ,Z c ) in the camera coordinate system to the camera pixel coordinates (u c ,v c ), the three-dimensional (X c ,Y c ,Z c ) of the object is preliminarily calculated.

[0022] Preferably, the parameters obtained by the system calibration in step 3 include the camera intrinsic parameters A c , the extrinsic parameters R c and t c , the projector intrinsic parameters A p , the extrinsic parameters R p and t p .

[0023] The conversion process of the coordinates (X c ,Y c ,Z c ) in the camera coordinate system to the pixel coordinates (u p ,v p ) under the projector is as follows:

[0024]

[0025] Let,

[0026]

[0027] After simplification, the conversion process is as follows:

[0028]

[0029] Integration gives:

[0030]

[0031] The conversion process of the coordinates (X c ,Y c ,Z c ) in the camera coordinate system into the camera pixel coordinates (u c ,v c ) is as follows:

[0032]

[0033]

[0034]

[0035] Preferably, according to the coordinates (u c ,v c ) of any point on the closed boundary in the camera pixel coordinate system, the pixel coordinate u p under the projector and the three-dimensional coordinates (X c ,Y c ,Z c ) of the point calculated initially are obtained, and the pixel coordinate v p under the projector is obtained as follows:

[0036]

[0037] Preferably, in step 5, the inside of the closed boundary in the projector pixel coordinate system is taken as 1 and the outside is taken as 0 to generate a binary mask as The maximum input grayscale image is regenerated as follows:

[0038]

[0039] i represents the number of iterations, and r represents the reduced grayscale level of the overexposed area.

[0040] Compared with the prior art, the present application has the following advantages:

[0041] The present application identifies the overexposed area through the collected maximum input grayscale image, reduces the maximum input grayscale level multiple times locally, and performs three-dimensional reconstruction after the final collected maximum input grayscale image is free of overexposed areas, thereby solving the problem of high light in the prior art. The present application does not need to adjust the exposure time multiple times to collect images for fusion, reduces the number of collected images, and simplifies the operation process. BRIEF DESCRIPTION OF DRAWINGS

[0042] ​In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly describe the drawings needed in the embodiments. The drawings are only some embodiments, and other drawings can also be obtained by those skilled in the art without creative labor under the premise of not paying creative labor.

[0043] Figure 1 The flow chart of the high dynamic three-dimensional measurement method based on overexposure connected domain projection intensity adaptive distribution of the present application.

[0044] Figure 2 The monocular stripe projection system imaging model in the embodiment of the present application.

[0045] Figure 3 The circular dot plane plate used in the embodiment of the present application.

[0046] Figure 4 The contrast before and after the projection intensity adaptive distribution in the embodiment of the present application, wherein (a) is the initial projection high-frequency stripe image, and (b) is the high-frequency stripe image after 3 iterations.

[0047] Figure 5 The reconstruction effect contrast before and after the projection intensity adaptive distribution in the embodiment of the present application, wherein (a) is the initial reconstruction result, and (b) is the reconstruction result after 3 iterations. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. Other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0049] A high dynamic three-dimensional measurement method based on overexposure connected domain intensity adaptive distribution, as shown in Figure 1 , the method comprises:

[0050] Step 1: Build a monocular imaging system, and the imaging model is as shown in Figure 2 .

[0051] In some embodiments, a white circular dot plane plate with large reflectivity change is used as the measured object, as shown in Figure 3 . Using computer-generated sinusoidal grating stripe image and maximum input gray scale, the monocular stripe projection system projects it onto the surface of the measured object and synchronously collects the corresponding image; the computer-generated four-step phase-shift sinusoidal grating optical expression is as follows,

[0052]

[0053] Image pixel depth is 8 bits, where n = 0, 1, 2, 3 is the phase shift index, (u p ,v p ) is the projector pixel coordinate, M migl is the maximum input grayscale image, a p represents the average intensity, b p represents the amplitude, a p = b p = 0.5, is the frequency of the projected sinusoidal fringe, and if the generated vertical fringe is represented as T shows the period of the projected space fringe, and U represents the resolution in the u direction of the projected space in pixels. The initial high-frequency fringe pattern is shown in Figure 4 (a).

[0054] Step 2: Identify the connected overexposed area from the collected maximum input grayscale image according to the threshold value, extract the boundary of the overexposed area in the camera pixel coordinate system and track it.

[0055] The collected maximum input grayscale image can be represented as I c (u c ,v c ), (u c ,v c ) is the projector pixel coordinate, according to the threshold value thr = 250, the pixel value greater than or equal to the threshold value is the overexposed area, and the overexposed area is identified by a binary image ,

[0056]

[0057] The binary image is analyzed for connected domains, each connected domain in the image is found and labeled, and smaller connected domains are removed. The Canny operator is used to extract the boundary of the remaining saturated pixel area to obtain the mask space

[0058] Step 3: Combine the parameters obtained by pre-calibration of the system and the sinusoidal grating stripes collected in step 1 to perform three-dimensional reconstruction of the object; the parameters obtained by pre-calibration of the system include the camera intrinsic parameters A c and extrinsic parameters R c and t c , the projector intrinsic parameters A p and extrinsic parameters R p and t p , the process of converting the coordinates (X c , Y c , Z c ) in the camera coordinate system to the pixel coordinates (u p , v p ) in the projector is:

[0059]

[0060] Let,

[0061]

[0062] Equation (3) simplifies the conversion process to,

[0063]

[0064] Further integration of equation (4) can be obtained,

[0065]

[0066] The process of converting the coordinates (X c ,Y c ,Z c ) in the camera coordinate system to the camera pixel coordinates (u c ,v c ) is,

[0067]

[0068] It can be obtained,

[0069]

[0070] The continuous phase diagram obtained can be recorded as Φ(u c ,v c ), according to the law of phase distribution, the horizontal position of the projector in the pixel coordinate system can be calculated as,

[0071]

[0072] Combining equation (5), (7) and equation (8), the three-dimensional (X c ,Y c ,Z c ) of the object is preliminarily calculated. Thus, the initial three-dimensional of the white dot plane panel is obtained as shown in Figure 5 (a).

[0073] Step 4: Map the closed boundary coordinates in the camera pixel coordinate system to the projector pixel coordinate system, track the coordinates of the closed boundary in the projector pixel coordinate system, and automatically connect to form a closed boundary;

[0074] Map the coordinates (u c ,v c ) of a point of the closed boundary in the camera pixel coordinate system obtained in step 2 to the projector pixel coordinate u p Equation (8) is used, and since the three-dimensional (Xc Y c Z c ), according to equation (4), its mapping to the projector pixel coordinate v p is,

[0075]

[0076] Thus the boundary in step 2 can be mapped to the projector pixel coordinate system one by one, and then the closed boundary is obtained by automatic connection

[0077] Step 5: Generate binary mask from the closed boundary The inside of the closed boundary in the projector pixel coordinate system is taken as 1, and the outside is taken as 0, and then the binary mask generated is The generated maximum input grayscale map is represented as,

[0078]

[0079] Here i represents the number of iterations, r represents the reduced gray level of the overexposed area, and the gray level is taken as 60. Further reduce the maximum input gray level inside the closed boundary, and regenerate the maximum input grayscale map and the sinusoidal grating fringe.

[0080] Step 6: Project the maximum input grayscale map and the sinusoidal grating fringe onto the object again and collect, execute steps 2 to 5, and so on for 3 iterations, and the newly generated high-frequency fringe is as shown in Figure 4 (b). The last projected maximum input grayscale map collected picture has no overexposed area, and the continuous phase obtained by the sinusoidal grating fringe is used to finally calculate the complete three-dimensional of the measured object by equations (5), (7) and (8), as shown in Figure 5 (b).​

Claims

1. A high-dynamic three-dimensional measurement method based on adaptive distribution of overexposed connected component intensity, characterized in that, The specific steps are as follows: Step 1: Use a computer to generate a maximum input grayscale image and a four-step phase-shift sinusoidal fringe pattern. Use a monocular fringe projection system to project the maximum input grayscale image and the four-step phase-shift sinusoidal fringe pattern onto the surface of the object being measured, and simultaneously acquire the sinusoidal fringe pattern and the maximum input grayscale image; the computer-generated four-step phase-shift sinusoidal grating fringes... The optical expression is as follows: Where n = 0, 1, 2, 3 are the phase shift indices, (u p ,v p M represents the pixel coordinates of the projector. migl For the largest input grayscale image, a p b represents the average intensity. p Indicates amplitude, It is the frequency of the projected sinusoidal fringes; Step 2: Identify connected overexposed regions from the acquired maximum input grayscale image, extract and track the boundaries of the overexposed regions in the camera pixel coordinate system; the acquired maximum input grayscale image is represented as I. c (u c ,v c ), (u c ,v c () represents the camera pixel coordinates. Based on a given threshold thr, values ​​greater than or equal to the threshold are considered overexposed, and this is verified using a binary image. Mark and identify overexposed areas. For binary images Connectivity analysis is performed to identify and label all connected components in the image. Smaller connected components are removed, and the Canny operator is used to extract the boundaries of the remaining saturated pixel regions to obtain the boundary mask space. Step 3: Combine the parameters obtained from system calibration and the continuous phase obtained from the acquired sinusoidal fringes to perform preliminary three-dimensional reconstruction of the object; Step 4: Map the closed boundary coordinates in the camera pixel coordinate system to the projector pixel coordinate system, track the coordinates of the closed boundary in the projector pixel coordinate system, and automatically connect them to form a closed boundary. Step 5: Generate a binary mask from the closed boundary, reduce the maximum input gray level inside the closed boundary, and regenerate the maximum input gray image and intensity-adjusted sine stripes; Step 6: Project the maximum input grayscale image and sine stripes onto the object again and collect the image. Repeat steps 2 to 5 until the image collected by projecting the maximum input grayscale image has no overexposed areas. Reconstruct the complete three-dimensional shape of the object using the continuous phase obtained from the sine stripes.

2. The high dynamic three-dimensional measurement method based on adaptive distribution of overexposed connected component intensity according to claim 1, characterized in that, The continuous phase map obtained in step 3 is Φ(u c ,v c Based on the phase distribution pattern, the horizontal position of the continuous phase map in the projector's pixel coordinate system is calculated as follows: U is the resolution in the u-direction of the projection space, and T is the period of the fringe in the projection space; Combining the horizontal position of the continuous phase map in the projector's pixel coordinate system and its coordinates in the camera coordinate system (X... c ,Y c Z c ) to pixel coordinates under the projector (u p ,v p The transformation relationship between the camera coordinate system and the coordinates in the camera coordinate system (X) c ,Y c Z c (u) to camera pixel coordinates (u) c ,v c The transformation relationship between X and Y is used to initially calculate the three dimensions (X, Y, X) of the object. c ,Y c Z c ).

3. The high dynamic three-dimensional measurement method based on adaptive distribution of overexposed connected component intensity according to claim 1, characterized in that, The parameters obtained from the system calibration in step 3 include camera intrinsic parameter A. c and external parameters R c and t c Projector internal parameters A p and external parameters R p and t p ; Camera coordinates (X) c ,Y c Z c ) to pixel coordinates under the projector (u p ,v p The conversion process is as follows: make, The simplified transformation process of the above formula is as follows: The results of integration are as follows: Camera coordinates (X) c ,Y c Z c (u) to camera pixel coordinates (u) c ,v c The conversion process is as follows: have to, 4. The high dynamic three-dimensional measurement method based on adaptive distribution of overexposed connected component intensity according to claim 3, characterized in that, Based on the coordinates (u) of any point on the closed boundary in the camera pixel coordinate system c ,v c Mapped to pixel coordinates u on the projector p And the points (u) calculated in the preliminary calculation c ,v c The corresponding three-dimensional (X) c ,Y c Z c ), thus obtaining its pixel coordinates v mapped to the projector. p for:

5. The high dynamic three-dimensional measurement method based on adaptive distribution of overexposed connected component intensity according to claim 1, characterized in that, In step 5, a binary mask is generated by setting 1 to the inside and 0 to the outside of the closed boundary in the projector pixel coordinate system. Regenerate the maximum input grayscale image: i represents the number of iterations, and r represents the gray level reduction in the overexposed area.

Images