A 3D point cloud acquisition method for highly reflective workpieces based on adaptive stripes

Abstract

The present invention discloses a method for obtaining a three-dimensional point cloud of a highly reflective workpiece based on adaptive stripes, which comprises the following steps: Step 1, determining a saturation region, projecting a small number of grayscale sequence images, marking the saturation region and establishing a surface coefficient lookup table corresponding to saturated pixels at the pixel level; Step 2, generating low-intensity sine stripes, marking the points with the largest change in grayscale value in the grayscale sequence according to the grayscale sequence images, interpolating and fitting to generate a low-intensity orthogonal stripe pattern and projecting it; Step 3, generating adaptive stripes, establishing a pixel coordinate correspondence between the camera and the projector according to the low-intensity orthogonal stripes, establishing a coordinate mapping of the camera and the projector, looking up the surface coefficient table to obtain the optimal projection intensity of saturated pixels, generating adaptive stripes and projecting them; Step 4, three-dimensional reconstruction. It can effectively determine the point cloud missing region of highly reflective components and effectively solve the problem of difficult three-dimensional reconstruction in highly reflective regions.

Description

Technical Field

[0001] The present invention relates to the single - time three - dimensional point cloud acquisition of highly reflective workpieces, and specifically to a method for three - dimensional point cloud acquisition of highly reflective workpieces based on adaptive stripes. Background Art

[0002] Existing three - dimensional topography measurement methods mainly fall into two categories: One is contact measurement represented by coordinate measuring machines. This measurement method has high accuracy but is very time - consuming. Measuring large parts takes several hours or even a day. The other is non - contact measurement represented by fringe projection technology, which has high accuracy, fast measurement speed, and is suitable for on - line measurement. However, the fringe projection technology has poor robustness in measuring objects with large surface reflectivity changes. For example, when simultaneously measuring objects with darker and brighter surfaces or complex curved surfaces with high - reflectivity surfaces, in the case of ensuring a relatively high modulation signal - to - noise ratio in darker regions, the high - reflectivity surface is likely to cause the camera response value to exceed the range of the sensor, resulting in image saturation and causing partial point cloud data loss. The three - dimensional reconstruction of the high - reflective area will directly affect the final measurement accuracy. The three - dimensional topography measurement of high - reflective surface components is one of the difficult problems in the field of optical three - dimensional measurement. Therefore, this paper conducts research and analysis on the three - dimensional reconstruction of highly reflective workpieces based on the adaptive stripe algorithm. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for three - dimensional point cloud acquisition of highly reflective workpieces based on adaptive stripes, which can effectively determine the point cloud missing area of highly reflective components and effectively solve the problem of difficult three - dimensional reconstruction in high - reflective areas.

[0004] The method for three - dimensional point cloud acquisition of highly reflective workpieces based on adaptive stripes according to the present invention includes the following steps:

[0005] Step 1, determining the saturated area of the measured component, including:

[0006] Step 1.1, projecting and collecting an image M with a uniform gray value of 255 255 (x,y);

[0007] Step 1.2, binarizing the image M with a uniform gray value of 255 projected and collected 255 (x,y), and recording the saturated area mask matrix as Q C (x,y);

[0008] Step 1.3, for the pixels in the saturated area, establishing a surface coefficient look - up table for a single pixel.

[0009] Step 2, generating low - intensity orthogonal stripes, including:

[0010] Step 2.1, marking the gray - level image sequence I collected by the camera ckThe point with the largest change in gray value is denoted as (x g , y g );

[0011] Step 2.2, interpolate and predict the best projection intensity of the fitting point (x g , y g ) according to the gray scale sequence.

[0012] Step 2.3, generate low-intensity orthogonal mapping stripes;

[0013] Step 3 Generate adaptive stripes, including:

[0014] Step 3.1, establish the coordinate mapping between the camera and the projector according to the low-intensity orthogonal mapping stripes;

[0015] Step 3.2, search the surface coefficient table to solve the best projection intensity of the pixels in the saturated area;

[0016] Step 3.3, generate adaptive stripes;

[0017] Step 4, 3D reconstruction: Perform 3D reconstruction according to the collected adaptive stripes in combination with the camera-projector joint calibration results.

[0018] Furthermore, Step 1.2 is specifically: Binarize the image M 255 (x, y) with a uniform gray value of 255 after projection and acquisition to determine the saturated area of the highly reflective workpiece under the current system pose. The gray value M 255 (x, y) greater than 250 is marked as 1, and the rest are marked as 0. Denote the mask matrix as Q C (x, y), and its calculation formula is:

[0019] Furthermore, Step 1.3 is specifically: Based on the saturated area marked in Step 1.2, calculate the ktr(x ) of each pixel point in the saturated area according to the calculation formula c , y c ), where I c (x c , y c ) is the gray value of a certain pixel point (x c , y c ) on the image collected by the camera, and I p (x p , y p ) is the gray value of the pixel point (x c , y c ) corresponding to the pixel point (x p , y p ) on the projected uniform gray image;

[0020] Define the surface coefficient α(x c , y c ), and let ktr(x c , y c ) = α(x c , y c ), and establish the corresponding surface coefficient lookup table.

[0021] Furthermore, step 2.1 is specifically as follows: First, find the corresponding saturated pixel region according to the saturation region mask matrix Q C (x, y), and sequentially traverse the saturated pixel regions in the collected gray-scale sequence I ck in order of decreasing gray-scale value. k is the number of the collected image sequence. If there is still saturation in the mask matrix Q ck in a certain gray-scale image I k (x, y) region, while there are no saturated pixels in the saturation region mask matrix Q c(k-1) in I k (x, y), mark the point with the largest change in gray-scale value, denoted as (x g , y g ).

[0022] Furthermore, step 2.2 is specifically as follows: According to the point (x g , y g ) with the largest change in gray-scale value, take the gray-scale value I ck corresponding to the coordinates (x g , y g ) in the gray-scale image sequence I c (x g , y g ), look up the surface coefficient table obtained in step 1.3, obtain the surface coefficient α(x g , y g ) of the point (x g , y g ), and calculate the optimal projection intensity of the low-intensity stripe

[0023] Furthermore, step 2.3 is specifically as follows: The computer generates low-intensity orthogonal stripe images with stripe periods of 90, 99, and 100, twelve vertical stripes and twelve horizontal stripes each, and its formula is:

[0024]

[0025] In the formula: (x p , y p ) represents the pixel coordinates of the projector image plane, represents the optimal projection intensity of the point (x g , y g ), represents the point (x g , yg ) The encoded phase value at the pixel position; N represents the number of phase-shifting steps of the sine fringe image; I p represents the gray value at the (x, y) position.

[0026] Further, step 3.2 is specifically: According to the surface coefficient look-up table established in step 1.3, through the mask matrix M c (x c , y c ) Obtain the gray value corresponding to the gray value sequence at the saturated pixel (x c , y c ). According to the formula

[0027] and

[0028]

[0029] According to the collected gray image sequence I ck , considering the non-linear response of the camera and the projector, use cubic B-spline interpolation to calculate the optimal projection intensity (MIGL(x p , y p )) for each pixel in the saturated pixel region.

[0030] Further, step 3.3 is specifically: According to the optimal projection intensity (MIGL(x p , y p )) of a single pixel in the saturated region obtained in step 3.2, through the coordinate mapping of the camera and the projector established by the low-intensity orthogonal fringes and the mask matrix M c (x c , y c ) Determine the projector pixel M p (x p , y p ) that needs to be adjusted, and generate an adaptive fringe according to the following formula:

[0031]

[0032] In the formula: (x p , y p ) represents the pixel coordinates of the projector image plane, A represents the background gray value, B represents the modulation degree of the sine fringe image, represents the encoded phase value at this pixel position; N represents the number of phase-shifting steps of the fringe pattern; MIGL(x p , y p ) is the optimal projection intensity at the pixel position of the point (x p , y p ), represents the intensity of the generated adaptive fringe.

[0033] The present invention has the following beneficial effects compared with the prior art:

[0034] 1. The present invention adaptively determines a low-intensity orthogonal sine stripe pattern through a uniform gray-scale sequence image, without relying on empirical values or manual settings, nor on complex dichotomy iteration. At the same time, considering the influence of ambient light and noise, a more accurate projection intensity model for the saturation region is established, ensuring the accuracy of the mapping of camera pixels and projector pixels in the saturation region of the component to be measured.

[0035] 2. The present invention determines the saturation region by projecting a 255 gray-scale image, establishes an accurate surface coefficient model for a single pixel in the saturation region through a uniform gray-scale sequence image, stores the surface coefficient of each pixel point into the surface coefficient lookup table, and can complete the three-dimensional reconstruction of the highly reflective region with high accuracy by projecting fewer images.

[0036] 3. The present invention calculates the optimal projection intensity that more accurately ensures that the pixels in the saturation region are not overexposed by looking up the surface coefficient of each pixel point in the saturation region and adopting cubic B-spline interpolation considering the non-linear response curves of the camera and the projector, and obtains the optimal projection intensity value for the saturation region not to be overexposed accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flowchart of the method for obtaining the three-dimensional point cloud of a highly reflective workpiece based on adaptive stripes according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0039] See Figure 1 , the method for obtaining the three-dimensional point cloud of a highly reflective workpiece based on adaptive stripes shown, which includes the following steps:

[0040] Step 1, determining the saturation region of the component to be measured, including:

[0041] Step 1.1, projecting a uniform gray-scale image sequence I pi ; First project a uniform gray-scale image with I max =255, and then project a uniform gray-scale image sequence, I pi =240 - G×(i - 1), and the gray-scale image sequence collected by the camera is I ck =(x c ,yc ), where i is the grayscale image sequence number, i = 1, 2, …, N; G is the grayscale difference between adjacent grayscale image sequences; k is the acquired image sequence number, k = 1, 2, …, N, N + 1; when k = 1, the acquired image is I max , (x c , y c ) is the image coordinate on the camera image plane.

[0042] Step 1.2, Binarize the image M with a uniform grayscale value of 255 obtained by projection and acquisition, (x, y), to determine the saturation area of the highly reflective workpiece under the current system pose. The grayscale value M(x, y) greater than 250 is marked as 1, and the rest are marked as 0. Denote the mask matrix as Q(x, y), and its calculation formula is: 255 (x, y) binarization, determine the saturation area of the high-reflectivity workpiece under the current system pose, the gray value M 255 (x, y) greater than 250 is marked as 1, and the rest are marked as 0. Denote the mask matrix as Q C (x, y), and its calculation formula is:

[0043] Step 1.3, For the pixels in the saturation area, establish a surface coefficient lookup table for a single pixel. Specifically: According to the saturation area marked in Step 1.2, based on the calculation formula calculate the ktr(x c , y c ) of each pixel point in the saturation area. In the formula, I(x c (x c , y c ) is the gray value of a certain pixel point (x c , y c ) on the camera-acquired image, and I(x p (x p , y p ) is the gray value of the pixel point (x c , y c ) corresponding to the pixel point (x p , y p ) on the projected uniform gray image; Define the surface coefficient α(x c , y c ), and let ktr(x c , y c ) = α(x c , y c ), and establish the corresponding surface coefficient lookup table.

[0044] Step 2 Generate low-intensity orthogonal fringes, including:

[0045] Step 2.1, Mark the point with the largest change in gray value in the grayscale image sequence I ck acquired by the camera, denoted as (x g , y g ). Specifically: First, according to the saturation area mask matrix Q C(x, y) finds the corresponding saturated pixel region and traverses the collected gray level sequence I in descending order of gray level values ck in the saturated pixel region, k is the number of the collected image sequence. If a certain gray level image I ck in the mask matrix Q k (x, y) region is still saturated, while in I c(k-1) the mask matrix Q of the saturated region k (x, y) has no saturated pixels, mark the point with the largest change in gray level value, denoted as (x g , y g ).

[0046] Step 2.2, predict and fit the best projection intensity of the point (x g , y g ) according to the gray level sequence interpolation Specifically: according to the point (x g , y g ), take the gray level value I ck in the gray level image sequence I corresponding to the coordinates (x g , y g ), look up the surface coefficient table obtained in Step 1.3, obtain the surface coefficient α(x c (x g , y g ), and calculate the best projection intensity of the low-intensity stripe g , y g ), calculate the best projection intensity of the low-intensity stripe g , y g )

[0047] Step 2.3, generate low-intensity orthogonal mapping stripes. Specifically: the computer generates low-intensity orthogonal stripe images with stripe periods of 90, 99, and 100, twelve vertical stripes and twelve horizontal stripes each, and its formula is:

[0048]

[0049] In the formula: (x p , y p ) represents the pixel coordinates of the projector image plane, represents the best projection intensity of the point (x g , y g ), represents the encoded phase value at the pixel position of the point (x g , y g ); N represents the number of phase steps of the sine stripe image; I p represents the gray level value at the (x, y) position.

[0050] Step 3 Generate adaptive stripes, including:

[0051] Step 3.1: Establish the coordinate mapping between the camera and the projector based on the low-intensity orthogonal mapping fringes. Specifically, obtain the absolute phase map of the measured object using the multi-frequency heterodyne algorithm, and find the coordinates of the overexposed area pixels in the projector's field of view. The mask matrix of the projector corresponding to the saturated area pixels of the camera is M p (x c ,y c ), and its calculation formula is:

[0052]

[0053]

[0054] In the formula: is the sum of the absolute phases in the horizontal direction is the sum of the absolute phases in the vertical direction, f is the number of cycles of the sinusoidal grating fringes; W and H are the width and height of the resolution of the sinusoidal fringe image respectively, and u and v are the column coordinate and row coordinate of the point (x c ,y c ) on the camera image plane corresponding to the projector image plane.

[0055] Step 3.2: Search the surface coefficient table to solve the optimal projection intensity of the saturated area pixels. Specifically, according to the surface coefficient lookup table established in Step 1.3, obtain the gray value corresponding to the gray level sequence at the saturated pixel (x c (x c ,y c ) through the mask matrix M c ,y c ). According to the formulas

[0056] and

[0057]

[0058] Considering the non-linear responses of the camera and the projector, use cubic B-spline interpolation to calculate the optimal projection intensity of each pixel in the saturated pixel area (MIGL(x ck , considering the acquired gray image sequence I p ,y p ))).

[0059] Step 3.3: Generate adaptive fringes. Specifically, according to the optimal projection intensity (MIGL(x p ,y p )) of a single pixel in the saturated area obtained in Step 3.2, through the coordinate mapping between the camera and the projector established by the low-intensity orthogonal fringes and the mask matrix M c (x c ,y c)Determine the projector pixel M to be adjusted p (x p ,y p ), generate an adaptive fringe according to the following formula:

[0060]

[0061] In the formula: (x p ,y p ) represents the pixel coordinates of the projector image plane, A represents the background gray value, B represents the modulation degree of the sine fringe image, represents the encoded phase value at this pixel position; N represents the number of phase steps of the fringe pattern; MIGL(x p ,y p ) is the optimal projection intensity at the pixel position of the point (x p ,y p ), represents the generated adaptive fringe intensity.

[0062] Step 4, 3D reconstruction: According to the collected adaptive fringes and combined with the camera-projector joint calibration results, perform 3D reconstruction.

[0063] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A three-dimensional point cloud acquisition method for highly reflective workpieces based on adaptive stripes, characterized in that, Including the following steps: Step 1, determining the saturation region of the element under test, including: Step 1.1, project and collect an image M with a uniform gray value of 255 255 (x, y); Step 1.2, binarize the image M with a uniform gray value of 255 obtained by projection and acquisition 255 at (x, y), and denote the saturation region mask matrix as Q C (x, y); Step 1.3, for the pixels in the saturation region, establishing a surface coefficient look-up table for a single pixel; Step 2, generating low-intensity orthogonal fringes, including: Step 2.1, mark the point with the largest change in grayscale value in the grayscale image sequence I collected by the camera, denoted as (x ck , y g , g ); Step 2.2, interpolate and predict the best projection intensity of the fitting point (x g , y g ) according to the gray scale sequence Step 2.3, generating low-intensity orthogonal mapped fringes; specifically: the computer generates low-intensity orthogonal fringe images with fringe periods of 90, 99, and 100, twelve vertical fringes and twelve horizontal fringes each, and its formula is: Where: (x p , y p ) represents the pixel coordinates of the projector image plane, represents the optimal projection intensity of the point (x g , y g ); represents the encoded phase value at the pixel position of the point (x g , y g ); N represents the number of phase-shifting steps of the sine fringe image; I p represents the gray value at the (x, y) position; Step 3, generating adaptive fringes, including: Step 3.1, establishing a coordinate mapping between the camera and the projector according to the low-intensity orthogonal mapped fringes; Step 3.2, looking up the surface coefficient look-up table to solve the optimal projection intensity of the pixels in the saturation region; Step 3.3, generating adaptive fringes; Step 4, three-dimensional reconstruction, performing three-dimensional reconstruction according to the collected adaptive fringes in combination with the camera-projector joint calibration results.

2. The three-dimensional point cloud acquisition method for highly reflective workpieces based on adaptive stripes according to claim 1, wherein, Step 1.2 specifically is: For the image M with a uniform gray value of 255 obtained by projection and acquisition 255 (x, y) is binarized to determine the saturated area of the highly reflective workpiece under the current system pose. The gray value M 255 (x, y) greater than 250 is marked as 1, and the rest are marked as 0. Denote the mask matrix as Q C (x, y), and its calculation formula is:

3. The three-dimensional point cloud acquisition method for highly reflective workpieces based on adaptive stripes according to claim 1 or 2, characterized in that, Step 1.3 specifically is: Based on the saturated regions marked in Step 1.2, according to the calculation formula ktr(x c ,y c ) of each pixel in the saturated region is calculated. In the formula, I c (x c ,y c ) is the gray value of a certain pixel (x c ,y c ) on the image collected by the camera, and I p (x p ,y p ) is the gray value of the pixel (x c ,y c ) corresponding to the pixel (x p ,y p ) on the projected uniform gray image; Define the surface coefficient α(x c , y c ), let ktr(x c , y c ) = α(x c , y c ), and establish the corresponding surface coefficient look-up table.

4. The method for obtaining a three-dimensional point cloud of a highly reflective workpiece based on adaptive stripes according to claim 1 or 2, characterized in that Step 2.1 specifically is: First, according to the saturation region mask matrix Q C (x, y), find the corresponding saturated pixel region, and sequentially traverse the collected gray level sequence I ck in the saturated pixel region. k is the number of the collected image sequence. If there is still saturation in the mask matrix Q ck in a certain gray level image I k (x, y) region, while there are no saturated pixels in the saturation region mask matrix Q c(k-1) in I k (x, y), mark the point with the largest change in its gray level value, denoted as (x g , y g ).

5. The method for obtaining a three-dimensional point cloud of a highly reflective workpiece based on adaptive stripes according to claim 4, wherein Step 2.2 specifically is: Based on the point (x g , y g ) with the largest change in gray value, take the gray value I ck in the gray image sequence I corresponding to the coordinates (x g , y g ), that is, I c (x g , y g ). Search the surface coefficient look-up table obtained in Step 1.3 to obtain the surface coefficient α(x g , y g ) of the point (x g , y g ), and calculate the optimal projection intensity of the low-intensity stripe 6. The three-dimensional point cloud acquisition method for highly reflective workpieces based on adaptive stripes according to claim 1, characterized in that Step 3.2 specifically is: according to the surface coefficient lookup table established in Step 1.3, through the mask matrix M c (x c ,y c ) to obtain the gray value corresponding to the gray scale sequence at the saturated pixel (x c ,y c ), according to the formula According to the acquired grayscale image sequence I ck , considering the non-linear responses of the camera and the projector, the optimal projection intensity of each pixel in the saturated pixel region is calculated by cubic B-spline interpolation (MIGL(x p , y p )).

7. The three-dimensional point cloud acquisition method for highly reflective workpieces based on adaptive stripes according to claim 6, characterized in that, Step 3.3 is specifically as follows: According to the best projection intensity (MIGL(x p ,y p )) of a single pixel in the saturation region obtained in Step 3.2, determine the projector pixel M c (x c ,y c ) that needs to be adjusted through the coordinate mapping between the camera and the projector established by the low-intensity orthogonal stripes and the mask matrix M p (x p ,y p ), and generate adaptive stripes according to the following formula: Where: (x p , y p ) represents the pixel coordinates of the projector image plane, A represents the background gray value, B represents the modulation degree of the sine stripe image, represents the encoded phase value at this pixel position; N represents the number of phase steps of the stripe pattern; MIGL(x p , y p ) is the optimal projection intensity at the pixel position of the point (x p , y p ), represents the generated adaptive stripe intensity.

Images