An adaptive projection method based on high-light boundary three-dimensional remapping

Abstract

The application discloses a kind of self-adapting projection methods based on high light boundary three-dimensional remapping.First, based on phase shift brightness statistics diagram, extract single or multiple overexposure connected region, extract the pixel group coordinates of the un-overexposed boundary of the region and reconstruct the corresponding three-dimensional coordinates of boundary based on calibration parameter;Then, based on the projection light path model, the three-dimensional coordinates are inversely mapped to the DMD target surface to obtain a closed-loop connected domain;High light closed-loop connected domain is screened using a two-dimensional image to obtain the boundary of the outermost circle high brightness range, and the projection light intensity is gradually reduced by high light boundary extraction and three-dimensional inverse mapping, which reduces the high light range, and finally completely eliminates the overexposure range.The method first maps the high light edge pixels to the DMD target surface pixels, and reduces the projection light to eliminate all high light areas.The complex splicing process of different three-dimensional point clouds in the traditional multi-exposure method is avoided, which improves the reconstruction rate while reducing the complexity of calculation.

Description

Technical Field

[0001] This invention relates to the field of structured light 3D measurement, and in particular to an adaptive projection method based on specular boundary 3D remapping. Background Technology

[0002] In the field of structured light 3D measurement, interference from highlight regions often has a fatal impact on the integrity of measurement results. Traditional methods for suppressing highlights in structured light measurement mainly include the following two: First, the multi-exposure compensation method, which completely eliminates highlight regions by projecting images at different brightness levels. This method is inefficient, requires projecting multiple sets of structured light, is time-consuming, and increases computational load by stitching together measurement results calculated at different exposure times. Second, the method of using low-density stripes instead of high-density stripes. While this method can suppress some highlight regions, it cannot completely eliminate their existence, and the use of low-density stripes results in reduced measurement accuracy. Compared to the first two methods, this method uses adaptive projection to completely eliminate the brightness of highlight regions, thus achieving measurement results from a single phase-shift measurement. This avoids the complex stitching process of different 3D point clouds and ensures measurement accuracy without increasing measurement time. Summary of the Invention

[0003] To further reduce computational complexity while improving measurement accuracy, this invention provides an adaptive projection method based on specular boundary 3D remapping.

[0004] The technical solution adopted in this invention is as follows: an adaptive projection method based on specular boundary 3D remapping, comprising the following steps:

[0005] Step 1: Based on the phase-shifted brightness map, extract the connected regions in the overexposed areas of the image;

[0006] Step 2: Extract the coordinates of the unexposed pixel group at the outer boundary of the overexposed connected region, and reconstruct the corresponding three-dimensional coordinates at the boundary based on the calibration parameters.

[0007] Step 3: Reverse map the 3D coordinates to the DMD target surface coordinates to obtain the closed-loop connected domain;

[0008] Step 4: Adjust the brightness of the projected light within the connected domain. If there are still overexposed areas, gradually reduce the brightness of the projected light until all the highlight areas are eliminated.

[0009] Preferably, step one specifically involves: reading all phase-shift images of the densest stripes captured by the camera, statistically extracting the regions with pixel values ​​greater than or equal to 255 in all phase-shift images, and recording the regions in all phase-shift images where the pixel value is greater than or equal to 255 as the highlight regions.

[0010] Preferably, step two specifically involves:

[0011] All pixel values ​​in the image with a value less than 255 are set to 0. An edge detection algorithm is used to extract the boundaries of all highlight regions. The extracted coordinates and corresponding phase values ​​are then used to solve for the three-dimensional coordinates. Based on the pinhole imaging model, we know that:

[0012]

[0013] Where f x_p with f y_p u is the focal length of the projector lens. p With v p These are the pixel coordinates of the camera, (u 0_p ,v 0_p ) represents the principal point position on the projector target surface, (u p ,v p ) represents the pixel coordinates of the projector's target surface, RT P3×4 This is the extrinsic parameter matrix of the projector;

[0014] The pixel coordinates of the projector are correlated with those of the camera using the camera's extrinsic parameter matrix and the projector's extrinsic parameter matrix.

[0015]

[0016] Where "-1" represents the pseudo-inverse of the matrix, RT c3×4 This represents the extrinsic parameter matrix in the camera coordinate system.

[0017] Therefore, we can conclude that:

[0018]

[0019] Combine the intrinsic parameter matrix and the extrinsic parameter matrix into a single matrix R, i.e.:

[0020]

[0021] Extracting the first row of formula (3) yields:

[0022] Z p u p =R(1,1)·X c +R(1,2)·Y c +R(1,3)·Z c +R(1,4) (5)

[0023] Z p It is derived from formula (2);

[0024] Starting from the camera, we can obtain two constraints:

[0025]

[0026] Combining formulas (5) and (6), the three-dimensional coordinates at the boundary can be obtained:

[0027]

[0028] Preferably, step three specifically involves:

[0029] First, P is obtained through equation (2). c (X,Y,Z) transforms into P p (X,Y,Z), and from equation (1), we can obtain:

[0030]

[0031] Finally, the pixel coordinates of the DMD target surface are:

[0032]

[0033] By connecting all DMD target surface pixel coordinates obtained by solving the problem according to the principle of closest distance, the connected domain of pixel coordinates of the DMD target surface highlight area is obtained.

[0034] Preferably, step four specifically involves:

[0035] The projected light intensity is graded based on the original projected light intensity:

[0036] [I,nI,n 2 I,n 3 I,n 4 I,…] (10)

[0037] Where I is the original light intensity, and n (n<1) is the reduction ratio. By gradually reducing the light intensity within the highlight region, the highlight region is divided according to the actual light intensity value. The higher the actual light intensity, the smaller the ratio.

[0038] Each time the projected light intensity is changed, the projection ratio coefficient of the corresponding area is recorded, and the process returns to step one to check whether there are still highlight areas. If so, steps two and three are repeated until all highlight areas are incorporated into the normal exposure area.

[0039] The present invention has the following advantages: The present invention proposes an adaptive projection method based on three-dimensional remapping of specular boundaries, which reduces the computational complexity compared with previous methods, while preserving the integrity of the measurement. Attached Figure Description

[0040] Figure 1 This is a system block diagram of an embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram of a method according to an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of the method for gradually reducing light intensity in an embodiment of the present invention. Figure 1 .

[0043] Figure 4 This is a schematic diagram of the method for gradually reducing light intensity in an embodiment of the present invention. Figure 2 .

[0044] Figure 5 This is a schematic diagram of the method for gradually reducing light intensity in an embodiment of the present invention. Figure 3 .

[0045] Figure 6 This is a schematic diagram of the method for gradually reducing light intensity in an embodiment of the present invention. Figure 4 . Detailed Implementation

[0046] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0047] like Figure 1 The diagram illustrates the steps of this embodiment: First, all phase shift images of the densest fringes are read in. Then, all regions with pixel values ​​exceeding 255 in the phase shift images are counted. Next, the coordinates of unexposed pixel groups at the outer boundaries of the overexposed connected regions are extracted. Based on calibration parameters, the corresponding three-dimensional coordinates at the boundaries are reconstructed. Then, the three-dimensional coordinates are inversely mapped to the DMD target surface coordinates to obtain a closed-loop connected region. Finally, the brightness of the structured light within the connected region is adjusted. If overexposed regions still exist, the brightness of the projected light is gradually reduced until all highlight regions are eliminated.

[0048] The specific steps are as follows:

[0049] Step 1: Extract the overexposed connected regions.

[0050] First, read all phase-shift images of the densest stripes captured by the camera. Statistically extract regions with pixel values ​​greater than or equal to 255 from all phase-shift images. Record the regions in all phase-shift images where the pixel value is greater than or equal to 255; these are the highlight regions.

[0051] Step 2: Selection and reconstruction of the boundary region.

[0052] All pixel values ​​less than 255 in the image are set to 0. Edge detection algorithms are used to extract the boundaries of all highlight regions. The extracted coordinates and corresponding phase values ​​are then used to calculate the 3D coordinates. Based on the pinhole camera model, we know that:

[0053]

[0054] Where f x_p with fy_p This refers to the focal length of the projector lens. p With v p These are the pixel coordinates of the camera. (u 0_p ,v 0_p (u) represents the principal point position on the projector target surface. p ,v p ) represents the pixel coordinates of the projector's target surface. RT P3×4 This is the extrinsic parameter matrix of the projector. The pixel coordinates of the projector are correlated with the pixel coordinates of the camera using the extrinsic parameter matrices of the camera and the projector.

[0055]

[0056] Where "-1" represents the pseudo-inverse of the matrix, RT c3×4 This represents the extrinsic parameter matrix in the camera coordinate system. From this, we can obtain:

[0057]

[0058] Combine the intrinsic parameter matrix and the extrinsic parameter matrix into a single matrix R, i.e.:

[0059]

[0060] Extracting the first row of the entire formula (3) yields:

[0061] Z p u p =R(1,1)·X c +R(1,2)·Y c +R(1,3)·Z c +R(1,4) (5)

[0062] Z p It can be derived from formula (2):

[0063] Starting from the camera, we can obtain two constraints:

[0064]

[0065] Combining formulas (5) and (6), the three-dimensional coordinates at the boundary can be obtained:

[0066]

[0067] Step 3: Solve for the closed-loop connected component of the pixel coordinates in the DMD specular region. First, P is obtained using equation (2). c (X,Y,Z) transforms into P p (X,Y,Z), and from equation (1), we can obtain:

[0068]

[0069] Finally, the pixel coordinates of the DMD target surface are:

[0070]

[0071] The pixel coordinates of the DMD target surface obtained by solving are connected by connecting the nearest adjacent pixels to obtain the connected domain of the pixel coordinates of the highlight region of the DMD target surface.

[0072] Step four, method for adjusting structured light brightness. The projected light intensity is graded based on the original projected light intensity:

[0073] [I,nI,n 2 I,n 3 I,n 4 I,…] (10)

[0074] Where I represents the original light intensity, and n (n<1) is the reduction ratio. By progressively reducing the light intensity within the highlight region, the highlight region is divided into areas (a1, a2, a3, ...) according to the actual light intensity value. The higher the actual light intensity, the smaller the ratio. Each time the projected light intensity is reduced, the projection ratio of the corresponding area is recorded. Then, the process returns to step one to check if any highlight regions still exist. If so, steps two and three are repeated until all highlight regions are incorporated into the normal exposure area A. Figures 3-6 This diagram illustrates a method for gradually reducing light intensity. The projected light intensity is graded, with different levels of intensity projected onto areas of varying brightness. Areas with higher light intensity have smaller scaling factors, such as area a3.

[0075] Obviously, the above embodiments of the present invention are merely illustrative examples to illustrate the invention and are not intended to limit the implementation of the invention. Other obvious variations or modifications derived from the essential spirit of the invention still fall within the protection scope of the invention.

Claims

1. An adaptive projection method based on specular boundary 3D remapping, characterized in that... The steps are as follows: Step 1: Based on the phase shift brightness map, extract the connected regions of the overexposed areas of the image. Specifically, read all phase shift maps of the densest stripes captured by the camera, extract the regions with pixel values ​​greater than or equal to 255 in all phase shift maps, and record the regions with pixel values ​​greater than or equal to 255 in all phase shift maps as the highlight regions. Step 2: Extract the coordinates of the unexposed pixel group at the outer boundary of the overexposed connected region, and reconstruct the corresponding three-dimensional coordinates at the boundary based on the calibration parameters. Step 3: Reverse map the three-dimensional coordinates to the DMD target surface coordinates to obtain the closed-loop connected domain; Step 4: Based on the original projected light intensity, classify the projected light intensity into levels. By gradually reducing the light intensity within the highlight area, the highlight area is divided according to the actual light intensity value. The higher the actual light intensity, the smaller the proportion coefficient of the area. Each time the projected light intensity is changed, record the projection proportion coefficient of the corresponding area, and return to Step 1 to check whether there are still highlight areas. If there are, repeat Step 2 and Step 3 until all highlight areas are incorporated into the normal exposure area.

2. The adaptive projection method based on specular boundary 3D remapping according to claim 1, characterized in that... Step two is as follows: All pixel values ​​in the image with a value less than 255 are set to 0. An edge detection algorithm is used to extract the boundaries of all highlight regions. The extracted coordinates and corresponding phase values ​​are then used to solve for the three-dimensional coordinates. Based on the pinhole imaging model, we know that: （1） in and The focal length of the projector lens. and These are the pixel coordinates of the camera. This is the principal point position on the projector target surface. Let be the pixel coordinates of the projector's target surface. This is the extrinsic parameter matrix of the projector; The pixel coordinates of the projector are correlated with those of the camera using the camera's extrinsic parameter matrix and the projector's extrinsic parameter matrix. （2） Where "-1" represents the pseudo-inverse of the matrix, This represents the extrinsic parameter matrix in the camera coordinate system. Therefore, we can conclude that: （3） Combine the intrinsic parameter matrix and the extrinsic parameter matrix into a single matrix. ,Right now: （4） Extracting the first row of formula (3) yields: （5） in It is derived from formula (2); Starting from the camera, we can obtain two constraints: （6） Combining formulas (5) and (6), the three-dimensional coordinates at the boundary can be obtained: （7）。 3. The adaptive projection method based on specular boundary 3D remapping according to claim 2, characterized in that... Step three specifically involves: First, through equation (2) Transform into And from equation (1), we can obtain: （8） Finally, the pixel coordinates of the DMD target surface are: （9） By connecting all DMD target surface pixel coordinates obtained by solving the problem according to the principle of closest distance, the connected domain of pixel coordinates of the DMD target surface highlight area is obtained.

4. The adaptive projection method based on specular boundary 3D remapping according to claim 1, characterized in that... In step four: The projected light intensity is graded based on the original projected light intensity: （10） in, Original light intensity To reduce the scaling factor, and .

Images