Multi-projection structured light coordinate system automatic correction method

Abstract

The application discloses a multi-projection structured light coordinate system automatic correction method, which discloses the following steps: step one: the whole ground glass screen is divided into X areas, from area 1 to area X, the height is continuously rising, and the rising height is the depth of field requirement of the whole system; step two: area 1 is placed at the lower depth of field position of the whole system to shoot, then the ground glass screen is rotated, so that the whole system shoots area 1 to area X in turn. When the last area is shot, it is just in the upper depth of field area; step 3: a circular ring mark plate is placed at the 0 plane position to shoot a group of pictures; step 4: calibration is carried out, and the corrected calibration parameters can be obtained.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a method for automatic correction of multi-projection structured light coordinate systems. Background Technology

[0002] Industrial robots are a crucial support for my country's industrial development, with widespread applications in manufacturing and logistics. Structured light 3D cameras, acting as the "eyes" of industrial robots, are a key factor influencing their application and performance. The measurement accuracy of structured light cameras is critical for 3D industrial modeling and defect detection. Calibration, as a crucial step in the production of 3D structured light cameras, significantly impacts their measurement accuracy and repeatability. In practical use, environmental factors such as stress, vibration, and temperature inevitably cause slight changes in the coordinate system parameters of the previously calibrated structured light system, which significantly affects high-precision measurements. This is especially true for multi-projection structured light systems (N projections, N>1, one camera), where coordinate system shifts can cause misalignment between the two projection systems, leading to layered noise during subsequent point cloud stitching and fusion. Therefore, secondary calibration of multi-projection structured light systems is essential. The current solution is recalibration, which offers the highest accuracy. However, in practical engineering applications, this solution has drawbacks such as being time-consuming and requiring reinstallation on a specialized calibration machine.

[0003] The current secondary correction schemes can be roughly divided into the following two categories:

[0004] Zhang's calibration process:

[0005] (1) Take pictures of the calibration board when the projector is not working; at the same position, project the stripe group and take pictures;

[0006] (2) Change the pose and repeat step (1);

[0007] (3) First, perform camera calibration based on the stripeless image. The calibration method is Zhang's 9-point method. Obtain the camera's intrinsic and extrinsic parameters.

[0008] (4) Based on the fringe pattern, the phase distribution on the calibration plate is obtained by phase decomposition, and the projector is calibrated according to the phase setting of the projector and the reverse process of the camera.

[0009] (5) For the same calibration plate position, calculate the positional relationship between the camera and the projector based on the camera's calibrated pose and the projector's calibrated pose.

[0010] (6) System calibration completed.

[0011] Calibration process based on a precision displacement stage:

[0012] Step (1): The entire calibration system requires the following: the optical axis of the camera is perpendicular to the calibration plate (which has a circular array pattern on it), and the direction of movement of the calibration plate is parallel to the optical axis of the camera. These conditions are very demanding and difficult to achieve in reality.

[0013] Step (2): Place the projector calibration plate at a suitable position on the high-precision translation stage, and use this position as the reference plane to establish a global spatial coordinate system, with the vertical upward direction as the positive direction of the Z-axis and the horizontal rightward direction as the positive direction of the X-axis. When the projector is working, take a picture of the fringe pattern and calculate the initial phase value; when the projector is not working, take a picture of the calibration plate.

[0014] Step (3): Move the calibration plate according to a certain step size, with a height difference of dh from the reference position. The projector is working to take a picture of the fringe pattern, calculate the phase value, and subtract the initial phase value to obtain dθ. The projector is not working and takes a picture of the calibration plate.

[0015] Step (4): Repeat step (3) to collect data at various altitudes, fit the equation, and obtain the relationship equation between phase value and altitude;

[0016] Step (6): Based on the relational equation in step (4), obtain the Z value of the center of the circular array pattern on the camera calibration board, generate three-dimensional coordinates (X, Y, Z) according to the calibration board parameters, and extract the center coordinate sequence (u, v) of the circular array projection according to the camera pattern.

[0017] Step (7): Repeat step (6) to obtain multiple sets of corresponding sequences and perform camera calibration.

[0018] However, both of these recalibration methods have drawbacks in actual engineering applications, such as requiring numerous images, being time-consuming, and being inconvenient to operate. In particular, for the high-precision displacement stage, the system needs to be completely disassembled, which is unacceptable in practical use. Summary of the Invention

[0019] The technical problem solved by this invention is to provide an automatic correction method for multi-projection structured light coordinate systems that can eliminate the layering effect of the system.

[0020] The technical solution adopted by this invention to solve its technical problem is: an automatic correction method for a multi-projection structured light coordinate system, comprising a frosted plate, wherein multiple regions with spirally increasing heights are arranged in a ring on the frosted plate, designated as region 1 to region X, where X ≥ 5; a circular calibration plate; and a monocular subsystem, wherein the monocular subsystem includes a camera and a projector used in conjunction. The specific method is as follows:

[0021] Step 1: Place region 1 in the lower depth of field of the entire shooting system and take a picture. Then rotate the frosted plate so that the entire system sequentially captures images of regions 1 to X. When region X is captured, region X is located in the upper depth of field of the entire shooting system, thereby obtaining the height value Z of each pixel. i (i = 1, 2, 3...X);

[0022] Step 2: In a monocular subsystem, for a single pixel, the phase value θ at this height is obtained through a four-step phase shift algorithm. i (i = 1, 2, 3...X);

[0023] Step 3: By adjusting the height value Z of a single pixel i and its own phase value θ i Establish higher-order functions to generate mapping relationships and complete the Z-axis correction;

[0024] Step 4: Place the circular marker board at the 0 plane position and take a picture. Obtain the pixel coordinates of the center of the circle using the center extraction algorithm. Obtain the phase value θ of the center of the circle using the four-step phase shift algorithm. Obtain the Z value of the center of the circle based on the mapping relationship corrected in Step 3. Obtain the XY coordinates of each marker point G on the circular marker board according to the design value of the circular marker points. Thus, the 3D coordinates (X, Y, Z) of all marker points can be obtained. Based on the center extraction of the image, the corresponding coordinates (u, v) of this 3D point in the image coordinate system can be obtained. Camera calibration is performed using the camera calibration formula to achieve XY axis correction.

[0025] Furthermore, in step three, the height value Z of a single pixel... i and its own phase value θ i The mapping relationship for higher-order functions is established as follows:

[0026] Z(m,n)=f (m,n) (θ)=a (m,n) θ 4 +b (m,n) θ 3 +c (m,n) θ 2 +d (m,n) θ 1 +e (m,n) ;

[0027] Where (m,n) represents the pixel position of a single pixel, and the coefficient a is calculated. (m,n) b (m,n) c (m,n) d (m,n) and e (m ,n) complete the Z-axis correction.

[0028] Furthermore, the camera calibration formula in step four is:

[0029]

[0030] The above f x f y c is the focal length parameter of the camera. x c y It is the principal point offset parameter of the camera, r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 t1, t2, and t3 are rotation parameters, representing the camera's extrinsic parameters; t1, t2, and t3 are translation parameters, representing the camera's extrinsic parameters.

[0031] Among them, (X) w Y w Z w Let f be the 3D coordinates (X, Y, Z) of the aforementioned marker point. Based on the data of multiple 3D coordinate points and their corresponding coordinates (u, v) in the image coordinate system, f is calculated. x f y c x c y r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 The values ​​of t1, t2, and t3.

[0032] Furthermore, it also includes a shooting system, which comprises a camera and N projectors, wherein the camera and one of the projectors constitute the above-mentioned monocular subsystem, and N≥2;

[0033] In step one, when shooting areas 1 to X, each area generates N Z values. i It is the average of N Z values.

[0034] Furthermore, the height intervals of the multiple spirally rising areas on the frosted plate are equal.

[0035] The beneficial effects of this invention are: this calibration method removes the stringent limitations of calibration based on a high-precision displacement stage, allowing for very convenient on-site calibration. After calibration, the layering effect of the system can be eliminated. Taking a dual-projection structured light system as an example, the effect is as follows: Figure 4As shown in the figure, the same frosted plate plane was reconstructed before and after the correction. The left figure shows that the Z-values ​​of the point clouds reconstructed by the two subsystems are layered by 120 micrometers. After the system correction, the point cloud Z-values ​​of the two subsystems are no longer layered. Due to random errors, the point cloud will jump by 10 micrometers. Attached Figure Description

[0036] Figure 1 This is a top view of the frosted plate according to an embodiment of this application.

[0037] Figure 2 This is a side view of the frosted plate according to an embodiment of this application.

[0038] Figure 3 This is a schematic diagram of the structure of the circular sign plate according to an embodiment of this application.

[0039] Figure 4 This is a layered diagram of the point cloud Z-values ​​before and after correction for the dual-projection structured light system in this application embodiment. Detailed Implementation

[0040] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0041] like Figures 1 to 3 As shown, embodiments of this application disclose an automatic correction method for a multi-projection structured light coordinate system, including a frosted plate with multiple spirally increasing regions arranged in a ring on the frosted plate, designated as region 1 to region X, where X ≥ 5. It also includes a circular calibration plate and a monocular subsystem comprising a camera and a projector. The specific method is as follows:

[0042] Step 1: Place region 1 in the lower depth of field of the entire shooting system and take a picture. Then rotate the frosted plate so that the entire system sequentially captures images of regions 1 to X. When region X is captured, region X is located in the upper depth of field of the entire shooting system, thereby obtaining the height value Z of each pixel. i (i = 1, 2, 3...X);

[0043] Step 2: In a monocular subsystem, for a single pixel, the phase value θ at this height is obtained through a four-step phase shift algorithm. i (i = 1, 2, 3...X);

[0044] In the two steps above, the captured pattern is a conventional 4-step phase-shift image from a 3D structured light camera, namely 4 phase-shift images + 8 Gray code images. This set of images allows for the convenient acquisition of the phase of each pixel in the image using a four-step phase-shift algorithm.

[0045] In this embodiment, X is defined as 6, meaning the entire rotating frosted plate is divided into 6 regions. Based on the 6 different heights of the frosted plate captured in the image, for each image pixel, 6 different Z values, Zi (i = 1, 2, 3, 4, 5, 6), can be obtained. Simultaneously, in each monocular subsystem, for a single pixel, the phase value θ at that height can be obtained through phase deconstruction, i.e., a four-step phase shift algorithm. i (i = 1, 2, 3, 4, 5, 6), the four-step phase shift algorithm described above is an existing algorithm and will not be repeated here.

[0046] Step 3: By adjusting the height value Z of a single pixel i and its own phase value θ i Establish higher-order functions to generate mapping relationships:

[0047] Z(m,n)=f (m,n) (θ)=a (m,n) θ 4 +b (m,n) θ 3 +c (m,n) θ 2 +d (m,n) θ 1 +e (m,n) ;

[0048] (m,n) represents the pixel position of a single pixel, using 6 different Zi values ​​and their corresponding θ values. i The value can be used to calculate the coefficient a. (m,n) b (m,n) c (m,n) d (m,n) and e (m,n)， This completes the Z-axis correction, that is, achieves the above-mentioned Z-axis correction. i and θ i The mapping relationship function is updated;

[0049] In actual use, due to factors such as vibration, temperature, and stress release, the coordinate systems of these monocular subsystems may become slightly separated, resulting in layered artifacts during subsequent point cloud stitching. Therefore, by re-establishing the higher-order function mapping relationship, the Z value calculated for the same pixel position in the image for each monocular subsystem is restored to be consistent.

[0050] Step 4: Place the circular marker board at the 0 plane position and take a picture. Obtain the pixel coordinates of the center of the circle using the center extraction algorithm. Obtain the phase value θ of the center of the circle using the four-step phase shift algorithm. Obtain the Z value of the center of the circle based on the mapping relationship corrected in Step 3. Obtain the XY coordinates of each marker point G on the circular marker board according to the design values ​​of the circular marker points. Thus, the 3D coordinates (X, Y, Z) of all marker points can be obtained. Based on the center extraction of the image, the corresponding coordinates (u, v) of this 3D point in the image coordinate system can be obtained. Camera calibration is performed using the camera calibration formula to achieve XY axis correction. The camera calibration formula is:

[0051]

[0052] The above f x f y c is the focal length parameter of the camera. x c y It is the principal point offset parameter of the camera, r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 t1, t2, and t3 are rotation parameters, representing the camera's extrinsic parameters; t1, t2, and t3 are translation parameters, representing the camera's extrinsic parameters.

[0053] In the above steps, the circular marker plate can be a standard part or a custom design; the principle is to extract the center of the circular ring. The so-called 0-plane refers to the 0-plane calibrated before the system leaves the factory. For example, if the plane calibrated before the system leaves the factory with a working distance of 280mm is the 0-plane, then the xyz coordinates and the Z coordinate on this plane are 0.

[0054] Among them, (X) w Y w Z w Let f be the 3D coordinates (X, Y, Z) of the aforementioned marker point. Based on the data of multiple 3D coordinate points and their corresponding coordinates (u, v) in the image coordinate system, f is calculated. x f y c x c y r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33、 The values ​​of t1, t2, and t3.

[0055] Specifically, it also includes a shooting system, which includes one camera and N projectors. One camera and one of the projectors constitute the above-mentioned monocular subsystem, where N≥2;

[0056] The aforementioned N projectors may or may not be at the same height, but it is preferable that they are at the same height.

[0057] In step one, when shooting areas 1 to X, each area generates N Z values. i It is the average of N Z values.

[0058] In the above method, for a single pixel, each shot generates N Z values, where N is the number of projectors and also the number of monocular subsystems. These N Z values ​​differ slightly (in actual use, the difference does not exceed 100 micrometers). Therefore, the average of these N subsystem Z values ​​can be calculated as the new Z value for all subsystems under this pixel, thereby ensuring the accuracy of the Z value in each region.

[0059] Based on the experimental structure, calibration can eliminate the layering effect of the system. Taking a dual-projection structured light system as an example, the effect is as follows: Figure 4 As shown in the figure, the same frosted plate plane was reconstructed before and after the correction. The left figure shows that the Z-values ​​of the point clouds reconstructed by the two subsystems are layered by 120 micrometers. After the system correction, the point cloud Z-values ​​of the two subsystems are no longer layered. Due to random errors, the point cloud will jump by 10 micrometers.

[0060] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-projection structured light coordinate system automatic correction method, characterized in that: The sanding plate comprises a plurality of regions with a height rising spirally arranged annularly, numbered as region 1 to region X, X≥5, further comprising a circular ring calibration plate, further comprising a monocular subsystem, the monocular subsystem comprises a camera and a projector used in cooperation, specifically, Step one: place region 1 at the lower depth of field position of the entire shooting system for shooting, then rotate the ground glass screen so that the entire system shoots region 1 to region X in turn, and when shooting region X, region X is located at the upper depth of field region of the entire shooting system, thereby obtaining the height value Z of each pixel i (i = 1, 2, 3... X); Step two: In the monocular subsystem, for a single pixel, the phase value θ at this height is obtained by a four-step phase shift algorithm i (i = 1, 2, 3... X); Step three: by the single pixel height value Z i and the phase value θ i Establish high-order function to produce mapping relationship to complete the correction of Z-axis; Step four: place the circular ring calibration plate in the 0 plane position for picture shooting, obtain the pixel coordinates of the center of the circle according to the center extraction algorithm, obtain the phase value θ of the center of the circle through the four-step phase shift algorithm, and obtain the Z value of the center of the circle according to the mapping relationship corrected in step three, obtain the XY coordinates of each mark point G on the circular ring calibration plate according to the design value of the circular ring mark point, so as to obtain the 3D coordinates (X, Y, Z) of all mark points, according to the center extraction of the image, the corresponding coordinates (u, v) of the 3D point coordinates in the image coordinate system can be obtained, and the camera calibration is carried out through the camera calibration formula, and the XY axis correction is realized.

2. The multi-projection structured light coordinate system automatic correction method of claim 1, wherein: In step three, the single pixel height value Z i and the phase value θ of itself i The high-order function is established to generate the mapping relationship: Z(m, n) = f (m,n) (θ) = a (m,n) θ 4 + b (m,n) θ 3 + c (m,n) θ 2 + d (m,n) θ 1 + e (m,n) ; where (m, n) represents the pixel position of the single pixel, coefficients a (m,n) , b (m,n) , c (m,n) , d (m,n) , and e (m,n) are calculated to complete the correction of the Z axis.

3. The method of claim 1, wherein: The camera calibration formula in the step four is: The above f x , f y is the focal length parameter of the camera, c x , c y is the principal point offset parameter of the camera, r 11 , r 12 , r 13 , r 21 , r 22 , r 23 , r 31 , r 32 , r 33 , is a rotation parameter, represents the extrinsic parameters of the camera, t1, t2 and t3 are translation parameters, and is the camera extrinsic parameter; wherein (X w , Y w , Z w ) are 3D coordinates (X, Y, Z) of the above-mentioned mark points, f x , f y , c x , c y , r 11 , r 12 , r 13 , r 21 , r 22 , r 23 , r 31 , r 32 , r 33 , t1, t2 and t3 are values calculated according to data of a plurality of 3D coordinate points and corresponding coordinates (u, v) under a corresponding image coordinate system.

4. The method of claim 1, wherein: Further comprising a shooting system, the shooting system comprises a camera and N projectors, a camera and one of the projectors constitute the monocular subsystem, and N≥2; In step one, when the regions 1 through X are photographed, each region produces N Z values, Z i is the average of the N Z values.

5. The method of claim 1, wherein: The height intervals of the plurality of regions with a height rising spirally on the sanding plate are equal.

Images