A Ring Structured Light Calibration Method Based on a Planar Checkerboard

Abstract

The present invention discloses a calibration method for annular structured light based on a planar checkerboard grid, comprising: (1) projecting the annular structured light onto the checkerboard calibration board, and changing the pose of the checkerboard calibration board to obtain multiple groups of calibration patterns; (2) obtaining the pixel coordinates of the intersection points of the annular structured light and the checkerboard straight lines in each group of calibration patterns; (3) using the principle of cross-ratio invariance, respectively converting the pixel coordinates of the intersection points of the annular structured light and the checkerboard straight lines in each group of calibration patterns into three-dimensional coordinates in the camera coordinate system; (4) fitting the three-dimensional coordinates of all the intersection points corresponding to the camera coordinate system into a conic surface equation; and completing the calibration by establishing a mathematical model of the annular structured light vision sensor based on the fitted conic surface equation. The present invention can provide a simple, efficient and fast calibration scheme for annular structured light.

Description

Technical Field

[0001] The present invention relates to the technical field of optical measurement and detection, and particularly to a calibration method for annular structured light based on a planar checkerboard grid. Background Art

[0002] With the continuous development of society and the continuous improvement of industrial production levels, the automation industry has put forward higher requirements for the measurement and inspection of the inner wall of a circular ring. As an important branch of computer vision, three-dimensional vision imaging technology can meet the needs of automated production inspection with its high efficiency and high precision. Structured light three-dimensional measurement technology has many advantages such as low cost, high precision, easy implementation, and non-contact, and has broad development space and practical application value in industrial inspection and production. Structured light is a system structure composed of a projector (point, line, plane) and a camera. After projecting specific light information onto the object surface and the background, it is collected by the camera, and the position and depth information of the object are calculated based on the changes in the light signal caused by the object, and then the entire three-dimensional space is restored. Structured light calibration is the primary task for realizing measurement, and its purpose is to determine the positional relationship between the laser and the camera.

[0003] Structured light patterns are divided into point structured light patterns, line structured light patterns, multi-line structured light patterns, surface structured light patterns, and the recently emerged phase method. Existing three-dimensional object measurements using structured light generally require the production of special targets, and the process of producing special targets is complex and costly. To reduce costs, in the prior art, there are also cases where traditional and easily generated checkerboard grids are used instead of special targets. For example, in the patent application with the application number 2021110778226 and the name "Three-dimensional Scanning Inner Wall Equipment and Method of Monocular Vision Plus Annular Structured Light Cluster", the structured light calibration projects annular light onto the checkerboard grid. However, in this technical solution, the track trolley is fixed according to the principle that the camera center and the optical center of the annular structured light emitter are on the same straight line, and the checkerboard grid is driven by a precision instrument stepping motor on the track trolley to move back and forth on the optical axis straight line. This solution directly assumes that what is projected onto the checkerboard grid is the circular ring, and thus the three-dimensional point cloud data obtained by calculation is inaccurate. Summary of the Invention

[0004] Object of the Invention: The present invention provides a calibration method for annular structured light based on a planar checkerboard grid, which is simple to operate and can quickly and accurately complete the calibration of annular structured light.

[0005] Technical Solution: The present invention provides a calibration method for annular structured light based on a planar checkerboard grid, specifically including:

[0006] (1) Project the annular structured light onto the checkerboard calibration board, and capture a calibration pattern with a camera. Keep the current pose of the checkerboard calibration board unchanged, turn off the annular structured light and capture another calibration pattern. Two calibration patterns in the same pose form a group of calibration patterns. Change the pose of the checkerboard calibration board to obtain multiple groups of calibration patterns.

[0007] (2) Perform masking processing on the obtained multiple groups of calibration patterns, and segment out the annular structured light in each group of calibration patterns. Adopt the laser center extraction algorithm to extract the pixel coordinates of the annular structured light in each group of calibration patterns. Obtain the pixel coordinates of the intersection points of the annular structured light and the checkerboard straight lines in each group of calibration patterns based on the pixel coordinates of the annular structured light.

[0008] (3) Utilize the principle of cross-ratio invariance to convert the pixel coordinates of the intersection points of the annular structured light and the checkerboard straight lines in each group of calibration patterns into three-dimensional coordinates in the world coordinate system respectively. Then, convert the three-dimensional coordinates in the world coordinate system corresponding to the intersection points of the annular structured light and the checkerboard straight lines obtained in each group of calibration patterns into three-dimensional coordinates in the camera coordinate system through camera calibration.

[0009] (4) Fit the three-dimensional coordinates of all the intersection points in the camera coordinate system into a conic surface equation. Establish the mathematical model of the annular structured light vision sensor based on the fitted conic surface equation to complete the calibration. The specific mathematical model is:

[0010]

[0011] In the formula, S is the scale transformation factor; (u, v, 1) are the pixel coordinates of the intersection points; f x , f y , u0, v0 are all camera internal parameters; (X c , Y c , Z c ) are the three-dimensional coordinates of the intersection points in the camera coordinate system; l0 to l 10 are all the coefficients of the fitted conic surface equation.

[0012] Furthermore, the acquisition of the pixel coordinates of the intersection points in step (2) is specifically as follows:

[0013] (2.1) For each group of calibration patterns, subtract the calibration pattern captured with the annular structured light on from the calibration pattern captured with the annular structured light off in the same pose to extract the mask of the annular structured light in this group of calibration patterns, and then segment out the annular structured light in each group of calibration patterns through image processing.

[0014] (2.2) Adopt the Steger center point sub-pixel extraction algorithm to extract the sub-pixel coordinates of the annular structured light in each group of calibration patterns.

[0015] (2.3) The sub-pixel coordinates of the extracted annular structured light corresponding to each set of calibration patterns are fitted into elliptical equations, and the straight lines on the chessboard are fitted into straight line equations. The elliptical equation and the straight line equation are combined to obtain the pixel coordinates of the intersection of the annular structured light and the straight lines on the chessboard in each set of calibration patterns.

[0016] Furthermore, step (3) specifically includes:

[0017] (3.1) The pixel coordinates of the intersection of the ring structured light and the chessboard straight line in each set of calibration patterns are converted into image coordinates. The formula is:

[0018]

[0019] Where 1 / dx and 1 / dy represent the camera intrinsic parameters, (x, y, z) represents the image coordinates, and z is normalized;

[0020] (3.2) Using the cross-ratio invariance principle, we transform (x, y, z) into (X) in the world three-dimensional coordinates based on the condition that the complex ratio in the image coordinate system is equal to the complex ratio in the world coordinate system. w ,Y w ,Z w );

[0021] (3.3) Use the camera's intrinsic parameters to transform the world's three-dimensional coordinates into the three-dimensional coordinates in the camera coordinate system. The formula is:

[0022]

[0023] Where R and T are the external parameters of the camera.

[0024] Furthermore, the three-dimensional coordinates of all intersection points in step (4) corresponding to the camera coordinate system are fitted into the conic surface equation by the least squares method.

[0025] Furthermore, in step (3), the camera is calibrated by using the calibration patterns in each group of calibration patterns photographed under the condition of turning off the annular structured light to obtain the intrinsic parameters and extrinsic parameters of the camera.

[0026] In addition, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program. A computer-readable storage medium stores a computer program thereon, wherein the computer program implements the steps of the above method when executed by the processor.

[0027] Beneficial effects: The annular structured light calibration method based on a planar checkerboard in the present invention has the following significant advantages compared with the prior art: 1. It uses a traditional and easily generated checkerboard to replace a special target, eliminating the complex process of manufacturing the target, with low cost and simple operation; 2. By photographing calibration patterns in multiple different poses, the sub-pixel coordinates of the annular structured light are accurately obtained using a sub-pixel extraction algorithm and fitted into an ellipse equation, which is more general and conducive to improving accuracy; 3. The calibration can be made more reliable by enriching the experimental data through the calibration patterns photographed in multiple different poses. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 The flowchart of the method of the present invention is shown;

[0029] Figure 2 The schematic diagram of the method of the present invention is shown;

[0030] Figure 3 The system architecture of the method of the present invention is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The technical solutions of the present invention will be further described below with reference to the accompanying drawings.

[0032] As Figure 1 shown, the present invention provides an annular structured light calibration method based on a planar checkerboard, which specifically includes: As Figure 3 shown, it includes a camera, an annular structured light vision sensor, and a checkerboard calibration plate; in this embodiment, the annular structured light vision sensor is used to emit a circular ring structured light, and the centers of the camera and the annular structured light vision sensor do not need to be on the same straight line. The pose of the checkerboard calibration plate can be adjusted according to the annular structured light and does not need to be fixed in a single direction.

[0033] Step 1: Project the annular structured light onto the checkerboard calibration plate and change the pose of the checkerboard to obtain multiple groups of calibration patterns. Specifically,

[0034] Each group of calibration patterns includes two patterns. One of them turns on the annular structured light, projects the annular structured light onto the checkerboard calibration plate, and takes a calibration pattern through the camera; then, without changing the current pose of the checkerboard, turn off the structured light and take another calibration pattern to obtain the second pattern; the two calibration patterns in the same pose form a group of calibration patterns.

[0035] Multiple groups of calibration patterns are required for one system calibration. The pose of the checkerboard needs to be changed for each group of calibration patterns. To ensure the reliability of the data, at least 15 groups of calibration patterns should be obtained.

[0036] Step 2: Obtain the intersection pixel coordinates of the straight lines on the checkerboard and the annular structured light. Specifically:

[0037] (1) For each group of calibration patterns, the mask of the annular structured light in the calibration pattern is extracted by subtracting the calibration pattern taken with the annular structured light turned on and the calibration pattern taken with the annular structured light turned off at the same posture, and then the annular structured light in each group of calibration patterns is segmented by image processing;

[0038] (2) using a laser center extraction algorithm to extract pixel coordinates of the annular structured light in each group of calibration patterns; specifically, in order to obtain accurate pixel coordinates of the annular structured light, using a Steger center point sub-pixel extraction algorithm to extract sub-pixel coordinates of the annular structured light in each group of calibration patterns;

[0039] (3) The sub-pixel coordinates of the extracted annular structured light corresponding to each calibration pattern are fitted into an ellipse equation, and the straight line on the chessboard is fitted into a straight line equation. The ellipse equation and the straight line equation are combined to obtain the pixel coordinates of the intersection of the annular structured light of each calibration pattern and the straight line on the chessboard. The specific formula is:

[0040] k1u2+k2v2+k3u+k4v+k5uv+k6=0

[0041]

[0042] Wherein, k1~k6 are the coefficients of the fitting ellipse equation, a1~a6 and b1~b6 are the coefficients of the fitting chessboard straight line equation; n represents the number of rows of the chessboard; the above ellipse equation and straight line equation can be combined to obtain the pixel coordinates (u, v) of the intersection of each straight line on the chessboard and the annular structured light.

[0043] Step 3: Using the cross ratio invariance principle, transform the pixel coordinates of the intersection of the ring structured light and the chessboard straight lines in each calibration pattern into three-dimensional spatial coordinates (i.e., three-dimensional coordinates in the world coordinate system), and then transform them into three-dimensional coordinates in the camera coordinate system through camera calibration. Specifically:

[0044] (1) Perform camera calibration on each group of closed ring structured light patterns separately to obtain the internal and external parameters of the camera, and then use the cross ratio invariance principle to obtain the spatial three-dimensional coordinates of the intersection. First, convert the pixel coordinates of the intersection of the ring structured light and the chessboard straight line in each group of calibration patterns into image coordinates, using the formula:

[0045]

[0046] Where 1 / dx and 1 / dy represent the camera intrinsic parameters, (x, y, z) represents the image coordinates, and z is normalized;

[0047] (2) By using the principle of cross-ratio invariance, the image coordinates (x, y, z) are transformed into the three-dimensional world coordinates (X w , Y w , Z w ) based on the condition that the cross-ratio in the image coordinate system is equal to the cross-ratio in the world coordinate system; since each checkerboard corresponding to each set of calibration patterns is the reference plane, the depth Z w is all 0;

[0048] (3) The three-dimensional world coordinates are transformed into the three-dimensional coordinates in the camera coordinate system by using the internal parameters of the camera. The formula is:

[0049]

[0050] In the formula, R and T are both the external parameters of the camera.

[0051] Step 4: As Figure 2 shown, the three-dimensional coordinates of all intersection points in the camera coordinate system are fitted into a conical surface equation by the least squares method; a mathematical model of the annular structured light vision sensor is established based on the fitted conical surface equation to complete the calibration. The specific mathematical model of the annular structured light vision sensor is:

[0052]

[0053] In the formula, S is the scale transformation factor; (u, v, 1) are the pixel coordinates of the intersection point; f x , f y , u0, v0 are all the internal parameters of the camera; (X c , Y c , Z c ) are the three-dimensional coordinates of the intersection point in the camera coordinate system; l0 to l 10 are all the coefficients of the fitted conical surface equation.

[0054] In summary, the method of the present invention does not need to manufacture a special planar target model, does not need to move the target with precision instruments, is simple to operate, the system cost is controllable, and has significant practical application value.

Claims

1. A calibration method for annular structured light based on a planar checkerboard, characterized in that The following steps are involved: (1) Projecting the annular structured light onto the checkerboard calibration plate, and taking a calibration pattern with the camera; keeping the current posture of the checkerboard calibration plate unchanged, turning off the annular structured light and taking another calibration pattern; two calibration patterns with the same posture form a set of calibration patterns; Change the posture of the checkerboard calibration plate to obtain multiple sets of calibration patterns; (2) performing mask processing on the obtained multiple groups of calibration patterns to segment the annular structured light in each group of calibration patterns; using a laser center extraction algorithm to extract the pixel coordinates of the annular structured light in each group of calibration patterns; and obtaining the pixel coordinates of the intersection of the annular structured light and the chessboard straight line in each group of calibration patterns according to the pixel coordinates of the annular structured light; (3) Using the cross ratio invariance principle, the pixel coordinates of the intersection points of the annular structured light and the checkerboard straight lines in each calibration pattern are converted into three-dimensional coordinates in the world coordinate system; then, the three-dimensional coordinates in the world coordinate system corresponding to the intersection points of the annular structured light and the checkerboard straight lines in each calibration pattern are converted into three-dimensional coordinates in the camera coordinate system through camera calibration; (4) Fit the three-dimensional coordinates of all intersection points in the camera coordinate system into a conic surface equation; establish a mathematical model of the annular structured light vision sensor based on the fitted conic surface equation to complete the calibration; the specific mathematical model of the annular structured light vision sensor is: where S is the scale transformation factor; (u, v, 1) are the intersection point pixel coordinates; f x , f y , u0, v0 are both the camera internal parameters; (X c , Y c , Z c ) are the three-dimensional coordinates of the intersection point in the camera coordinate system; l0 to l 10 are all the coefficients of the fitted conical surface equation.

2. The annular structured light calibration method based on a planar checkerboard according to claim 1, wherein The pixel coordinates of the intersection point in step (2) are obtained as follows: (2.1) For each group of calibration patterns, the mask of the annular structured light in the calibration pattern is extracted by subtracting the calibration pattern taken with the annular structured light turned on and the calibration pattern taken with the annular structured light turned off at the same posture, and then the annular structured light in each group of calibration patterns is segmented by image processing; (2.2) Using the Steger center point sub-pixel extraction algorithm, extract the sub-pixel coordinates of the annular structured light in each set of calibration patterns; (2.3) The sub-pixel coordinates of the extracted annular structured light corresponding to each set of calibration patterns are fitted into elliptical equations, and the straight lines on the chessboard are fitted into straight line equations. The elliptical equation and the straight line equation are combined to obtain the pixel coordinates of the intersection of the annular structured light and the straight lines on the chessboard in each set of calibration patterns.

3. The ring structured light calibration method based on a planar checkerboard according to claim 1, wherein Step (3) specifically includes: (3.1) The pixel coordinates of the intersection of the ring structured light and the chessboard straight line in each set of calibration patterns are converted into image coordinates. The formula is: Where 1 / dx and 1 / dy represent the camera intrinsic parameters, (x, y, z) represents the image coordinates, and z is normalized; (3.2) Using the principle of cross-ratio invariance, (x, y, z) is transformed into (X w , Y w , Z w ) in the world three-dimensional coordinates based on the condition that the cross-ratio in the image coordinate system is equal to the cross-ratio in the world coordinate system; (3.3) Use the camera's intrinsic parameters to transform the world's three-dimensional coordinates into the three-dimensional coordinates in the camera coordinate system. The formula is: Where R and T are the external parameters of the camera.

4. The annular structured light calibration method based on a planar checkerboard according to claim 1, characterized in that The three-dimensional coordinates of all intersection points in step (4) corresponding to the camera coordinate system are fitted into the conic surface equation by the least squares method.

5. The ring structured light calibration method based on a planar checkerboard according to claim 1, wherein In step (3), the camera is calibrated by using the calibration patterns photographed under the condition of turning off the annular structured light in each group of calibration patterns to obtain the intrinsic and extrinsic parameters of the camera.

6. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method described in any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images