A cross structure light plane calibration method

Abstract

The application provides a cross structured light plane calibration method, which is used to solve the technical problem that when cross structured light is used as a light source for shooting, the laser stripe contained in the shot image belongs to two laser planes, which causes matching error in the process of laser plane calibration. The cross structured light plane calibration method of the application first divides the laser line image into two parts of laser stripe and background, then extracts the laser stripe center line on each segmented laser line image by using an image thinning algorithm, so that the accurate extraction of the laser stripe center line is realized; and then based on the principal component analysis method of K nearest neighbors and the normal vector consistency, a classification criterion is constructed, which can quickly and accurately determine whether the laser point on the laser stripe center line belongs to the first laser plane or the second laser plane, successfully solves the point cloud belonging ambiguity problem of the double-plane laser line, and realizes the accurate solution and calibration of the cross laser plane parameters.

Description

Technical Field

[0001] This invention relates to a planar calibration method for lasers, specifically a planar calibration method for a cross-shaped structured light. Background Technology

[0002] One of the core tasks in the field of machine vision is to acquire image information through a camera and calculate the geometric features of objects in three-dimensional space, thereby achieving object reconstruction and recognition. The correspondence between the three-dimensional coordinates of a point on the surface of an object in space and that point in the image is determined by the camera's imaging geometry model, i.e., camera parameters. Camera parameters are usually obtained through camera calibration experiments, mainly including the camera's geometric and optical parameters, as well as the camera's pose relative to the world coordinate system. Laser plane calibration aims to obtain the plane equation of the laser plane in the camera coordinate system, thereby determining the depth information of pixels on the laser line in the image. Only after completing both camera calibration and laser plane calibration can the three-dimensional information of the target object be accurately obtained.

[0003] Structured light, a commonly used laser, uses two laser planes that complement each other, providing richer edge and directional information for target measurement. Furthermore, the different incident angles of the two laser planes compensate for the loss of a single laser line. However, when using structured light as a light source for imaging, the laser stripes in the captured image belong to two separate laser planes, leading to matching errors during straight-line fitting in laser plane calibration. Summary of the Invention

[0004] The purpose of this invention is to solve the technical problem that when using cross-shaped structured light as a light source for shooting, the laser stripes in the captured image belong to two laser planes, which leads to matching errors when performing straight line fitting during the laser plane calibration process. The invention provides a cross-shaped structured light plane calibration method.

[0005] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0006] A planar calibration method for a cross-shaped structured light, characterized by the following steps:

[0007] Step 1: Prepare the chessboard calibration board and collect multiple sets of image data of the chessboard calibration board in different postures; each set of image data includes one normal light image and one laser line image in the same posture. The normal light image is the image collected with the ambient light on and the laser off; the laser line image is the image collected with the ambient light off and the laser on.

[0008] Step 2: Calibrate the camera intrinsic parameter matrix and obtain the 3D coordinates of all corner points of the checkerboard calibration board in the world coordinate system under each pose based on all normal light images; solve the extrinsic parameter matrix corresponding to each normal light image based on the camera intrinsic parameter matrix and the 3D coordinates of all corner points of the checkerboard calibration board in the world coordinate system under each pose.

[0009] Step 3: Divide the multiple laser line images into two parts: laser stripes and background.

[0010] Step 4: Use an image thinning algorithm to extract the center line of the laser stripe in each segmented laser line image;

[0011] Step 5: Using the principal component analysis method based on K-nearest neighbors, calculate the normal vector of all laser points on the center line of each laser stripe, and classify all laser points on the center line of each laser stripe based on the consistency of the normal vectors to determine whether they are located in the first laser plane or the second laser plane of the cross structured light.

[0012] Step 6: Use the least squares method to perform line fitting on the laser points located in the first laser plane and the laser points located in the second laser plane after classifying each laser stripe center line, to obtain the first target fitting line and the second target fitting line corresponding to all laser stripe center lines.

[0013] Step 7: Combine the camera's intrinsic parameter matrix and the extrinsic parameter matrix corresponding to each normal light image, and use the least squares method to perform plane fitting on the first target fitting line and the second target fitting line corresponding to the center line of all laser stripes, respectively, to obtain the fitted first laser plane and the second laser plane, thus completing the plane calibration of the cross structured light.

[0014] Furthermore, step 3 specifically involves:

[0015] Step 3.1: Median filtering is used to denoise multiple laser line images;

[0016] Step 3.2: Perform contrast stretching on each denoised laser line image to obtain a contrast-stretched laser line image; define the original grayscale value range of the denoised laser line image as... The target grayscale value range is For each original gray value in the denoised laser line image Its stretched grayscale value for:

[0017] ;

[0018] Step 3.3: Calculate the grayscale histogram of the contrast-stretched laser line image, and determine the segmentation threshold of the corresponding contrast-stretched laser line image by observing the peaks and valleys of the corresponding grayscale histogram; according to the segmentation threshold, the corresponding contrast-stretched laser line image is segmented into two parts: laser stripes and background.

[0019] Furthermore, step 4 specifically involves:

[0020] Step 4.1: Select any segmented laser line image for scanning;

[0021] Step 4.2: Record any pixel on the laser stripe of the currently segmented laser line image as a foreground pixel. Foreground pixels Centered on, The matrix formed by it and its 8 neighboring pixels can be represented as , Foreground pixel The 8-neighbor pixels; let =1, if If it is a background pixel, then =0, otherwise =1, k=1,2,…8;

[0022] Step 4.3, Define the first judgment value Second judgment value , , k=1,2,…8; Determine the current foreground pixel Is it a boundary point? and and and Then it is considered that the current foreground pixel For boundary points, delete the current foreground pixels. Otherwise, retain the current foreground pixel. ;

[0023] Step 4.4, further determine which current foreground pixels to retain. Is it a boundary point? and and and Then it is considered that the current foreground pixel For boundary points, delete the current foreground pixels. Otherwise, retain the current foreground pixel. ;

[0024] Step 4.5: Traverse all pixels on the laser stripes of the current segmented laser line image, and delete all pixels that are judged as boundary points on the laser stripes of the current segmented laser line image according to the methods in Steps 4.2-4. Extract the coordinates of all remaining pixels on the laser stripes of the current segmented laser line image to obtain the center line of the laser stripes of the current segmented laser line image.

[0025] Step 4.6: Traverse all segmented laser line images and delete the pixels that are judged as boundary points on the laser stripes of each segmented laser line image according to the methods in Steps 4.1-4.5. Extract the coordinates of all remaining pixels on the laser stripes of each segmented laser line image to obtain the center line of the laser stripes of all segmented laser line images.

[0026] Furthermore, step 5 specifically includes:

[0027] Step 5.1: Select any laser point on the center line of any laser stripe as the current laser point, and determine all K nearest neighbors of the current laser point;

[0028] Step 5.2: Calculate the mean of all K nearest neighbors of the current laser point to obtain the covariance matrix of the current laser point; perform eigenvalue decomposition on the covariance matrix of the current laser point to obtain the minimum eigenvalue, and the eigenvector corresponding to the minimum eigenvalue is the normal vector of the current laser point.

[0029] Step 5.3: Calculate the normal vector of all laser points on the current laser stripe center line according to the method in steps 5.1-5.2;

[0030] Step 5.4: Take any laser point on the current laser stripe center line as the reference laser point, and define the plane containing the reference laser point as the first laser plane of the cross-shaped structured light; calculate the cosine of the angle between the normal vector of other laser points on the current laser stripe center line and the normal vector of the reference laser point. ;

[0031] Step 5.5, Set the plane determination threshold ,when When the angle is in the first laser plane, the laser point corresponding to the included angle is located in the first laser plane of the cross structured light; otherwise, the laser point corresponding to the included angle is located in the second laser plane of the cross structured light.

[0032] Step 5.6: Using the methods in steps 5.1-5.5, classify all laser points on the center line of each laser stripe and determine whether they are located on the first laser plane or the second laser plane of the cross structured light.

[0033] Furthermore, step 6 specifically includes:

[0034] Step 6.1: Arbitrarily select a laser stripe center line. Denote the equation of the straight line fitted by all laser points on the first laser plane of the cross-shaped structured light along the current laser stripe center line as the first target fitted line. The equation of the first target fitted line is then: , The slope of the fitted line for the first objective. The y-intercept of the line fitted to the first target is defined as: The laser point on the first laser plane of the cross-shaped structured light is defined as... , n is the number of laser points on the first laser plane; all laser points If each observation point is used as an observation point, then the distance between each observation point and the fitted line of the first target is... for:

[0035] ;

[0036] Step 6.2: Accumulate the deviations of all observation points from the fitted line to the first target. Then we have:

[0037] ;

[0038] Step 6.3, taking the first derivative of the above equation and finding it to be zero, yields:

[0039] ;

[0040] Then the slope of the first target fitted line The y-intercept of the first target fitted line They are respectively:

[0041] ;

[0042] Complete the first target fitted straight line solution for the current laser stripe centerline;

[0043] Step 6.4: Denote the equation of the straight line fitted by all laser points on the second laser plane of the cross structured light on the current laser stripe center line as the second target fitted line, and solve for the second target fitted line of the current laser stripe center line using the methods in steps 6.1-6.3.

[0044] Step 6.5: Traverse all laser stripe center lines and solve for the first target fitting line and the second target fitting line corresponding to each laser stripe center line according to the methods in steps 6.1-6.4.

[0045] Furthermore, step 7 specifically includes:

[0046] Step 7.1, let the plane equation of the first laser plane be... , The three parameters together constitute the normal vector of the first laser plane. It is related to the distance from the first laser plane to the origin of the coordinate system;

[0047] Step 7.2: Select the two laser points at both ends of the first target fitting line corresponding to the center lines of all laser stripes. Using the camera's intrinsic parameter matrix and the extrinsic parameter matrix corresponding to each normal light image, transform the two laser points corresponding to the first target fitting line of each laser stripe center line to the corresponding world coordinate system, forming a three-dimensional laser point. Use this three-dimensional laser point as the observation point for solving the first laser plane equation. The observation point can then be represented as... , , The number of image sets of the chessboard calibration board under different poses;

[0048] Step 7.3, calculate the observation point using the following formula. centroid coordinates ( ):

[0049] ;

[0050] Step 7.4, based on the observation point centroid coordinates ( Solve for the covariance matrix. :

[0051]

[0052] in , , , , , ;

[0053] Step 7.5, for the covariance matrix Perform eigenvalue decomposition to obtain the minimum eigenvalue, and the eigenvector corresponding to the minimum eigenvalue. As the normal vector of the first laser plane;

[0054] Step 7.6, solve for the plane equation of the first laser plane using the following formula:

[0055]

[0056] Step 7.7: Solve the plane equation of the second laser plane using the methods in Steps 7.1-7.6 to complete the plane calibration of the cross structured light.

[0057] Furthermore, in step 2, the camera intrinsic parameter matrix is ​​calibrated using Zhang's calibration method.

[0058] Furthermore, in step 2, the extrinsic parameter matrix corresponding to each normal light image is solved using the PnP (Perspective-n-Point) algorithm.

[0059] Furthermore, in step 3.3, the segmentation threshold is 228.

[0060] Further, in step 5.5, the determination threshold... =0.93.

[0061] Compared with the prior art, the present invention has the following beneficial technical effects:

[0062] 1. This invention provides a planar calibration method for cross-shaped structured light. First, the laser line image is segmented into two parts: laser stripes and background. Then, an image thinning algorithm is used to extract the center line of the laser stripes on each segmented laser line image, effectively overcoming the shortcomings of the traditional threshold method in extracting the laser center line and achieving accurate extraction of the laser stripe center line. Furthermore, a classification criterion is constructed based on the K-nearest neighbor principal component analysis method and normal vector consistency, which can quickly and accurately determine whether the laser points on the laser stripe center line belong to the first laser plane or the second laser plane, successfully solving the problem of point cloud classification ambiguity in dual-plane laser lines. Then, through the classification results, line fitting and plane fitting are performed to achieve accurate solution and calibration of the cross-shaped laser plane parameters.

[0063] 2. The planar calibration method for cross-shaped structured light provided by this invention is convenient to operate and solves the matching problem when performing straight line fitting during laser planar calibration. It can be widely applied to the planar calibration of various cross-shaped structured lights. Attached Figure Description

[0064] Figure 1 This is a flowchart of a planar calibration method for a cross-shaped structured light according to the present invention;

[0065] Figure 2 The images shown are normal light images and laser line images of the checkerboard calibration board in different postures in step 1 of the present invention. Among them, (a) is the normal light image of the checkerboard calibration board in the first posture, (b) is the laser line image of the checkerboard calibration board in the first posture, (c) is the normal light image of the checkerboard calibration board in the second posture, and (d) is the laser line image of the checkerboard calibration board in the second posture.

[0066] Figure 3 A schematic diagram illustrating coordinate system transformation during the camera imaging process;

[0067] Figure 4The diagrams shown are schematic diagrams of the first and second target fitting lines obtained by the checkerboard calibration plate under different postures in step 6 of the present invention. (a) is a schematic diagram of the first and second target fitting lines corresponding to the center line of the laser stripe extracted by the checkerboard calibration plate in the first posture. (b) is a schematic diagram of the first and second target fitting lines corresponding to the center line of the laser stripe extracted by the checkerboard calibration plate in the second posture. In the diagrams, the green line is the first target fitting line and the blue line is the second target fitting line.

[0068] Figure 5 This is a schematic diagram of the geometric model of laser intersection imaging in step 7.2 of this embodiment of the invention. Detailed Implementation

[0069] To make the objectives, advantages, and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that these embodiments are merely used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0070] A planar calibration method for cross-shaped structured lights, such as Figure 1 As shown, the specific steps include:

[0071] Step 1, image data acquisition.

[0072] A high-precision checkerboard calibration board is prepared, with known physical dimensions of the checkerboard and the number of internal corner points. Multiple sets of image data are acquired from the checkerboard calibration board in different poses. Each set of image data includes one normal light image and one laser line image for the same pose, and the pose of the checkerboard calibration board differs between different sets of image data. The normal light image, acquired with ambient light on and laser off, is used to identify the corner points of the checkerboard calibration board; the laser line image, acquired with ambient light off and laser on, is used to extract laser stripes. Figure 2 (a) is the normal light image of the checkerboard calibration board in the first posture, (b) is the laser line image of the checkerboard calibration board in the first posture, (c) is the normal light image of the checkerboard calibration board in the second posture, and (d) is the laser line image of the checkerboard calibration board in the second posture.

[0073] When acquiring laser line images, the laser line must clearly pass through multiple grid squares on the checkerboard calibration board; different orientations of the checkerboard calibration board can cause the laser line to appear in different positions in the laser line image.

[0074] Step 2: Obtain the camera's intrinsic parameter matrix and the extrinsic parameter matrix corresponding to each normal light image.

[0075] Camera calibration yields the geometric model of camera imaging, i.e., the correspondence between points in three-dimensional space and points in a two-dimensional image. The camera imaging process can be understood as a transformation from the world coordinate system to the pixel coordinate system. Without considering distortion, this transformation process is as follows: Figure 3 As shown, the rotation and translation relationship between the camera coordinate system and the world coordinate system is the camera extrinsic parameter matrix, and the transformation relationship from the camera coordinate system to the image coordinate system and then to the pixel coordinate system is the camera intrinsic parameter matrix. The elements in the camera intrinsic parameter matrix mainly include focal length, pixel size, etc. In reality, due to radial distortion and tangential distortion, the elements in the camera intrinsic parameter matrix also need to consider distortion parameters.

[0076] Therefore, the camera intrinsic parameter matrix is ​​first calibrated using Zhang's calibration method; then, the three-dimensional coordinates of all corner points of the checkerboard calibration board in each pose are obtained in the world coordinate system based on all normal light images. Specifically, this is done according to the preset parameters of the checkerboard calibration board. For example, in this embodiment, the grid width dx=25mm, the grid height dy=25mm, and the number of internal corner points is 11×8. An origin is set, such as setting the upper left corner point as the origin (0,0,0) in the world coordinate system. Along the horizontal direction of the checkerboard calibration board, the next corner point is located in the world coordinate system. The three-dimensional coordinates of the next corner point in the world coordinate system are (dx, 0, 0). The three-dimensional coordinates of the next corner point in the world coordinate system are (2dx, 0, 0). Along the longitudinal direction of the checkerboard calibration board, the three-dimensional coordinates of the next corner point in the world coordinate system are (0, dy, 0), and the three-dimensional coordinates of the next corner point in the world coordinate system are (0, 2dy, 0), and so on. By analogy, the three-dimensional coordinates of all corner points of the checkerboard calibration board in each pose can be obtained in the world coordinate system. This realizes the transformation of all corner points of the checkerboard calibration board in each pose from the camera coordinate system to the world coordinate system.

[0077] Then, based on the camera intrinsic parameter matrix and the 3D coordinates of all corner points in the world coordinate system under each pose of the checkerboard calibration board, the extrinsic parameter matrix of the camera coordinate system relative to the world coordinate system under each pose of the checkerboard calibration board is solved using the PnP (Perspective-n-Point) algorithm. This solves for the extrinsic parameter matrix corresponding to each normal light image. The specific formula is P_camera = R * P_world + t, where P_camera is the 3D coordinate of the corner point in the camera coordinate system, P_world is the 3D coordinate of the corner point in the world coordinate system, R is the rotation matrix of the extrinsic parameter matrix, and t is the translation vector of the extrinsic parameter matrix.

[0078] Step 3, image preprocessing, divides the laser line image into two parts: laser stripes and background.

[0079] After solving for the extrinsic parameter matrix corresponding to each normal light image, laser plane calibration first requires the extraction of the laser stripe center line. In order to accurately obtain the basic features of the original image, image preprocessing is often required.

[0080] Step 3.1: First, median filtering is used to denoise multiple laser line images.

[0081] Step 3.2: Perform contrast stretching on each denoised laser line image to obtain contrast-stretched laser line images. The contrast between the laser stripes and the background is enhanced by adjusting the grayscale value range of the image.

[0082] Specifically, the original grayscale value range of the denoised laser line image is defined as follows: The target grayscale value range is For each original gray value in the denoised laser line image Its stretched grayscale value for:

[0083] ;

[0084] Step 3.3: Calculate the grayscale histogram of the contrast-stretched laser line image. Determine the segmentation threshold of the contrast-stretched laser line image by observing the peaks and troughs of the corresponding grayscale histogram. In this embodiment, the segmentation threshold is set to 228. Finally, the contrast-stretched laser line image is segmented into two parts: laser stripes and background, based on the segmentation threshold.

[0085] Step 4: Use an image thinning algorithm to extract the center line of the laser stripes.

[0086] Step 4.1: Select any segmented laser line image for scanning.

[0087] Step 4.2: Record any pixel on the laser stripe of the currently segmented laser line image as a foreground pixel. ; with the current foreground pixel Centered on, The matrix formed by it and its 8 neighboring pixels can be represented as , Let P0 be the 8 neighboring pixels of the current foreground pixel P0; =1, if If it is a background pixel, then =0, otherwise =1, k=1,2,…8.

[0088] Step 4.3, Define the first judgment value Second judgment value , , k=1,2,…8; Determine the current foreground pixel Is it a boundary point? and and and Then it is considered that the current foreground pixel For boundary points, delete the current foreground pixels. Otherwise, retain the current foreground pixel. .in, Used to ensure the current foreground pixel There are enough foreground pixels in the neighborhood of the current foreground pixel. It is neither an isolated point nor a point about to break off; =1 is used to ensure the current foreground pixel The neighborhood of the laser stripe is singly connected, ensuring that the laser stripe will not break after deletion. Used to ensure At least one of the three pixels is a background pixel, meaning at least one is a true boundary point; Used to ensure At least one of the three pixels is a background pixel.

[0089] Step 4.4, further determine which current foreground pixels to retain. Is it a boundary point? and and and Then it is considered that the current foreground pixel The boundary point indicates that the laser stripes can be further refined, so the current foreground pixel is deleted. Otherwise, it means the laser stripe has been refined to a center line with a width of one pixel, retaining the current foreground pixel. .

[0090] Step 4.5: Traverse all pixels on the laser stripes of the currently segmented laser line image, and delete all pixels that are judged as boundary points on the laser stripes of the currently segmented laser line image according to the methods in steps 4.2-4. Extract the coordinates of all remaining pixels on the laser stripes of the currently segmented laser line image to obtain the center line of the laser stripes of the currently segmented laser line image.

[0091] Step 4.6: Traverse all segmented laser line images and delete the pixels that are judged as boundary points on the laser stripes of each segmented laser line image according to the methods in Steps 4.1-4.5. Extract the coordinates of all remaining pixels on the laser stripes of each segmented laser line image to obtain the center line of the laser stripes of all segmented laser line images.

[0092] Step 5: Classify the laser points based on the principal component analysis method of K-nearest neighbors and the consistency of normal vectors.

[0093] Since the extracted laser stripe centerlines belong to two laser planes of the cross-shaped structured light, they need to be classified. This invention employs principal component analysis based on K-nearest neighbors to calculate the normal vectors of all laser points on each laser stripe centerline. Based on the consistency of these normal vectors, all laser points on each laser stripe centerline are classified to determine whether they lie on the first or second laser plane of the cross-shaped structured light. Specifically:

[0094] Step 5.1: Select any laser point on the center line of any laser stripe as the current laser point, and determine all K nearest neighbors of the current laser point.

[0095] Step 5.2: Calculate the mean of all K nearest neighbors of the current laser point to obtain the covariance matrix of the current laser point; perform eigenvalue decomposition on the covariance matrix of the current laser point to obtain the minimum eigenvalue, and the eigenvector corresponding to the minimum eigenvalue is the normal vector of the current laser point.

[0096] Step 5.3: Calculate the normal vector of all laser points on the center line of the current laser stripe according to the method in steps 5.1-5.2.

[0097] Step 5.4: Take any laser point on the current laser stripe center line as the reference laser point, and define the plane containing the reference laser point as the first laser plane of the cross-shaped structured light; calculate the normal vectors of other laser points on the current laser stripe center line respectively. Normal vector of the reference laser point The cosine of the angle between them , .

[0098] Step 5.5, Set the plane determination threshold After experimental testing, the plane determination threshold was determined. The classification effect is best when the value is 0.93. Values ​​less than 0.93 may result in misclassification of some laser points, but these misclassifications will be corrected after subsequent laser line fitting. In this embodiment... =0.93, when When the angle is in the first position, the laser point corresponding to the angle is located on the first laser plane of the cross-shaped structured light; otherwise, the laser point corresponding to the angle is located on the second laser plane of the cross-shaped structured light.

[0099] Step 5.6: Using the methods in steps 5.1-5.5, classify all laser points on the center line of each laser stripe and determine whether they are located on the first laser plane or the second laser plane of the cross structured light.

[0100] Step 6, laser line fitting.

[0101] The least squares method is used to fit a straight line to the laser points located in the first laser plane after classifying each laser stripe center line, obtaining the first target fitted line. At the same time, a straight line is fitted to the laser points located in the second laser plane after classifying each laser stripe center line, obtaining the second target fitted line, as follows:

[0102] Step 6.1: Arbitrarily select a laser stripe center line. Denote the equation of the straight line fitted by all laser points on the first laser plane of the cross-shaped structured light along the current laser stripe center line as the first target fitted line. The equation of the first target fitted line is then: , The slope of the fitted line for the first objective. The y-intercept of the line fitted to the first target is defined as: The laser point on the first laser plane of the cross-shaped structured light is defined as... , n is the number of laser points on the first laser plane; all laser points If each observation point is used as an observation point, then the distance between each observation point and the fitted line of the first target is... for:

[0103] .

[0104] Step 6.2: Accumulate the deviations of all observation points from the fitted line to the first target. Then we have:

[0105] .

[0106] Step 6.3, taking the first derivative of the above equation and finding it to be zero, yields:

[0107] ;

[0108] Then the slope of the first target fitted line The y-intercept of the first target fitted line They are respectively:

[0109] ;

[0110] Complete the first target fitted straight line solution for the current laser stripe center line.

[0111] Step 6.4: Denote the equation of the straight line fitted by all laser points on the second laser plane of the cross structured light on the current laser stripe center line as the second target fitted line, and solve for the second target fitted line of the current laser stripe center line using the methods in steps 6.1-6.3.

[0112] Step 6.5: Traverse all laser stripe center lines and solve for the first target fitting line and the second target fitting line corresponding to each laser stripe center line according to the methods in steps 6.1-6.4.

[0113] The results of fitting the checkerboard calibration board under different postures are as follows: Figure 4 As shown, (a) is a schematic diagram of the first target fitting line and the second target fitting line corresponding to the center line of the laser stripe extracted by the checkerboard calibration board in the first posture, and (b) is a schematic diagram of the first target fitting line and the second target fitting line corresponding to the center line of the laser stripe extracted by the checkerboard calibration board in the second posture; wherein, the green line is the first target fitting line, and the laser point on it is the laser point on the first laser plane, and the blue line is the second target fitting line, and the laser point on it is the laser point on the second laser plane.

[0114] Step 7: Calculate the laser plane parameters.

[0115] By combining the camera's intrinsic parameter matrix and the extrinsic parameter matrix corresponding to each normal light image, and using the least squares method to perform plane fitting on the first target fitting line corresponding to the center lines of all laser stripes, a fitted first laser plane is obtained. Simultaneously, plane fitting is performed on the second target fitting line corresponding to the center lines of all laser stripes, resulting in a fitted second laser plane, as detailed below:

[0116] Step 7.1, let the plane equation of the first laser plane be... , The three parameters together constitute the normal vector of the first laser plane. This is related to the distance from the first laser plane to the origin. The first laser plane can be represented by a normal vector and any point on the first laser plane. Therefore, solving the plane equation of the first laser plane is equivalent to solving for the normal vector of the first laser plane and any point on the first laser plane.

[0117] Step 7.2, taking the first laser plane in the cross-shaped structured light as an example, the geometric model of its laser intersection imaging is as follows: Figure 5 As shown, where If the laser plane is used to acquire N sets of images of the checkerboard calibration board in different poses, then... These are the calibration planes of the chessboard grid under different orientations. These are the fitted straight lines for the first target under different postures, that is, the fitted straight lines for the first target corresponding to the center lines of different laser stripes.

[0118] Select two laser points at both ends of the first target fitting line corresponding to the center line of all laser stripes. The coordinates of the two selected laser points are as follows: and Using the camera's intrinsic parameter matrix and the extrinsic parameter matrix corresponding to each normal light image, the two laser points corresponding to the first target fitting line of each laser stripe centerline are transformed to the corresponding world coordinate system, forming a three-dimensional laser point. This three-dimensional laser point is then used as the observation point for solving the first laser plane equation. The observation point can then be represented as... , .

[0119] Step 7.3, calculate the observation point using the following formula. centroid coordinates ( ):

[0120] .

[0121] Step 7.4, based on the observation point centroid coordinates ( Solve for the covariance matrix. :

[0122]

[0123] in, , , , , , .

[0124] Step 7.5, for the covariance matrix Perform eigenvalue decomposition to obtain the minimum eigenvalue, and the eigenvector corresponding to the minimum eigenvalue. As the normal vector of the first laser plane;

[0125] Step 7.6, solve for the plane equation of the first laser plane using the following formula:

[0126]

[0127] Step 7.7: Solve the plane equation of the second laser plane using the methods in Steps 7.1-7.6 to complete the plane calibration of the cross structured light.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present invention.

Claims

1. A planar calibration method for a cross-shaped structured light, characterized in that, Includes the following steps: Step 1: Prepare the chessboard calibration board and collect multiple sets of image data of the chessboard calibration board in different postures; each set of image data includes one normal light image and one laser line image in the same posture. The normal light image is the image collected with the ambient light on and the laser off; the laser line image is the image collected with the ambient light off and the laser on. Step 2: Calibrate the camera intrinsic parameter matrix and obtain the 3D coordinates of all corner points of the checkerboard calibration board in the world coordinate system under each pose based on all normal light images; solve the extrinsic parameter matrix corresponding to each normal light image based on the camera intrinsic parameter matrix and the 3D coordinates of all corner points of the checkerboard calibration board in the world coordinate system under each pose. Step 3: Divide each laser line image into two parts: laser stripes and background. Step 4: Use an image thinning algorithm to extract the center line of the laser stripe in each segmented laser line image; Step 5: Using the principal component analysis method based on K-nearest neighbors, calculate the normal vector of all laser points on the center line of each laser stripe, and classify all laser points on the center line of each laser stripe based on the consistency of the normal vectors to determine whether they are located in the first laser plane or the second laser plane of the cross structured light. Step 6: Use the least squares method to perform line fitting on the laser points located in the first laser plane and the laser points located in the second laser plane after classifying each laser stripe center line, to obtain the first target fitting line and the second target fitting line corresponding to all laser stripe center lines. Step 7: Combine the camera's intrinsic parameter matrix and the extrinsic parameter matrix corresponding to each normal light image, and use the least squares method to perform plane fitting on the first target fitting line and the second target fitting line corresponding to the center line of all laser stripes, respectively, to obtain the fitted first laser plane and the second laser plane, thus completing the plane calibration of the cross structured light.

2. The planar calibration method for a cross-shaped structured light according to claim 1, characterized in that, Step 3 specifically involves: Step 3.1: Median filtering is used to denoise multiple laser line images; Step 3.2: Perform contrast stretching on each denoised laser line image to obtain a contrast-stretched laser line image; define the original grayscale value range of the denoised laser line image as... The target grayscale value range is For each original gray value in the denoised laser line image Its stretched grayscale value for: ； Step 3.3: Calculate the grayscale histogram of the contrast-stretched laser line image, and determine the segmentation threshold of the corresponding contrast-stretched laser line image by observing the peaks and valleys of the corresponding grayscale histogram; according to the segmentation threshold, the corresponding contrast-stretched laser line image is segmented into two parts: laser stripes and background.

3. The planar calibration method for a cross-shaped structured light according to claim 2, characterized in that, Step 4 is as follows: Step 4.1: Select any segmented laser line image for scanning; Step 4.2: Record any pixel on the laser stripe of the currently segmented laser line image as a foreground pixel. Foreground pixels Centered on, The matrix formed by its 8 neighboring pixels is represented as , ~ Foreground pixel The 8-neighbor pixels; let =1, if If it is a background pixel, then =0, otherwise =1, k=1,2,…8; Step 4.3, Define the first judgment value Second judgment value , , , k=1,2,…8; Determine the current foreground pixel Is it a boundary point? and and and Then it is considered that the current foreground pixel For boundary points, delete the current foreground pixels. Otherwise, retain the current foreground pixel. ; Step 4.4, further determine which current foreground pixels to retain. Is it a boundary point? and and and Then it is considered that the current foreground pixel For boundary points, delete the current foreground pixels. Otherwise, retain the current foreground pixel. ; Step 4.5: Traverse all pixels on the laser stripes of the current segmented laser line image, and delete all pixels that are judged as boundary points on the laser stripes of the current segmented laser line image according to the methods in Steps 4.2-4. Extract the coordinates of all remaining pixels on the laser stripes of the current segmented laser line image to obtain the center line of the laser stripes of the current segmented laser line image. Step 4.6: Traverse all segmented laser line images and delete the pixels that are judged as boundary points on the laser stripes of each segmented laser line image according to the methods in Steps 4.1-4.

5. Extract the coordinates of all remaining pixels on the laser stripes of each segmented laser line image to obtain the center line of the laser stripes of all segmented laser line images.

4. The planar calibration method for a cross-shaped structured light according to claim 3, characterized in that, Step 5 specifically involves: Step 5.1: Select any laser point on the center line of any laser stripe as the current laser point, and determine all K nearest neighbors of the current laser point; Step 5.2: Calculate the mean of all K nearest neighbors of the current laser point to obtain the covariance matrix of the current laser point; perform eigenvalue decomposition on the covariance matrix of the current laser point to obtain the minimum eigenvalue, and the eigenvector corresponding to the minimum eigenvalue is the normal vector of the current laser point. Step 5.3: Calculate the normal vector of all laser points on the current laser stripe center line according to the method in steps 5.1-5.2; Step 5.4: Take any laser point on the current laser stripe center line as the reference laser point, and define the plane containing the reference laser point as the first laser plane of the cross-shaped structured light; calculate the cosine of the angle between the normal vector of other laser points on the current laser stripe center line and the normal vector of the reference laser point. ; Step 5.5, Set the plane determination threshold ,when When the angle is in the first laser plane, the laser point corresponding to the included angle is located in the first laser plane of the cross structured light; otherwise, the laser point corresponding to the included angle is located in the second laser plane of the cross structured light. Step 5.6: Using the methods in steps 5.1-5.5, classify all laser points on the center line of each laser stripe and determine whether they are located on the first laser plane or the second laser plane of the cross structured light.

5. The planar calibration method for a cross-shaped structured light according to claim 4, characterized in that, Step 6 specifically involves: Step 6.1: Arbitrarily select a laser stripe center line. Denote the equation of the straight line fitted by all laser points on the first laser plane of the cross-shaped structured light along the current laser stripe center line as the first target fitted line. The equation of the first target fitted line is then: , The slope of the fitted line for the first objective. The y-intercept of the line fitted to the first target is defined as: The laser point on the first laser plane of the cross-shaped structured light is defined as... , n is the number of laser points on the first laser plane; all laser points If each observation point is used as an observation point, then the distance between each observation point and the fitted line of the first target is... for: ； Step 6.2: Accumulate the deviations of all observation points from the fitted line to the first target. Then we have: ； Step 6.3, taking the first derivative of the above equation and finding it to be zero, yields: ； Then the slope of the first target fitted line The y-intercept of the first target fitted line They are respectively: ； Complete the first target fitted straight line solution for the current laser stripe centerline; Step 6.4: Denote the equation of the straight line fitted by all laser points on the second laser plane of the cross structured light on the current laser stripe center line as the second target fitted line, and solve for the second target fitted line of the current laser stripe center line using the methods in steps 6.1-6.

3. Step 6.5: Traverse all laser stripe center lines and solve for the first target fitting line and the second target fitting line corresponding to each laser stripe center line according to the methods in steps 6.1-6.

4.

6. The planar calibration method for a cross-shaped structured light according to claim 5, characterized in that, Step 7 specifically includes: Step 7.1, let the plane equation of the first laser plane be... , The three parameters together constitute the normal vector of the first laser plane. It is related to the distance from the first laser plane to the origin of the coordinate system; Step 7.2: Select the two laser points at both ends of the first target fitting line corresponding to the center lines of all laser stripes. Using the camera's intrinsic parameter matrix and the extrinsic parameter matrix corresponding to each normal light image, transform the two laser points corresponding to the first target fitting line of each laser stripe center line to the corresponding world coordinate system, forming a three-dimensional laser point. Use this three-dimensional laser point as the observation point for solving the first laser plane equation. The observation point is then represented as... , , The number of image sets of the chessboard calibration board under different poses; Step 7.3, calculate the observation point using the following formula. centroid coordinates ( ): ； Step 7.4, based on the observation point centroid coordinates ( Solve for the covariance matrix. : ； in, , , , , , ; Step 7.5, for the covariance matrix Perform eigenvalue decomposition to obtain the minimum eigenvalue, and the eigenvector corresponding to the minimum eigenvalue. As the normal vector of the first laser plane; Step 7.6, solve for the plane equation of the first laser plane using the following formula: ； Step 7.7: Solve the plane equation of the second laser plane using the methods in Steps 7.1-7.6 to complete the plane calibration of the cross structured light.

7. The planar calibration method for a cross-shaped structured light according to claim 1, characterized in that: In step 2, the camera intrinsic parameter matrix is ​​calibrated using Zhang's calibration method.

8. The planar calibration method for a cross-shaped structured light according to claim 7, characterized in that: In step 2, the extrinsic parameter matrix corresponding to each normal light image is solved using the PnP (Perspective-n-Point) algorithm.

9. The planar calibration method for a cross-shaped structured light according to claim 2, characterized in that: In step 3.3, the segmentation threshold is 228.

10. The planar calibration method for a cross-shaped structured light according to claim 4, characterized in that: In step 5.5, the determination threshold =0.93.

Images