Calibration Method and Three-Dimensional Reconstruction Method Based on High-Precision Line Laser 3D Camera

Abstract

The present invention discloses a calibration method and a three-dimensional reconstruction method based on a high-precision line laser 3D camera. The method includes placing a calibration board on a working plane according to a preset rule and photographing the calibration board to obtain a plurality of first calibration board images; converting the first calibration board images into corresponding grayscale images, numbering the circular fiducial points on the calibration board in the grayscale images, and calculating the internal parameters of the camera according to the circular fiducial points and their numbers; establishing a reference world coordinate system and a temporary coordinate system for the calibration board, and calculating the external parameters of the camera according to the reference world coordinate system and the temporary coordinate system; fitting a light plane based on the laser lines irradiated on different planes, and calculating a pose transformation matrix of the light plane relative to the coordinate system origin in the reference world coordinate system. The technical solution of the present invention reduces the difficulty of calibration operation and improves the calibration accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of 3D camera calibration and three-dimensional reconstruction, and in particular to a calibration method and a three-dimensional reconstruction method based on a high-precision line laser 3D camera. Background Art

[0002] The calibration and three-dimensional reconstruction methods of high-precision line laser 3D cameras can be used in many fields and processes of the manufacturing industry. They are applicable to metal processing, furniture manufacturing, medical research and other industries, with a wide range of applications. Three-dimensional reconstruction technology can be divided into monocular camera vision method, binocular camera vision method and multi-camera vision method according to the number of cameras, and can be divided into active vision method and passive vision method according to the nature of the light source. As for the camera calibration method, it can be divided into zero-dimensional self-calibration, one-dimensional calibration, two-dimensional calibration and three-dimensional calibration methods according to the spatial dimension. Spherical targets, multi-faceted stereo targets and virtual stereo targets are used in three-dimensional calibration; Letraset quick-engraving boards, checkerboard patterns and Gray code patterns are used in two-dimensional calibration; one-dimensional calibration method uses a one-dimensional calibration rod.

[0003] Currently, all optical camera lenses have distortion problems. Distortion is a geometric distortion in the imaging process. It is a phenomenon of image distortion caused by different magnifications of the image in different areas on the focal plane of the lens. The degree of this deformation gradually increases from the center of the image to the edge of the image, especially at the edge of the image.

[0004] At present, the commonly used technologies for line structured light calibration mainly include: the calibration method proposed by Huynh and Xu Guangyou respectively to obtain the calibration points of the light plane using the principle of cross ratio invariance; the calibration method proposed by Liu Zhen to solve the equation of the light plane by combining the Plücker matrix of the light strips on the light plane. The target reference objects required by both are relatively simple, usually chessboard targets, and have the advantages of low cost and high precision. However, the calibration method based on the principle of cross ratio invariance requires the light strips projected by the line structured light to intersect with the chessboard on the chessboard target, which is not conducive to extracting the chessboard corner points and the center points of the light strips, and fewer feature points are extracted at the same time. The calibration method for solving the Plücker matrix of the light plane requires combining three plane equations to calculate the Plücker matrix of multiple light strips on the light plane, which is more complicated to calculate.

[0005] In 3D reconstruction technology, the cost of multiple cameras is much higher than that of a single camera. Moreover, during the calibration process, coordinate transformation between different cameras and determination of the actual 3D coordinate system of space involve more considerations, with complex structures, high calibration difficulty, low efficiency, and higher installation and maintenance costs. For calibration methods, it is difficult to produce 3D stereo targets, and the cost of calibration equipment is relatively high; Letraset rubdown lettering sheets and checkerboard patterns in 2D calibration belong to pixel-level patterns with low calibration accuracy; 1D calibration targets are not flexible enough and have weak applicability in different scenarios; the mathematical operations of 0D self-calibration methods are complex and the accuracy is not high. Summary of the Invention

[0006] The present invention provides a calibration and 3D reconstruction method based on a high-precision line laser 3D camera, which reduces the difficulty of calibration operations and improves the calibration accuracy.

[0007] An embodiment of the present invention provides a calibration and 3D reconstruction method based on a high-precision line laser 3D camera. During the calibration process, the surface scanning camera and the laser projector are fixed on the profile, and the relative positions of the surface scanning camera and the laser projector remain unchanged; the following calibration steps are included:

[0008] Place the calibration board on the working plane according to a preset rule and take pictures of the calibration board to obtain a number of first calibration board images;

[0009] Convert the first calibration board images into corresponding grayscale images, number the circular fiducial points on the calibration board in the grayscale images, and calculate the internal parameters of the camera according to the circular fiducial points and their numbers;

[0010] Establish a reference world coordinate system and a temporary coordinate system for the calibration board, and calculate the external parameters of the camera according to the reference world coordinate system and the temporary coordinate system;

[0011] Fit the laser lines irradiated on different planes to obtain the light planes, and calculate the pose transformation matrix of the light planes relative to the origin of the coordinate system in the reference world coordinate system.

[0012] Further, the internal parameters of the camera are calculated according to the following steps:

[0013] Place the calibration board on the working plane according to a preset rule and take pictures of the calibration board to obtain a number of first calibration board images;

[0014] Convert the first calibration board images into corresponding grayscale images, number the circular fiducial points on the calibration board in the grayscale images, and establish a calibration board coordinate system according to the circular fiducial points and their numbers;

[0015] Obtain the position of the calibration board and the center positions of the circular fiducial points on the calibration board in the first calibration board image according to the calibration board coordinate system, and calculate the pose of the calibration board relative to the camera in the actual three-dimensional space based on the position of the calibration board and the center positions of the circular fiducial points on the calibration board;

[0016] Establish a linear equation of the internal parameters according to the orthogonality of the pose and the rotation matrix, and calculate the internal parameters of the camera through the linear equation.

[0017] Furthermore, calculate the external parameters of the camera according to the reference world coordinate system and the temporary coordinate system, including the following steps:

[0018] Lower the center point of the first calibration board by one calibration board thickness in the direction perpendicular to the working plane, and use the lowered center point as the origin of the reference world coordinate system. Define the direction perpendicular to the calibration board plane and upward as the positive direction of the Z axis of the reference world coordinate system, and determine the positive directions of the X axis and Y axis according to the direction markers on the calibration board; Calculate the first pose of the calibration board in the camera coordinate system;

[0019] Lower the center point of the second calibration board by one calibration board thickness in the direction perpendicular to the working plane, and use the lowered center point as the origin of the temporary coordinate system. Define the direction perpendicular to the calibration board plane and upward as the positive direction of the Z axis of the temporary coordinate system, and determine the positive directions of the X axis and Y axis according to the direction markers on the calibration board; Calculate the second pose of the calibration board in the camera coordinate system;

[0020] Calculate the external parameters of the camera according to the first pose and the second pose; The first calibration board is placed on the working plane and horizontally centered at the center of the camera's field of view; The second calibration board is placed on the working plane after being raised by a first distance and horizontally centered at the center of the camera's field of view.

[0021] Furthermore, the steps of fitting a light plane based on the laser lines irradiated on different planes and calculating the pose transformation matrix of the light plane relative to the coordinate system origin in the reference world coordinate system include:

[0022] Irradiate the laser line of the laser projector on the first plane, make the laser line clearly presented in the camera's field of view, and take a picture of the laser line irradiated on the first plane through the camera to obtain a first laser picture;

[0023] Use the gray centroid method to calculate the first center coordinates of the laser stripe in the first laser picture, calculate the first pixel coordinate points of the laser line in the first laser picture according to the first center coordinates, and convert the first pixel coordinate points into three-dimensional coordinate points in the reference world coordinate system through the calibrated transformation matrix;

[0024] Irradiate the laser line of the laser projector on the second plane, so that the laser line is clearly presented in the camera's field of view and photograph the laser line irradiated on the second plane through the camera to obtain a second laser picture;

[0025] Use the gray centroid method to calculate the second center coordinates of the laser stripe in the second laser picture, calculate the second pixel coordinate points of the laser line in the second laser picture according to the second center coordinates, and convert the second pixel coordinate points into three-dimensional coordinate points in the temporary coordinate system through the calibrated transformation matrix;

[0026] Fit an optical plane based on the first laser line and the second laser line, and calculate the pose transformation matrix of the optical plane relative to the coordinate system origin in the reference world coordinate system; the first laser line is the laser line irradiated on the first plane, and the second laser line is the laser line irradiated on the second plane.

[0027] Further, the first plane is the plane where the first calibration plate is located, and the second plane is the plane where the second calibration plate is located.

[0028] Further, the distance between the camera and the working plane is a second distance, and the second distance is greater than the first distance.

[0029] Further, the calibration plate includes a plurality of circular fiducial points, the arrangement of the circular fiducial points on the calibration plate is an array arrangement, and the calibration plate is provided with a direction mark.

[0030] Further, the internal parameters include 3 radial distortion parameters and 2 tangential distortion parameters.

[0031] Another embodiment of the present invention provides a three-dimensional reconstruction method based on a high-precision line laser 3D camera for three-dimensional reconstruction after calibration according to the calibration method based on the high-precision line laser 3D camera, including the following steps:

[0032] Calculate the motion direction vector of the object to be measured;

[0033] Scan the object to be measured through the laser projector to obtain a plurality of scan pictures, extract the pixel coordinates of the laser line for each scan picture, and convert the pixel coordinates into three-dimensional coordinate points in the reference world coordinate system;

[0034] Piece together the single contours of the object to be measured one by one according to the motion direction vector of the object to be measured to form a three-dimensional point cloud model; the single contour is the three-dimensional coordinate points of the laser line extracted from the scan picture each time.

[0035] Further, calculate the motion direction vector of the object to be measured according to the following steps:

[0036] Place the calibration board at the first position of the conveyor belt, and take pictures of the calibration board at the first position through a camera to obtain a number of second calibration board images;

[0037] Calculate the first origin coordinates of the center of the calibration board according to the position of the calibration board and the coordinate values of the center pixels of the circular marker points in the second calibration board images;

[0038] Move the calibration board to the second position through the conveyor belt, and take pictures of the calibration board at the second position through the camera to obtain a number of third calibration board images;

[0039] Calculate the second origin coordinates of the center of the calibration board according to the position of the calibration board and the coordinate values of the center pixels of the circular marker points in the third calibration board images;

[0040] Calculate the translation vector of the calibration board in the reference world coordinate system according to the first origin coordinates, the second origin coordinates and the number of steps for the calibration board to move from the first position to the second position, and the translation vector is the motion direction vector of the object to be measured.

[0041] The embodiments of the present invention have the following beneficial effects:

[0042] The present invention provides a calibration method and a three-dimensional reconstruction method based on a high-precision line laser 3D camera. This calibration method can complete the camera calibration process using a calibration board, avoiding problems such as the computational complexity of three-dimensional calibration methods and the poor accuracy of one-dimensional calibration methods, greatly reducing the calibration difficulty, and improving the calibration efficiency and calibration accuracy. The calibration board is small and compact, convenient to carry and use, and the calibration result can be easily restored, avoiding the disadvantages of high cost, inconvenient carrying and use, and high installation and maintenance costs of three-dimensional calibration equipment. It is suitable for various three-dimensional model application scenarios such as industrial part defect detection and spraying and grinding in the footwear and clothing industries, with strong adaptability, and improves the disadvantage of the single application scenario of existing calibration equipment. Description of the Drawings

[0043] Figure 1 is a schematic flowchart of a calibration method based on a high-precision line laser 3D camera provided by an embodiment of the present invention;

[0044] Figure 2 is a schematic flowchart of a three-dimensional reconstruction method based on a high-precision line laser 3D camera provided by an embodiment of the present invention;

[0045] Figure 3 is a schematic diagram of a calibration board of a three-dimensional reconstruction method based on a high-precision line laser 3D camera provided by an embodiment of the present invention;

[0046] Figure 4It is a schematic diagram for determining the positive directions of the X-axis and Y-axis of the reference world coordinate system in the three-dimensional reconstruction method based on a high-precision line laser 3D camera provided by an embodiment of the present invention. Detailed implementation manners

[0047] The following will clearly and completely describe the technical solutions in the present invention with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0048] As Figure 1 shown, a calibration method based on a high-precision line laser 3D camera provided by an embodiment of the present invention, during the calibration process, the surface scanning camera and the laser projector are fixed on the profile, and the relative positions of the surface scanning camera and the laser projector remain unchanged; the calibration steps include the following:

[0049] Step S101: Place the calibration board on the working plane according to a preset rule and photograph the calibration board to obtain a plurality of first calibration board images.

[0050] Step S102: Convert the first calibration board image into a corresponding grayscale image, number the circular fiducial points on the calibration board in the grayscale image, and calculate the internal parameters of the camera according to the circular fiducial points and their numbers.

[0051] Step S103: Establish a reference world coordinate system and a temporary coordinate system for the calibration board, and calculate the external parameters of the camera according to the reference world coordinate system and the temporary coordinate system.

[0052] Step S104: Fit the light planes according to the laser lines irradiated on different planes, and calculate the pose transformation matrix of the light planes relative to the coordinate system origin in the reference world coordinate system.

[0053] As one of the embodiments, the internal parameters of the camera are calculated according to the following steps:

[0054] Place the calibration board on the working plane according to a preset rule and photograph the calibration board to obtain a plurality of first calibration board images;

[0055] Convert the first calibration board image into a corresponding grayscale image, number the circular fiducial points on the calibration board in the grayscale image, and establish a calibration board coordinate system according to the circular fiducial points and their numbers;

[0056] Obtain the position of the calibration board and the center positions of the circular fiducial points on the calibration board in the first calibration board image according to the calibration board coordinate system, and calculate the pose of the calibration board relative to the camera in the actual three-dimensional space based on the position of the calibration board and the center positions of the circular fiducial points on the calibration board;

[0057] Establish a linear equation of the internal parameters according to the orthogonality of the pose and the rotation matrix, and calculate the internal parameters of the camera through the linear equation.

[0058] As one of the embodiments, calculate the external parameters of the camera according to the reference world coordinate system and the temporary coordinate system, including the following steps:

[0059] Lower the center point of the first calibration board by the thickness of the calibration board in the direction perpendicular to the working plane, and use the lowered center point as the origin of the reference world coordinate system. Define the direction perpendicular to the calibration board plane and upward as the positive direction of the Z axis of the reference world coordinate system, and determine the positive directions of the X axis and the Y axis according to the direction marks on the calibration board; calculate the first pose of the calibration board in the camera coordinate system;

[0060] Lower the center point of the second calibration board by the thickness of the calibration board in the direction perpendicular to the working plane, and use the lowered center point as the origin of the temporary coordinate system. Define the direction perpendicular to the calibration board plane and upward as the positive direction of the Z axis of the temporary coordinate system, and determine the positive directions of the X axis and the Y axis according to the direction marks on the calibration board; calculate the second pose of the calibration board in the camera coordinate system;

[0061] Calculate the external parameters of the camera according to the first pose and the second pose; the first calibration board is placed on the working plane and horizontally centered at the center of the camera's field of view; the second calibration board is placed on the working plane after being raised by a first distance and horizontally centered at the center of the camera's field of view. Specifically, the first distance in this embodiment is 100 mm.

[0062] As one of the embodiments, calculate the pose of the light plane according to the following steps:

[0063] Irradiate the laser line of the laser projector on the first plane, make the laser line clearly presented in the camera's field of view and take a picture of the laser line irradiated on the first plane through the camera to obtain the first laser picture;

[0064] Calculate the first center coordinates of the laser stripe in the first laser picture by using the gray center of gravity method, calculate the first pixel coordinate points of the laser line in the first laser picture according to the first center coordinates, and convert the first pixel coordinate points into three-dimensional coordinate points in the reference world coordinate system through the calibrated transformation matrix;

[0065] Irradiate the laser line of the laser projector on the second plane, so that the laser line is clearly presented in the camera's field of view and photograph the laser line irradiated on the second plane through the camera to obtain a second laser image;

[0066] Use the gray centroid method to calculate the second central coordinates of the laser stripe in the second laser image, calculate the second pixel coordinate points of the laser line in the second laser image according to the second central coordinates, and convert the second pixel coordinate points into three-dimensional coordinate points in the temporary coordinate system through the calibrated transformation matrix;

[0067] Fit an optical plane based on the first laser line and the second laser line, and calculate the pose transformation matrix of the optical plane relative to the coordinate system origin in the reference world coordinate system; the first laser line is the laser line irradiated on the first plane, and the second laser line is the laser line irradiated on the second plane.

[0068] The first plane is the plane where the first calibration plate is located, and the second plane is the plane where the second calibration plate is located.

[0069] As one of the embodiments, the distance between the camera and the working plane is a second distance, and the second distance is greater than the first distance. Preferably, the second distance is 400 mm. In other embodiments, the second distance can be other heights greater than the first distance by 100 mm.

[0070] As one of the embodiments, the calibration plate includes a plurality of circular fiducial points, the arrangement of the circular fiducial points on the calibration plate is an array arrangement, and the calibration plate is provided with a direction flag bit. Preferably, as Figure 3 shown, the calibration plate uses a 70 mm calibration plate with 49 circular fiducial points, the arrangement of the circular fiducial points on the calibration plate is 7 rows and 7 columns, and the sharp corner in the upper left corner of the calibration plate is the direction flag of the calibration plate.

[0071] As one of the detailed embodiments, complete the calibration work before 3D reconstruction according to step A01-step A03.

[0072] Step A01: Calculate the internal parameters of the camera. Specifically, it includes the following sub-steps:

[0073] Sub-step A011: Place the calibration plate on the working plane according to certain rules and obtain a number of first calibration plate images by photographing. Specifically, it includes the following sub-steps:

[0074] Sub-step A0111: Press the edge of the calibration plate against the edge of the camera image field of view, place and photograph it in the lower left corner, upper left corner, upper right corner, and lower right corner of the camera image field of view in sequence, and place the sharp corner part of the calibration plate (i.e. Figure 3The upper left corner of the calibration plate) faces the four directions of the image field of view.

[0075] Sub-step A0112: Rotate the calibration plate counterclockwise 30° each time around the normal vector coordinate axis of the working plane, and place it in turn at the edge of the camera image field of view (excluding the lower left corner, upper left corner, upper right corner, and lower right corner) for photographs, and rotate it in turn in the area near the center of the image field of view (excluding the edge part) for photographs, so that the placement position of the calibration plate in sub-steps A0111 and A0112 covers the entire field of view of the camera image.

[0076] Sub-step A0113: Place the calibration plate at an angle of 60° to the plane on which it is located in the middle of the four edge lines of the camera's field of view (i.e., the upper, lower, left and right edge lines of the camera's field of view), with the outer edge of the calibration plate close to the edge of the camera's field of view, and the sharp corners of the calibration plate facing the four edge lines in turn, and take a photo.

[0077] Sub-step A0114: Place the calibration plate at an angle of 60° to the plane where it is located, in the center of the camera's field of view, close to the four edge lines, with the sharp corners of the calibration plate facing the four sharp corners of the field of view in turn, and take a photo.

[0078] Sub-step A0115: Place the calibration plate horizontally on the upper surface of the object to be measured and take a photo.

[0079] Sub-step A0116: Place the calibration plate on the work plane and horizontally center it in the center of the camera's field of view, and take a photo.

[0080] Sub-step A0117: Place a regular cube on the work plane, and place the calibration plate on the upper surface of the cube so that the calibration plate is raised by 100 mm relative to the previous step. At the same time, place it horizontally and centered in the center of the camera's field of view, and take a photo.

[0081] The operations in sub-steps A0116 and A0117 are for taking pictures of the laser lines of two working surfaces at different heights so as to fit a laser plane.

[0082] Sub-step A012: convert the first calibration plate image into a corresponding grayscale image, number the circular marking points on the calibration plate in the grayscale image, and establish a calibration plate coordinate system according to the circular marking points and their numbers. Specifically, the following sub-steps are included:

[0083] Sub-step A0121: According to PAL, i.e., phase alternating line television standard, the three primary color values of R, G, and B of each pixel in the first calibration plate image (the first calibration plate image is a color image) are calculated by the following formula to obtain a corresponding brightness value Y=0.222R+0.707G+0.071B;

[0084] And use this brightness value as the brightness value of the pixel at the corresponding position in the grayscale image, and finally obtain the grayscale image corresponding to the original color image of the first calibration plate image.

[0085] Sub-step A0122: Obtain the edge position of the image according to the mutation position of the grayscale value in the grayscale image, and obtain the position of the calibration plate in the image and the position of the center pixel coordinate point of the circular fiducial point on the calibration plate.

[0086] Sub-step A0123: Encode the circular fiducial points on the calibration plate one by one in sequence. Define the circular fiducial point closest to the sharp corner of the calibration plate as the No. 1 feature point. Assuming that the sharp corner of the calibration plate is in the upper left corner, then define the circular fiducial point to the right of the No. 1 feature point as the No. 2 feature point, and so on. Mark the circular fiducial points in the first row up to the No. 7 feature point; the leftmost circular fiducial point in the second row is the No. 8 feature point, and the circular fiducial point to its right is the No. 9 feature point, and so on. Mark the circular fiducial points row by row to the right until all 49 fiducial points on the calibration plate are marked.

[0087] Sub-step A0124: Define the direction along the No. 1, 2, and 3 fiducial points as the positive x-axis direction of the calibration plate coordinate system, and the direction along the No. 1, 8, and 15 fiducial points as the positive y-axis direction of the calibration plate coordinate system. Define the center coordinate of the No. 25 fiducial point as the origin position of the calibration plate coordinate system, that is, define the center coordinate of the center point of the 49 circular fiducial points as the origin position of the calibration plate coordinate system.

[0088] Sub-step A013: Obtain the position of the calibration plate in the first calibration plate image and the center position of the circular fiducial points on the calibration plate according to the calibration plate coordinate system, and calculate the pose of the calibration plate relative to the camera in the actual three-dimensional space according to the position of the calibration plate and the center position of the circular fiducial points on the calibration plate.

[0089] Sub-step A014: Establish a linear equation of the internal parameters according to the orthogonality of the pose and the rotation matrix, and calculate the internal parameters of the camera through the linear equation. The internal parameters include 3 radial distortion parameters and 2 tangential distortion parameters. Specifically, the linear equation is as follows:

[0090] x distorted = x(1 + k1r 2 + k2r 4 + k3r 6 ) + 2p1xy + p2(r 2 + 2x 2 );

[0091] y distorted = y(1 + k1r 2 + k2r 4 + k3r 6) + p1(r 2 + 2y 2 ) + 2p2xy;

[0092] In the formula, k1, k2, and k3 are radial distortion parameters, p1 and p2 are tangential distortion parameters, (x, y) is the original coordinate of the distorted point on the image, and (x distorted , y distorted ) is the new coordinate after correcting the distortion.

[0093] Step A02: Calculate the external parameters of the camera. The external parameters include a first transformation matrix from the camera coordinate system to the reference world coordinate system and a second transformation matrix from the camera coordinate system to the temporary coordinate system. The camera coordinate system takes the optical center of the camera as the coordinate origin, the direction along the optical axis away from the camera is the Z-axis direction, and the X-axis and Y-axis are parallel to the X-axis and Y-axis of the image coordinate system respectively. Specifically, it includes the following sub-steps:

[0094] Sub-step A021: Lower the center point of the calibration plate in sub-step A0116 by the thickness of the calibration plate in the direction perpendicular to the working plane, and use the lowered center point as the origin of the reference world coordinate system. Define the direction perpendicular to the calibration plate plane and upward as the positive direction of the Z-axis of the reference world coordinate system, and determine the positive directions of the X-axis and Y-axis according to the direction marks on the calibration plate. Lowering the center point of the calibration plate by the thickness of the calibration plate is to define the origin of the reference world coordinate system on the working plane to exclude the influence of the calibration plate thickness.

[0095] As Figure 4 shown, determine the positive directions of the X-axis and Y-axis according to the direction marks on the calibration plate. Specifically: Take the opposite direction of the calibration plate's pointed corner as the direction of the diagonal where the calibration plate's pointed corner is located, and decompose the diagonal where the calibration plate's pointed corner is located into two mutually perpendicular components, and define the two components as the positive directions of the X-axis and Y-axis of the world coordinate system respectively.

[0096] Sub-step A022: Calculate the first pose of the calibration plate in the camera coordinate system, that is, the first transformation matrix from the camera coordinate system to the reference world coordinate system.

[0097] Sub-step A023: Lower the center point of the calibration plate in sub-step A0117 by the thickness of the calibration plate in the direction perpendicular to the working plane, and use the lowered center point as the origin of the temporary coordinate system. Define the direction perpendicular to the calibration plate plane and upward as the positive direction of the Z-axis of the temporary coordinate system, and determine the positive directions of the X-axis and Y-axis according to the direction marks on the calibration plate. Specifically, take the opposite direction of the calibration plate's pointed corner as the direction of the diagonal where the calibration plate's pointed corner is located, and decompose the diagonal where the calibration plate's pointed corner is located into two mutually perpendicular components, and define the two components as the positive directions of the X-axis and Y-axis of the world coordinate system respectively.

[0098] Sub-step A024: Calculate the second pose of the calibration board in the camera coordinate system, that is, the second transformation matrix from the camera coordinate system to the temporary coordinate system.

[0099] Step A03: Calculate the pose of the light plane. Specifically, it includes the following sub-steps:

[0100] Sub-step A031: Project the laser line of the laser projector onto the first plane, make the laser line clearly presented in the camera's field of view, and take a picture of the laser line projected onto the first plane through the camera to obtain the first laser picture. Specifically, remove the calibration board, turn on the laser projector and reduce the camera exposure time, project the laser line onto the plane where the calibration board is located in sub-step A0116, and make the laser line clearly presented in the center of the camera's field of view.

[0101] Sub-step A032: Use the gray-scale centroid method to calculate the first center coordinates of the laser stripe in the first laser picture, calculate the first pixel coordinate point of the laser line in the first laser picture according to the first center coordinates, and convert the first pixel coordinate point into a three-dimensional coordinate point in the reference world coordinate system through the calibrated transformation matrix. Specifically, use the gray-scale centroid method to obtain the first center coordinates of the laser stripe in the first laser picture, and the principle is as follows:

[0102] Calculate the laser stripe column by column, and take the ordinate of the gray-scale centroid of the laser stripe calculated for each column as its center coordinate. Along the direction perpendicular to the laser line, set the coordinate position of the k-th column of the laser stripe cross-section as (x k , y i ), then the gray value corresponding to the coordinate of this column is f(x k , y i ), where the variable i = 1,..., M, and M represents the cross-sectional width of the laser stripe in this column, usually taken as an odd number. The calculation formula is as follows:

[0103]

[0104] In the formula, k represents the length variable of the laser stripe, and its range of variation is determined according to the image size; y k refers to the center column coordinate value of the k-th column of the laser stripe. Further, by removing pseudo-maximum values, eliminating step points, and connecting break points, the center line is made smoother and more continuous. Convert the pixel coordinate points of the extracted laser line into three-dimensional coordinate points (x ai , y ai , z ai)。Since the laser line photo obtained by taking pictures is not a thin line, but an area composed of many pixel points. For the accuracy of subsequent 3D reconstruction, it is necessary to find a thin line to more accurately replace this laser line area, and this thin line is called the center line of the laser line. Formula (1) is used to determine the specific position of the laser center line in this area.

[0105] Sub-step A033: Irradiate the laser line of the laser projector on the second plane, make the laser line clearly presented in the camera's field of view, and take pictures of the laser line irradiated on the second plane through the camera to obtain a second laser picture.

[0106] Sub-step A034: Use the gray centroid method to calculate the second center coordinates of the laser stripe in the second laser picture, calculate the second pixel coordinate points of the laser line in the second laser picture according to the second center coordinates, and convert the second pixel coordinate points into three-dimensional coordinate points in the temporary coordinate system through the calibrated transformation matrix. In sub-step A034, the same gray centroid method as in sub-step A032 is used to calculate the second center coordinates.

[0107] Sub-step A035: Fit an optical plane based on the first laser line and the second laser line, and calculate the pose transformation matrix of the optical plane relative to the coordinate system origin in the reference world coordinate system; the first laser line is the laser line irradiated on the first plane, and the second laser line is the laser line irradiated on the second plane. Since the first laser line and the second laser line come from the same laser projector and the laser projector remains stationary during the irradiation process, the first laser line and the second laser line are parallel to each other.

[0108] The first plane is the plane where the calibration board is located in sub-step A0116, and the second plane is the plane where the calibration board is located in sub-step A0117.

[0109] The embodiment of the present invention can complete the camera calibration process using a calibration board, avoiding problems such as the computational complexity of the 3D calibration method and the poor accuracy of the 1D calibration method, greatly reducing the calibration difficulty, and improving the calibration efficiency and calibration accuracy. The calibration board is small and compact, convenient to carry and use, and the calibration result can be easily restored, avoiding the disadvantages of high cost of 3D calibration equipment, inconvenient carrying and use, and high installation and maintenance costs. It is suitable for various 3D model application scenarios such as industrial part defect detection and spraying and grinding in the footwear and clothing industries, with strong adaptability, and improves the disadvantage of single application scenario of existing calibration equipment.

[0110] Based on the above-described embodiment of the calibration method invention, the present invention correspondingly provides an embodiment of a 3D reconstruction method, as Figure 2 shown;

[0111] Another embodiment of the present invention provides a three-dimensional reconstruction method based on a high-precision line laser 3D camera for three-dimensional reconstruction after calibration according to the calibration method invention embodiment described above, including the following steps:

[0112] Step S105: Calculate the motion direction vector of the object to be measured;

[0113] Step S106: Scan the object to be measured through a laser projector to obtain a number of scanned pictures. For each scanned picture, extract the pixel coordinates of the laser line and convert the pixel coordinates into three-dimensional coordinate points in the reference world coordinate system;

[0114] Step S107: Piece together the single contours of the object to be measured one by one according to the motion direction vector of the object to be measured to form a three-dimensional point cloud model; the single contour is the three-dimensional coordinate points of the laser line extracted from the scanned picture each time.

[0115] As one of the embodiments, calculate the motion direction vector of the object to be measured according to the following steps:

[0116] Place the calibration plate at the first position on the conveyor belt and take pictures of the calibration plate at the first position through the camera to obtain a number of second calibration plate images;

[0117] Calculate the first origin coordinates of the center of the calibration plate according to the position of the calibration plate and the coordinate values of the center pixels of the circular marking points in the second calibration plate image;

[0118] Move the calibration plate to the second position through the conveyor belt and take pictures of the calibration plate at the second position through the camera to obtain a number of third calibration plate images;

[0119] Calculate the second origin coordinates of the center of the calibration plate according to the position of the calibration plate and the coordinate values of the center pixels of the circular marking points in the third calibration plate image;

[0120] Calculate the translation vector of the calibration plate in the reference world coordinate system according to the first origin coordinates, the second origin coordinates and the number of steps for the calibration plate to move from the first position to the second position. The translation vector is the motion direction vector of the object to be measured.

[0121] As one of the detailed embodiments, complete the three-dimensional reconstruction work after calibration according to step A04 - step A07.

[0122] Step A04: Calculate the motion direction vector of the object to be measured. Specifically, it includes the following sub-steps:

[0123] Sub-step A041: Place the calibration plate at the first position on the conveyor belt, and take pictures of the calibration plate at the first position through the camera to obtain a number of second calibration plate images. Specifically, turn off the laser projector, adjust the camera exposure time so that the calibration plate can be clearly imaged in the camera's field of view, place the calibration plate at the first position on the conveyor belt and take pictures to obtain a number of second calibration plate images.

[0124] Sub-step A042: Calculate the first origin coordinates of the center of the calibration plate based on the position of the calibration plate in the second calibration plate image and the coordinate values of the centers of the circular marker points in pixels. Specifically, obtain the position of the calibration plate in the second calibration plate image and the coordinate values of the centers of the circular marker points in pixels according to the method in sub-step A0122.

[0125] Sub-step A043: Move the calibration plate to the second position through the conveyor belt, and take pictures of the calibration plate at the second position through the camera to obtain a number of third calibration plate images.

[0126] Sub-step A044: Calculate the second origin coordinates of the center of the calibration plate based on the position of the calibration plate in the third calibration plate image and the coordinate values of the centers of the circular marker points in pixels. Specifically, obtain the position of the calibration plate in the third calibration plate image and the coordinate values of the centers of the circular marker points in pixels according to the method in sub-step A0122.

[0127] Sub-step A045: Calculate the translation vector of the calibration plate in the reference world coordinate system (i.e., the vector matrix of the moving direction of the object to be measured) based on the first origin coordinates, the second origin coordinates, and the number of steps (e.g., the number of steps from the first position to the second position is n) for the calibration plate to move from the first position to the second position. The translation vector is the motion direction vector of the object to be measured.

[0128] Step A05: Scan the object to be measured through the laser projector to obtain a number of scanned pictures, calculate the third center coordinates of the laser stripes in the scanned pictures using the gray-scale centroid method, then calculate the third pixel coordinate points of the laser line based on the third center coordinates, and convert the third pixel coordinate points into three-dimensional coordinate points in the reference world coordinate system. Specifically, remove the calibration plate, turn on the laser projector and adjust the camera exposure time, irradiate the laser line on the object to be measured so that the laser line is clearly presented in the center of the camera's field of view; place the object to be measured on one side of the laser line, turn on the conveyor belt so that the object to be measured can be completely scanned by the laser line, and the camera takes pictures every 1 step; for the pictures taken, use the gray-scale centroid method to obtain the third center point coordinates of the laser stripes, and convert the extracted third pixel coordinate points of the laser line into three-dimensional coordinate points in the reference world coordinate system.

[0129] Step A06: Piece together the single contours of the object under test one by one according to the motion direction vector of the object under test to form a three-dimensional point cloud model; the single contour is the three-dimensional coordinate points of the laser line extracted from the scanned picture each time. Specifically, take the three-dimensional coordinate points of the laser line extracted each time in Step A05 as the single contour of the object under test, and piece together the extracted single contours one by one according to the motion direction vector of the object under test to form a three-dimensional point cloud model with noise points.

[0130] Step A07: Use the connected component point number screening method or the radius filter to perform filtering and denoising processing on the three-dimensional point cloud model to obtain a three-dimensional point cloud model of the object under test with fewer irrelevant noise points.

[0131] In the embodiment of the present invention, the camera calibration process can be completed using a calibration board, avoiding problems such as the computational complexity of the three-dimensional calibration method and the poor accuracy of the one-dimensional calibration method, greatly reducing the calibration difficulty, and improving the calibration efficiency and calibration accuracy. The calibration board is small and compact, convenient to carry and use, and the calibration result can be easily restored, avoiding the disadvantages of high cost of three-dimensional calibration equipment, inconvenient carrying and use, and high installation and maintenance costs. It is suitable for various three-dimensional model application scenarios such as industrial part defect detection and spraying and grinding in the footwear and clothing industries, with strong adaptability, and improves the disadvantage of the single application scenario of the existing calibration equipment.

[0132] The above is the preferred implementation manner of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

[0133] Those of ordinary skill in the art can understand that all or part of the processes in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above embodiments. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

Claims

1. A calibration method based on a high-precision line laser 3D camera, characterized in that, During the calibration process, the surface scanning camera and the laser projector are fixed on the profile, and the relative positions of the surface scanning camera and the laser projector remain unchanged; It includes the following calibration steps: Place the calibration board on the working plane according to the preset rules and take pictures of the calibration board to obtain a number of first calibration board images; Convert the first calibration board image into the corresponding grayscale image, number the circular fiducial points on the calibration board in the grayscale image, and establish a calibration board coordinate system according to the circular fiducial points and their numbers; obtain the position of the calibration board and the center positions of the circular fiducial points in the calibration board in the first calibration board image according to the calibration board coordinate system, and calculate the pose of the calibration board in the actual three-dimensional space relative to the camera according to the position of the calibration board and the center positions of the circular fiducial points in the calibration board; establish a linear equation of the internal parameters of the camera according to the orthogonality of the pose and the rotation matrix, and calculate the internal parameters of the camera through the linear equation; Establish a reference world coordinate system and a temporary coordinate system for the calibration board, and calculate the external parameters of the camera according to the reference world coordinate system and the temporary coordinate system. Specifically: lower the center point of the first calibration board by the thickness of a calibration board in the direction perpendicular to the working plane, and use the lowered center point as the origin of the reference world coordinate system. Define the direction perpendicular to the calibration board plane and upward as the positive direction of the Z axis of the reference world coordinate system, and determine the positive directions of the X axis and the Y axis according to the direction marks of the calibration board; calculate the first pose of the calibration board in the camera coordinate system; lower the center point of the second calibration board by the thickness of a calibration board in the direction perpendicular to the working plane, and use the lowered center point as the origin of the temporary coordinate system. Define the direction perpendicular to the calibration board plane and upward as the positive direction of the Z axis of the temporary coordinate system, and determine the positive directions of the X axis and the Y axis according to the direction marks of the calibration board; calculate the second pose of the calibration board in the camera coordinate system; calculate the external parameters of the camera according to the first pose and the second pose; the first calibration board is placed on the working plane and horizontally centered at the center of the camera's field of view; the second calibration board is placed on the working plane after being raised by a first distance and horizontally centered at the center of the camera's field of view; Fit the light plane according to the laser lines irradiated on different planes, and calculate the pose transformation matrix of the light plane relative to the origin of the coordinate system in the reference world coordinate system.

2. The calibration method based on a high-precision line laser 3D camera according to claim 1, wherein The step of fitting the light plane according to the laser lines irradiated on different planes and calculating the pose transformation matrix of the light plane relative to the origin of the coordinate system in the reference world coordinate system includes the following steps: Irradiate the laser line of the laser projector on the first plane, make the laser line clearly presented in the camera's field of view and take pictures of the laser line irradiated on the first plane through the camera to obtain a first laser picture; The first central coordinate of the laser stripe in the first laser image is calculated by using the gray - level centroid method. The first pixel coordinate point of the laser line in the first laser image is calculated according to the first central coordinate, and the first pixel coordinate point is converted into a three - dimensional coordinate point in the reference world coordinate system through the calibrated transformation matrix; The laser line of the laser projector is irradiated on the second plane, so that the laser line is clearly presented in the camera's field of view and the laser line irradiated on the second plane is photographed by the camera to obtain a second laser image; The second central coordinate of the laser stripe in the second laser image is calculated by using the gray - level centroid method. The second pixel coordinate point of the laser line in the second laser image is calculated according to the second central coordinate, and the second pixel coordinate point is converted into a three - dimensional coordinate point in the temporary coordinate system through the calibrated transformation matrix; A light plane is fitted according to the first laser line and the second laser line, and the pose transformation matrix of the light plane relative to the coordinate system origin in the reference world coordinate system is calculated; the first laser line is the laser line irradiated on the first plane, and the second laser line is the laser line irradiated on the second plane.

3. The calibration method based on a high-precision line laser 3D camera according to claim 2, wherein The first plane is the plane where the first calibration plate is located, and the second plane is the plane where the second calibration plate is located.

4. The calibration method based on a high-precision line laser 3D camera according to claim 3, wherein The distance between the camera and the working plane is the second distance, and the second distance is greater than the first distance.

5. The calibration method based on a high-precision line laser 3D camera according to claim 4, wherein, The calibration plate includes a number of circular fiducial points, the arrangement of the circular fiducial points on the calibration plate is an array arrangement, and the calibration plate is provided with a direction mark.

6. The calibration method based on a high-precision line laser 3D camera according to any one of claims 1 to 5, characterized in that The internal parameters include 3 radial distortion parameters and 2 tangential distortion parameters.

7. A three-dimensional reconstruction method based on a high-precision line laser 3D camera, characterized in that, For three - dimensional reconstruction after calibration according to the calibration method of the high - precision line - laser 3D camera according to any one of claims 1 to 6, it includes the following steps: Calculate the motion direction vector of the object to be measured; The object to be measured is scanned by the laser projector to obtain a number of scanned images. For each scanned image, the pixel coordinates of the laser line are extracted, and the pixel coordinates are converted into three - dimensional coordinate points in the reference world coordinate system; According to the motion direction vector of the object to be measured, the single contours of the object to be measured are stitched together one by one to form a three - dimensional point cloud model; the single contour is the three - dimensional coordinate points of the laser line extracted from each scanned image.

8. The three-dimensional reconstruction method based on a high-precision line laser 3D camera according to claim 7, characterized in that, The motion direction vector of the object to be measured is calculated according to the following steps: Place the calibration plate at the first position on the conveyor belt, and photograph the calibration plate at the first position through the camera to obtain a number of second calibration plate images; Calculate the first origin coordinate of the calibration plate center according to the position of the calibration plate in the second calibration plate image and the coordinate values of the centers of the circular fiducial points of the pixels; Move the calibration plate to the second position through the conveyor belt, and photograph the calibration plate at the second position through the camera to obtain a number of third calibration plate images; Calculate the second origin coordinate of the calibration plate center according to the position of the calibration plate in the third calibration plate image and the coordinate values of the centers of the circular fiducial points of the pixels; Calculate the translation vector of the calibration plate in the reference world coordinate system according to the first origin coordinates, the second origin coordinates, and the number of steps for the calibration plate to move from the first position to the second position, where the translation vector is the motion direction vector of the object under test.

Images