Calibration method and three-dimensional reconstruction method based on line laser scanning

Abstract

The invention discloses a calibration method and a three-dimensional reconstruction method based on line laser scanning, and the method comprises the steps: calibrating a three-dimensional reconstruction device to determine calibration information, and specifically, calibrating a camera of the three-dimensional reconstruction device to at least determine the internal reference of the camera, calibrating a light plane of a laser module of the three-dimensional reconstruction device so as to at least determine parameters of the light plane; performing line laser scanning on the three-dimensional calibration plate to obtain a light strip image of line laser; extracting a light bar in the light bar image based on a light bar extraction algorithm, and performing three-dimensional reconstruction on the three-dimensional calibration plate based on the extracted light bar and the calibration information to obtain a three-dimensional reconstruction result; acquiring model information of the three-dimensional calibration board, registering the three-dimensional reconstruction result of the three-dimensional calibration board with the model information of the three-dimensional calibration board to obtain difference information, and optimizing the calibration information based on the difference information; the optimized calibration information obtained through the method is more accurate.

Description

Technical Field

[0001] The invention relates to the field of machine vision, and in particular to a calibration method and a three-dimensional reconstruction method based on line laser scanning. Background Art

[0002] 3D reconstruction technology is one of the main research areas of machine vision. It can efficiently measure the 3D information and 3D posture of an object or environment by restoring its surface 3D structure. 3D reconstruction technology is widely used, including product measurement, detection, positioning and identification.

[0003] The purpose of 3D reconstruction is to obtain surface information of an object. According to the type of light source, it can be divided into passive light source and active light source. Among them, structured light is an active light source, and structured light measurement is an important direction in non-contact measurement. Structured light includes point structured light, line structured light, and coded structured light. Please refer to Figure 1 , which is a schematic diagram of line structured light measurement; the principle and process of measurement based on laser triangulation is as follows: fix the relative position of the camera and the laser transmitter, project a line laser line onto the surface of the object to be measured through the line laser sensor, and then use the camera to obtain the reflected light. The shape change of the object surface will be reflected in the light strip image received by the camera (the image of the reflected light on the image is called the light strip), causing the position of the light strip to shift, and the magnitude of this shift corresponds to the depth change of the object surface. Therefore, the offset of the light strip on the image can be calculated, and then the depth information of the object surface can be indirectly obtained - that is: when the line laser is projected onto the surface of the object, the bumps and depressions on the surface of the object to be measured will cause the light strip obtained by the camera to undergo corresponding displacements. Through this corresponding displacement, the bumps and depressions on the surface of the object can be calculated - for example Figure 1 When the surface of the object to be measured is at point D1, the corresponding image on the imaging plane is F1, and when the surface of the object to be measured is at point D2, the corresponding image on the imaging plane is F2. The light transmission path is a triangle in space, so it is called triangulation.

[0004] When performing three-dimensional reconstruction based on line laser scanning, the three-dimensional reconstruction device needs to be calibrated so that the two-dimensional coordinates of the light strip obtained based on the scan on the image can be correctly and accurately converted into the depth information of the object surface. Otherwise, the reconstructed result will be incorrect and inaccurate. Summary of the invention

[0005] In view of the above problems, the present application proposes a calibration method and a three-dimensional reconstruction method based on line laser scanning, which are described in detail below.

[0006] According to the first aspect, an embodiment provides a calibration method applied to a three-dimensional reconstruction device based on line laser scanning, the calibration method comprising:

[0007] Calibrate the three-dimensional reconstruction device to determine calibration information; wherein the calibration includes: calibrating a camera of the three-dimensional reconstruction device to at least determine an intrinsic parameter of the camera, and calibrating a light plane of a laser module of the three-dimensional reconstruction device to at least determine a parameter of the light plane; wherein the calibration information includes the intrinsic parameter of the camera and the parameter of the light plane;

[0008] Perform line laser scanning on the three-dimensional calibration plate to obtain a light strip image of the line laser;

[0009] Extracting light stripes in the light stripe image based on a light stripe extraction algorithm, and performing three-dimensional reconstruction on the three-dimensional calibration plate based on the extracted light stripes and the calibration information to obtain a three-dimensional reconstruction result;

[0010] The model information of the three-dimensional calibration plate is obtained, and the three-dimensional reconstruction result of the three-dimensional calibration plate is registered with the model information thereof to obtain difference information, and the calibration information is optimized based on the difference information.

[0011] In one embodiment, the calibration further includes: using a first model to perform distortion correction on the camera to obtain distortion parameters; and the calibration information also includes the distortion parameters.

[0012] In one embodiment, the first model includes:

[0013]

[0014] Among them, (u,v) T represents the coordinates before distortion correction in the imaging plane coordinate system, represents the coordinates after distortion correction in the imaging plane coordinate system, and k represents the radial distortion parameter;

[0015] or,

[0016]

[0017] Among them, (u,v) T represents the coordinates before distortion correction in the imaging plane coordinate system, represents the coordinates after distortion correction in the imaging plane coordinate system, k1, k2, and k3 represent distortion parameters.

[0018] In one embodiment, the optimizing the calibration information based on the difference information includes: establishing an objective function based on the difference information and optimizing the calibration information based on the objective function; wherein the objective function is:

[0019]

[0020] u and v are the coordinates in the pixel coordinate system; c y and c x Table 2 is the internal parameter of the camera, indicating the optical center; f x and f y The s x and y Countdown, s x Indicates the size of the camera sensor in pixels on the v axis, s y represents the size of the unit pixel of the camera sensor on the u axis; A, B and C are three parameters of the light plane, and the light plane is Ax+By+Cz=1; i represents the sequence number of the pixel point, and n represents the total number of pixels; z c is the coordinate of the Z axis in the camera coordinate system.

[0021] In one embodiment, the light stripe extraction algorithm includes:

[0022] The light stripe image is processed column by column, and for any column of the light stripe image:

[0023] Calculate a first gradient parameter of each pixel point in the column of the light stripe image;

[0024] According to the first gradient parameter of each pixel point in the column, a starting point and an end point are selected from the column to define a light bar area of ​​the column;

[0025] The point with the largest second gradient parameter in the light strip area of ​​the column, the starting point or the end point is taken as the first candidate point;

[0026] Selecting a pixel point in the light strip area of ​​the column as a second candidate point by a preset method;

[0027] For any column of the light bar image, at least according to the width of the light bar region of the column, a position of one of the first candidate point and the second candidate point of the column is selected as the position of the center point of the light bar of the column.

[0028] In one embodiment, selecting a starting point and an end point from the column to define the light bar area of ​​the column based on the first gradient parameter of each pixel point in the column includes: traversing the pixel points in the direction of the column, taking the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and continuing to traverse to find the first pixel point whose grayscale value is less than or equal to the starting point as the end point; or, taking the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and taking the pixel point whose first gradient parameter is greater than or equal to the second threshold as the end point.

[0029] In one embodiment, the step of selecting a pixel point in the light bar area of ​​the column as the second candidate point by a preset method includes:

[0030] Calculating the first-order gradient and the second-order gradient of each pixel point in the light strip area of ​​the column; fitting a quadratic function based on the grayscale value, the first-order gradient and the second-order gradient of each pixel point in the light strip area of ​​the column as parameters of the quadratic function, and solving the local maximum value and the corresponding position of the quadratic function of the pixel point; comparing the local maximum values ​​of the quadratic function of each pixel point in the column to obtain the largest local maximum value among these local maximum values, and taking the position corresponding to the largest local maximum value as the position of the second candidate point;

[0031] or,

[0032] Obtaining the grayscale value and weight of each pixel point in the light strip area of ​​the column, and calculating a weighted average grayscale value, wherein the weight obeys a Gaussian distribution; taking the pixel point in the light strip area of ​​the column whose grayscale value is greater than the weighted average grayscale value as the second candidate point;

[0033] or,

[0034] Obtain the grayscale values ​​of the pixel with the largest grayscale value in the light strip area of ​​the column and two pixels in its neighborhood, and fit a quadratic function of the grayscale value with respect to the position based on these three pixels, obtain the position corresponding to the local maximum of the quadratic function, and use this position as an offset plus the position of the pixel with the largest grayscale value to calculate the position obtained as the position of the second candidate point.

[0035] In one embodiment, the step of selecting one of the positions of the first candidate point and the second candidate point of the column as the position of the center point of the light bar of the column at least according to the width of the light bar region of the column includes:

[0036] At least according to the width of the light strip area of ​​the column, it is determined whether the light strip area of ​​the column belongs to the first type of area or the second type of area; wherein, according to the width of the light strip area of ​​the column and the width of the light strip area of ​​the columns in the neighborhood of the column, it is determined whether the light strip area of ​​the column belongs to the first type of area or the second type of area: when the width of the light strip area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light strip area of ​​the column belongs to the first type of area; or, when the width of the light strip area of ​​the column is less than the first width threshold , and the number of columns whose widths in its neighborhood are greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or, when the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns whose widths in its neighborhood are greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or, when the width of the light bar area of ​​the column is less than the first width threshold, and the number of columns whose widths in its neighborhood are greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar area of ​​the column belongs to the second type of area;

[0037] If it belongs to the first type of area, the position of the first candidate point in the column is selected as the position of the center point of the light bar in the column;

[0038] If it belongs to the second type area, the position of the second candidate point in the column is selected as the position of the center point of the light bar in the column.

[0039] In one embodiment, the three-dimensional calibration plate includes: a main plate body and an edge portion; the edge portion and the plate body both have a first side, the first side of the edge portion and the plate body are oriented in the same direction and are parallel to the thickness direction of the three-dimensional calibration plate; a step structure is formed between the edge portion and the plate body at least on the first side, the first side of the plate body is distributed with calibration protrusions and / or calibration recesses in an array form, and the first side of the edge portion is also distributed with calibration protrusions and / or calibration recesses in an array form.

[0040] According to the second aspect, an embodiment provides a three-dimensional reconstruction method based on line laser scanning, which is applied to a three-dimensional reconstruction device. The three-dimensional reconstruction method includes:

[0041] Perform laser scanning on the object to be measured to obtain a light strip image;

[0042] Extracting light stripes from the light stripe image based on a light stripe extraction algorithm;

[0043] The object to be measured is three-dimensionally reconstructed based on the extracted light strips and calibration information to obtain a reconstruction result; wherein the calibration information is optimized calibration information obtained by the calibration method according to any one of claims 1 to 9.

[0044] According to a third aspect, an embodiment provides a computer-readable storage medium, comprising a program, wherein the program can be executed by a processor to implement a method as described in any embodiment herein.

[0045] According to the calibration method, the three-dimensional reconstruction method based on line laser scanning and the computer-readable storage medium of the above-mentioned embodiments, the three-dimensional reconstruction device is first calibrated to determine the calibration information, and then the three-dimensional calibration plate is line laser scanned and reconstructed based on the calibration information, and the reconstruction result is aligned with the model information of the three-dimensional calibration plate to optimize the calibration information, thereby making the optimized calibration information more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic structural diagram of a three-dimensional reconstruction device based on line laser scanning according to an embodiment;

[0047] Figure 2 A flowchart of a calibration method according to an embodiment;

[0048] Figure 3 A flowchart of a light stripe extraction algorithm according to an embodiment;

[0049] Figure 4 A flowchart of how to process a light bar image column by column in one embodiment;

[0050] Figure 5 A three-dimensional diagram of an embodiment of a three-dimensional calibration plate;

[0051] Figure 6 for Figure 5 The three-dimensional image of the 3D calibration plate when viewed from the bottom;

[0052] Figure 7 for Figure 5 The main view of

[0053] Figure 8 for Figure 7 AA section view;

[0054] Fig. 9 for Figure 7 B-direction view;

[0055] Fig.10 for Figure 5 A partial enlarged view of point C in the middle;

[0056] Fig.11 The present invention is a flowchart of a three-dimensional reconstruction method based on line laser scanning according to an embodiment. DETAILED DESCRIPTION

[0057] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0058] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0059] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0060] Please refer to Figure 1 , is a three-dimensional reconstruction device based on line laser scanning in some embodiments, which includes a laser module 10 and an imaging component 30. The laser module 10 is used to emit a line laser to the object to be measured, and the imaging component 30 is used to receive the light obtained by the line laser reflected by the object to be measured to convert the light signal into an electrical signal to obtain a light strip image; the imaging component 30 has a camera; in some embodiments, the three-dimensional reconstruction device also has a moving component 20 and / or a controller 40. Figure 1 A in the figure represents a schematic diagram of the object to be tested.

[0061] In some embodiments, the moving assembly 20 is used to carry and move the object to be measured, so that the line laser emitted by the laser module 10 can complete the scanning of the object to be measured - in such an embodiment, the laser module 10 and the imaging assembly 30 can be fixed.

[0062] In some embodiments, the mobile component 20 can also carry the laser module 10 and the imaging component 30, so that the line laser emitted by the laser module 10 can complete the scanning of the object to be measured, and the imaging component 30 can continuously perform imaging during the scanning process - in such an embodiment, the laser module 10 and the imaging component 30 are mobile, but the relative position between the two is fixed, and at the same time, the position of the object to be measured is also fixed.

[0063] In some embodiments, the controller 40 can control the various components to work and process the light stripe image, such as extracting the light stripe and performing three-dimensional reconstruction based on the extracted light stripe.

[0064] In some embodiments, the controller 40 is capable of executing the methods and steps described in any embodiment herein.

[0065] During specific three-dimensional reconstruction, it is necessary to calibrate the imaging component 30, such as internal parameters and / or external parameters, calibrate the light plane of the laser module 10, and even calibrate the mobile component 20. Then, the mobile component 20 is controlled to work, thereby collecting light strip images. One light strip can be extracted from each light strip image. Based on all the extracted light strips, a world coordinate system is established (for example, in an example in which the mobile component 20 carries the object to be measured and moves the object to be measured, a world coordinate system can be established based on the initial position of the mobile component 20). Based on laser triangulation and the line-surface model, the world coordinates corresponding to the light strip coordinates in each light strip image are solved. Finally, the three-dimensional surface of the object to be measured can be reconstructed based on the obtained point cloud data.

[0066] In some embodiments of the present application, the three-dimensional reconstruction device is first calibrated to determine the calibration information, such as the intrinsic parameters of the camera, the extrinsic parameters of the camera, and the parameters of the light plane of the laser module; then the three-dimensional calibration plate is subjected to line laser scanning and reconstructed based on the calibration information, and the reconstruction result is aligned with the model information of the three-dimensional calibration plate to optimize the calibration information.

[0067] In some embodiments of the present application, when optimizing the calibration information, a new three-dimensional calibration plate is proposed, whose structure can meet the calibration requirements of different ranges, does not require repeated placement of standard blocks, can simplify operations, and is easy to ensure calibration accuracy.

[0068] In some embodiments of the present application, the light stripe extraction algorithm proposed in the present application is used when performing three-dimensional reconstruction. When extracting light stripes from a light stripe image, for each column, the starting point and the end point of the light stripe on the column in its width direction are first located to define the light stripe area of ​​each column, and then two types of candidate points (first candidate points and second candidate points) are defined based on the light stripe area, and the position of one of the two types of candidate points is selected as the position of the center point of the light stripe in the column based on the properties of the light stripe area. Through the scheme of the present application, when facing a non-opaque object surface (such as a translucent object surface), the light stripes can be extracted from the light stripe image more accurately.

[0069] The details are explained below.

[0070] In some embodiments, a calibration method is disclosed, which can be applied to a three-dimensional reconstruction device based on line laser scanning, such as Figure 1 Please refer to the 3D reconstruction device shown in Figure 2 , the calibration method of some embodiments includes the following steps:

[0071] Step S100: calibrate the 3D reconstruction device to determine calibration information.

[0072] In some embodiments, the calibration in step S100 includes: calibrating a camera of the three-dimensional reconstruction device to at least determine the intrinsic parameters of the camera, and calibrating a light plane of a laser module of the three-dimensional reconstruction device to at least determine the parameters of the light plane; accordingly, the calibration information includes the intrinsic parameters of the camera and the parameters of the light plane.

[0073] In some embodiments, the calibration in step S100 further includes: using the first model to perform distortion correction on the camera to obtain distortion parameters; accordingly, the calibration information also includes the distortion parameters.

[0074] In some embodiments, the first model includes:

[0075]

[0076] Among them, (u,v) T represents the coordinates before distortion correction in the imaging plane coordinate system, represents the coordinates after distortion correction in the imaging plane coordinate system, and k represents the radial distortion parameter.

[0077] In some embodiments, the first model includes:

[0078]

[0079] Among them, (u,v) T represents the coordinates before distortion correction in the imaging plane coordinate system, represents the coordinates after distortion correction in the imaging plane coordinate system, k1, k2, and k3 represent distortion parameters.

[0080] The following is a more detailed description of step S100.

[0081] The purpose of calibration is mainly to determine the matrix mapping corresponding to the point position in the three-dimensional space and the two-dimensional image of the camera; while the correction of distortion parameters is to determine the distortion coefficient of the camera imaging to correct the image.

[0082] Specifically, the world coordinate system, camera coordinate system, imaging plane coordinate system and pixel coordinate system are introduced, and the ultimate goal is to obtain the transformation relationship between the pixel coordinate system and the world coordinate system.

[0083] The world coordinate system is used to describe the three-dimensional position information of an object in space. The camera coordinate system is also a three-dimensional rectangular coordinate system, with the optical center of the camera lens as the center of the circle, the optical axis as the Z axis, and perpendicular to the plane where the photosensitive film / photosensitive array is located. The imaging plane coordinate system is a two-dimensional coordinate system established with the photosensitive film / photosensitive array as the plane, perpendicular to the optical axis, and used to connect the pixel coordinate system to the camera coordinate system. The pixel coordinate system is used to describe the position information of the midpoint of the image; the difference from the previous three coordinate systems is that the unit is pixel, the coordinate value is the row and column of the pixel point in the image, and the units of the above three coordinate systems are all actual physical sizes.

[0084] First, from the world coordinate system P w =(x w ,y w ,z w ) T Transform to the camera coordinate system P c =(x c ,y c ,z c ) T : Then from the camera coordinate system P c =(x c ,y c ,z c ) T Transform to the imaging plane coordinate system q c =(u,v) T : Where f is the focal length of the lens.

[0085] The imaging plane coordinate system before distortion correction is q c =(u,v) T , the lens distortion is corrected by a division model or a polynomial module, and the imaging plane coordinate system after the correction of the distortion is obtained as

[0086] From the imaging plane coordinate system Transform to pixel coordinate system q i =(r,c) T : Among them: r represents the row, c represents the column; s y Indicates the size of the sensor's unit pixel on the v-axis, in units such as mm / pixel; s x Indicates the size of the sensor's unit pixel on the u axis, in units such as mm / pixel; c y and c x It represents the optical center, that is, the intersection of the camera optical axis and the image plane, which is usually located at the center of the image, so its value is often half of the resolution. In addition, it can be understood that without considering the distortion, from the imaging plane coordinate system q c =(u,v) T Transform to pixel coordinate system q i =(r,c) T :

[0087]

[0088] Without considering the distortion, the expression of the total conversion can be obtained:

[0089]

[0090] Where: R is a 3×3 rotation matrix, t is a 3×1 translation vector, 0 T is the zero vector representing 1×3, z c Represents the object coordinates in the camera coordinate system.

[0091] The parameters obtained by the above solution are used as initial values ​​to further obtain the nonlinear model parameters of the camera taking the distortion factor into consideration, so that the parameters of the camera, such as intrinsic parameters and extrinsic parameters, can be finally obtained.

[0092] Generally, in the equation on the right side of the above total conversion expression, the first two matrices in the equation are called the intrinsic parameters of the camera, and the third matrix in the equation is called the extrinsic parameter of the camera.

[0093] It should be noted that, in the above, r and c are used to represent the coordinates in the pixel coordinate system, and u and v are used to represent the imaging plane coordinates. It can be understood that u and v can also be used to represent the coordinates in the pixel coordinate system, but u and v in the pixel coordinate system and u and v in the imaging plane coordinates still need to be transformed. Which letters are used to represent which coordinate system does not affect the definition of the corresponding coordinate system and its essential connotation; for example, x and y can also be used to represent the coordinates in the pixel coordinate system.

[0094] The above are some instructions for calibrating camera parameters. The following describes how to calibrate the light plane.

[0095] The laser module is turned on, and the laser line is projected onto the surface of the calibration plate (the three-dimensional calibration plate or other calibration plate of the present application), and the position set of the laser line in the image is obtained by a light strip extraction algorithm such as that of some embodiments of the present application. 1,i}, and the position of the feature points of the calibration object in the image {p 2,i}; Based on the position of the feature points and the calibrated camera intrinsic parameters, the coordinates of the calibration plate in the camera coordinate system can be calculated, including the placement matrix R and the translation vector t of the above total transformation expression; therefore, the three-dimensional position of the point on the calibration plate plane in the camera coordinate system and its three-dimensional coordinates in the imaging plane coordinate system can be calculated, and the following transformation relationship can be obtained:

[0096] p c =[Rt]K -1 p r ;

[0097] p c represents the three-dimensional coordinates of the point on the calibration plate captured in the image in the camera coordinate system; p r represents the point coordinates in the pixel coordinate system after distortion correction (i.e., the pixel coordinate system); K represents the intrinsic parameter matrix.

[0098] The position set of laser lines in the image {p 1,i}Substitute p in the above transformation relationship r , get the three-dimensional coordinates of the point where the laser plane intersects the calibration plate in the reference coordinate system of the imaging system Each point is represented by (x, y, z); by changing the posture of the calibration plate and taking multiple images, multiple sets of different represents the jth point in the i-th group; further construct equations and solve them to obtain the values ​​of A, B and C in the coordinate expression Ax+By+Cz=1 of the light plane.

[0099] The above is some description of step S100 of calibrating the 3D reconstruction device to determine the calibration information.

[0100] Step S200: performing line laser scanning on the three-dimensional calibration plate to obtain a light stripe image of the line laser.

[0101] Step S300: extracting light stripes from the light stripe image based on a light stripe extraction algorithm, and performing three-dimensional reconstruction on the three-dimensional calibration plate based on the extracted light stripes and the calibration information to obtain a three-dimensional reconstruction result.

[0102] Step S400: Acquire the model information of the three-dimensional calibration plate, and register the three-dimensional reconstruction result of the three-dimensional calibration plate with its model information to obtain difference information, and optimize the calibration information based on the difference information.

[0103] In some examples, the reconstructed structure is aligned with the model information of the 3D calibration plate (e.g., the file information of the 3D calibration plate set by CAD), and the calibration information is optimized based on the error between the two; for example, (x c ,y c ,z c ) is calculated as follows:

[0104]

[0105]

[0106]

[0107] Where u and v are pixel coordinates, c y and c x Table 2 is the internal parameter of the camera, indicating the optical center; f x and f y The s x and y Countdown, s x Indicates the size of the camera sensor in pixels on the v axis, s y represents the size of the unit pixel of the camera sensor on the u axis; A, B and C are three parameters of the light plane, and the light plane is Ax+By+Cz=1; i represents the sequence number of the pixel point, and n represents the total number of pixels; z c is the coordinate of the Z axis in the camera coordinate system.

[0108] Based on the above calculation results and the model information of the three-dimensional calibration plate, the internal parameters and A, B, and C are optimized. The external parameter matrix can also be optimized based on the optimization results.

[0109] Therefore, in some embodiments, step S400 establishes an objective function based on the difference information and optimizes the calibration information based on the objective function; wherein the objective function is:

[0110]

[0111] u and v are the coordinates in the pixel coordinate system; c y and c x Table 2 is the internal parameter of the camera, indicating the optical center; f x and f y The s x and y Countdown, sx Indicates the size of the camera sensor in pixels on the v axis, s y represents the size of the unit pixel of the camera sensor on the u axis; A, B and C are three parameters of the light plane, and the light plane is Ax+By+Cz=1; i represents the sequence number of the pixel point, and n represents the total number of pixels; z c is the coordinate of the Z axis in the camera coordinate system.

[0112] In some embodiments, the model information of the three-dimensional calibration plate may be a design file of the three-dimensional calibration plate, such as file information of the three-dimensional calibration plate set by CAD.

[0113] The light stripe extraction algorithm mentioned above may adopt an existing or future algorithm, or the algorithm mentioned in this article.

[0114] In some embodiments, please refer to Figure 3 , the light stripe extraction algorithm includes the following steps:

[0115] Step S310: Process the light bar image column by column.

[0116] It can be understood that a two-dimensional image includes rows and columns, and rows and columns are relative on a two-dimensional plane: if the horizontal direction is the row direction, the vertical direction is the column direction; if the horizontal direction is the column direction, the vertical direction is the row direction. In order to make it easier to process, the direction roughly parallel to the light strip direction can be defined as the row direction, and the direction roughly perpendicular to the light strip direction can be defined as the column direction.

[0117] The following is a detailed description of the column-by-column processing of the light strip image in step S310. Figure 4 In some embodiments, for any column of the light bar image (for the convenience of discussion, let's call it C[x]), the following steps are performed:

[0118] Step S311: Calculate the first gradient parameter of each pixel in the light stripe image sequence C[x].

[0119] There are many ways to calculate the first gradient parameter of a pixel point, and several of them are listed below.

[0120] In some embodiments, step S311 may extract the first-order gradient of each pixel in column C[x] as the first gradient parameter of each pixel by using a one-dimensional gradient operator.

[0121] The one-dimensional gradient operator may be, for example, a Sobel operator, a Roberts operator, or a Prewitt operator. The gradient perpendicular to the light strip direction, that is, the column direction, is extracted by the one-dimensional gradient operator, which can be expressed by the following formula:

[0122] I' x,y =f(Ix,y );

[0123] Among them I' x,y Represents pixel I x,y The first-order gradient of ; x and y represent rows and columns respectively, and f represents a convolution operation, where the convolution kernel used can be, for example, a one-dimensional Y-direction Sobel operator:

[0124]

[0125] In some embodiments, step S311 can also calculate the first-order gradient, second-order gradient and third-order gradient of each pixel point in column C[x]; for each pixel point in column C[x], a quadratic function is fit based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as parameters of the quadratic function, and the local maximum of the quadratic function is solved as the first gradient parameter of the pixel point.

[0126] The first-order gradient, second-order gradient and third-order gradient of the pixel point can be calculated by using the existing known or future methods; then the quadratic function is fitted based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as the parameters of the quadratic function. An example can be the following formula:

[0127]

[0128] Among them I' x,y , I” x,y ,I”' x,y Represents pixel I x,y The first-order gradient, second-order gradient, and third-order gradient of ; x and y represent rows and columns respectively; and then solve the local maximum of the quadratic function of the above pixel point as the first gradient parameter of the pixel point.

[0129] Based on the first-order gradient, second-order gradient and third-order gradient of the pixel point, a quadratic function is fitted and the first gradient parameter of the pixel point is finally calculated. This method calculates the gradient with high accuracy and good robustness.

[0130] Step S313: According to the first gradient parameter of each pixel point in column C[x], a starting point and an end point are selected from column C[x] to define a light bar region of column C[x].

[0131] In some embodiments, step S313 determines the starting point through the first gradient parameter, and determines the end point in combination with the grayscale information of the light bar image; for example, step S313 traverses the pixel points in the column direction, takes the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and continues to traverse to find the first pixel point whose grayscale value is less than or equal to the starting point as the end point.

[0132] In some embodiments, step S313 may also determine the starting point and the end point through the first gradient parameter, for example, taking the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and taking the pixel point whose first gradient parameter is greater than or equal to the second threshold as the end point.

[0133] Through the above steps, the light bar area of ​​column C[x] is defined, that is, the range of the width of the light bar on column C[x].

[0134] Step S315: taking the point with the largest second gradient parameter, the starting point or the end point in the light bar area of ​​column C[x] as the first candidate point.

[0135] In some embodiments, step S315 may extract the first-order gradient of each pixel in column C[x] as the second gradient parameter of each pixel by using a one-dimensional gradient operator.

[0136] The one-dimensional gradient operator may be, for example, a Sobel operator, a Roberts operator, or a Prewitt operator. The gradient perpendicular to the light strip direction, that is, the column direction, is extracted by the one-dimensional gradient operator, which can be expressed by the following formula:

[0137] I' x,y =f(I x,y );

[0138] Among them I' x,y Represents pixel I x,y The first-order gradient of ; x and y represent rows and columns respectively, and f represents a convolution operation, where the convolution kernel used can be, for example, a one-dimensional Y-direction Sobel operator:

[0139]

[0140] In some embodiments, step S315 can calculate the first-order gradient, second-order gradient and third-order gradient of each pixel point in column C[x]; for each pixel point in column C[x], a quadratic function is fit based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as parameters of the quadratic function, and the local maximum of the quadratic function is solved as the second gradient parameter of the pixel point.

[0141] The first-order gradient, second-order gradient and third-order gradient of the pixel point can be calculated by using the existing known or future methods; then the quadratic function is fitted based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as the parameters of the quadratic function. An example can be the following formula:

[0142]

[0143] Among them I' x,y , I” x,y ,I”'x,y Represents pixel I x,y The first-order gradient, second-order gradient, and third-order gradient of ; x and y represent rows and columns respectively; and then solve the local maximum of the quadratic function of the above pixel point as the first gradient parameter of the pixel point.

[0144] Based on the first-order gradient, second-order gradient and third-order gradient of the pixel point, a quadratic function is fitted and the second gradient parameter of the pixel point is finally calculated. This method calculates the gradient with high accuracy and good robustness.

[0145] Step S317: Select a pixel point in the light bar area of ​​column C[x] as the second candidate point by a preset method.

[0146] In some embodiments, step S317 calculates the first-order gradient and second-order gradient of each pixel point in the light strip area of ​​column C[x]; for each pixel point in the light strip area of ​​column C[x], a quadratic function is fitted based on the grayscale value, first-order gradient and second-order gradient of the pixel point as parameters of the quadratic function, and the local maximum and corresponding position of the quadratic function of the pixel point are solved; the local maximum values ​​of the above-mentioned quadratic function of each pixel point in column C[x] are compared to obtain the largest local maximum value among these local maximum values, and the position corresponding to the largest local maximum value is used as the position of the second candidate point.

[0147] For each pixel in the light strip area of ​​column C[x], a quadratic function is fitted based on the grayscale value, first-order gradient and second-order gradient of the pixel as parameters of the quadratic function. An example may be the following formula:

[0148]

[0149] Among them I x,y , I' x,y , I” x,y Represents pixel I x,y The gray value, first-order gradient, and second-order gradient of the pixel are respectively represented by x and y. Then the point corresponding to the local maximum value of the quadratic function of the above pixel point is solved as the second candidate point, and its position is correspondingly used as the position of the second candidate point.

[0150] The second candidate point obtained by this method has very high accuracy, and the algorithm has a faster processing speed.

[0151] In some embodiments, step S317 obtains the grayscale value and weight of each pixel point in the light strip area of ​​column C[x], and calculates a weighted average grayscale value, wherein the weight obeys a Gaussian distribution; and takes the pixel point in the light strip area of ​​column C[x] whose grayscale value is greater than the weighted average grayscale value as the second candidate point.

[0152] The second candidate point obtained by this method has very high accuracy, and the algorithm has a faster processing speed.

[0153] In some embodiments, step S317 obtains the grayscale values ​​of the pixel with the largest grayscale value in the light strip area of ​​column C[x] and the grayscale values ​​of two pixels in its neighborhood (it can be understood that the neighborhood here refers to the neighborhood in the column direction), and fits a quadratic function of the grayscale value with respect to the position based on these three pixels, obtains the position corresponding to the local maximum of the above quadratic function, and uses this position as an offset plus the position of the pixel with the largest grayscale value to calculate the position obtained as the position of the second candidate point.

[0154] The above scheme is essentially based on local grayscale information estimation, using quadratic functions for fitting, taking the grayscale values ​​of the point with the highest grayscale value in the area and the two points in its neighborhood, fitting the quadratic function of the pixel grayscale value with respect to the coordinate position, taking the highest value of the quadratic function as the peak grayscale value, and saving it with the position information as the brightness of the light point, thereby also obtaining the position of the light point with the highest brightness (the point with the highest grayscale value in the area, whose coordinates plus the offset are equal to the position of the light point with the highest brightness), which improves the accuracy of grayscale estimation. It can be seen that in the process of extracting the center point, not only the grayscale distribution of the light strip is taken into account, but also the brightness is determined, taking into account the brightness changes of the laser, so as to be more conducive to solving imaging problems such as translucent or highly reflective objects.

[0155] For example, the following is an example:

[0156] In an example, let I0 represent the point with the highest grayscale value in the above-mentioned area, and the two points in its neighborhood are I -1 and I1; through these three pixels I -1 , I0 and I1 gray value and position fit a quadratic function I g =ax 2 +bx+c, and get the values ​​of a, b, and c; these three pixels I -1 The grayscale values ​​of I0, I1 are substituted into the I g The corresponding x value can be solved to obtain the three pixel points I -1 The relative coordinates of I0 and I1 are a-b+c, c, a+b+c respectively; therefore, the offset obtained by further calculation is:

[0157]

[0158] The position of pixel I0 in the column direction (i.e., row coordinate, for example Figure 2 The X-coordinate in the ) plus the offset is equal to the position of the light point with the highest brightness (that is, the row coordinate, for example Figure 2The X-coordinate in .

[0159] The processing of a certain column in the light stripe image is completed through the above steps S311 to S317. Steps S311 to S317 can be performed on each column one by one, so that the light stripe area, the first candidate point (position) and the second candidate point (position) of each column in the light stripe image can be obtained.

[0160] Step S320: For any column of the light bar image (for the sake of convenience, let's call it C[x]), select one of the first and second candidate points of the column as the position of the center point of the light bar of the column based on at least the width of the light bar area of ​​the column.

[0161] In some embodiments, there are multiple implementations of how step S320 selects one of the first candidate point and the second candidate point in column C[x] as the center point of the light bar of column C[x] (or selects the position of one as the position of the center point of the light bar of column C[x]), and a few of them are listed below.

[0162] In some embodiments, step S320 determines whether the light bar region of column C[x] belongs to the first type of region or the second type of region based on at least the width of the light bar region of column C[x]; if it belongs to the first type of region, step S320 selects the position of the first candidate point of column C[x] as the position of the center point of the light bar of column C[x]; if it belongs to the second type of region, step S120 selects the position of the second candidate point of column C[x] as the position of the center point of the light bar of column C[x]. This process involves how to determine whether the light bar region of column C[x] belongs to the first type of region or the second type of region. In some embodiments, step S320 determines whether the light bar region of column C[x] belongs to the first type of region or the second type of region based on the width of the light bar region of column C[x] and the width of the light bar region of the columns in the neighborhood of column C[x], for example:

[0163] When the width of the light bar area of ​​column C[x] is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar area of ​​column C[x] belongs to the first type of area;

[0164] When the width of the light bar region of column C[x] is less than the first width threshold, and the number of columns in its neighborhood whose widths are greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar region of column C[x] belongs to the first type of region;

[0165] When the width of the light bar region of column C[x] is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar region of column C[x] belongs to the first type of region;

[0166] When the width of the light bar region of column C[x] is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar region of column C[x] belongs to the second type region.

[0167] It can be understood that the width of the light strip area refers to the length of the light strip area in the column direction, that is, the distance between the starting point and the end point of the light strip area.

[0168] Based on experiments and theoretical conclusions, the applicant concluded that for any column C[x], if the width of its own light stripe area is relatively large, and there are a large number of columns with relatively large light stripe area widths in its neighborhood (referring to some adjacent columns of column C[x], where the columns N columns away from column C[x] can be specifically defined as the columns in its neighborhood, and N can be taken according to actual conditions, such as 1, 2, 3, etc.), then it means that column C[x] corresponds to the area of ​​the non-opaque position (such as the semi-transparent position) of the object to be measured; if the width of the light stripe area of ​​column C[x] itself is relatively small, but there are a large number of columns with relatively large light stripe area widths in its neighborhood, then it means that column C[x] corresponds to the area of ​​the non-opaque position (such as the semi-transparent position) of the object to be measured. The calculation of the light bar area itself may be affected by problems, therefore, column C[x] is still regarded as the area corresponding to the non-opaque position (e.g., semi-transparent position) of the object to be measured, which helps to eliminate the influence of interference factors; if the width of the light bar area of ​​column C[x] itself is large, although the number of columns with large light bar area widths in its neighborhood is small, column C[x] should still be regarded as the area corresponding to the non-opaque position (e.g., semi-transparent position) of the object to be measured; if the width of the light bar area of ​​column C[x] itself is small, and the number of columns with large light bar area widths in its neighborhood is small, it means that column C[x] corresponds to the area of ​​the opaque position of the object to be measured.

[0169] In some other embodiments, step S320 selects one of the first candidate point and the second candidate point of column C[x] as the position of the center point of the light bar of column C[x] according to the width of the light bar area of ​​column C[x] and the width of the light bar area of ​​columns in the neighborhood of column C[x]. For example:

[0170] When the width of the light bar area of ​​column C[x] is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, the position of the first candidate point of column C[x] is selected as the position of the center point of the light bar of column C[x];

[0171] When the width of the light bar area of ​​column C[x] is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, the position of the first candidate point of column C[x] is selected as the position of the center point of the light bar of column C[x];

[0172] When the width of the light bar area of ​​column C[x] is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, the position of the first candidate point of column C[x] is selected as the position of the center point of the light bar of column C[x];

[0173] When the width of the light bar area of ​​column C[x] is less than the first width threshold, and the number of columns in its neighborhood whose widths are greater than or equal to the second width threshold is less than the threshold, the position of the second candidate point of column C[x] is selected as the position of the center point of the light bar of the column.

[0174] Through the above method, the center point of the light stripes in each column can be extracted, and the center points of the light strips in each column constitute the extracted light strips.

[0175] The light stripe extraction method of the present application can extract light stripes from a light stripe image more accurately when facing a non-opaque object surface (such as a translucent object surface); some schemes use gradients, which are suitable for using common hardware acceleration methods such as FPGA, GPU, etc.; in addition, a first candidate point and a second candidate point are determined from the light stripe area, and the position of one of them is finally selected as the position of the center point of the light stripe of the column, which takes into account both opaque positions and non-opaque positions (such as translucent positions), which makes the use environment of the scheme of the present application more universal and eliminates the complicated process of manually distinguishing regional features.

[0176] After calibrating the 3D reconstruction device to obtain the parameters of the camera and the light plane, considering that there may be a large height difference on the surface of the object during line laser scanning, in order to ensure the reconstruction accuracy of different heights, standard blocks of different sizes, such as 1mm, 5mm, and 10mm standard blocks, can be used, combined with the full range of the two-dimensional target online laser for secondary calibration of each height. The specific secondary calibration process is: adjust the height of the workbench so that the working distance of the laser transmitter can cover the full range; adjust the posture of the two-dimensional target and save the corresponding photos; place a 1mm standard block, and place the standard block on the two-dimensional target, adjust the posture, and save the corresponding photos. Since the three-dimensional size of the standard block is known, the known size of the standard block can be compared with the size measured by 3D imaging to complete the calibration of 1mm height; and so on, complete the calibration of all heights in the full range; this secondary calibration process requires repeated placement of standard blocks, which is cumbersome to operate, and is easily affected by the placement accuracy of the standard blocks, and the calibration accuracy is also easily affected.

[0177] In some embodiments of the present application, an improved three-dimensional calibration plate is proposed, which can be used as the three-dimensional calibration plate in this article, such as steps S200 to S400. Different parts of this improved three-dimensional calibration plate can form height differences in the thickness direction of the three-dimensional calibration plate, and the calibration convex parts and / or calibration concave parts can form more height differences. In this way, the three-dimensional calibration plate can form the height direction dimension difference required for calibration by itself, and there is no need to repeatedly set standard blocks of different sizes, which is easy to operate.

[0178] Please refer to Figures 5 to 10 In one embodiment, the three-dimensional calibration plate includes a first plate body portion and a second plate body portion, wherein the first plate body portion is a plate body 100 and the second plate body portion is an edge portion 200. In the illustrated embodiment, please refer to Figure 5 According to the arrows in the figure, the first plate part and the second plate part have the first side and the second side in the thickness direction, respectively. The first sides of the two sides are parallel to the thickness direction of the three-dimensional calibration plate, and the first sides of the first plate part and the second plate part have the same orientation, both facing upward. It should be noted that the top and bottom in this article are for the purpose of more clearly explaining the specific implementation methods and technical solutions of the present application, and more clearly indicating the positional relationship between the various components, and do not limit the specific implementation methods and technical solutions of the present application to be arranged in this way.

[0179] Please refer to Figure 7 In one embodiment, the shape of the three-dimensional calibration plate is L-shaped, and the top corners are rounded. The shape can be designed according to the shape of the actual object to be measured. In addition, the width of the edge portion 200 can also be designed as needed. The edge portion 200 surrounds the plate body 100 and has an L-shaped outline, which is consistent with the outline of the plate body 100.

[0180] A step structure may be formed between the edge portion 200 and the board body 100 at least on the first side, and the first side of the board body 100 may be provided with calibration protrusions 310 and / or calibration recesses 320 in an array, and the first side of the edge portion 200 may also be provided with calibration protrusions 310 and / or calibration recesses 320 in an array. Figure 6 In a specific embodiment, the bottom surface of the plate body 100 is coplanar with the bottom surface of the edge portion 200, and the top surface forms a step structure, which is simpler in structure and can also facilitate the stable placement of the three-dimensional calibration plate and the positioning with the matching positioning structure. In some preferred embodiments, the calibration protrusions 310 and / or the calibration recesses 320 can be arranged in a rectangular array, and the row direction and column direction of the rectangular array can be parallel to the side of the three-dimensional calibration plate. Of course, other array forms are not excluded.

[0181] In a specific embodiment, the first side of the board body 100 is only provided with the calibration concave portion 320, the edge portion 200 forms a sinking structure around the board body 100, and the edge portion 200 is only provided with the calibration convex portion 310. For details, please refer to Figure 5 , Figure 6 and Figure 7 , the calibration recesses 320 are distributed in arrays on the top surface (i.e., the side surface of the first side) and the bottom surface (i.e., the side surface of the second side) of the plate body 100; and the calibration protrusions 310 are distributed in arrays on the top surface (i.e., the side surface of the first side) of the edge portion 200, and the bottom surface (i.e., the side surface of the second side) of the edge portion 200 is a flat surface, without the calibration protrusions 310 and the calibration recesses 320. Since the thickness of the edge portion 200 is relatively thin, the calibration protrusions 310 are provided to ensure the structural strength, and the depth of the calibration recesses 320 is not limited by the thickness of the edge portion 200 when the calibration recesses 320 are used; in addition, the flat surface of the bottom surface of the edge portion 200 is also conducive to avoiding the influence of the dimensional stability due to the thin edge portion 200. In other embodiments, the calibration recess 320 at the bottom of the three-dimensional calibration plate may not distinguish between the plate body 100 and the edge portion 200. The side surfaces of the plate body 100 and the edge portion 200 facing away from the first side are located on the same plane, and the calibration recesses 320 are distributed in an array on the plane, that is, the calibration recesses 320 are arranged on the bottom surface of the entire three-dimensional calibration plate.

[0182] It should be noted that, in some other embodiments, the calibration protrusion 310 on the top surface (i.e., the side surface of the first side) of the plate body 100 and / or the edge portion 200 can also be replaced by a calibration recess 320, and the calibration recess 320 on the bottom surface (i.e., the side surface of the second side) of the plate body 100 can also be replaced by a calibration protrusion 310. Since the three-dimensional calibration plate needs to have a high processing accuracy, generally it needs to reach micron-level accuracy, so the processing cost is very high, and it needs to have a high rigidity to maintain dimensional stability. In the present application, the calibration protrusion 310 and the calibration recess 320 are simultaneously provided on the three-dimensional calibration plate, and the calibration of the two convex and concave structures can be realized using one calibration plate, which is conducive to reducing costs.

[0183] In one embodiment, the calibration recesses 320 on the edge portion 200 are provided in one row, which can calibrate the position closest to the edge and improve the accuracy of edge detection. In addition, since the processing accuracy requirements of the three-dimensional calibration plate are very high, only one row of calibration recesses 320 is also conducive to reducing processing costs. In other embodiments, the calibration recesses 320 on the edge portion 200 can also be provided in more than two rows, and the calibration recesses 320 on each row can be aligned along the extension direction of the edge portion 200, or can be staggered along the extension direction of the edge portion 200. In addition, considering the field of view of the imaging module, the size of the edge portion 200 and the number of columns of the calibration convex portion 310 / calibration recess 320 also need to match the field of view of the imaging module.

[0184] In one embodiment, in the protruding direction of the calibration protrusion 310, the top surface of the calibration protrusion 310 on the edge portion 200 is lower than the surface of the board body 100 facing the first side, so that a richer height difference can be formed to better meet the calibration requirements. However, in some other embodiments, the calibration protrusion 310 on the edge portion 200 can also be the same height as the board body 100 or higher than the board body 100.

[0185] In the present application, the shapes and sizes of the calibration protrusions on the plate body 100 and the edge portion 200 are consistent. In one embodiment, the calibration protrusion 310 has a top surface and an inclined side surface 330 connected to the top surface. The inclined side surfaces 330 are provided at four locations, and each inclined side surface 330 forms a quadrangular pyramid shape. The inclined side surfaces 330 provided on the calibration protrusion can better reflect the laser, avoid the sudden changes in the vertical intersection surface that affect the light reflection effect, and can achieve linear calibration. At the same time, the provision of four inclined side surfaces 330 can achieve detection in four different directions, which is more conducive to improving the overall detection accuracy.

[0186] Similarly, in one embodiment, the calibration recess 320 has a bottom surface and an inclined side surface 330 connected to the bottom surface, and the inclined side surfaces 330 are provided at four locations, each of which forms a quadrangular pyramid shape. In other embodiments, the number of the inclined side surfaces 330 on the calibration protrusion 310 and the calibration recess 320 may also be three or more than five.

[0187] In a specific embodiment, the calibration protrusion 310 includes a base portion 311, the base portion 311 is located at the bottom of the quadrangular pyramid, the cross section of the base portion 311 is square, and each side of the base portion 311 corresponds to each side of the quadrangular pyramid. This structure can form a richer surface shape and is easy to process, which is conducive to the actual implementation of product design.

[0188] In order to further improve the calibration effect of the three-dimensional calibration plate, in one embodiment, a transition surface 400 is provided between the plate body 100 and the edge portion 200, and the transition surface 400 is an inclined surface. The transition surface 400 provided between the plate body 100 and the edge portion 200 can form a gradual height change to achieve linear calibration. In a specific embodiment, calibration convex portions 310 are distributed on the transition surface 400, and the convex direction of the calibration convex portions 310 on the transition surface 400 is perpendicular to the transition surface 400. The calibration convex portions 310 on the inclined transition surface 400 can form a richer height change to achieve a better calibration effect. It should be noted that, in theory, a calibration recess 320 can also be set on the transition surface 400. However, in order to better ensure accuracy, the three-dimensional calibration plate in this embodiment is made of a whole piece of material and processed by a tool machine. Since the transition surface 400 is an inclined surface, the protruding calibration protrusion 310 on the transition surface 400 is more convenient for processing. If a calibration recess 320 is set, the tool needs to have a smaller size, the processing difficulty will be higher, and it may even be impossible to process.

[0189] During current calibration, the coordinate relationship between the camera (imaging module) and the laser plane is usually calibrated through a plane calibration plate, and three-dimensional reconstruction is completed based on the intersection of the intersecting surfaces. There will be certain errors, and the calibration block is used to calculate the true error, and the error compensation is performed directly without correcting the system parameters; however, in many cases the compensation is not linear. In this application, the system rescans the calibration block, calculates the true error, and then corrects the coordinate relationship between the camera and the laser plane, which is equivalent to correcting the system coefficient to make the system error as small as possible, rather than directly compensating the error, thereby achieving better detection effects.

[0190] In other embodiments of a three-dimensional calibration plate in the present application, the first plate portion and the second plate portion of the three-dimensional calibration plate may also be in other combinations, for example, the first plate portion and the second plate portion may be in a stepped shape arranged side by side in the horizontal direction. For another example, the first plate portion and the second plate portion may form a step on the top surface (i.e., the side surface of the first side) of the three-dimensional calibration plate, and at the same time form a step on the bottom surface (i.e., the side surface of the second side) of the three-dimensional calibration plate.

[0191] In some embodiments of the present application, a three-dimensional reconstruction method based on line laser scanning is also disclosed, which is applied to a three-dimensional reconstruction device, and the three-dimensional reconstruction device can be the three-dimensional reconstruction device described in this article. Fig.11 , the three-dimensional reconstruction method in some embodiments comprises the following steps:

[0192] Step S1000: performing laser scanning on the object to be measured to obtain a light strip image.

[0193] Step S1100: extracting light stripes from the light stripe image based on a light stripe extraction algorithm. Step S1100 involves a light stripe extraction algorithm, which may adopt an existing or future algorithm, or the algorithm mentioned in this article.

[0194] Step S1200: Perform three-dimensional reconstruction of the object to be measured based on the extracted light strips and the calibration information to obtain a reconstruction result. In some embodiments, the calibration information in step S1200 is optimized calibration information obtained by the calibration method of any embodiment of this document.

[0195] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, various operating steps and components for performing the operating steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or considering any number of cost functions associated with the operation of the system.

[0196] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. In addition, as understood by those skilled in the art, the principles of this article can be reflected in a computer program product on a computer-readable storage medium, which is pre-installed with a computer-readable program code. Any tangible, non-temporary computer-readable storage medium can be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD to ROM, DVD, Blue Ray disks, etc.), flash memory and / or the like. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer or other programmable data processing device to form a machine, so that these instructions executed on a computer or other programmable data processing device can generate a device that implements a specified function. These computer program instructions can also be stored in a computer-readable memory, which can instruct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory can form a manufactured product, including an implementation device that implements a specified function. Computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on a computer or other programmable device to generate a computer-implemented process, so that the instructions executed on a computer or other programmable device can provide steps for implementing a specified function.

[0197] Although the principles of this invention have been shown in various embodiments, many modifications of structures, arrangements, proportions, elements, materials and components particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this invention. The above modifications and other changes or amendments will be included in the scope of this invention.

[0198] The foregoing specific description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of the present disclosure. Therefore, the consideration of the present disclosure will be illustrative rather than restrictive, and all these modifications will be included in its scope. Similarly, the advantages, other advantages and solutions to the problems of various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more clear, should not be interpreted as critical, necessary or necessary. The term "include" and any other variants used in this article are all non-exclusive inclusions, so that the process, method, article or device including the list of elements not only includes these elements, but also includes other elements that are not explicitly listed or do not belong to the process, method, system, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0199] Those skilled in the art will appreciate that many changes may be made to the details of the above-described embodiments without departing from the basic principles of the invention. Therefore, the scope of the present invention should be determined solely by the claims.

Claims

1. A calibration method, characterized in that: Applicable to a three-dimensional reconstruction device based on line laser scanning, characterized in that the calibration method comprises: Calibrate the three-dimensional reconstruction device to determine calibration information; wherein the calibration includes: calibrating a camera of the three-dimensional reconstruction device to at least determine an intrinsic parameter of the camera, and calibrating a light plane of a laser module of the three-dimensional reconstruction device to at least determine a parameter of the light plane; wherein the calibration information includes the intrinsic parameter of the camera and the parameter of the light plane; Perform line laser scanning on the three-dimensional calibration plate to obtain a light strip image of the line laser; Extracting light stripes in the light stripe image based on a light stripe extraction algorithm, and performing three-dimensional reconstruction on the three-dimensional calibration plate based on the extracted light stripes and the calibration information to obtain a three-dimensional reconstruction result; The model information of the three-dimensional calibration plate is obtained, and the three-dimensional reconstruction result of the three-dimensional calibration plate is registered with the model information thereof to obtain difference information, and the calibration information is optimized based on the difference information.

2. The calibration method according to claim 1, characterized in that: The calibration further includes: using a first model to perform distortion correction on the camera to obtain distortion parameters; the calibration information also includes the distortion parameters.

3. The calibration method according to claim 2, characterized in that: The first model includes: Among them, (u,v) T represents the coordinates before distortion correction in the imaging plane coordinate system, represents the coordinates after distortion correction in the imaging plane coordinate system, and k represents the radial distortion parameter; or, Among them, (u,v) T represents the coordinates before distortion correction in the imaging plane coordinate system, represents the coordinates after distortion correction in the imaging plane coordinate system, k1, k2, and k3 represent distortion parameters.

4. The calibration method according to claim 1 or 2, characterized in that: The optimizing the calibration information based on the difference information includes: establishing an objective function based on the difference information and optimizing the calibration information based on the objective function; wherein the objective function is: u and v are the coordinates in the pixel coordinate system; c y and c x Table 2 is the internal parameter of the camera, indicating the optical center; f x and f y The s x and y Countdown, s x Indicates the size of the camera sensor in pixels on the v axis, s y represents the size of the unit pixel of the camera sensor on the u axis; A, B and C are three parameters of the light plane, and the light plane is Ax+By+Cz=1; i represents the sequence number of the pixel point, and n represents the total number of pixels; z c is the coordinate of the Z axis in the camera coordinate system.

5. The calibration method according to claim 1, characterized in that: The light stripe extraction algorithm comprises: The light stripe image is processed column by column, and for any column of the light stripe image: Calculate a first gradient parameter of each pixel point in the column of the light stripe image; According to the first gradient parameter of each pixel point in the column, a starting point and an end point are selected from the column to define a light bar area of ​​the column; The point with the largest second gradient parameter within the light strip area of ​​the column, the starting point or the end point is taken as the first candidate point; Selecting a pixel point in the light strip area of ​​the column as a second candidate point by a preset method; For any column of the light bar image, at least according to the width of the light bar region of the column, a position of one of the first candidate point and the second candidate point of the column is selected as the position of the center point of the light bar of the column.

6. The calibration method according to claim 5, characterized in that: The method of selecting a starting point and an end point from the column to define the light bar area of ​​the column according to the first gradient parameter of each pixel point in the column includes: traversing the pixel points in the direction of the column, taking the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and continuing to traverse to find the first pixel point whose grayscale value is less than or equal to the starting point as the end point; or taking the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and taking the pixel point whose first gradient parameter is greater than or equal to the second threshold as the end point.

7. The calibration method according to claim 5, characterized in that: The step of selecting a pixel point in the light bar area of ​​the column as the second point to be selected by a preset method includes: Calculating the first-order gradient and the second-order gradient of each pixel point in the light strip area of ​​the column; fitting a quadratic function based on the grayscale value, the first-order gradient and the second-order gradient of each pixel point in the light strip area of ​​the column as parameters of the quadratic function, and solving the local maximum value and the corresponding position of the quadratic function of the pixel point; comparing the local maximum values ​​of the quadratic function of each pixel point in the column to obtain the largest local maximum value among these local maximum values, and taking the position corresponding to the largest local maximum value as the position of the second candidate point; or, Obtaining the grayscale value and weight of each pixel point in the light strip area of ​​the column, and calculating a weighted average grayscale value, wherein the weight obeys a Gaussian distribution; taking the pixel point in the light strip area of ​​the column whose grayscale value is greater than the weighted average grayscale value as the second candidate point; or, Obtain the grayscale values ​​of the pixel with the largest grayscale value in the light strip area of ​​the column and two pixels in its neighborhood, and fit a quadratic function of the grayscale value with respect to the position based on these three pixels, obtain the position corresponding to the local maximum of the quadratic function, and use this position as an offset plus the position of the pixel with the largest grayscale value to calculate the position obtained as the position of the second candidate point.

8. The calibration method according to claim 5, characterized in that: The step of selecting one of the positions of the first candidate point and the second candidate point of the column as the position of the center point of the light bar of the column at least according to the width of the light bar area of ​​the column comprises: At least according to the width of the light strip area of ​​the column, it is determined whether the light strip area of ​​the column belongs to the first type of area or the second type of area; wherein, according to the width of the light strip area of ​​the column and the width of the light strip area of ​​the columns in the neighborhood of the column, it is determined whether the light strip area of ​​the column belongs to the first type of area or the second type of area: when the width of the light strip area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light strip area of ​​the column belongs to the first type of area; or, when the width of the light strip area of ​​the column is less than the first width threshold , and the number of columns whose widths in its neighborhood are greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or, when the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns whose widths in its neighborhood are greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or, when the width of the light bar area of ​​the column is less than the first width threshold, and the number of columns whose widths in its neighborhood are greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar area of ​​the column belongs to the second type of area; If it belongs to the first type of area, the position of the first candidate point in the column is selected as the position of the center point of the light bar in the column; If it belongs to the second type area, the position of the second candidate point in the column is selected as the position of the center point of the light bar in the column.

9. The calibration method according to claim 1, characterized in that: The three-dimensional calibration plate includes: a main plate and an edge portion; the edge portion and the plate body both have a first side, the edge portion and the first side of the plate body are oriented in the same direction and are parallel to the thickness direction of the three-dimensional calibration plate; a step structure is formed between the edge portion and the plate body at least on the first side, the first side of the plate body is distributed with calibration protrusions and / or calibration recesses in an array form, and the first side of the edge portion is also distributed with calibration protrusions and / or calibration recesses in an array form.

10. A three-dimensional reconstruction method based on line laser scanning, applied to a three-dimensional reconstruction device, characterized in that: The three-dimensional reconstruction method comprises: Perform laser scanning on the object to be measured to obtain a light strip image; Extracting light stripes from the light stripe image based on a light stripe extraction algorithm; The object to be measured is three-dimensionally reconstructed based on the extracted light strips and calibration information to obtain a reconstruction result; wherein the calibration information is optimized calibration information obtained by the calibration method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: The method comprises a program which can be executed by a processor to implement the method according to any one of claims 1 to 10.

Images