Automatic calibration method and system based on line structured light measurement system, medium and equipment

Abstract

The invention relates to the field of vision measurement, and discloses an automatic calibration method, system, medium and equipment based on a line structured light measurement system.The automatic calibration method comprises the steps that internal parameters of a camera are determined, laser stripes emitted by laser emitters form light planes, and the laser emitters are adjusted to enable the laser stripes to intersect; two light plane coordinate systems with a space conversion relation are established, intersecting light planes form intersecting lines in a three-dimensional space, the checkerboard calibration plate is placed at the position of a third plane, and pixel coordinates of laser stripe center points are extracted; pixel coordinates of the stripe center point are converted into a camera coordinate system, specific space coordinates of the stripe point cloud on the calibration plate are obtained, and then the actual length of each section of the laser stripe is obtained; one-to-one corresponding space coordinates of the stripe point clouds in the light plane coordinate system and the camera coordinate system are obtained according to space conversion, an expression of a light plane in the light plane coordinate system in the camera coordinate system is obtained according to a conversion matrix between the two coordinate systems, and calibration is completed.

Description

Technical Field

[0001] The present invention relates to the field of visual measurement technology, and in particular to an automatic calibration method, system, medium and equipment based on a line structured light measurement system. Background Art

[0002] In machine vision measurement technology, 3D vision measurement can detect the 3D shape and posture of products more comprehensively than 2D vision measurement methods, overcoming the limitations of 2D measurement and becoming the current mainstream non-contact measurement technology. In the process of implementing 3D vision measurement, internal parameter calibration is required to ensure the effective realization of the measurement function. By solving the relevant parameters from 2D images to 3D space, an accurate mapping relationship between images and physical objects is established. The accuracy of parameter solution directly determines the accuracy of the measurement system.

[0003] To ensure the accuracy and reliability of the measurement, a variety of calibration methods have been proposed and applied, such as the wire drawing method, the toothed target method, the method based on cross ratio invariance, and the 2D (3D stereo) target method. Each method has its own unique advantages and disadvantages. For example, the wire drawing method is suitable for visible light camera sensors, but not for invisible structured light; the method based on cross ratio invariance is easy to operate, but requires unified calibration points, which makes the algorithm complex. Therefore, in practical applications, the most appropriate calibration method needs to be selected according to specific needs and conditions. The current line structured light 3D reconstruction technology generally requires multiple shots to complete the calibration of the entire system, and it is difficult to automate the calibration process. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide an automatic calibration method, system, medium and equipment based on a line structured light measurement system, which solves the calibration problems such as complex light plane calibration operation, easy interference in extracting feature points, and difficulty in calibrating multiple light planes.

[0005] To achieve the above-mentioned purpose, in the first aspect, the technical solution adopted by the present invention is: an automatic calibration method based on a line structured light measurement system, which includes: using a standard checkerboard calibration plate, using Zhang's calibration method to determine the internal parameters of the camera, and at the same time, the laser stripes emitted by each laser emitter form each light plane, and adjust the projection position of the laser emitter so that the laser stripes emitted by each laser emitter intersect; establish two light plane coordinate systems with a spatial transformation relationship for the light plane formed by the laser, the intersecting light planes form an intersection line L in three-dimensional space, and assume that there is a third plane below the intersection line L, place the checkerboard calibration plate at the third plane, capture the image of the laser stripes, and extract The pixel coordinates of the center point of the laser stripe; the external parameter matrix between the camera and the checkerboard calibration plate is obtained through the homography matrix and the internal parameters of the camera to convert the pixel coordinates of the center point of the stripe to the camera coordinate system, and the intersection of the light and the calibration plate plane is solved according to the inverse light path to obtain the specific spatial coordinates of the stripe point cloud on the calibration plate, and then the actual length of each segment of the laser stripe is obtained; according to the plane distance formula and space transformation, the one-to-one corresponding spatial coordinates of the stripe point cloud in the light plane coordinate system and the camera coordinate system are obtained at the same time, and the transformation matrix between the two coordinate systems is solved according to the corresponding relationship, and the expression of the light plane in the light plane coordinate system in the camera coordinate system is obtained from the transformation matrix to complete the calibration.

[0006] In a possible implementation, each light plane is formed by laser stripes emitted by each laser emitter, and the projection position of the laser emitter is adjusted so that the laser stripes emitted by each laser emitter intersect, including:

[0007] The lasers emitted by two laser emitters with the same fixed emission angle are vertically incident on the horizontal plane at the same height to form parallel laser stripes, and one of the laser emitters is rotated until the two laser stripes intersect;

[0008] Take the emission point as the origin O l , the angular bisector of the emission angle is taken as Z l Axis, select perpendicular to Z l The straight line of the axis is the X l Axis, then take Y l Axis perpendicular to X l Axis and Z l Axis, forming the optical plane coordinate system O l -X l Y l Z l , so that any laser stripe projected onto the plane is aligned with X l Y l Z l The two sides of the in-plane emission angle form a triangular light plane.

[0009] In one possible implementation, in the light plane coordinate system of the non-rotating laser emitter, an expression for the intersection line L and the two sides of the emission angle are obtained, keeping the projection position unchanged. At this time, if there is a third plane within the height range of the intersection line L, the laser stripes projected on the plane must intersect and the intersection point is located on the intersection line L, and the four endpoints of the two laser stripes must be on the two sides of the emission angle.

[0010] In one possible implementation, obtaining specific spatial coordinates of the fringe point cloud on the calibration plate includes:

[0011] In three-dimensional space, laser stripes are the intersections of two light planes and a third plane. There are five intersection points and endpoints of the stripes. If the length between each point is known, the specific coordinates can be solved according to the plane distance formula:

[0012] (X li -X li ) 2 +(Z li -Z lj ) 2 = l k

[0013] In the formula, (Xl i ,Z li ) and (X lj ,Z lj ) are the coordinates of the endpoints or intersection points, l k is the distance between two points, i, i, k = 1, 2,…, 5, i≠j.

[0014] In a possible implementation, according to the plane distance formula and the space conversion, the space coordinates of the fringe point cloud in the light plane coordinate system and the camera coordinate system are obtained one by one, including:

[0015] The coordinates of other stripe points between the intersection and endpoints of the stripes can be solved according to the stripe slope and distance formula, and then the point cloud distribution of each stripe on the same plane can be obtained;

[0016] According to the spatial transformation relationship between the two optical plane coordinate systems, the point cloud on the same stripe is transformed to its corresponding optical plane, and the spatial point cloud distribution of different laser stripes in the same optical plane coordinate system is obtained. At this time, each point of the stripe is distributed on the third plane.

[0017] In a possible implementation, the spatial transformation relationship between the two light plane coordinate systems is:

[0018]

[0019] Among them, the two light plane coordinate systems are light O l -X l Yl Z l and l -x l y l z l , let the coordinates of a point in one of the coordinate systems be (X l ,Y l ,Z l ), which is expressed in another coordinate system as (x l ,y l ,z l ), ε is the rotation angle of one of the rotating laser emitters, and t is the distance between the two parallel laser stripes when not rotating.

[0020] In a possible implementation, a transformation matrix between the two coordinate systems is solved according to the corresponding relationship, and an expression of the light plane in the light plane coordinate system in the camera coordinate system is obtained from the transformation matrix, including:

[0021] Let the transformation matrix between the two coordinate systems be M. For the one-to-one corresponding point cloud coordinate sets A and B, A = MB;

[0022] Calculate the center point μ of point cloud coordinate sets A and B respectively A , μ B , to establish the three-dimensional covariance matrix H;

[0023] The three-dimensional covariance matrix H is subjected to singular value decomposition to obtain three orthogonal matrices U, S and V. Therefore, according to the transformation matrix M, the X in the light plane coordinate system can be obtained. l O l Z l The expression of the face in the camera coordinate system.

[0024] In the second aspect, the technical solution adopted by the present invention is: an automatic calibration system based on a line structured light measurement system, which includes: an internal parameter and laser adjustment module, which adopts a standard checkerboard calibration plate, and uses Zhang's calibration method to determine the internal parameters of the camera. At the same time, the laser stripes emitted by each laser emitter form each light plane, and adjust the projection position of the laser emitter so that the laser stripes emitted by each laser emitter intersect; an intersection line setting module, which establishes two light plane coordinate systems with a spatial transformation relationship for the light plane formed by the laser, and the intersecting light planes form an intersection line L in three-dimensional space, and it is assumed that there is a third plane below the intersection line L, and the checkerboard calibration plate is placed on the third plane, and the image of the laser stripes is captured, and the laser stripes are extracted. The pixel coordinates of the center point of the laser stripe are obtained by the stripe point cloud acquisition module, which obtains the extrinsic parameter matrix between the camera and the checkerboard calibration plate through the homography matrix and the internal parameters of the camera, so as to convert the pixel coordinates of the center point of the stripe into the camera coordinate system, and solve the intersection of the light and the calibration plate plane according to the inverse light path to obtain the specific spatial coordinates of the stripe point cloud on the calibration plate, and then obtain the actual length of each segment of the laser stripe; the extrinsic parameter and light plane calibration module, according to the plane distance formula and space transformation, simultaneously obtains the one-to-one corresponding spatial coordinates of the stripe point cloud in the light plane coordinate system and the camera coordinate system, solves the transformation matrix between the two coordinate systems according to the corresponding relationship, and obtains the expression of the light plane in the light plane coordinate system in the camera coordinate system from the transformation matrix to complete the calibration.

[0025] In a third aspect, the technical solution adopted by the present invention is: a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes any one of the above methods.

[0026] In a fourth aspect, the technical solution adopted by the present invention is: a computing device, comprising: one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the above methods.

[0027] The present invention adopts the above technical solution, which has the following advantages:

[0028] 1. The present invention uses image processing technology and mathematical algorithms to accurately detect feature points, calculate light plane parameters to improve accuracy, and enhance the accuracy of the entire measurement system.

[0029] 2. The present invention can quickly acquire all point cloud data through only one double (multiple) chessboard image, realize rapid calibration, reduce time and labor costs and improve the overall efficiency of the measurement system.

[0030] 3. The present invention utilizes laser measurement, which does not require direct contact with the object to be measured, thus avoiding the risk of error and damage. It is suitable for surfaces of complex shapes and has a wide range of applications.

[0031] 4. The present invention is easy to implement, has clear steps and processes, is easy to operate, does not require complex equipment and professional skills, and is easy to promote in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 1 is a schematic diagram of the overall process of an automatic calibration method based on a line structured light measurement system in an embodiment of the present invention;

[0033] Figure 2 is a schematic diagram of the relationship between the light plane and the lens imaging in an embodiment of the present invention;

[0034] Figure 3 The non-rotating laser emitter in the embodiment of the present invention constitutes the light plane space coordinate system O l -X l O l Z l Schematic diagram of

[0035] Figure 4 is a schematic diagram of the position of the intersection line L formed by the intersection of laser stripes in three-dimensional space in the light plane of the non-rotating laser emitter in an embodiment of the present invention;

[0036] Figure 5 It is the length constraint of the two laser stripes by the third plane within the intersection height range in the embodiment of the present invention;

[0037] Figure 6 It is the imaging of the intersecting laser stripes in the checkerboard calibration plate in the embodiment of the present invention;

[0038] Figure 7 This is an automatic calibration image of the production line in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] With the rapid development of science and technology, machine vision is widely used in industrial automation, intelligent manufacturing, biomedicine and other fields, playing a key role. Its non-contact, high-efficiency, low-cost and highly automated measurement method effectively solves the core measurement problem of deformable materials. Compared with traditional manual inspection, machine vision avoids inefficiency, subjective errors and omissions, and gradually replaces manual operations.

[0040] As an important 3D visual measurement method, structured light measurement technology has important applications in the fields of industry and science and technology. Its basic principle is to project a light source with a known structure onto the surface of the target object, and then use a camera to capture the projected image to derive the 3D shape and depth information of the target object surface. Structured light measurement technology provides a reliable solution for high-precision and high-efficiency 3D shape reconstruction and analysis.

[0041] Therefore, the present invention proposes an automatic calibration method, system, medium and equipment based on a line structured light measurement system, which integrates the calibration process of camera external parameters and light plane, and realizes automatic calibration of the entire line structured light measurement system in the production line to meet the calibration requirements of different types and application scenarios, improve the accuracy and efficiency of calibration, expand its scope of application, and further promote the application and development of machine vision in various fields.

[0042] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0044] In one embodiment of the present invention, an automatic calibration method based on a line structured light measurement system is provided. In this embodiment, the line structured light measurement system is composed of an industrial camera and two or more laser emitters. The camera collects deformed laser stripe images projected on the surface of the object to be measured, and then processes the stripe images to achieve three-dimensional measurement of the object to be measured.

[0045] like Figure 1 As shown, the automatic calibration method includes the following steps:

[0046] 1) To obtain the parameters of line structured light, a standard checkerboard calibration plate is used, and the Zhang calibration method is used to determine the internal parameters of the camera. At the same time, the laser stripes emitted by each laser emitter form each light plane, and the projection position of the laser emitter is adjusted so that the laser stripes emitted by each laser emitter intersect, such as Figure 2 As shown;

[0047] 2) Establish two light plane coordinate systems with spatial transformation relationship for the light plane formed by the laser (such as Figure 3 As shown in Figure 2), the intersecting light planes form an intersection line L in three-dimensional space, and assuming that there is a third plane below the intersection line L, the checkerboard calibration plate is placed on the third plane, and the image of the laser stripes is captured (as shown in Figure 2). Figure 6 As shown), extract the pixel coordinates of the center point of the laser stripe;

[0048] 3) The external parameter matrix between the camera and the checkerboard calibration plate is obtained through the homography matrix and the internal parameters of the camera, so as to transform the pixel coordinates of the center point of the stripe into the camera coordinate system, solve the intersection of the light and the calibration plate plane according to the inverse light path, obtain the specific spatial coordinates of the stripe point cloud on the calibration plate, and then obtain the actual length of each segment of the laser stripe;

[0049] 4) According to the plane distance formula and space transformation, the spatial coordinates of the stripe point cloud in the light plane coordinate system and the camera coordinate system are obtained one by one. The transformation matrix between the two coordinate systems is solved according to the corresponding relationship. The expression of the light plane in the light plane coordinate system in the camera coordinate system is obtained from the transformation matrix to complete the calibration.

[0050] In the above step 1), the laser stripes emitted by each laser emitter form each light plane, and the projection position of the laser emitter is adjusted so that the laser stripes emitted by each laser emitter intersect, which includes the following steps:

[0051] 1.1) Two laser emitters with the same fixed emission angle are vertically incident on a horizontal plane at the same height to form parallel laser stripes. The distance between the two parallel laser stripes is represented by t. One of the laser emitters is rotated until the two laser stripes intersect; wherein the rotation angle ε can be arbitrarily selected.

[0052] 1.2) Establish a coordinate system O for the light plane formed by the above laser l -X l Y l Z l and l -x l y l z l ; Take the launch point as the origin O l , the angular bisector of the emission angle is taken as Z l Axis, select perpendicular to Z l The straight line of the axis is the X l Axis, then take Y l Axis perpendicular to X l Axis and Z l Axis, forming the optical plane coordinate system O l -X l Y l Z l, so that any laser stripe projected onto the plane is aligned with X l Y l Z l The two sides of the in-plane emission angle form a triangular light plane.

[0053] Among them, in the light plane coordinate system of the non-rotating laser emitter, the expression of the intersection line L and the expressions of the two sides of the emission angle are obtained; keeping the projection position unchanged, if there is a third plane within the height range of the intersection line L, the laser stripes projected on the plane must intersect and the intersection point is located on the intersection line L, and the four endpoints of the two laser stripes must be on the two sides of the emission angle.

[0054] When the laser stripes intersect, the plane triangle projected by the laser emitter forms an intersection line L in three-dimensional space, as shown in Figure 4 As shown, the intersection of the laser stripes is at the end of the intersection line. For a non-rotating laser transmitter, the intersection line is at X l Y l Z l The expression of the surface is:

[0055]

[0056] At the same time, expressions are established for the two sides of the emission angle:

[0057]

[0058] Where α is the laser emission angle.

[0059] In the above step 2), the checkerboard calibration plate is placed on the third plane, the image of the laser stripes is captured, and the pixel coordinates of the center point of the laser stripes are extracted. Specifically, the image is corrected according to the distortion parameters to restore it to the image under the linear imaging model, and the pixel coordinates of the center point of the laser stripes are extracted.

[0060] In this embodiment, in order to ensure the accuracy of the backlight projection of the fringe center in the camera coordinate system, the actual calibration process generally uses maximum likelihood estimation to solve the camera distortion parameters and optimize the imaging model to improve the calibration accuracy of the camera. Assume that n checkerboard images are taken, each image has m feature points, the pixel coordinates corresponding to the feature points in the three-dimensional space are x, and the pixel coordinates after the intrinsic and extrinsic matrix transformation are x′, and considering the nonlinearity of the distortion parameters in the calibration process, combined with the radial distortion parameter k 1 With k 2 , solve the maximum likelihood estimation problem by minimizing the positions of x and x′:

[0061]

[0062] The Levenberg-Marquardt (LM) algorithm is used for iteration, k 1 With k 2 The initial value of is set to 0, and a stable approximation is obtained after convergence.

[0063] In the above step 3), the specific spatial coordinates of the stripe point cloud on the calibration plate are obtained, specifically: in three-dimensional space, the laser stripe is the intersection of two light planes and a third plane, and the intersection and endpoints of the stripes are five (such as Figure 5 As shown), if the length between each point is known, the specific coordinates can be solved according to the plane distance formula:

[0064] (X li -X lj ) 2 +(Z li -Z lj ) 2 = l k

[0065] In the formula, (X li ,Z li ) and (X lj ,Z lj ) are the coordinates of the endpoints or intersection points, l k is the distance between two points, i, j, k = 1, 2,…, 5, i≠j.

[0066] In the above step 3), the external parameter matrix between the camera and the checkerboard calibration plate can be immediately obtained through the homography matrix and the camera internal parameters. The center point of the laser stripe and the origin of the camera coordinate system form a set of vectors, and the intersection with the third plane (i.e., the checkerboard plane) is the actual position of the laser stripe on the checkerboard. Solve the actual three-dimensional coordinates and length relationship of this set of vectors in the camera coordinate system.

[0067] In the above step 4), according to the plane distance formula and space conversion, the spatial coordinates of the fringe point cloud in the light plane coordinate system and the camera coordinate system are obtained one by one, including the following steps:

[0068] 4.1.1) The coordinates of other fringe points between the intersection and endpoints of the fringe can be solved according to the fringe slope and distance formula, and then the point cloud distribution of each fringe on the same plane can be obtained;

[0069] 4.1.2) According to the spatial transformation relationship between the two optical plane coordinate systems, the point cloud on the same stripe is transformed to its corresponding optical plane, and the spatial point cloud distribution of different laser stripes in the same optical plane coordinate system is obtained. At this time, the points of the stripes are distributed on the third plane.

[0070] In this embodiment, the spatial transformation relationship between the two light plane coordinate systems is:

[0071]

[0072] Among them, the two light plane coordinate systems are light O l -X l Y l Z l and l -x l y l z l , let the coordinates of a point in one of the coordinate systems be (X l ,Y l ,Z l ), which is expressed in another coordinate system as (x l ,y l ,z l ), ε is the rotation angle of one of the rotating laser emitters, and t is the distance between the two parallel laser stripes when not rotating.

[0073] In the above step 4), the transformation matrix between the two coordinate systems is solved according to the corresponding relationship, and the expression of the light plane in the light plane coordinate system in the camera coordinate system is obtained from the transformation matrix, including the following steps:

[0074] 4.2.1) Let the transformation matrix between the two coordinate systems be M. For the one-to-one corresponding point cloud coordinate sets A and B, A = MB;

[0075] The laser stripe center point cloud coordinates are transformed by using the transformation matrix M to obtain the three-dimensional point coordinates of the center points of the two laser stripes in the same light plane coordinate system. This set of center point cloud coordinates corresponds one-to-one to the center point cloud coordinate data in the camera coordinate system.

[0076] 4.2.2) Calculate the center point μ of the point cloud coordinate sets A and B respectively A , μ B , to establish the three-dimensional covariance matrix H;

[0077] in:

[0078]

[0079] Where N is the number of coordinate points, is the coordinate value.

[0080] Then the three-dimensional covariance matrix H is:

[0081]

[0082] 4.2.3) The three-dimensional covariance matrix H is subjected to singular value decomposition to obtain three orthogonal matrices U, S and V. Therefore, according to the transformation matrix M, the X in the light plane coordinate system can be obtained l O lZ l The expression of the face in the camera coordinate system.

[0083] in:

[0084]

[0085] In the formula, R l and t l are the rotation matrix and translation vector in the transformation matrix M.

[0086] X in the light plane coordinate system l O l Z l The expression of the surface in the camera coordinate system is:

[0087]

[0088] Determine the plane equation parameter A of the calibration plate in the camera coordinate system 1 , B 1 , C 1 , D 1 , A 2 , B 2 , C 3 , D 4 , thus completing the calibration of the light plane by dual (multi) laser stripes.

[0089] In the above embodiments, the line structured light measurement system is composed of only an industrial camera and two or more laser emitters, which greatly reduces the system debugging time and cost.

[0090] In the above embodiments, the size of the standard checkerboard calibration plate is arbitrary, which is convenient for users to customize and adjust according to actual needs.

[0091] In the above embodiments, all point cloud data can be obtained through only one double (multi) chessboard image, and the process is fully automated.

[0092] In the above embodiments, by solving the actual three-dimensional coordinates and length relationship of the above vectors in the camera coordinate system, the actual three-dimensional coordinates and length relationship of the laser stripes in the camera coordinate system can be obtained.

[0093] In the embodiment, the line structured light measurement system is basically composed of an industrial camera and two or more laser emitters. The automatic calibration method in this embodiment includes the following steps:

[0094] 1) A 9*12 standard alumina checkerboard calibration plate with a standard length of 20 mm per grid is selected, and the Zhang calibration method is used to complete the intrinsic calibration of the camera used.

[0095] 2) Take 15 laser-free images of the calibration plate in different poses for feature extraction. According to the 15 groups of 2D image pixel coordinates of key points on the calibration plate and the corresponding 3D coordinate points in the world coordinate system, the camera intrinsic parameter matrix M can be obtained. 1 , camera distortion coefficient k 1 and k 2 :

[0096]

[0097] k 1 =-0.0263

[0098] k 2 =0.0886

[0099] 3) Turn on two laser transmitters with an emission angle of α=45°, and the laser lines are incident vertically on the horizontal plane at the same height to form laser stripes, and the relative positions of the laser stripes are adjusted.

[0100] 4) Align the transmitting head of the above laser transmitter to ensure that the laser stripes are in the camera field of view. First, make the two laser stripes parallel and the translation distance is t=40mm.

[0101] 5) Rotate one of the laser emitters at an angle of ε until the two laser stripes intersect. Then make the laser incident vertically on the horizontal plane, at which point the angle formed by the stripes is equal to the angle between the two light planes.

[0102] 6) When the above angle is adjusted to ε=28°, the two light plane coordinate systems have a spatial transformation relationship:

[0103]

[0104] 7) Completely tighten the above-mentioned laser transmitter and industrial camera to ensure that their relative position relationship does not change during the movement of the mobile platform of the production line, thus completing the debugging preparation process of the experiment.

[0105] 8) Start the calibration of the line structured light to the light plane. Place the calibration plate in the laser stripe area, adjust the camera exposure time to ensure that the stripe image is normally exposed, and take an image.

[0106] 9) Perform distortion correction on the pixel coordinates in the above image to obtain corrected pixel coordinates.

[0107] 10) According to the fitting of multiple three-dimensional coordinate points, the plane equation of the calibration plate in the camera coordinate system is obtained:

[0108]

[0109] And determine the plane equation parameter A of the calibration plate in the camera coordinate system 1, B 1 , C 1 , D 1 , A 2 , B 2 , C 3 , D 4 , thus completing the calibration of line structured light to the light plane.

[0110] The expression of the two light planes in the camera coordinate system obtained in this embodiment is:

[0111]

[0112] From the expressions of the above two planes, it can be seen that the angle is about 28.29°, which is close to the measured value prepared for debugging. Therefore, the optical plane accuracy obtained using the standard calibration plate is relatively high, and subsequent measurement experiments can be carried out.

[0113] The embodiment of the present invention implements an automatic calibration method based on a line structured light measurement system in the case of a checkerboard calibration plate with a standard length of each grid of 20 mm. The automatic calibration method based on a line structured light measurement system is the same for calibration plates of other specifications.

[0114] In summary, the present invention determines all the unknown parameters of the system according to the positional relationship between the camera imaging model and the light plane and the measurement principle; realizes the calibration of the internal parameters of the camera based on Zhang's calibration method, and clarifies that when the internal parameters of the camera remain unchanged, the external parameters can be directly determined; and based on the calibration principle of the internal and external parameters of the camera, the geometric relationship between the stripes in the light plane and the calibration plate is utilized to integrate the camera extrinsic parameters and the light plane calibration process together, thereby realizing the automatic calibration of the entire line structured light measurement system in the production line.

[0115] The debugging process of the present invention in the industrial production line is as follows: after the internal parameters of the camera are determined, the plane calibration plate used for external parameter calibration can be simplified and attached to the mobile platform of the industrial production line. The laser transmitter and the industrial camera are turned on to ensure that the intersecting laser stripes are in the camera field of view. When the calibration pattern moves to the intersecting field of view, it is directly photographed. The image obtained is as follows: Figure 7 As shown, the entire line structured light non-contact measurement system can be automatically calibrated through calculation. Therefore, the automatic calibration method in this paper greatly simplifies the debugging operation of the system and greatly improves the system versatility and the degree of automation of the production line.

[0116] Furthermore, the automatic calibration method of the present invention is based on the line structured light measurement system, has no requirements on the installation position between the laser and the camera, and is applicable to the calibration of the light plane by any form of structured light; the acquisition method of the light plane calibration image is simple, the process is simple, and it is easy to operate; and the data of multiple groups of calibration plate feature points are used to improve the accuracy of the calibration result.

[0117] In one embodiment of the present invention, an automatic calibration system based on a line structured light measurement system is provided, comprising:

[0118] The internal reference and laser adjustment module uses a standard checkerboard calibration plate and Zhang's calibration method to determine the internal parameters of the camera. At the same time, the laser stripes emitted by each laser emitter form each light plane, and the projection position of the laser emitter is adjusted so that the laser stripes emitted by each laser emitter intersect;

[0119] The intersection line setting module establishes two light plane coordinate systems with a spatial transformation relationship for the light plane formed by the laser. The intersecting light planes form an intersection line L in three-dimensional space. It is assumed that there is a third plane below the intersection line L. The checkerboard calibration plate is placed on the third plane, and the image of the laser stripes is captured to extract the pixel coordinates of the center point of the laser stripes.

[0120] The stripe point cloud acquisition module obtains the external parameter matrix between the camera and the checkerboard calibration plate through the homography matrix and the internal parameters of the camera, so as to convert the pixel coordinates of the stripe center point into the camera coordinate system, solve the intersection of the light and the calibration plate plane according to the inverse light path, obtain the specific spatial coordinates of the stripe point cloud on the calibration plate, and then obtain the actual length of each segment of the laser stripe;

[0121] The external parameter and light plane calibration module obtains the spatial coordinates of the fringe point cloud in the light plane coordinate system and the camera coordinate system one by one according to the plane distance formula and space transformation, solves the transformation matrix between the two coordinate systems according to the corresponding relationship, and obtains the expression of the light plane in the light plane coordinate system in the camera coordinate system from the transformation matrix to complete the calibration.

[0122] In the above embodiment, the laser stripes emitted by each laser emitter form each light plane, and the projection position of the laser emitter is adjusted so that the laser stripes emitted by each laser emitter intersect, including:

[0123] The lasers emitted by two laser emitters with the same fixed emission angle are vertically incident on the horizontal plane at the same height to form parallel laser stripes, and one of the laser emitters is rotated until the two laser stripes intersect;

[0124] Take the emission point as the origin O l , the angular bisector of the emission angle is taken as Z l Axis, select perpendicular to Z l The straight line of the axis is the X l Axis, then take Y l Axis perpendicular to X l Axis and Z l Axis, forming the optical plane coordinate system O l -X l Y l Z l , so that any laser stripe projected onto the plane is aligned with Xl Y l Z l The two sides of the in-plane emission angle form a triangular light plane.

[0125] In this embodiment, in the light plane coordinate system of the non-rotating laser emitter, the expression of the intersection line L and the expressions of the two sides of the emission angle are obtained, and the projection position is kept unchanged. At this time, if there is a third plane within the height range of the intersection line L, the laser stripes projected on the plane must intersect and the intersection point is located on the intersection line L, and the four endpoints of the two laser stripes must be on the two sides of the emission angle.

[0126] In the above embodiment, obtaining the specific spatial coordinates of the fringe point cloud on the calibration plate includes:

[0127] In three-dimensional space, laser stripes are the intersections of two light planes and a third plane. There are five intersection points and endpoints of the stripes. If the length between each point is known, the specific coordinates can be solved according to the plane distance formula:

[0128] (X li -X lj ) 2 +(Z li -Z lj ) 2 = l k

[0129] In the formula, (X li ,Z li ) and (X lj ,Z lj ) are the coordinates of the endpoints or intersection points, l k is the distance between two points, i, j, k = 1, 2,…, 5, i≠j.

[0130] In this embodiment, according to the plane distance formula and space conversion, the spatial coordinates of the fringe point cloud in the light plane coordinate system and the camera coordinate system are obtained one by one, including:

[0131] The coordinates of other stripe points between the intersection and endpoints of the stripes can be solved according to the stripe slope and distance formula, and then the point cloud distribution of each stripe on the same plane can be obtained;

[0132] According to the spatial transformation relationship between the two optical plane coordinate systems, the point cloud on the same stripe is transformed to its corresponding optical plane, and the spatial point cloud distribution of different laser stripes in the same optical plane coordinate system is obtained. At this time, each point of the stripe is distributed on the third plane.

[0133] In this embodiment, the spatial transformation relationship between the two light plane coordinate systems is:

[0134]

[0135] Among them, the two light plane coordinate systems are light O l -X l Y l Z l and l -x l y l z l , let the coordinates of a point in one of the coordinate systems be (X l ,Y l ,Z l ), which is expressed in another coordinate system as (x l ,y l ,z l ), ε is the rotation angle of one of the rotating laser emitters, and t is the distance between the two parallel laser stripes when not rotating.

[0136] In the above embodiment, the transformation matrix between the two coordinate systems is solved according to the corresponding relationship, and the expression of the light plane in the light plane coordinate system in the camera coordinate system is obtained from the transformation matrix, including:

[0137] Let the transformation matrix between the two coordinate systems be M. For the one-to-one corresponding point cloud coordinate sets A and B, A = MB;

[0138] Calculate the center point μ of point cloud coordinate sets A and B respectively A , μ B , to establish the three-dimensional covariance matrix H;

[0139] The three-dimensional covariance matrix H is subjected to singular value decomposition to obtain three orthogonal matrices U, S and V. Therefore, according to the transformation matrix M, the X in the light plane coordinate system can be obtained. l O l Z l The expression of the face in the camera coordinate system.

[0140] The system provided in this embodiment is used to execute the above-mentioned method embodiments. Please refer to the above-mentioned embodiments for specific processes and detailed contents, which will not be repeated here.

[0141] In one embodiment of the present invention, a computing device is provided, which may be a terminal, and may include: a processor, a communication interface, a memory, a display screen, and an input device. Among them, the processor, the communication interface, and the memory communicate with each other through a communication bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory, and the non-volatile storage medium stores an operating system and a computer program. When the computer program is executed by the processor, the method in each of the above embodiments is implemented; the internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a management network, NFC (near field communication) or other technologies. The display screen may be a liquid crystal display screen or an electronic ink display screen, and the input device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computing device, or an external keyboard, touchpad or mouse, etc. The processor may call the logic instructions in the memory.

[0142] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0143] In one embodiment of the present invention, a computer program product is provided, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments.

[0144] In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores server instructions, and the computer instructions enable a computer to execute the methods provided in the above embodiments.

[0145] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0146] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0147] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic calibration method based on a line structured light measurement system, characterized in that: include: A standard checkerboard calibration plate is used to determine the internal parameters of the camera using Zhang's calibration method. At the same time, the laser stripes emitted by each laser emitter form each light plane, and the projection position of the laser emitter is adjusted so that the laser stripes emitted by each laser emitter intersect; Two light plane coordinate systems with a spatial transformation relationship are established for the light plane formed by the laser. The intersecting light planes form an intersection line L in three-dimensional space. It is assumed that there is a third plane below the intersection line L. The checkerboard calibration plate is placed on the third plane, and the image of the laser stripes is captured to extract the pixel coordinates of the center point of the laser stripes. The external parameter matrix between the camera and the checkerboard calibration plate is obtained through the homography matrix and the internal parameters of the camera, so as to transform the pixel coordinates of the center point of the stripe into the camera coordinate system, and solve the intersection of the light and the calibration plate plane according to the inverse light path to obtain the specific spatial coordinates of the stripe point cloud on the calibration plate, and then obtain the actual length of each segment of the laser stripe; According to the plane distance formula and space transformation, the spatial coordinates of the fringe point cloud in the light plane coordinate system and the camera coordinate system are obtained one by one. The transformation matrix between the two coordinate systems is solved according to the corresponding relationship, and the expression of the light plane in the light plane coordinate system in the camera coordinate system is obtained from the transformation matrix to complete the calibration.

2. The automatic calibration method based on the line structured light measurement system according to claim 1, characterized in that: The laser stripes emitted by each laser emitter form each light plane, and the projection position of the laser emitter is adjusted so that the laser stripes emitted by each laser emitter intersect, including: The lasers emitted by two laser emitters with the same fixed emission angle are vertically incident on the horizontal plane at the same height to form parallel laser stripes, and one of the laser emitters is rotated until the two laser stripes intersect; Take the emission point as the origin O l , the angular bisector of the emission angle is taken as Z l Axis, select perpendicular to Z l The straight line of the axis is the X l Axis, then take Y l Axis perpendicular to X l Axis and Z l Axis, forming the optical plane coordinate system O l -X l Y l Z l , so that any laser stripe projected onto the plane is aligned with X l Y l Z l The two sides of the in-plane emission angle form a triangular light plane.

3. The automatic calibration method based on the line structured light measurement system according to claim 2, characterized in that: In the light plane coordinate system of the non-rotating laser emitter, the expression of the intersection line L and the expressions of the two sides of the emission angle are obtained, keeping the projection position unchanged. At this time, if there is a third plane within the height range of the intersection line L, the laser stripes projected on this plane must intersect and the intersection point is located on the intersection line L, and the four endpoints of the two laser stripes must be on the two sides of the emission angle.

4. The automatic calibration method based on the line structured light measurement system according to claim 1, characterized in that: Obtain the specific spatial coordinates of the fringe point cloud on the calibration plate, including: In three-dimensional space, laser stripes are the intersections of two light planes and a third plane. There are five intersection points and endpoints of the stripes. If the length between each point is known, the specific coordinates can be solved according to the plane distance formula: (X li -X lj ) 2 +(Z li -Z lj ) 2 =l k In the formula, (X li ,Z li ) and (X lj ,Z lj ) are the coordinates of the endpoints or intersection points, l k is the distance between two points, i, j, k = 1, 2,…, 5, i≠j.

5. The automatic calibration method based on the line structured light measurement system according to claim 4, characterized in that: According to the plane distance formula and space conversion, the spatial coordinates of the fringe point cloud in the light plane coordinate system and the camera coordinate system are obtained one by one, including: The coordinates of other stripe points between the intersection and endpoints of the stripes can be solved according to the stripe slope and distance formula, and then the point cloud distribution of each stripe on the same plane can be obtained; According to the spatial transformation relationship between the two optical plane coordinate systems, the point cloud on the same stripe is transformed to its corresponding optical plane, and the spatial point cloud distribution of different laser stripes in the same optical plane coordinate system is obtained. At this time, each point of the stripe is distributed on the third plane.

6. The automatic calibration method based on the line structured light measurement system according to claim 4, characterized in that: The spatial transformation relationship between the two light plane coordinate systems is: Among them, the two light plane coordinate systems are light O l -X l Y l Z l and l -x l y l z l , let the coordinates of a point in one of the coordinate systems be (X l ,Y l ,Z l ), which is expressed in another coordinate system as (x l ,y l ,z l ), ε is the rotation angle of one of the rotating laser emitters, and t is the distance between the two parallel laser stripes when not rotating.

7. The automatic calibration method based on the line structured light measurement system according to claim 1, characterized in that: According to the corresponding relationship, the transformation matrix between the two coordinate systems is solved, and the expression of the light plane in the light plane coordinate system in the camera coordinate system is obtained from the transformation matrix, including: Let the transformation matrix between the two coordinate systems be M. For the one-to-one corresponding point cloud coordinate sets A and B, A = MB; Calculate the center point μ of point cloud coordinate sets A and B respectively A , μ B , to establish the three-dimensional covariance matrix H; The three-dimensional covariance matrix H is subjected to singular value decomposition to obtain three orthogonal matrices U, S and V. Therefore, according to the transformation matrix M, the X in the light plane coordinate system can be obtained. l O l Z l The expression of the face in the camera coordinate system.

8. An automatic calibration system based on a line structured light measurement system, characterized in that: include: The internal reference and laser adjustment module uses a standard checkerboard calibration plate and Zhang's calibration method to determine the internal parameters of the camera. At the same time, the laser stripes emitted by each laser emitter form each light plane, and the projection position of the laser emitter is adjusted so that the laser stripes emitted by each laser emitter intersect; The intersection line setting module establishes two light plane coordinate systems with a spatial transformation relationship for the light plane formed by the laser. The intersecting light planes form an intersection line L in three-dimensional space. It is assumed that there is a third plane below the intersection line L. The checkerboard calibration plate is placed on the third plane, and the image of the laser stripes is captured to extract the pixel coordinates of the center point of the laser stripes. The stripe point cloud acquisition module obtains the external parameter matrix between the camera and the checkerboard calibration plate through the homography matrix and the internal parameters of the camera, so as to convert the pixel coordinates of the stripe center point into the camera coordinate system, solve the intersection of the light and the calibration plate plane according to the inverse light path, obtain the specific spatial coordinates of the stripe point cloud on the calibration plate, and then obtain the actual length of each segment of the laser stripe; The external parameter and light plane calibration module obtains the spatial coordinates of the fringe point cloud in the light plane coordinate system and the camera coordinate system one by one according to the plane distance formula and space transformation, solves the transformation matrix between the two coordinate systems according to the corresponding relationship, and obtains the expression of the light plane in the light plane coordinate system in the camera coordinate system from the transformation matrix to complete the calibration.

9. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any one of the methods of claims 1 to 7.

10. A computing device, characterized in that include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods described in claims 1 to 7.

Images