A method for light plane calibration for multi-line laser three-dimensional measurement system

Abstract

This invention discloses a method for optical plane calibration of a multi-line laser 3D measurement system. The method includes acquiring multiple sets of checkerboard calibration images at different locations, capturing clear checkerboard images under normal exposure, capturing laser stripe images of various orders falling on the checkerboard plane under low exposure parameters, and calibrating the camera's intrinsic and extrinsic parameter matrices using the set of clear checkerboard images acquired under normal exposure. The method generates line structured light through grating diffraction, adjusts the aperture width to select the working laser stripe diffraction order, redistributes laser stripe brightness using a phase plate, and achieves system calibration using the diffraction equation of the multi-line laser. This improves the calibration accuracy of the optical plane of the multi-line laser system, thereby enhancing measurement accuracy and efficiency. The diffraction grating generates multiple diffracted laser lines on the object, allowing simultaneous acquisition of surface 3D information at multiple line positions on the object's surface. Furthermore, the diffraction order can be selected based on the object's surface shape.

Description

Technical Field

[0001] This invention relates to the field of optical measurement, and more specifically, to a method for optical plane calibration in a multi-line laser three-dimensional measurement system. Background Technology

[0002] Line structured light 3D measurement based on the laser triangulation principle offers high-precision measurement advantages. Improving measurement efficiency has become a key research focus when performing structured light scanning measurements. To achieve rapid measurement, multi-line structured light 3D measurement is the most frequently studied approach, aiming to improve measurement efficiency while maintaining accuracy.

[0003] Researchers have proposed a three-dimensional measurement technique that uses a semiconductor laser module to simultaneously project multiple light planes onto the surface of the object under test, and have provided a mathematical model and related algorithms for multi-line structured light system measurement. In this method, the sensor can be installed in a relatively random location, making system installation and adjustment very easy and flexible. By calibrating each light plane independently, it can be considered as multiple line lasers working independently. Some scholars have proposed a new non-contact measurement method for multi-structured linear illumination (or multi-light scalpel) based on laser scanning measurement technology. To improve the accuracy of the developed multi-light scalpel measurement device, virtual network mapping and least squares methods are used to calibrate the device across the entire measurement field, calibrating each line separately. However, this method is not very accurate and the calibration process is complex. Other researchers have modified existing calibration plates and used the Random Sample Consensus Algorithm (RANSAC) to transform the centerline of each stripe into a three-dimensional point cloud fitted to a plane. This reduces the relative error of the overall fitting result of the multi-line structured light plane and improves the overall calibration accuracy of the light plane from the perspective of angular errors between calibration light planes. Similarly, this method still calibrates the optical plane independently, which is more prone to errors. In addition to calibrating the system through the optical plane, some researchers have proposed measurement models for multi-parallel laser systems, introducing the concept of multiple base planes to derive the measurement system formulas, thereby simplifying the calibration process; however, this type of calibration method based on the system's mathematical model depends on the system's installation accuracy.

[0004] Current calibration methods for multi-line laser planes either calibrate each laser plane independently or are based on the system's mathematical model. These methods often suffer from reduced calibration accuracy due to optical path or mechanical errors. Gratings, as precision optical components, can generate various structured light beams to illuminate objects and utilize the generated fine structured light distribution to achieve 3D reconstruction. Calibration methods based on grating diffraction constraints can achieve higher measurement accuracy. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a method for optical plane calibration in a multi-line laser three-dimensional measurement system. This method generates line structured light through grating diffraction, adjusts the aperture width to select the diffraction order of the working laser stripes, redistributes the brightness of the laser stripes using a phase plate, and achieves system calibration by constraining the diffraction equation of the multi-line laser. This improves the calibration accuracy of the optical plane in the multi-line laser system, thereby enhancing measurement accuracy and efficiency.

[0006] To achieve the above objectives, the technical method employed in this invention is as follows:

[0007] This invention discloses a method for optical plane calibration in a multi-line laser three-dimensional measurement system, comprising:

[0008] Acquire multiple sets of checkerboard calibration images at different locations, capture clear checkerboard images under normal exposure, and capture images of laser stripes of various orders falling on the checkerboard plane under low exposure parameters;

[0009] The camera's intrinsic and extrinsic parameter matrices were calibrated by acquiring a set of clear checkerboard images under normal exposure.

[0010] Calculate the plane equation of the camera coordinate system for the corresponding position of the checkerboard plane using the camera's extrinsic parameter matrix;

[0011] Images of laser stripes of various grades falling on the checkerboard plane were acquired under the aforementioned low exposure parameters, and the center line extraction algorithm was used to calculate the pixel coordinates of the center line of each grade of laser stripe.

[0012] The homogeneous coordinates of the laser stripe centerline in the camera coordinate system are calculated using the camera intrinsic parameter matrix and the pixel coordinates of the laser stripe centerlines of each order.

[0013] By using the extrinsic parameters of each camera position, the checkerboard plane equation in the world coordinate system at the corresponding position is transformed into the plane equation in the camera coordinate system.

[0014] By combining the homogeneous coordinates of the laser stripe centerline camera with the plane equations of the checkerboard in the camera coordinate system, the homogeneous coordinates of the laser stripe centerline camera are transformed into three-dimensional coordinates in the camera coordinate system of the laser stripe centerline.

[0015] Determine the order of the diffraction laser line corresponding to each laser stripe in the laser stripe image falling on the checkerboard plane acquired under low exposure parameters, and substitute the camera coordinates of the center line of the corresponding diffraction laser stripe into the laser stripe light plane equation of the corresponding order to obtain the initial value of the coefficients of the light plane equation of each diffraction laser stripe.

[0016] The squared distance from the center point of the laser fringe to the equation of the optical plane is selected as the objective function, and the diffraction angle relationship of the grating equation is used as the constraint. The normal vector (a) of each plane is obtained through a constrained least squares optimization algorithm.i b i c i The optimized value of )

[0017] Based on the fact that each diffracted light plane intersects a straight line in space, the position parameters of each light plane are optimized by expressing the spatial straight line equation and substituting the normal vector optimization value of each light plane.

[0018] As a further improvement, the acquisition of multiple sets of chessboard grid calibration images at different locations described in this invention specifically includes:

[0019] The camera and line laser remain in the same position, while the checkerboard is moved. When the checkerboard is in the same position, two calibration images are captured: a clear photo of the checkerboard is captured under normal camera exposure, and an image of the laser stripes falling on the checkerboard plane is captured under low camera exposure parameters. After capturing the two images, the checkerboard is moved to capture multiple sets of images.

[0020] As a further improvement, the present invention substitutes the camera coordinates of the center line of the corresponding diffraction laser fringes into the equation of the light plane where each laser fringe is located, specifically as follows:

[0021] The camera coordinates of the laser fringe centerline on the optical plane πi corresponding to the i-th order diffraction fringe are:

[0022] (x ik y ik , z ik ), i=0, ±1,...±n, k=1, 2,...m;

[0023] In the above formula, m is the number of center points on each order of laser stripes. If the normal vector of the light plane is (a i b i c i The average value of the planar point cloud in each direction is (x) i0 y i0 , z i0 The plane equation can be expressed as:

[0024] a i (xx i0 )+b i (yy i0 )+c i (zz i0 ) = 0;

[0025] Coordinates of all laser stripe centerline points (x ik y ik , z ik Substituting into the above formula and expressing it in matrix form, we get:

[0026]

[0027] The normal vectors of each plane (a) are obtained using the SVD algorithm. i b i c i The initial value of ).

[0028] As a further improvement, the present invention uses the squared distance from the center point of the laser stripe to the equation of the light plane as the objective function, and the diffraction angle relationship of the grating equation as the constraint, and obtains the normal vector (a) of each plane through a constrained least squares optimization algorithm. i b i c i The optimized value is as follows:

[0029] Select the coordinates of the center point of the laser stripe (x ik y ik , z ik The squared distance from the light plane equation to the objective function is taken as the objective function.

[0030]

[0031] At the same time, the angle θ between the i-order light plane and the 0-order light plane i Satisfy the following relationship

[0032]

[0033]

[0034] In the above formula, λ is the center wavelength of the laser, and d g The grating constant;

[0035] The normal vectors (a) of each plane are obtained through a constrained least squares optimization algorithm. i b i c i The optimized value of ).

[0036] As a further improvement, the present invention optimizes the position parameters of each diffraction plane by expressing the result of the intersection of each diffraction plane with a straight line in space and substituting the normal vector optimization value of each plane into the equation of the straight line in space. Specifically, the optimization is achieved as follows:

[0037] Collinearity of diffracted laser fringes within the optical plane space can be represented as a combination of two base planes.

[0038] λ(e 11 x+e 12 y+e 13 z+e 14 )+μ(e 21 x+e 22 y+e 23 z+e 24) = 0;

[0039] The obtained plane method (a) i b i c i Substituting the vectors into the above equation, we can obtain the following matrix.

[0040]

[0041] By fitting the base plane equation and the coefficient combination of the light plane of each laser stripe, the light plane equation of each laser stripe can be obtained from the corresponding combination coefficient and the two base plane equations, thus completing the optimization of the light plane calibration.

[0042] As a further improvement, the calibration method of the present invention is implemented based on the following apparatus:

[0043] Line lasers are used to emit line laser stripes;

[0044] Multi-line laser fringes are generated by a line laser through grating diffraction; the diffraction grating located in front of the line laser is used to divide the line laser fringes into multiple orders of line laser fringes with a certain spatial distribution; the fringe distribution can be adjusted according to the grating parameters.

[0045] A rectangular aperture located in front of the diffraction grating is used to block the high-order low-energy laser stripes, so that the generated multi-beam linear laser stripes of a specific order are projected onto the object surface; and the phase plate makes the intensity of the generated laser stripes uniformly distributed.

[0046] A camera that captures images of deformed multi-line laser stripes from another angle;

[0047] A translation stage for placing and moving objects for scanning and measurement;

[0048] An industrial control computer connected to the drive motors of the camera and the translation stage respectively.

[0049] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0050] This paper proposes a novel 3D measurement system based on multi-line structured light generated by a diffraction grating. A diffraction grating is added after the laser line path in front of the object to generate multi-line structured light.

[0051] Compared to traditional multi-line laser 3D sensors, diffraction gratings generate multiple diffraction laser lines on an object, allowing for the simultaneous acquisition of 3D surface information at multiple line locations. Furthermore, the diffraction order can be selected based on the object's surface shape.

[0052] Because grating diffraction generates multi-line structured light, each order of laser lines has a precise spatial light field distribution.

[0053] Because the optical plane of a multi-line system has precise constraints, calibrating the system using these constraints can improve the calibration accuracy of the optical plane, thereby improving the system's measurement accuracy.

[0054] This system not only improves the accuracy of the measurement system, but also has the advantage of a compact light source, which facilitates the miniaturization of the device and has great application value. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the structure of the device of the present invention;

[0056] In the diagram, 1 is a line laser, 2 is a diffraction grating, 3 is a rectangular aperture, 4 is a translation stage and its drive motor, 5 is a camera, 6 is an industrial computer, and 7 is multiple diffracted rays passing through the diffraction grating.

[0057] Figure 2 This is a schematic diagram illustrating the working principle of the method of this invention;

[0058] Figure 3 This is a data flow diagram of the method of the present invention. Detailed Implementation

[0059] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0060] System Composition

[0061] This system is based on the principle of laser triangulation, determining the three-dimensional coordinates of the target through the triangular relationship between the projection point and the imaging point. The multi-line laser three-dimensional measurement system based on grating diffraction consists of: a line laser 1 emitting a line laser; a diffraction grating 2 with a precise grating constant located in front of the line laser 1, used to divide the line laser into multiple orders of line laser planes with a certain spatial distribution; a rectangular aperture 3 with adjustable width located in front of the diffraction grating 2 blocking the higher-order, lower-energy laser lines, causing multiple diffracted rays 7 of specific orders generated by the grating to be projected onto the object surface; and an observation camera 5 acquiring the deformed multi-line laser image from another angle.

[0062] Camera 5 transmits image data to industrial control computer 6 via network cable, and the translation stage and its drive motor 4 are connected to industrial control computer 6 via data cable.

[0063] The industrial computer 6 obtains the intrinsic and extrinsic parameters of the camera 5 through a calibration algorithm. The optical plane calibration algorithm is based on the parameter constraints of the grating diffraction equation and simultaneously calibrates the optical planes of all laser lines of all orders. The industrial computer 6 extracts the coordinates of the twisted laser line fringes through the centerline extraction algorithm, and simultaneously solves the optical plane equations of the corresponding laser lines of the orders. The surface contours at the locations of these modulated and curved laser lines are then calculated. The object is fixed on a translation stage, and the industrial computer 6 drives the motor to move the translation stage. The camera 5 scans and processes multiple frames of images, calculates, and repeats the above calculation process to obtain the object surface contours at different positions. The direction vector of the translation stage is obtained through the translation stage calibration algorithm, and the point clouds of multiple frames of data are registered to obtain high-density, high-precision three-dimensional contour data of the object surface.

[0064] The hardware drivers and hardware motion control, including camera 5 and translation stage, as well as the software tools for camera 5 acquisition, calibration, stripe centerline extraction, calculation, registration, and display, all run on a standard industrial computer 6.

[0065] Light plane calibration method

[0066] The optical plane calibration method is a core component of a multi-line laser 3D measurement system. By combining the optical plane equation with the position coordinates of the modulated light rays from the object in the actual measurement, the 3D coordinates at that location can be calculated, thereby achieving 3D measurement.

[0067] The calibration method proposed in this paper is an optical plane optimization method based on grating diffraction constraints, which can improve the calibration accuracy of the optical plane and thus improve the measurement accuracy.

[0068] Calibration process:

[0069] (1) Camera 5 acquires two calibration images at the same position; a clear checkerboard photo is acquired under normal exposure for camera 5 calibration, and a laser stripe image falling on the checkerboard plane is acquired under low exposure parameters; camera 5 and the laser line remain in the same position, while the position of the checkerboard is moved, and two calibration images are acquired when the checkerboard is in the same position: a clear checkerboard photo is acquired under normal exposure, and an image of the laser stripe falling on the checkerboard plane is acquired under low exposure parameters. After the two images are taken, the checkerboard is moved to acquire multiple sets of images.

[0070] (2) Collect multiple sets of calibration images by moving the chessboard grid;

[0071] (3) Camera 5 calibration; The intrinsic parameter matrix A, extrinsic parameter RT matrix, and distortion coefficient K of camera 5 are calibrated using the above checkerboard image;

[0072] (4) Laser line center coordinate system transformation; extract the center line of the laser stripe in the low exposure mode, and calculate the coordinate system equation of the corresponding checkerboard plane of the camera 5 through the corresponding RT matrix of the camera 5. Combine the pixel coordinates of the center line and the intrinsic parameters of the camera 5 to transform the pixel coordinates of the center line into the coordinate system of the camera 5.

[0073] (5) Determine the order of the diffraction laser line corresponding to the coordinates of each laser fringe; the coordinates of the center camera 5 on the light plane πi corresponding to the i-th order diffraction fringe are...

[0074] (x ik y ik , z ik ), i=0, ±1,...±n, k=1, 2...m

[0075] (6) The normal vector of the light plane is (a i b i c i The average value of the planar point cloud in each direction is (x) i0 y i0 , z i0 The plane equation can be expressed as:

[0076] a i (xx i0 )+b i (yy i0 )+c i (zz i0 ) = 0

[0077] (7) Calculate and initialize parameter values ​​for a single plane; coordinates of all laser line center points (x... ik y ik , z ik Substituting into the above formula and expressing it in matrix form, we get:

[0078]

[0079] The normal vectors of each plane (a) are obtained using the SVD algorithm. i b i c i The initial value of ).

[0080] (8) Multi-plane optimization objective function; select points (x) on each light plane. in y in , z in The squared distance from the light plane equation to the objective function is taken as the objective function.

[0081]

[0082] At the same time, the angle θ between the i-order light plane and the 0-order light plane iSatisfy the following relationship

[0083]

[0084]

[0085] In the above formula, λ is the center wavelength of the laser, and d g is the grating constant.

[0086] (9) Obtain the normal vectors (a, b) of each plane using a constrained least squares optimization algorithm. i b i c i The optimized value of ).

[0087] (10) The collinearity of the optical plane of a diffracted laser line in space can be represented as a combination of two base planes.

[0088] λ(e 11 x+e 12 y+e 13 z+e 14 )+μ(e 21 x+e 22 y+e 23 z+e 24 Substituting the obtained plane normal vector into the above equation, we can obtain the following matrix: ) = 0

[0089]

[0090] By fitting the equations of the base plane and the coefficient combinations of each optical plane, the optical plane equations of each laser line can be obtained from the corresponding combination coefficients and the two base plane equations, thus completing the optimization of the optical plane calibration.

[0091] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the core technical features of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for optical plane calibration in a multi-line laser three-dimensional measurement system, characterized in that, include: Acquire multiple sets of checkerboard calibration images at different locations, capture clear checkerboard images under normal exposure, and capture images of laser stripes of various orders falling on the checkerboard plane under low exposure parameters; The intrinsic and extrinsic parameter matrix of the camera (5) is calibrated by acquiring a clear checkerboard image set under normal exposure as described above. The plane equation of the camera (5) coordinate system of the corresponding position chessboard plane is calculated by using the extrinsic parameter matrix of the camera (5); Images of laser stripes of various grades falling on the checkerboard plane were acquired under the aforementioned low exposure parameters, and the center line extraction algorithm was used to calculate the pixel coordinates of the center line of each grade of laser stripe. The homogeneous coordinates of the laser stripe centerline in the camera (5) coordinate system are calculated using the camera (5) intrinsic parameter matrix and the pixel coordinates of the laser stripe centerlines of each order. The checkerboard plane equation in the world coordinate system at each position is transformed into the plane equation in the camera (5) coordinate system by using the extrinsic parameters of each position of the camera (5); Combine the homogeneous coordinates of the camera (5) and the plane equation of the checkerboard in the coordinate system of the camera (5) to transform the homogeneous coordinates of the camera (5) of the laser stripe centerline into three-dimensional coordinates in the coordinate system of the camera (5) of the laser stripe centerline. Determine the order of each diffraction laser line corresponding to each laser stripe in the laser stripe image captured under low exposure parameters and place the three-dimensional coordinates of the center line of the corresponding diffraction laser stripe in the camera (5) coordinate system into the laser stripe light plane equation of the corresponding order to obtain the initial value of the coefficients of the light plane equation of each diffraction laser stripe. The objective function is the squared distance from the center point of the laser fringe to the equation of the optical plane, and the diffraction angle relationship of the grating equation is used as a constraint. The normal vectors of each plane are obtained through a constrained least squares optimization algorithm. The optimized value; Based on the fact that each diffracted light plane intersects a straight line in space, the position parameters of each light plane are optimized by expressing the spatial straight line equation and substituting the normal vector optimization value of each light plane.

2. The optical plane calibration method for a multi-line laser three-dimensional measurement system according to claim 1, characterized in that, The acquisition of multiple sets of chessboard grid calibration images at different locations specifically includes: The camera (5) and the line laser (1) remain in the same position, while the position of the checkerboard is moved. When the checkerboard is in the same position, two calibration images are collected: a clear checkerboard photo is collected under normal exposure of the camera (5), and a laser stripe image falling on the plane of the checkerboard is collected under low exposure parameters of the camera (5). After the two images are taken, the checkerboard is moved to collect multiple sets of images.

3. The optical plane calibration method for a multi-line laser three-dimensional measurement system according to claim 1 or 2, characterized in that, The process involves substituting the three-dimensional coordinates of the camera (5) coordinate system corresponding to the center line of the diffraction laser stripe into the corresponding order of the laser stripe light plane equation to obtain the initial values ​​of the coefficients of the light plane equation for each diffraction laser stripe, specifically as follows: The coordinates of the center line point of the laser fringe on the optical plane πi corresponding to the i-th order diffraction fringe are: ； In the above formula, m is the number of center points on each order of laser stripes. If the normal vector of the light plane is... Average the values ​​of the planar point cloud in each direction. The equation of the plane can be expressed as ； Coordinates of all laser stripe centerline points Substituting into the above formula and expressing it in matrix form, we get: ； The normal vectors of each plane are obtained using the SVD algorithm. The initial value.

4. The optical plane calibration method for a multi-line laser three-dimensional measurement system according to claim 3, characterized in that, The objective function is the squared distance from the center point of the laser stripe to the equation of the optical plane, and the diffraction angle relationship of the grating equation is used as a constraint. The normal vectors of each plane are obtained through a constrained least squares optimization algorithm. The optimized values ​​are as follows: Select the coordinates of the center line point of the laser stripe The squared distance to the light plane equation is used as the objective function, i.e. ； At the same time, the angle between the i-order light plane and the 0-order light plane Satisfy the following relationship ； ； In the above formula, λ is the center wavelength of the laser. The grating constant; The normal vectors of each plane are obtained using a constrained least squares optimization algorithm. The optimized value.

5. The optical plane calibration method for a multi-line laser three-dimensional measurement system according to claim 1, 2, or 4, characterized in that, The optimization of the position parameters of each diffracted light plane is achieved by using the fact that each diffracted light plane intersects a straight line in space, expressing the result through the equation of the straight line in space, and substituting the normal vector optimization value of each light plane. Specifically: Collinearity of diffracted laser fringes within the optical plane space can be represented as a combination of two base planes. ； The obtained plane normal vector Substituting the optimized value into the above formula, we obtain the following matrix: ； By fitting the base plane equation and the coefficient combination of the light plane of each laser stripe, the light plane equation of each laser stripe can be obtained from the corresponding combination coefficient and the two base plane equations, thus completing the optimization of the light plane calibration.

6. The optical plane calibration method for a multi-line laser three-dimensional measurement system according to claim 1, characterized in that, The calibration method described above is implemented using the following device: A line laser (1) is used to emit line laser stripes; The multi-line laser stripes are generated by a line laser (1) through grating diffraction; the diffraction grating (2) located in front of the line laser (1) is used to divide the line laser stripes into multi-level line laser stripes with a certain spatial distribution. The stripe distribution can be adjusted according to the grating parameters; The rectangular aperture (3) located in front of the diffraction grating (2) is used to block the high-order low-energy laser stripes, so that the generated multi-beam linear laser stripes of a specific order are projected onto the surface of the object; and the phase plate makes the intensity of the generated laser stripes uniformly distributed. (5) A camera that acquires images of deformed multi-line laser stripes from another angle; A translation stage for placing and moving objects for scanning and measurement; An industrial control computer (6) is connected to the drive motors of the camera (5) and the translation stage respectively.

Images