Laser stripe three-dimensional space coordinate acquisition method, system, device and storage medium

Abstract

The present invention relates to the field of image analysis technology, and specifically to a method, system, device and storage medium for acquiring three-dimensional spatial coordinates of laser stripes. In order to solve the technical problem of low accuracy in acquiring three-dimensional data of laser stripes in the prior art, the present invention first extracts the centerline points of laser stripes based on laser stripe images on calibration plates with different postures respectively acquired by two cameras, so as to more comprehensively cover the spatial distribution of the laser stripe light plane. Then, based on the centerline sets of the laser stripes of the two cameras, the straight line equation of each laser stripe in the camera coordinate system is calculated, so as to acquire the equation of the light plane where the laser stripe is located. Finally, by projecting the three-dimensional coordinates of the laser stripe points on the image taken by the first camera on the light plane where each laser stripe is located onto the light plane of the second camera, the matching points are acquired and the three-dimensional coordinates of the corresponding laser stripe points are calculated, so as to obtain accurate three-dimensional data of the laser stripe image.

Description

Technical Field

[0001] The present invention relates to the field of image analysis technology, and in particular to a method, system, device and storage medium for acquiring three-dimensional spatial coordinates of laser stripes. Background Art

[0002] Camera calibration and object 3D spatial coordinate reconstruction based on laser stripe images are important research directions in computer stereo vision. At present, the structured light plane generated by the laser is often used to determine the spatial position of the center point of the laser stripe. Therefore, the parameters of the light plane where the laser stripe is located will affect the accuracy of spatial coordinate extraction, and the extraction of the center point of the laser stripe is a key step in calculating the spatial coordinates. However, the existing methods are not adaptable enough to the laser stripe scene on the checkerboard calibration plate, resulting in poor fitting effect of the laser stripe center line and inaccurate positioning of the laser stripe center point, which in turn affects the accuracy of the spatial coordinates. Summary of the invention

[0003] The purpose of the present invention is to provide a method, system, device and storage medium for acquiring three-dimensional spatial coordinates of laser stripes.

[0004] The technical solution of the present invention is as follows:

[0005] A method for acquiring three-dimensional spatial coordinates of laser stripes, comprising the following operations:

[0006] S1. The laser projects line structured light onto a calibration plate at different positions. The first camera and the second camera respectively acquire images on the calibration plate to obtain a number of first camera images and a number of second camera images. The laser stripe center lines in the number of first camera images and the number of second camera images are respectively extracted to form a first camera laser stripe center line set and a second camera laser stripe center line set.

[0007] S2. Based on the center line set of the laser stripes of the first camera and the center line set of the laser stripes of the second camera, obtain the optical plane equations of the first camera center and the second camera center and each laser stripe center line, respectively, to form the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set; in the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set, each camera center laser stripe plane equation is combined with the calibration plate plane equation of the corresponding calibration plate posture to obtain the straight line equation of each laser stripe on the calibration plate with different postures of different cameras in the camera coordinate system; the straight line equation of the same laser stripe on the calibration plate with different postures of the same camera in the camera coordinate system is used as a laser stripe camera coordinate system straight line equation set; based on each laser stripe camera coordinate system straight line equation set, obtain the light plane equation of each laser stripe; based on the light plane equation of all laser stripes, perform light plane calibration on the laser, the first camera and the second camera;

[0008] S3. Obtain the spatial coordinates of each laser stripe point on the light plane where each laser stripe is located on the image taken by the first camera, and take the spatial coordinates corresponding to the same laser stripe point as a spatial coordinate set to obtain several spatial coordinate sets; project the several spatial coordinate sets onto the light plane of the second camera respectively, and match them with the laser stripes on the image taken by the second camera to obtain several matching points; based on the matching points and the corresponding laser stripe points, obtain the three-dimensional coordinates of the corresponding laser stripe points; the three-dimensional coordinates of all laser stripe points form the three-dimensional data of the laser stripe image.

[0009] In the first camera laser stripe center line set of S1, the method for obtaining the laser stripe center line in the current first camera image is specifically as follows: the current first camera image is grayed and Gaussian filtered to obtain the current first camera filtered image; the region of interest in the current first camera filtered image is screened out to obtain the first camera region of interest image; the Hessian matrix of each pixel point in the first camera region of interest image is obtained; based on the Hessian matrix of each pixel point, the respective eigenvalues ​​and eigenvectors are obtained; the pixel point whose eigenvalue meets the eigenvalue condition of the center point and whose eigenvector direction is the same as the eigenvector direction of other pixel points in the neighborhood is taken as the laser stripe center point; all the laser stripe center points are arranged in order from small to large in the horizontal coordinate position to obtain a number of sorted laser stripe centers; two adjacent laser stripe center points whose horizontal coordinate position difference is less than the position difference threshold are grouped to obtain a number of laser stripe center point groups; the least squares fitting is performed on the several laser stripe center point groups to obtain a number of laser stripe center lines in the current first camera image.

[0010] The light plane equation between the center of the first camera and the center line of the current laser stripe in S2 is obtained by the following formula: , is the focal length of the first camera, , , Respectively i The horizontal coordinate coefficient, vertical coordinate coefficient and vertical coordinate coefficient of the center line of the current laser stripe on the pose calibration plate are , , It is based on i The data of the laser stripe center point corresponds to the center line of the current laser stripe on the pose calibration plate.

[0011] The plane equation of the calibration plate in S2 is obtained based on the world coordinate system established on the calibration plate, and the rotation matrix and translation matrix of the camera coordinate system and the world coordinate system; the camera coordinate system is the first camera coordinate system or the second camera coordinate system.

[0012] The operation of obtaining the spatial coordinates of the current laser stripe point on the light plane where the first laser stripe is located in S3 is specifically as follows: based on the coordinate formulas of the laser stripe point in the pixel coordinate system and the first camera coordinate system, a laser stripe point coordinate transformation matrix is ​​obtained; the laser stripe point coordinate transformation matrix is ​​combined with the light plane equation of the light plane where the first laser stripe is located in the first camera coordinate system to obtain the laser stripe point combined formula; the current laser stripe point is substituted into the laser stripe point combined formula to obtain the spatial coordinates of the current laser stripe point on the light plane where the first laser stripe is located.

[0013] In S3, the operation of projecting the first spatial coordinates from the current spatial coordinate set onto the light plane of the second camera is specifically as follows: obtaining the transformation matrix from the first camera to the second camera, and obtaining the projection matrix after inversion; and obtaining the projection point of the first spatial coordinates on the light plane of the second camera based on the projection matrix, the first spatial coordinates, and the second camera internal parameters.

[0014] The laser projected line structured light does not contact the checkerboard on the calibration plate.

[0015] A laser stripe three-dimensional space coordinate acquisition system, used to implement the above-mentioned laser stripe three-dimensional space coordinate acquisition method, comprising:

[0016] The camera laser stripe center line set generation module is used for projecting line structured light of the laser onto a calibration plate in different positions, and the first camera and the second camera respectively acquire images on the calibration plate to obtain a plurality of first camera images and a plurality of second camera images; the laser stripe center lines in the plurality of first camera images and the plurality of second camera images are respectively extracted to form a first camera laser stripe center line set and a second camera laser stripe center line set;

[0017] The light plane equation generation and light plane calibration module is used to obtain the light plane equations of the first camera center and the second camera center and each laser stripe center line based on the first camera laser stripe center line set and the second camera laser stripe center line set, so as to form the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set; in the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set, each camera center laser stripe plane equation is combined with the calibration plate plane equation of the corresponding calibration plate posture to obtain the straight line equation of each laser stripe on the calibration plate with different postures of different cameras in the camera coordinate system; the straight line equation of the same laser stripe on the calibration plate with different postures of the same camera in the camera coordinate system is used as a laser stripe camera coordinate system straight line equation set; based on each laser stripe camera coordinate system straight line equation set, the light plane equation of each laser stripe is obtained; based on the light plane equation of all laser stripes, the laser, the first camera and the second camera are calibrated on the light plane;

[0018] The laser stripe image three-dimensional data generation module is used to obtain the spatial coordinates of each laser stripe point on the image taken by the first camera on the light plane where each laser stripe is located, and take the spatial coordinates corresponding to the same laser stripe point as a spatial coordinate set to obtain a plurality of spatial coordinate sets; project the plurality of spatial coordinate sets onto the light plane of the second camera respectively, and match them with the laser stripes on the image taken by the second camera to obtain a plurality of matching points; based on the matching points and the corresponding laser stripe points, obtain the three-dimensional coordinates of the corresponding laser stripe points; the three-dimensional coordinates of all laser stripe points form the three-dimensional data of the laser stripe image.

[0019] A laser stripe three-dimensional space coordinate acquisition device comprises a processor and a memory, wherein the processor implements the above-mentioned laser stripe three-dimensional space coordinate acquisition method when executing a computer program stored in the memory.

[0020] A computer-readable storage medium is used to store a computer program, wherein the computer program implements the above-mentioned method for acquiring three-dimensional spatial coordinates of laser stripes when executed by a processor.

[0021] The beneficial effects of the present invention are:

[0022] The present invention provides a method for acquiring three-dimensional spatial coordinates of laser stripes. First, based on the laser stripe images on a calibration plate with different postures respectively acquired by a first camera and a second camera, the centerline points of the laser stripes are extracted to form centerline data of the laser stripes, which can more comprehensively cover the spatial distribution of the laser stripe light plane and enhance the richness of data samples, thereby improving the subsequent calibration accuracy and the accuracy of three-dimensional coordinate reconstruction. Then, based on the centerline set of the laser stripes of the first camera and the centerline set of the laser stripes of the second camera, the plane equations of the centers of the two cameras and each centerline of the laser stripes are constructed respectively, and are combined with the plane equations of the calibration plate, so that the information provided by the two cameras can be fully utilized, the laser stripes can be constrained from different perspectives, the position of the laser stripes in space can be determined more accurately, and then the straight line equation of each laser stripe in the camera coordinate system can be accurately calculated. , which helps to determine the position and posture of the light plane more accurately and reduce the error of light plane calibration; at the same time, when calculating the light plane equation, the straight line equation of the same laser stripe in the camera coordinate system under different calibration plate postures of the same camera is solved, which can integrate the measurement data under multiple postures, effectively reduce the influence of single measurement errors, avoid error accumulation, and thus improve the overall accuracy of light plane calibration; finally, by projecting the three-dimensional coordinates of the laser stripe points on the image taken by the first camera on the light plane where each laser stripe is located onto the light plane of the second camera, the corresponding matching points of the laser stripe points on the second camera are obtained; and based on the matching points and the corresponding laser stripe points, the three-dimensional coordinates of the corresponding laser stripe points are obtained, thereby obtaining accurate three-dimensional data of the laser stripe image; the accuracy and adaptability of the method for acquiring three-dimensional data of laser stripe images are improved. DETAILED DESCRIPTION

[0023] This embodiment provides a method for obtaining three-dimensional spatial coordinates of laser stripes, including the following operations:

[0024] S1. The laser projects line structured light onto a calibration plate at different positions. The first camera and the second camera respectively acquire images on the calibration plate to obtain a number of first camera images and a number of second camera images. The laser stripe center lines in the number of first camera images and the number of second camera images are respectively extracted to form a first camera laser stripe center line set and a second camera laser stripe center line set.

[0025] S2. Based on the center line set of the laser stripes of the first camera and the center line set of the laser stripes of the second camera, obtain the optical plane equations of the first camera center and the second camera center and each laser stripe center line, respectively, to form the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set; in the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set, each camera center laser stripe plane equation is combined with the calibration plate plane equation of the corresponding calibration plate posture to obtain the straight line equation of each laser stripe on the calibration plate with different postures of different cameras in the camera coordinate system; the straight line equation of the same laser stripe on the calibration plate with different postures of the same camera in the camera coordinate system is used as a laser stripe camera coordinate system straight line equation set; based on each laser stripe camera coordinate system straight line equation set, obtain the light plane equation of each laser stripe; based on the light plane equation of all laser stripes, perform light plane calibration on the laser, the first camera and the second camera;

[0026] S3. Obtain the spatial coordinates of each laser stripe point on the light plane where each laser stripe is located on the image taken by the first camera, and take the spatial coordinates corresponding to the same laser stripe point as a spatial coordinate set to obtain several spatial coordinate sets; project the several spatial coordinate sets onto the light plane of the second camera respectively, and match them with the laser stripes on the image taken by the second camera to obtain several matching points; based on the matching points and the corresponding laser stripe points, obtain the three-dimensional coordinates of the corresponding laser stripe points; the three-dimensional coordinates of all laser stripe points form the three-dimensional data of the laser stripe image.

[0027] S1. The laser projects line structured light onto a calibration plate in different positions. The first camera and the second camera respectively acquire images on the calibration plate to obtain a number of first camera images and a number of second camera images. The center lines of the laser stripes in the first camera images and the second camera images are respectively extracted to form a first camera laser stripe center line set and a second camera laser stripe center line set.

[0028] Based on the laser stripe images on the calibration plate with different postures obtained by the first camera and the second camera respectively, the midline points of the laser stripes are extracted to form the centerline data of the laser stripes, which can more comprehensively cover the spatial distribution of the laser stripe light plane and enhance the richness of the data samples, thereby improving the subsequent calibration accuracy and the accuracy of three-dimensional coordinate reconstruction.

[0029] After calibrating the internal and external parameters of the first camera and the second camera on both sides of the laser, the laser projects several line structured lights onto a calibration plate (preferably a checkerboard calibration plate) in different positions. The first camera and the second camera respectively capture images on the calibration plate to obtain several first camera images and several second camera images, each of which contains several laser stripes.

[0030] Then, the center lines of the laser stripes in several first camera images and several second camera images are extracted respectively to obtain different center lines of the laser stripes on calibration plates with different postures of different cameras. The different center lines of the laser stripes on calibration plates with different postures of different cameras are combined into a data set, thus forming a first camera laser stripe center line set and a second first camera laser stripe center line set.

[0031] Among them, the center lines of the laser stripes of the first camera are concentrated, and the method for obtaining the center lines of several laser stripes in the current first camera image (on the current posture calibration board) is as follows.

[0032] Step 1: The current first camera image is grayed and Gaussian filtered to reduce the amount of calculation, focus on the brightness information in the image, and remove noise to obtain the current first camera filtered image.

[0033] Step 2: To filter out the interference noise, accurately extract the center point of each laser stripe, screen out the region of interest in the current first camera filtered image, and obtain the first camera region of interest image. The region of interest image can be determined according to the pixel coordinates of the corner points of the chessboard.

[0034] Step 3: Obtain the Hessian matrix of each pixel in the image of the region of interest of the first camera; the Hessian matrix of the pixel is obtained based on the first-order partial derivative and the second-order partial derivative of the pixel.

[0035] Step 4. Based on the Hessian matrix of each pixel point, obtain the respective eigenvalues ​​and eigenvectors; the pixel point whose eigenvalue satisfies the center point eigenvalue condition (the center point eigenvalue condition can be determined according to the specific situation of the image and historical experience) and whose eigenvector direction is the same as the eigenvector direction of other pixel points in the neighborhood is taken as the center point of the laser stripe.

[0036] Step 5: Arrange all laser stripe center points in ascending order of horizontal coordinate position to obtain a number of sorted laser stripe center points; group two adjacent sorted laser stripe centers whose horizontal coordinate position difference is less than the position difference threshold, that is, take two adjacent sorted laser stripe centers with similar horizontal coordinate distance as points on the same laser stripe center line to obtain a number of laser stripe center point groups. The laser stripe center points in the same laser stripe center point group are on the same laser stripe center line.

[0037] Step 6: perform least squares fitting on several laser stripe center point groups respectively, that is, obtain a laser stripe center line equation according to the data in one laser stripe center point group, and obtain several laser stripe center lines in the current first camera image.

[0038] In addition, the laser projected line structured light can be located on the checkerboard of the calibration plate, or the laser projected line structured light can be not in contact with the checkerboard of the calibration plate in order to prevent the white square light strips on the checkerboard from becoming thicker, thereby improving the calculation efficiency.

[0039] S2. Based on the center line set of the laser stripes of the first camera and the center line set of the laser stripes of the second camera, obtain the optical plane equations of the first camera center and the second camera center and each laser stripe center line respectively, so as to form the laser stripe plane equation set of the first camera center and the laser stripe plane equation set of the second camera center; in the laser stripe plane equation set of the first camera center and the laser stripe plane equation set of the second camera center, each camera center laser stripe plane equation is combined with the calibration plate plane equation of the corresponding calibration plate posture to obtain the straight line equation of each laser stripe on the calibration plate with different postures of different cameras in the camera coordinate system; the straight line equation of the same laser stripe on the calibration plate with different postures of the same camera in the camera coordinate system is used as a laser stripe camera coordinate system straight line equation set; based on each laser stripe camera coordinate system straight line equation set, obtain the light plane equation of each laser stripe; based on the light plane equation of all laser stripes, perform light plane calibration on the laser, the first camera and the second camera.

[0040] Based on the center line set of the laser stripes of the first camera and the center line set of the laser stripes of the second camera, plane equations of the two camera centers and the center line of each laser stripe are constructed, and combined with the plane equation of the calibration plate, the information provided by the two cameras can be fully utilized to constrain the laser stripes from different perspectives, and the position of the laser stripes in space can be more accurately determined, and then the straight line equation of each laser stripe in the camera coordinate system can be accurately calculated, which is helpful to more accurately determine the position and posture of the light plane and reduce the error of the light plane calibration; at the same time, when calculating the light plane equation, the straight line equation of the same laser stripe in the camera coordinate system under different calibration plate postures of the same camera is solved, which can integrate the measurement data under multiple postures, effectively reduce the influence of single measurement errors, avoid error accumulation, and thus improve the overall accuracy of the light plane calibration.

[0041] Firstly, based on the center line set of laser stripes of the first camera and the center line set of laser stripes of the second camera, the light plane equations of the first camera center and the second camera center and each laser stripe center line are obtained respectively, that is, the light plane formed by the camera center and each laser stripe center line is obtained as the light plane where the corresponding laser stripe is located, and the light plane equations of different laser stripes on the calibration plates of the same camera with different postures are combined into a data set, thus forming the plane equation set of laser stripes at the center of the first camera and the plane equation set of laser stripes at the center of the second camera.

[0042] Among them, taking the first camera as an example, in the laser stripe plane equation set at the center of the first camera, the light plane equation between the center of the first camera and the center line of the current laser stripe is as follows:

[0043] ,

[0044] is the focal length of the first camera, , , Respectively i The horizontal coordinate coefficient, vertical coordinate coefficient and vertical coordinate coefficient of the center line of the current laser stripe on the pose calibration plate are , , It is based on i The data of the laser stripe center point corresponds to the center line of the current laser stripe on the pose calibration plate.

[0045] The method for obtaining the plane equation of the laser stripe at the center of the second camera is similar to that of the first camera, and will not be repeated here to save space.

[0046] Then, in the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set, each camera center laser stripe plane equation is combined with the calibration plate plane equation of the corresponding calibration plate posture to obtain the straight line equation of each laser stripe on the calibration plate with different postures of different cameras in the camera coordinate system.

[0047] The above calibration plate plane equation is obtained based on the world coordinate system established on the calibration plate, and the rotation matrix and translation matrix of the camera coordinate system (the first camera coordinate system or the second camera coordinate system) and the world coordinate system.

[0048] Among them, taking the first camera as an example, i The straight line equation of the current laser stripe on the pose calibration plate in the first camera coordinate system is as follows:

[0049] ,

[0050] ,

[0051] , , Respectively i The plane equation coefficients of the plane where the calibration plate is located in each pose.

[0052] Next, the straight line equation of the same laser stripe on the calibration plate of the same camera in different poses in the camera coordinate system is used as a set of straight line equations of the laser streak camera coordinate system; based on each set of straight line equations of the laser streak camera coordinate system, the light plane equations of the respective laser stripes are obtained.

[0053] Finally, based on the light plane equations where all laser stripes are located, the laser, the first camera and the second camera are calibrated on the light plane.

[0054] S3. Obtain the spatial coordinates of each laser stripe point on the light plane where each laser stripe is located on the image taken by the first camera, take the spatial coordinates corresponding to the same laser stripe point as a spatial coordinate set, and obtain several spatial coordinate sets; project the several spatial coordinate sets onto the light plane of the second camera respectively, and match them with the laser stripes on the image taken by the second camera to obtain several matching points; based on the several matching points and the corresponding laser stripe points, obtain the three-dimensional coordinates of the corresponding laser stripe points; the three-dimensional coordinates of all laser stripe points form the three-dimensional data of the laser stripe image.

[0055] The three-dimensional coordinates of the laser stripe points on the image taken by the first camera on the light plane where each laser stripe is located are projected onto the light plane of the second camera, and the corresponding matching points of the laser stripe points on the second camera are obtained; and based on the matching points and the corresponding laser stripe points, the three-dimensional coordinates of the corresponding laser stripe points are obtained, thereby obtaining the three-dimensional data of the laser stripe image.

[0056] First, the spatial coordinates of each laser stripe point on the image taken by the first camera after the light plane calibration are obtained on the light plane where each laser stripe is located (the light plane equation where the laser stripe is located has been obtained in S2), and the spatial coordinates corresponding to the same laser stripe point are taken as a spatial coordinate set to obtain several spatial coordinate sets.

[0057] Among them, the operation of obtaining the spatial coordinates of the current laser stripe point on the light plane where the first laser stripe is located is specifically as follows: based on the coordinate formulas of the laser stripe point in the pixel coordinate system and the first camera coordinate system, respectively, a laser stripe point coordinate transformation matrix is ​​obtained; the laser stripe point coordinate transformation matrix is ​​combined with the light plane equation of the light plane where the first laser stripe is located in the first camera coordinate system to obtain the laser stripe point combined formula; the current laser stripe point is substituted into the laser stripe point combined formula to obtain the spatial coordinates of the current laser stripe point on the light plane where the first laser stripe is located.

[0058] Then, several sets of spatial coordinates are projected onto the light plane of the second camera, and matched with the laser stripes on the image captured by the second camera to obtain several matching points. During the matching process, if the projection point formed by projecting the spatial coordinate onto the light plane of the second camera is located on the laser stripes of the image captured by the second camera, the corresponding projection point is the matching point.

[0059] Among them, the operation of projecting the first spatial coordinate in the current spatial coordinate set onto the light plane of the second camera is specifically as follows: obtaining the transformation matrix from the first camera to the second camera, and obtaining the projection matrix after inversion; based on the projection matrix, the first spatial coordinate and the second camera intrinsic parameter, obtaining the projection point of the first spatial coordinate on the light plane of the second camera.

[0060] Finally, based on the matching points and the corresponding laser stripe points, as well as the light plane equations where the corresponding laser stripes are located, the three-dimensional coordinates of the corresponding laser stripe points are obtained; the three-dimensional coordinates of all laser stripe points form the three-dimensional data of the laser stripe image.

[0061] This embodiment further provides a laser stripe three-dimensional space coordinate acquisition system, which is used to implement the above-mentioned laser stripe three-dimensional space coordinate acquisition method, including:

[0062] The camera laser stripe center line set generation module is used for projecting line structured light of the laser onto a calibration plate in different positions, and the first camera and the second camera respectively acquire images on the calibration plate to obtain a plurality of first camera images and a plurality of second camera images; the laser stripe center lines in the plurality of first camera images and the plurality of second camera images are respectively extracted to form a first camera laser stripe center line set and a second camera laser stripe center line set;

[0063] The light plane equation generation and light plane calibration module is used to obtain the light plane equations of the first camera center and the second camera center and each laser stripe center line based on the first camera laser stripe center line set and the second camera laser stripe center line set, so as to form the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set; in the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set, each camera center laser stripe plane equation is combined with the calibration plate plane equation of the corresponding calibration plate posture to obtain the straight line equation of each laser stripe on the calibration plate with different postures of different cameras in the camera coordinate system; the straight line equation of the same laser stripe on the calibration plate with different postures of the same camera in the camera coordinate system is used as a laser stripe camera coordinate system straight line equation set; based on each laser stripe camera coordinate system straight line equation set, the light plane equation of each laser stripe is obtained; based on the light plane equation of all laser stripes, the laser, the first camera and the second camera are calibrated on the light plane;

[0064] The laser stripe image three-dimensional data generation module is used to obtain the spatial coordinates of each laser stripe point on the image taken by the first camera on the light plane where each laser stripe is located, and take the spatial coordinates corresponding to the same laser stripe point as a spatial coordinate set to obtain a plurality of spatial coordinate sets; project the plurality of spatial coordinate sets onto the light plane of the second camera respectively, and match them with the laser stripes on the image taken by the second camera to obtain a plurality of matching points; based on the matching points and the corresponding laser stripe points, obtain the three-dimensional coordinates of the corresponding laser stripe points; the three-dimensional coordinates of all laser stripe points form the three-dimensional data of the laser stripe image.

[0065] This embodiment further provides a laser stripe three-dimensional space coordinate acquisition device, including a processor and a memory, wherein the processor implements the above-mentioned laser stripe three-dimensional space coordinate acquisition method when executing a computer program stored in the memory.

[0066] This embodiment further provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned method for acquiring the three-dimensional spatial coordinates of laser stripes.

[0067] The present embodiment provides a method for acquiring three-dimensional spatial coordinates of laser stripes. First, based on the laser stripe images on a calibration plate with different postures acquired by a first camera and a second camera respectively, the centerline points of the laser stripes are extracted to form centerline data of the laser stripes, which can more comprehensively cover the spatial distribution of the laser stripe light plane and enhance the richness of data samples, thereby improving the subsequent calibration accuracy and the accuracy of three-dimensional coordinate reconstruction. Then, based on the centerline set of the laser stripes of the first camera and the centerline set of the laser stripes of the second camera, plane equations of the centers of the two cameras and the centerline of each laser stripe are constructed respectively, and the plane equations are combined with the plane equations of the calibration plate. The information provided by the two cameras can be fully utilized to constrain the laser stripes from different perspectives, and the position of the laser stripes in space can be determined more accurately, thereby accurately calculating the straight line square of each laser stripe in the camera coordinate system. The program helps to determine the position and posture of the light plane more accurately and reduce the error of light plane calibration; at the same time, when calculating the light plane equation, the straight line equation of the same laser stripe in the camera coordinate system under different calibration plate postures of the same camera is solved, which can integrate the measurement data under multiple postures, effectively reduce the influence of single measurement errors, avoid error accumulation, and thus improve the overall accuracy of light plane calibration; finally, the three-dimensional coordinates of the laser stripe points on the image taken by the first camera on the light plane where each laser stripe is located are projected onto the light plane of the second camera, and the corresponding matching points of the laser stripe points on the second camera are obtained; and based on the matching points and the corresponding laser stripe points, the three-dimensional coordinates of the corresponding laser stripe points are obtained, thereby obtaining accurate three-dimensional data of the laser stripe image; the accuracy and adaptability of the method for acquiring three-dimensional data of laser stripe images are improved.

Claims

1. A method for acquiring three-dimensional spatial coordinates of laser stripes, characterized in that: The following operations are included: S1. The laser projects line structured light onto a calibration plate at different positions. The first camera and the second camera respectively acquire images on the calibration plate to obtain a number of first camera images and a number of second camera images. The laser stripe center lines in the number of first camera images and the number of second camera images are respectively extracted to form a first camera laser stripe center line set and a second camera laser stripe center line set. S2, based on the first camera laser stripe center line set and the second camera laser stripe center line set, obtaining the light plane equations of the first camera center and the second camera center and each laser stripe center line, respectively, to form the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set; In the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set, each camera center laser stripe plane equation is combined with the calibration plate plane equation of the corresponding calibration plate posture to obtain the straight line equation of each laser stripe on the calibration plate with different postures of different cameras in the camera coordinate system; the straight line equation of the same laser stripe on the calibration plate with different postures of the same camera in the camera coordinate system is used as a laser stripe camera coordinate system straight line equation set; based on each laser stripe camera coordinate system straight line equation set, the light plane equation of each laser stripe is obtained; Based on the light plane equations where all laser stripes are located, the laser, the first camera and the second camera are calibrated on the light plane; S3. Obtain the spatial coordinates of each laser stripe point on the light plane where each laser stripe is located on the image taken by the first camera, and take the spatial coordinates corresponding to the same laser stripe point as a spatial coordinate set to obtain several spatial coordinate sets; project the several spatial coordinate sets onto the light plane of the second camera respectively, and match them with the laser stripes on the image taken by the second camera to obtain several matching points; based on the matching points and the corresponding laser stripe points, obtain the three-dimensional coordinates of the corresponding laser stripe points; the three-dimensional coordinates of all laser stripe points form the three-dimensional data of the laser stripe image.

2. The method for acquiring three-dimensional spatial coordinates of laser stripes according to claim 1, characterized in that: In the first camera laser stripe center line set of S1, the method for obtaining the laser stripe center line in the current first camera image is specifically as follows: The current first camera image is grayed and Gaussian filtered to obtain the current first camera filtered image; the region of interest in the current first camera filtered image is screened to obtain the first camera region of interest image; Obtaining the Hessian matrix of each pixel in the image of the region of interest of the first camera; Based on the Hessian matrix of each pixel point, the eigenvalue and eigenvector of each pixel point are obtained; the pixel point whose eigenvalue satisfies the eigenvalue condition of the center point and whose eigenvector direction is the same as that of the eigenvectors of other pixel points in the neighborhood is taken as the center point of the laser stripe; All laser stripe center points are arranged in ascending order of the horizontal coordinate position to obtain a number of sorted laser stripe centers; two adjacent laser stripe center points whose horizontal coordinate position difference is less than a position difference threshold are grouped together to obtain a number of laser stripe center point groups; and a number of laser stripe center point groups are respectively fitted by the least squares method to obtain a number of laser stripe center lines in the current first camera image.

3. The method for acquiring three-dimensional spatial coordinates of laser stripes according to claim 1, characterized in that: In S2, the light plane equation between the center of the first camera and the center line of the current laser stripe is obtained by the following formula: ， is the focal length of the first camera, , , Respectively i The horizontal coordinate coefficient, vertical coordinate coefficient and vertical coordinate coefficient of the center line of the current laser stripe on the pose calibration plate are , , It is based on i The data of the laser stripe center point corresponds to the center line of the current laser stripe on the pose calibration plate.

4. The method for acquiring three-dimensional space coordinates of laser stripes according to claim 1, characterized in that: In S2, the calibration plate plane equation is obtained based on the world coordinate system established on the calibration plate, and the rotation matrix and translation matrix of the camera coordinate system and the world coordinate system; the camera coordinate system is the first camera coordinate system or the second camera coordinate system.

5. The method for acquiring three-dimensional space coordinates of laser stripes according to claim 1, characterized in that: In S3, the operation of obtaining the spatial coordinates of the current laser stripe point on the light plane where the first laser stripe is located is specifically: Based on the coordinate formulas of the laser stripe point in the pixel coordinate system and the first camera coordinate system respectively, a laser stripe point coordinate transformation matrix is ​​obtained; the laser stripe point coordinate transformation matrix is ​​combined with the light plane equation of the light plane where the first laser stripe is located in the first camera coordinate system to obtain a laser stripe point simultaneous formula; the current laser stripe point is substituted into the laser stripe point simultaneous formula to obtain the spatial coordinates of the current laser stripe point on the light plane where the first laser stripe is located.

6. The method for acquiring three-dimensional space coordinates of laser stripes according to claim 1, characterized in that: In S3, the operation of projecting the first spatial coordinates onto the light plane of the second camera in the current spatial coordinate set is specifically as follows: The transformation matrix from the first camera to the second camera is obtained, and the projection matrix is ​​obtained after inversion. Based on the projection matrix, the first spatial coordinates and the second camera intrinsic parameters, the projection point of the first spatial coordinates on the light plane of the second camera is obtained.

7. The method for acquiring three-dimensional space coordinates of laser stripes according to claim 1, characterized in that: The laser projected line structured light does not contact the checkerboard on the calibration plate.

8. A laser stripe three-dimensional space coordinate acquisition system, used to implement the laser stripe three-dimensional space coordinate acquisition method according to claim 1, characterized in that: include: The camera laser stripe center line set generation module is used for projecting line structured light of the laser onto a calibration plate in different positions, and the first camera and the second camera respectively acquire images on the calibration plate to obtain a plurality of first camera images and a plurality of second camera images; the laser stripe center lines in the plurality of first camera images and the plurality of second camera images are respectively extracted to form a first camera laser stripe center line set and a second camera laser stripe center line set; The light plane equation generation and light plane calibration module is used to obtain the light plane equations of the first camera center and the second camera center and each laser stripe center line based on the first camera laser stripe center line set and the second camera laser stripe center line set, so as to form the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set; in the first camera center laser stripe plane equation set and the second camera center laser stripe plane equation set, each camera center laser stripe plane equation is combined with the calibration plate plane equation of the corresponding calibration plate posture to obtain the straight line equation of each laser stripe on the calibration plate with different postures of different cameras in the camera coordinate system; the straight line equation of the same laser stripe on the calibration plate with different postures of the same camera in the camera coordinate system is used as a laser stripe camera coordinate system straight line equation set; based on each laser stripe camera coordinate system straight line equation set, the light plane equation of each laser stripe is obtained; based on the light plane equation of all laser stripes, the laser, the first camera and the second camera are calibrated on the light plane; The laser stripe image three-dimensional data generation module is used to obtain the spatial coordinates of each laser stripe point on the image taken by the first camera on the light plane where each laser stripe is located, and take the spatial coordinates corresponding to the same laser stripe point as a spatial coordinate set to obtain a plurality of spatial coordinate sets; project the plurality of spatial coordinate sets onto the light plane of the second camera respectively, and match them with the laser stripes on the image taken by the second camera to obtain a plurality of matching points; based on the matching points and the corresponding laser stripe points, obtain the three-dimensional coordinates of the corresponding laser stripe points; the three-dimensional coordinates of all laser stripe points form the three-dimensional data of the laser stripe image.

9. A laser stripe three-dimensional space coordinate acquisition device, characterized in that: The method comprises a processor and a memory, wherein the processor implements the method for acquiring three-dimensional spatial coordinates of laser stripes as described in any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method for acquiring three-dimensional spatial coordinates of laser stripes according to any one of claims 1 to 7 is implemented.

Images