Automatic three-dimensional laser scanning system and scanning method thereof

Abstract

The invention discloses an automatic three-dimensional laser scanning system and a scanning method thereof, belongs to the technical field of computer vision, and solves the problems of low efficiency, high cost, complex operation and the like of a traditional automatic three-dimensional scanning system. The scanning system comprises a portal frame, a plurality of visual laser detectors, a guide rail, a scanned object, a controller and a computer. Compared with a photographing type three-dimensional scanner, three-dimensional reconstruction can be completed in the advancing process of a scanned object, the detection efficiency can completely meet the automatic detection requirement of a production line, the scanning area and the depth of field are greatly improved, the use number of the visual laser detectors is effectively reduced, and the purpose of reducing the cost is achieved; compared with a line laser three-dimensional scanner, the requirement for hardware stability is lowered, multi-line laser scanning can be achieved, and the scanning efficiency is multiplied; meanwhile, the device is low in operation difficulty and suitable for being widely used on an automatic production line.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to an automated three-dimensional laser scanning system and a scanning method thereof. Background Art

[0002] Three-dimensional scanning technology can calculate the spatial distribution of point clouds based on the surface information of an object collected without contacting the object to be scanned. Through a series of surface reconstruction methods, the point cloud data is integrated into a triangular mesh model of the object to be scanned in a computer. This technology is widely used in auxiliary manufacturing and inspection in the industrial design field, surgical positioning and rehabilitation in the medical field, virtual reality modeling and simulation in the game entertainment field, and site protection and restoration in the archaeological field, etc.

[0003] Compared with the three-dimensional detection method that requires manual participation, automated three-dimensional detection requires a high detection beat, low cost, and prohibits the use of positioning fiducial points on the surface of the object to be scanned. Currently, the three-dimensional scanning technologies applied to the field of automated detection mainly include photogrammetric three-dimensional scanners and line laser three-dimensional scanners. Among them, the photogrammetric three-dimensional scanner projects multiple raster patterns with optical coding characteristics continuously onto the surface of the object to be scanned by a projector and uses a binocular camera to collect images. The computer performs unified optical decoding on all the collected images to obtain the three-dimensional point cloud data of the object to be scanned in the area covered by the projected raster. When the size of the object to be scanned is larger than the raster coverage area, it is necessary to change the relative position of the scanner and the object to be scanned for multiple scans and stitching. In automated three-dimensional detection, usually, a single scanner is mounted on a transmission device such as a robotic arm or multiple scanners are arranged in space to obtain the complete point cloud of the object to be scanned. The single-scanner scheme based on the transmission device can achieve automated detection, but the detection efficiency still cannot meet the beat requirements. The scheme of multiple scanners working together not only has a high cost but also greatly increases the difficulty of mechanical design and collaborative control, and cannot be widely promoted and applied. The line laser three-dimensional scanner adopts a hardware scheme of "single camera + single-line laser" and is mainly applied to application fields with low detection efficiency requirements and flat object shapes to be scanned, such as circuit boards and sheet metal casings. This type of scanner only uses one camera for three-dimensional reconstruction and has extremely high requirements for the stability of the positional relationship between the camera and the laser line, which greatly increases the difficulty and cost of mechanical design. Moreover, this type of scanner only projects a single laser line onto the surface of the object to be scanned, which limits the point cloud generation efficiency and detection efficiency of a single-frame image.

[0004] With the advancement of industrial automation and intelligence, the demand for automated three-dimensional detection in production lines is increasing. Currently, the existing three-dimensional detection technologies are generally restricted by problems such as low efficiency, high cost, and complex operation, and are only limited to the three-dimensional detection mode with manual participation. Therefore, there is an urgent need in the detection field for an automated three-dimensional scanning system with high cost performance and strong versatility. Summary of the Invention

[0005] In order to solve the problems existing in the prior art, the present invention provides an automated three-dimensional laser scanning system and its scanning method, which overcomes the problems of low efficiency, high cost, complex operation, etc. existing in the traditional automated three-dimensional scanning system.

[0006] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0007] An automated three-dimensional laser scanning system, the system includes: a gantry, a plurality of vision laser detectors, a guide rail, an object to be scanned, a controller and a computer; the vision laser detector consists of a binocular camera and a laser that can emit one or more laser lines; the gantry is fixed on the ground, and the plurality of vision laser detectors are distributed on the gantry; the guide rail is arranged directly below the gantry, the object to be scanned is arranged on the guide rail, and the guide rail drives the object to be scanned to move in a straight line; the computer controls the opening and closing of the vision laser detector and the opening and closing of the guide rail through the controller, and receives the images collected by the vision laser detector and processes the images to generate point cloud data.

[0008] A calibration method for an automated three-dimensional laser scanning system includes the following steps:

[0009] Step 1: Respectively use the Zhang-Zhengyou calibration method for each of the vision laser detectors to obtain the equivalent focal length of the lens, the projection point coordinates of the lens optical axis on the image, and the distortion coefficient of the lens, use the binocular calibration method to obtain the pose transformation matrix between the binocular cameras of each of the vision laser detectors, and the plane equation of the laser line.

[0010] Step 2: Select one camera from each of the vision laser detectors in two adjacent vision laser detectors to form a binocular camera imaging system, and use the binocular calibration method to obtain the pose transformation matrix between the binocular cameras in the binocular camera imaging system as the pose transformation matrix of the two adjacent vision laser detectors; the two adjacent vision laser detectors are two vision laser detectors that are adjacent in spatial position on the gantry and have a common field of view;

[0011] For each of the two adjacent vision laser detectors in the system, the above method is used to obtain the pose transformation matrix of each of the two adjacent vision laser detectors.

[0012] Step 3: Set the tracking object on the guide rail. The guide rail drives the tracking object to move linearly. Select any one of the multiple vision laser detectors to obtain the motion trajectory coordinates of the tracking object in the coordinate system of this vision laser detector in real time. After the guide rail movement ends, perform linear fitting on the motion trajectory coordinates, and use the direction vector of the straight line as the motion direction vector of the guide rail;

[0013] The tracking object is an object with multiple feature points on its surface. The feature points are points that can be recognized by the vision laser detector. The coordinates of all feature points on the tracking object in the tracking object coordinate system are known.

[0014] A scanning method for an automated three-dimensional laser scanning system includes the following steps:

[0015] Step 1: The computer controls the movement of the guide rail through the controller, controls each vision laser detector to obtain the local point cloud on the surface of the object to be scanned in the coordinate system of each vision laser detector, and saves it to the computer;

[0016] Step 2: The computer calculates the coordinates of the local point cloud on the surface of the object to be scanned in each vision laser detector coordinate system in the world coordinate system according to the pose transformation matrix of each adjacent two vision laser detectors and the motion direction vector of the guide rail, and saves them to the point cloud library, thereby implementing a scanning method for an automated three-dimensional laser scanning system.

[0017] Preferably, in Step 1, all vision laser detectors work only once in one cycle, and any two vision laser detectors do not work simultaneously. After completing one cycle of cyclic work, it automatically enters the cyclic work of the next cycle until all vision laser detectors are turned off after scanning ends.

[0018] Preferably, the method for calculating the coordinates of the local point cloud on the surface of the object to be scanned in the world coordinate system in Step 2 is as follows:

[0019] Q ij =P ij ×T i -V×D ij +V×L

[0020] In the formula, P ij is the coordinate of the jth group of local point clouds on the surface of the object to be scanned collected by the ith vision laser detector in the coordinate system of the ith vision laser detector, Q ij is the coordinate of the jth group of local point clouds on the surface of the object to be scanned collected by the ith vision laser detector in the world coordinate system, T i$T_{i}$ is the pose transformation matrix from the coordinate system of the $i$-th vision laser detector to the world coordinate system, $V$ is the motion direction vector of the guide rail in the world coordinate system, and $D$ ij is the motion distance of the guide rail when the $i$-th vision laser detector collects the local point cloud of the $j$-th scanned object surface, and $L$ is the motion stroke of the guide rail during the scanning process.

[0021] The beneficial effects of the present invention are as follows: Compared with the photographic three-dimensional scanner, the present invention can complete three-dimensional reconstruction during the movement of the scanned object, and the detection efficiency can fully meet the automated detection requirements of the production line. Moreover, the present invention uses line laser as the scanning light source, which can greatly increase the scanning area and depth of field, thereby effectively reducing the number of vision laser detectors used and achieving the purpose of cost reduction; compared with the line laser three-dimensional scanner, the vision laser detector of the present invention uses the hardware design of a binocular camera, which not only reduces the requirements for hardware stability, but also can achieve multi-line laser scanning, doubling the scanning efficiency; at the same time, the present invention is easy to operate, only need to place the scanned object on the guide rail or conveyor belt, and is suitable for wide use on the automated production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic structural diagram of an automated three-dimensional laser scanning system of the present invention.

[0023] Figure 2 is a flowchart of a calibration method for an automated three-dimensional laser scanning system of the present invention.

[0024] Figure 3 is a schematic structural diagram of a tracking object (pasted with circular feature points) in the calibration method of the automated three-dimensional laser scanning system of the present invention

[0025] Figure 4 is a flowchart of a scanning method for an automated three-dimensional laser scanning system of the present invention.

[0026] In the figure: 1. Computer, 2. Controller, 3. Scanned object, 4. Guide rail, 5. Gantry, 6. Vision laser detector, 7. Vision laser detector, 8. Vision laser detector, 9. Vision laser detector. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The present invention will be further described in detail below with reference to the drawings and embodiments.

[0028] As Figure 1As shown in the figure, an automated three-dimensional laser scanning system includes: a gantry 5, visual laser detectors 6, 7, 8, 9, a guide rail 4, an object to be scanned 3, a controller 2, and a computer 1; the visual laser detectors 6, 7, 8, 9 are composed of a binocular camera and a laser that can emit one or more laser lines; the gantry 5 is fixed on the ground, and the multiple visual laser detectors 6, 7, 8, 9 are distributed on the gantry 5; the guide rail 4 is arranged directly below the gantry 5, the object to be scanned 3 is arranged on the guide rail 4, and the guide rail 4 drives the object to be scanned 3 to move in a straight line; the computer 1 controls the opening and closing of the visual laser detectors 6, 7, 8, 9 and the opening and closing of the guide rail 4 through the controller 2, and receives the images collected by the visual laser detectors 6, 7, 8, 9 and processes the images to generate point cloud data.

[0029] A calibration method based on an automated three-dimensional laser scanning system, as Figure 2 shown, the method includes the following steps:

[0030] Step 1: Respectively use the Zhang Zhengyou calibration method for each of the visual laser detectors 6, 7, 8, 9 to obtain the equivalent focal length of the lens, the coordinates of the projection points of the lens optical axis on the image, and the distortion coefficients of the lens. Use the binocular calibration method to obtain the pose transformation matrix between the binocular cameras of each of the visual laser detectors 6, 7, 8, 9, and the plane equation of the laser line.

[0031] Step 2: Select one camera from each of the visual laser detectors 6, 7, 8, 9 in two adjacent visual laser detectors 6, 7, 8, 9 to form a binocular camera imaging system. Use the binocular calibration method to obtain the pose transformation matrix between the binocular cameras in the binocular camera imaging system as the pose transformation matrix of the two adjacent visual laser detectors 6, 7, 8, 9; it should be noted that the two adjacent visual laser detectors 6, 7, 8, 9 are two visual laser detectors 6, 7, 8, 9 that are adjacent in space position on the gantry 5 and have a common field of view. As Figure 1 shown, the visual laser detector 9 and the visual laser detector 7 are two adjacent visual laser detectors. Similarly, the visual laser detector 6 and the visual laser detector 7, and the visual laser detector 6 and the visual laser detector 8 also form two adjacent visual laser detectors;

[0032] For each of the two adjacent visual laser detectors 6, 7, 8, 9 in the system, the above method is used to obtain the pose transformation matrix of each of the two adjacent visual laser detectors 6, 7, 8, 9.

[0033] Step 3: Set the tracking object on the guide rail 4. The guide rail 4 drives the tracking object to move linearly. Select any one of the multiple vision laser detectors 6, 7, 8, 9 to obtain the motion trajectory coordinates of the tracking object in the coordinate system of this vision laser detector 6, 7, 8, 9 in real time. After the movement of the guide rail 4 ends, perform linear fitting on the motion trajectory coordinates, and use the direction vector of the straight line as the motion direction vector of the guide rail 4;

[0034] The tracking object is an object with multiple feature points on its surface. The feature points are points that can be recognized by the vision laser detectors 6, 7, 8, 9. The coordinates of all the feature points on the tracking object in the tracking object coordinate system are known. Figure 3 It is a schematic structural diagram of a tracking object. The tracking object is a flat plate pasted with circular feature points. Placing this flat plate on the moving slider of the guide rail 4 can be used to obtain the motion trajectory coordinates of the tracking object in the coordinate system of the vision laser detector 6, 7, 8, 9 in real time.

[0035] Based on a scanning method of an automated three-dimensional laser scanning system, as Figure 4 shown, the method includes the following steps:

[0036] Step 1: The computer 1 controls the movement of the guide rail 4 through the controller 2, controls each of the vision laser detectors 6, 7, 8, 9 to obtain the local point cloud on the surface of the scanned object 3 in the coordinate system of each vision laser detector 6, 7, 8, 9, and saves it to the computer 1;

[0037] Step 2: The computer 1 calculates the coordinates of the local point cloud on the surface of the scanned object 3 in each vision laser detector 6, 7, 8, 9 coordinate system in the world coordinate system according to the pose transformation matrix of each adjacent two vision laser detectors 6, 7, 8, 9 and the motion direction vector of the guide rail 4, and saves them to the point cloud library, realizing a scanning method of an automated three-dimensional laser scanning system.

[0038] In Step 1, all the vision laser detectors 6, 7, 8, 9 work only once in a cycle, and any two vision laser detectors 6, 7, 8, 9 do not work simultaneously. After completing a cycle of cyclic work, it automatically enters the cyclic work of the next cycle until the scanning ends and all the vision laser detectors 6, 7, 8, 9 are turned off.

[0039] The method for calculating the coordinates of the local point cloud on the surface of the scanned object 3 in the world coordinate system in Step 2 is as follows:

[0040] Q ij =P ij ×Ti -V×D ij +V×L

[0041] wherein, P ij is the coordinate of the local point cloud on the surface of the scanned object 3 in the j-th group collected by the i-th vision laser detector 6, 7, 8, 9 in the coordinate system of the i-th vision laser detector 6, 7, 8, 9, and Q ij is the coordinate of the local point cloud on the surface of the scanned object 3 in the j-th group collected by the i-th vision laser detector 6, 7, 8, 9 in the world coordinate system, and T i is the pose transformation matrix from the coordinate system of the i-th vision laser detector 6, 7, 8, 9 to the world coordinate system, V is the motion direction vector of the guide rail 4 in the world coordinate system, and D ij is the motion distance of the guide rail 4 when the i-th vision laser detector 6, 7, 8, 9 collects the local point cloud on the surface of the scanned object 3 in the j-th group, and L is the motion stroke of the guide rail 4 during the scanning process.

[0042] In the above technical solution, when creating the world coordinate system, it is necessary to ensure that the pose transformation matrix between the world coordinate system and the coordinate systems of the vision laser detectors 6, 7, 8, 9 can be accurately described in the form of a mathematical matrix; the motion distance D ij of the guide rail 4 when the i-th vision laser detector 6, 7, 8, 9 collects the j-th group of local point clouds can be obtained through encoder feedback or calculated according to the working time interval of the vision laser detectors 6, 7, 8, 9 and the speed of the guide rail 4.

Claims

1. An automated three-dimensional laser scanning system, characterized in that: The system includes: a gantry, multiple visual laser detectors, a guide rail, a scanned object, a controller and a computer; the visual laser detector is composed of a binocular camera and a laser that can emit one or more laser lines; the gantry is fixed on the ground, and the multiple visual laser detectors are distributed on the gantry; the guide rail is arranged directly below the gantry, the scanned object is arranged on the guide rail, and the guide rail drives the scanned object to move in a straight line; the computer controls the opening and closing of the visual laser detector and the opening and closing of the guide rail through the controller, and receives images collected by the visual laser detector to process the images to generate point cloud data.

2. A calibration method for an automated three-dimensional laser scanning system according to claim 1, characterized in that: The steps include: Step 1: Use Zhang Zhengyou calibration method to obtain the equivalent focal length of the lens, the projection point coordinates of the lens optical axis on the image and the distortion coefficient of the lens for each of the visual laser detectors, and use binocular calibration method to obtain the pose transformation matrix between the binocular cameras of each of the visual laser detectors, as well as the plane equation of the laser line; Step 2: Select one camera from each of the two adjacent visual laser detectors to form a binocular camera imaging system, and use a binocular positioning method to obtain a pose transformation matrix between binocular cameras in the binocular camera imaging system as the pose transformation matrix of the two adjacent visual laser detectors; the two adjacent visual laser detectors are two visual laser detectors that are adjacent in space on the gantry and have a common field of view; The above method is used for each of the two adjacent visual laser detectors in the system to obtain the posture transformation matrix of each of the two adjacent visual laser detectors. Step 3: The tracking object is set on the guide rail, and the guide rail drives the tracking object to move in a straight line. One visual laser detector is selected from the multiple visual laser detectors to obtain the motion trajectory coordinates of the tracking object in the coordinate system of the visual laser detector in real time. After the guide rail moves, the motion trajectory coordinates are fitted with a straight line, and the direction vector of the straight line is used as the motion direction vector of the guide rail. The tracking object is an object with multiple feature points attached to its surface, and the feature points are points that can be recognized by the visual laser detector. The coordinates of all the feature points on the tracking object in the tracking object coordinate system are known.

3. A scanning method based on an automated three-dimensional laser scanning system according to claim 1, characterized in that: The steps include: Step 1: The computer controls the movement of the guide rail through the controller, controls each of the visual laser detectors to obtain a local point cloud of the scanned object surface in the coordinate system of each of the visual laser detectors and saves it to the computer; Step 2: The computer calculates the coordinates of the local point cloud of the scanned object surface in the coordinate system of each visual laser detector described in step 1 in the world coordinate system according to the posture transformation matrix of each of the two adjacent visual laser detectors and the motion direction vector of the guide rail, and saves them in the point cloud library, thereby realizing a scanning method for an automated three-dimensional laser scanning system.

4. The scanning method according to claim 3, characterized in that: In the step 1, all visual laser detectors work only once in one cycle, and any two visual laser detectors do not work at the same time. After completing one cycle of cyclic work, they automatically enter the next cycle of cyclic work until the scanning is completed and all visual laser detectors are turned off.

5. The scanning method according to claim 3, characterized in that: The method for calculating the coordinates of the local point cloud of the scanned object surface in each visual laser detector coordinate system in the world coordinate system in step 2 is: Q ij =P ij ×T i -V×D ij +V×L Where P ij is the coordinate of the jth group of local point clouds of the scanned object surface collected by the i-th visual laser detector in the i-th visual laser detector coordinate system, Q ij is the coordinates of the jth group of local point clouds of the scanned object surface collected by the i-th visual laser detector in the world coordinate system, T i is the pose transformation matrix from the i-th visual laser detector coordinate system to the world coordinate system, V is the motion direction vector of the guide rail in the world coordinate system, D ij is the movement distance of the guide rail when the i-th visual laser detector collects the j-th group of local point clouds on the surface of the scanned object, and L is the movement distance of the guide rail during the scanning process.

Images