Automated three-dimensional laser scanner and scanning method thereof

Abstract

The present invention discloses an automated three-dimensional laser scanner and a scanning method thereof, which belongs to the field of computer vision technology and overcomes the problem that the reflection and absorption of the object surface affect the scanning results and the problem that the scanner cannot be promoted for use on automated production lines. The scanner comprises: a laser group, a binocular camera, a turntable, a controller and a computer. Compared with the photographic three-dimensional scanner based on grating projection, the present invention can overcome the influence of the reflective and absorbing object surface on the scanning effect because it uses a line laser as the scanning light source, and it also has better adaptability to harsh working environments; compared with the handheld three-dimensional scanner based on multi-line laser, the present invention can obtain complete point cloud data of the object to be measured when the positions of the scanner and the object to be measured are relatively fixed, and there is no need to stick marker points in advance or manual participation during scanning, so it can be widely used on automated production lines.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to an automated three-dimensional laser scanner and a scanning method thereof. Background Art

[0002] For a long time, the precise measurement of the geometric dimensions of real objects has mainly relied on contact-based manual measurement methods that rely on measuring tools such as vernier calipers, micrometers, and angle rulers. This measurement method cannot measure the surface of objects with irregular shapes and also has technical bottlenecks for non-contact measurement objects such as cultural relics and historical sites. With the development of related disciplines and driven by new technologies and new demands, three-dimensional scanning technology has emerged. This technology can calculate the spatial distribution of point clouds based on the collected surface information of the object without contacting the object being measured. Through a series of surface reconstruction methods, the point cloud data is integrated into a triangular mesh model of the object being measured in the computer. This technology is widely used in auxiliary manufacturing and inspection in the field of industrial design, surgical positioning and rehabilitation in the medical field, real-scene modeling and simulation in the field of game entertainment, and site protection and restoration in the field of archaeology.

[0003] At present, the three-dimensional scanning technologies used in the measurement field mainly include photographic three-dimensional scanners based on grating projection and handheld three-dimensional scanners based on multi-line lasers. Among them, the photographic three-dimensional scanner uses a projector to continuously project multiple light and dark stripes with optical coding characteristics onto the surface of the object to be measured, and uses a binocular camera to synchronously collect the reflected light of each light and dark stripe on the surface of the object, and uniformly optically decodes all collected images to obtain three-dimensional point cloud data. This type of technology requires that the position of the scanner and the object to be measured is relatively fixed during scanning, and is an automated three-dimensional scanner. However, the optical power of the projector is generally low. When scanning reflective objects and dark objects, the light will be reflected and absorbed, resulting in a large amount of optical power loss and an inability to present a clear image in the camera, which in turn leads to the inability to obtain high-quality point cloud data. Handheld 3D scanners that use lasers as light sources have high optical power and can effectively overcome the influence of reflective and light-absorbing objects on the scanning effect. This type of technology requires sticking marking points on the surface of the object to be measured. During the scanning process, the operator holds the scanner and traverses the surface of the object to be measured to obtain complete point cloud data. Since this type of technology requires sticking marking points before scanning and the scanning process requires manual participation, it is only suitable for scanning large objects or sampling inspections, and cannot be promoted and used on automated production lines.

[0004] With the advancement of industrial automation and intelligence, the demand for automated 3D inspection of production lines is increasing. In addition, the surfaces of industrial products generally have reflective and light-absorbing properties. Therefore, the inspection field urgently needs an automated 3D scanner that can effectively solve the reflection and light absorption problems. Summary of the Invention

[0005] In order to solve the problems existing in the prior art, the present invention provides an automated three-dimensional laser scanner and a scanning method thereof, which overcomes the problem that reflection and absorption of the object surface affect the scanning results and the problem that the scanner cannot be promoted and used on automated production lines.

[0006] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0007] An automated three-dimensional laser scanner comprises: a laser group, a binocular camera, a turntable, a controller, and a computer; the laser group is disposed on the turntable, which drives the laser group to perform a swinging motion with the turntable's rotation axis as the central axis; the binocular cameras are fixed to both ends of the turntable and collect images projected by the laser group onto an object and reflected back; the computer controls the laser group to be turned on or off, controls the binocular camera to collect the images, controls the turntable's rotation angle, receives the images collected by the binocular camera, processes the images, and generates point cloud data through the controller.

[0008] Preferably, the binocular cameras are fixed to both ends of the turntable through a supporting structure.

[0009] A scanning method for an automated three-dimensional laser scanner, the method comprising the following steps:

[0010] Step 1: Calculate and set the speed and range of the electronically controlled turntable;

[0011] Step 2: The computer controls the turntable to rotate through the controller, and the laser group emits laser light onto the object to be measured. The computer controls the laser group to project laser light onto the object through the controller, and the reflected image is captured by the binocular camera to obtain multiple stereo image pairs.

[0012] Step 3: The computer calculates the laser line calibration surface equation at each moment of the stereo image pair acquisition described in step 2;

[0013] Step 4: Using the laser line calibration surface equation described in step 3, calculate the three-dimensional coordinate value of the laser line in each of the stereo image pairs and save it to the point cloud library, thereby realizing an automated three-dimensional laser scanning method.

[0014] Preferably, the speed of the turntable in step 1 is:

[0015]

[0016] Where F is the acquisition frame rate of the automated 3D laser scanner, r is the set point cloud resolution, and D is the maximum scanning distance of the automated 3D laser scanner;

[0017] The range of the turntable in step 1 is that all laser lines projected by the laser group appear simultaneously in the stereo image pair. Preferably, the laser line calibration surface equation in step 3 is the plane equation formed by the laser line in space in the camera coordinate system; the laser line calibration surface equation when the turntable is at zero position is calibrated and known. When the laser group rotates with the turntable, the laser line calibration surface equation changes in real time. Assuming that the laser line calibration surface equation when the turntable is at zero position is A0(X-x0)+B0(Y-y0)+C0(Z-z0)=0, during the rotation of the turntable, at the kth moment, the laser line calibration surface equation is A k (Xx k )+B k (Yy k )+C k (Zz k )=0, the relationship between the two calibration surface equation parameters is:

[0018]

[0019]

[0020] Where R k is the rotation matrix of the turntable relative to the zero position at time k, (P x , P y , P z ) is a point on the axis of the turntable in the camera coordinate system; A0(X-x0)+B0(Y-y0)+C0(Z-z0)=0 is the normal equation of the plane at zero position, and the normal vector A k (Xx k )+B k (Yy k )+C k (Zz k )=0 is the point normal equation of the plane at time k, and the normal vector

[0021] The rotation matrix R of the turntable relative to the zero position at time k k The calculation formula is:

[0022]

[0023] Where, (N x , N y , N z ) is the vector of the turntable axis in the camera coordinate system, and θ is the rotation angle of the turntable relative to the zero position at time k.

[0024] Preferably, in step four, the step of calculating the three-dimensional coordinate value of the laser line in each of the stereo image pairs is as follows: extracting a plurality of two-dimensional laser line patterns from the left and right viewpoint images of each of the stereo image pairs respectively; in the polar plane, any point on the two-dimensional laser line selected from the left and right viewpoint images respectively can be reconstructed into a three-dimensional space point; only points whose distance to the laser line calibration surface is less than a threshold value are saved in the point cloud library.

[0025] The beneficial effects of the present invention are: compared with the photographic 3D scanner based on grating projection, the present invention can overcome the influence of reflective and light-absorbing object surfaces on the scanning effect due to the use of line laser as the scanning light source, and it also has better adaptability to harsh working environments; compared with the handheld 3D scanner based on multi-line laser, the present invention can obtain complete point cloud data of the object to be measured when the positions of the scanner and the object to be measured are relatively fixed, and there is no need to stick marker points in advance or manual participation during scanning, and can be widely used on automated production lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic structural diagram of an automated three-dimensional laser scanner according to the present invention.

[0027] Figure 2 A flow chart of an automated three-dimensional laser scanning method of the present invention.

[0028] Figure 3 The present invention is an original left viewpoint image in a stereo image pair.

[0029] Figure 4 The original right viewpoint image in a stereo image pair of the present invention

[0030] Figure 5 The present invention Figure 3 The extracted two-dimensional laser line pattern.

[0031] Figure 6 The present invention Figure 4 The extracted two-dimensional laser line pattern.

[0032] In the figure: 1. Parallel laser lines, 2. Binocular camera, 3. Laser group, 4. Turntable, 5. Controller, 6. Support structure, 7. Computer. DETAILED DESCRIPTION

[0033] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0034] like Figure 1As shown, an automated three-dimensional laser scanner includes: a laser group 3, a binocular camera 2, a turntable 4, a controller 5, a support structure 6, and a computer 7; the laser group 3 is arranged on the turntable 4, and the turntable 4 drives the laser group 3 to perform a swinging motion with the rotation axis of the turntable 4 as the central axis; the binocular camera 2 is fixed to both ends of the turntable 4 through the support structure 6, and collects images projected by the laser group 3 onto the object to be measured and reflected back; the computer 7 controls the opening or closing of the laser group 3 through the controller 5, controls the binocular camera 2 to collect the images, controls the rotation angle of the turntable 4, and receives the images collected by the binocular camera 2, processes the images, and generates point cloud data.

[0035] Based on a scanning method of an automated 3D laser scanner, such as Figure 2 As shown, the method includes the following steps:

[0036] Step 1: Calculate and set the rotation speed and rotation range of the turntable 4; wherein the speed of the turntable 4 in step 1 is:

[0037]

[0038] F is the acquisition frame rate of the automated 3D laser scanner, r is the set point cloud resolution, and D is the maximum scanning distance of the automated 3D laser scanner; the rotation range of the turntable 4 is determined by the hardware of the automated 3D laser scanner, and it must be ensured that within the rotation range of the turntable 4, all parallel laser lines 1 projected by the laser group 3 installed on the turntable 4 can appear simultaneously in the stereo image pair.

[0039] Step 2: The computer 7 controls the turntable 4 to rotate through the controller 5, and the laser group 3 emits a parallel laser line 1 onto the object to be measured. The computer 7 controls the laser group 3 through the controller 5 to project the reflected image onto the object to be measured, and the binocular camera 2 captures the reflected image to obtain multiple stereo image pairs;

[0040] Step 3: The computer 7 calculates the laser line calibration surface equation at each moment of collecting the stereo image pair described in step 2; the laser line calibration surface equation is the plane equation formed by the laser line in space in the camera coordinate system; the laser line calibration surface equation when the turntable 4 is at zero position is calibrated and known, and when the laser group rotates with the turntable, the laser line calibration surface equation changes in real time, and the laser line calibration surface equation when the turntable is at zero position is A0(X-x0)+B0(Y-y0)+C0(Z-z0)=0. During the rotation of the turntable, at the kth moment, the laser line calibration surface equation is A k (Xx k )+B k (Yyk )+C k (Zz k )=0, the relationship between the two calibration surface equation parameters is:

[0041]

[0042]

[0043] Where R k is the rotation matrix of the turntable relative to the zero position at time k, (P x , P y , P z ) is the point on the axis of the turntable in the camera coordinate system, A0(X-x0)+B0(Y-y0)+C0(Z-z0)=0 is the point normal equation of the plane at zero position, and the normal vector A k (Xx k )+B k (Yy k )+C k (Zz k )=0 is the point normal equation of the plane at time k, and the normal vector

[0044] The rotation matrix R of the turntable relative to the zero position at time k k The calculation formula is:

[0045]

[0046] Where, (N x , N y , N z ) is the vector of the turntable axis in the camera coordinate system, and θ is the rotation angle of the turntable relative to the zero position at time k.

[0047] Step 4: Using the laser line calibration surface equation described in step 3, calculate the three-dimensional coordinate value of the laser line in each of the stereo image pairs and save it to the point cloud library to realize an automated three-dimensional laser scanning method; the step of calculating the three-dimensional coordinate value of the laser line in each of the stereo image pairs is as follows: extract multiple two-dimensional laser line patterns from the left viewpoint and right viewpoint images of each of the stereo image pairs, wherein Figure 3 is the original left view image, Figure 4 is the original right view image, Figure 5 for Figure 3 The two-dimensional laser line pattern extracted from Figure 6 for Figure 4Based on the epipolar constraint and triangulation principle, any point on the 2D laser line extracted from the left and right viewpoint images can be randomly selected in the epipolar plane to reconstruct a 3D spatial point. Only points whose distance to the laser line calibration surface is less than a threshold are saved in the point cloud library, and points exceeding the threshold are discarded.

Claims

1. A scanning method of an automated three-dimensional laser scanner, characterized in that: The scanner includes: a laser group, a binocular camera, a turntable, a controller, and a computer; the laser group is arranged on the turntable, and the turntable drives the laser group to perform a swinging motion with the turntable's rotation axis as the central axis; the binocular cameras are respectively fixed at both ends of the turntable to collect images projected by the laser group onto the object to be measured and reflected back; the computer controls the laser group to be turned on or off through the controller, controls the binocular camera to collect the images, controls the rotation angle of the turntable, receives the images collected by the binocular camera, processes the images, and generates point cloud data; The method comprises the following steps: Step 1: Calculate and set the speed and range of the turntable; The speed of the turntable in step 1 is: Where F is the acquisition frame rate of the automated 3D laser scanner, r is the set point cloud resolution, and D is the maximum scanning distance of the automated 3D laser scanner; The range of the turntable in step 1 is such that all laser lines projected by the laser group appear simultaneously in the stereoscopic image pair; Step 2: The computer controls the turntable to rotate through the controller, and the laser group emits laser light onto the object to be measured. The computer controls the laser group to project laser light onto the object to be measured, and the reflected image is captured by the binocular camera to obtain multiple stereo image pairs. Step 3: The computer calculates the laser line calibration surface equation at each moment of the stereo image pair acquisition described in step 2; The laser line calibration surface equation in step 3 is the plane equation formed by the laser line in space in the camera coordinate system; the laser line calibration surface equation when the turntable is at zero position is calibrated and known. When the laser group rotates with the turntable, the laser line calibration surface equation changes in real time. Assuming that the laser line calibration surface equation when the turntable is at zero position is A0(X-x0)+B0(Y-y0)+C0(Z-z0)=0, during the rotation of the turntable, at the kth moment, the laser line calibration surface equation is A k (Xx k )+B k (Yy k )+C k (Zz k )=0, the relationship between the two calibration surface equation parameters is: Where R k is the rotation matrix of the turntable relative to the zero position at time k, (P x , P y , P z ) is the point on the axis of the turntable in the camera coordinate system, A0(X-x0)+B0(Y-y0)+C0(Z-z0)=0 is the point normal equation of the plane at zero position, and the normal vector A k (Xx k )+B k (Yy k )+C k (Zz k )=0 is the point normal equation of the plane at time k, and the normal vector The rotation matrix R of the turntable relative to the zero position at time k k The calculation formula is: Where, (N x , N y , N z ) is the vector of the turntable axis in the camera coordinate system, and θ is the rotation angle of the turntable relative to the zero position at time k; Step 4: Using the laser line calibration surface equation described in step 3, calculate the three-dimensional coordinate value of the laser line in each of the stereo image pairs and save it to the point cloud library, thereby realizing an automated three-dimensional laser scanning method.

2. The scanning method according to claim 1, wherein: The binocular cameras are fixed to both ends of the turntable through a supporting structure.

3. The scanning method according to claim 1, wherein: In step four, the step of calculating the three-dimensional coordinate value of the laser line in each of the stereo image pairs is as follows: a plurality of two-dimensional laser line patterns are extracted from the left and right viewpoint images of each of the stereo image pairs, and in the polar plane, any point on the two-dimensional laser line selected from the left and right viewpoint images can be reconstructed into a three-dimensional space point, and only points whose distance to the laser line calibration surface is less than a threshold are saved in the point cloud library.

Images