Method for determining the current position and / or orientation of a laser radar relative to an object under measurement.

Abstract

The present disclosure is directed to a method for determining a current position and / or orientation of a laser radar (3) relative to an object (2) by a computer. The laser radar is fitted with a camera system (11, 11'). In the method, the position of the camera system is determined by acquiring one or more images of the object. At least three reference targets (12) having a fixed spatial relationship with the target are observed in the one or more images. The relative spatial relationship and / or position of the at least three reference targets on the object relative to each other is retrieved from a storage device. The position of the reference targets relative to the camera is calculated using the relative spatial relationship of the at least three reference targets relative to each other and the observed spatial orientations of the aforementioned reference targets defined with respect to the camera fixed coordinate system. Furthermore, the spatial relationship between the camera system and the laser radar is retrieved from the storage device. The spatial position and / or orientation of the object relative to the laser radar is calculated using the position and orientation of the object relative to the camera system and the spatial relationship between the camera system and the laser radar.

Description

Technical Field

[0001] The present invention relates to a method for determining the current position and / or orientation of a measurement system relative to a measured object using a measurement system including a lidar.

Background Art

[0002] Lidar is known in the prior art and may be a device for measuring XYZ coordinates with a computer-controlled steerable laser beam. The distance to a point of interest is determined by some characteristic of the reflected laser beam from the aforementioned point, such as a phase shift. The direction to the point of interest is determined from the current orientation of a beam steering mechanism, typically a mirror or prism rotatable about two perpendicular axes.

[0003] As shown in FIG. 1, the XYZ coordinates of point 1 on object 2 can be calculated from the distance and direction information from lidar 3. On top of the lidar body 5, a steerable mirror or prism 4 that directs beam 6 towards point 1 and records range 7 is shown. The XYZ values of point 1 are determined from the readings of the vertical angle 8 and horizontal angle 9 of the mirror or prism relative to the body 5, as well as the range or distance data from the laser rangefinder.

[0004] Lidar, also called LIDAR or LADAR, is commercially available and is commonly used to very quickly scan objects and surfaces in industries, construction, civil engineering, and other industries, i.e., to very quickly measure a large number of points of interest.

[0005] Lidar typically directs their laser beams over a wide range of angles and can record such angles with high precision, for example, well covering the inside of a cuboid. However, lidar typically relies on a direct line of sight and thus cannot measure any part of an object that is not in direct line of sight from the current position. For example, to cover the entire outer surface of a stationary cuboid, the lidar has to be moved.

[0006] The XYZ data of a laser radar may be relative to the position of the device itself, and if the device is moved from a known position and orientation, accurate information of the new position and orientation is required in order to correlate the data acquired before and after the move into a common coordinate system.

[0007] Robots are also known in the prior art and can be understood herein as computer-controlled mechanical mechanisms capable of moving a mechanical interface to a number of positions and orientations. The aforementioned mechanical interface used to mount a tool or other device to be moved is often called a tool center point (TCP). Various robotic embodiments exist that provide TCP motion with a variable number of degrees of freedom using means of linear motion, rotational motion, or a combination thereof. The position and orientation of the TCP at a given time are usually determined by simultaneous readings from a set of sensors or encoders assigned to each of the axial or linear motions.

[0008] As shown in Figure 2, when the radar unit is moved around object 2 by robot 10, it is advantageous that robot 10 can be used to measure a larger portion of the object than would be possible from a single position. The robot-mounted laser radar can combine information from the laser radar with information from the robot controller to determine a consistent set of XYZ data from any area of ​​the object. Unfortunately, while most robots have sufficient precision for industrial tasks such as painting, welding, and assembly, only special and very expensive robots have the precision required to provide sufficiently accurate movement information without significantly degrading the overall accuracy of the XYZ data from the laser radar moved by the robot. High-precision robots are also often of a moderate size, have limited degrees of freedom, cannot operate accurately in the extended temperature range encountered on production lines, and may require frequent maintenance or calibration, which is unacceptable in a production environment.

[0009] When using a laser radar moved by a robot for accurate measurements, it is common to use the laser radar itself to determine its current position and orientation. This is typically done by measuring at least three features (often tooling balls) at known positions on or around the object after each robot has moved.

[0010] However, this method has drawbacks, for example, when used within automotive manufacturing lines and similar applications. These have limited cycle times, meaning the object being measured is stationary for only a finite amount of time, typically 30-90 seconds. Manufacturing cells intended for dimensional monitoring need to measure as many features as possible on the object during the available cycle time. Therefore, the process of using the laser radar to determine its current position and orientation after each movement limits the applicability of the laser radar. For example, to measure a single feature in a hard-to-reach area of ​​a vehicle body, such as a slot, it may be necessary to move the laser radar to a dedicated position, then measure at least three known features, such as tooling balls, to determine the precise position of the laser radar, and then measure the slot itself. Clearly, a significant portion of the time is lost in the preparation process steps rather than measuring the desired object feature. This is particularly relevant for objects requiring multiple robotic movements, such as an entire automobile body, where this challenge significantly limits the applicability of the laser radar.

[0011] From prior art, it is known that the position and orientation of a robot-mounted surface measuring device, such as an optical scanner, can be determined very quickly by firmly mounting the scanner within a navigation frame and embedding numerous targets on the aforementioned frame so that at least three large, separated targets are reliably visible from any direction. The frame can then be observed by multiple cameras or pairs of cameras operating in the object's coordinate system, and thus, by observing the navigation frame, the current position and orientation of the scanner can be determined. Systems operating as described herein are commercially available, and the same technique can be adapted to solve the laser radar problems described above.

[0012] However, while such a "navigation frame" solution offers the advantage that the scanner can be observed from any direction by the camera, the scanner's field of view is fixed and requires a true omnidirectional tracking solution, so the solution becomes costly to implement because many targets and many cameras are usually required to ensure the coverage needed in any orientation. Laser radar, on the other hand, has a maneuverable laser beam, making it suitable for measuring a given feature over a wider range of position and orientation, and a truly omnidirectional solution is not required.

[0013] Furthermore, laser radar is known to exhibit some variation in its internal coordinate system as a function of orientation. These specifications are typically valid only within a given range of orientations, and beyond this range, their internal coordinate system may exhibit an offset, as may be required for access in, for example, automotive manufacturing cells.

[0014] When the laser radar itself is used to measure a reference target that enters the object's coordinate system, this effect is largely eliminated because the reference target defining the object's coordinate system is also measured in this potentially offset coordinate system, and subsequent laser radar measurements taken from the same position are offset by the same amount, compensating for the error. However, when the navigation frame method is applied to the laser radar, this error is not compensated for, and accuracy is reduced. Therefore, adapting known methods for high-speed device navigation does not provide optimal results for laser radar.

[0015] U.S. Patent Application Publication No. 2014 / 0211999 describes a measurement system comprising a laser tracker, an image detection unit, and a contact detection tool. The contact detection tool has six light-emitting diodes and one reflector. The image detection unit uses the light-emitting diodes to determine the precise orientation of the contact detection tool with six degrees of freedom, and the laser tracker measures the distance to the reflector. Instead of the laser radar described above and below, U.S. Patent Application Publication No. 2014 / 0211999 describes a laser tracker. European Patent No. 0607303 demonstrates that the relative position and orientation of an object including at least three base points and a camera observing those base points can be determined in all six degrees of freedom with respect to the camera's fixed internal coordinate system. The camera can measure the spatial direction to each of the base points and determine the position and orientation of the object including the base points. This requires that the base points are not collinear and that the positions of the base points are known relative to each other, i.e., that the base points are in a predetermined or known pattern. However, European Patent No. 0607303 does not show how errors can be compensated when moving the laser radar. [Overview of the project]

[0016] The technical problem to be solved is to overcome the aforementioned shortcomings. In particular, the technical problem to be solved is to provide rapid and accurate spatial measurement of objects. The technical problem to be solved is solved by the features of the independent claims.

[0017] One aspect of the present invention relates to a method for determining the current position and / or orientation of a laser radar relative to an object. The method may be performed by a computer. The laser radar is fitted with a camera system. In this method, the position and orientation of the camera system relative to the object are determined by acquiring images of one or more of at least three reference targets having a fixed spatial relationship with respect to the object. At least three reference targets are detected in the image. The relative spatial relationships of at least three reference targets are retrieved from memory. at least three The position and orientation of the reference targets are determined by the relative spatial relationship of at least three reference targets to each other, and the distance from the camera to each of the at least three reference targets. detection Using the spatial direction calculation Furthermore, the spatial relationship between the camera system and the laser radar is obtained from the storage device. The spatial position and orientation of an object relative to the laser radar are calculated using the object's position and orientation relative to the camera system and the spatial relationship between the camera system and the laser radar. The object is scanned with a laser radar.

[0018] This provides a method that enables a measurement system, including a laser radar, to accurately and quickly determine its position relative to an object. Since the camera system provides immediate measurement of the laser radar's current position and / or orientation relative to the object, inaccuracies in the robot moving the laser radar (and camera system) are avoided. Thus, even when multiple robot movements are required, a laser radar-based measurement system is provided that enables rapid but accurate XYZ measurements from a robot-mounted laser radar, making the use of laser radar in applications where it was previously unsuitable. This allows for the use of simple and inexpensive robots.

[0019] At least three reference targets are not aligned in a line. The storage device may be an electronic storage device having, for example, volatile or non-volatile memory. Data in the storage device may be retrieved from a server. The calculation of the position and / or orientation of the reference targets relative to the camera system may be in the coordinate system of the camera system. The storage device for the spatial relationship between the camera system and the laser radar may be the same storage device as described above, or an alternative and different storage device.

[0020] In a preferred embodiment, the method further includes the step of moving the camera system from a first position to a second position, where the second position is the current position, by a robot.

[0021] In a preferred embodiment, the current position and / or orientation of the laser radar is determined in accordance with the movement of the camera and laser radar by the robot. In a particularly preferred embodiment, the position and / or orientation is determined in accordance with the determination that the movement has ended. This ensures that the current position and / or orientation of the laser radar is always known. Additionally or alternatively, the position is determined during the movement by the robot.

[0022] In a preferred embodiment, the camera system is fixedly mounted to the beam steering mechanism of the laser radar. The beam steering mechanism is, in particular, a mirror or a prism. For example, the camera system may be mounted behind a mirror. The mirror may be inverted when positioning is performed by the camera. In this embodiment, it is easy to determine the current position and orientation of the camera system. In this case, the orientation of the mirror (or prism) affects the spatial relationship between the coordinate system of the laser radar and the camera system. The spatial relationship (e.g., a transformation matrix) can be calculated from the position offset of the camera system and the current orientation of the mirror.

[0023] In a preferred embodiment, the camera system is fixedly attached to the body of the lidar such that the spatial relationship between the camera system and the lidar is fixed. This may mean that the spatial relationship between the camera system coordinate system and the lidar coordinate system is fixed.

[0024] In a preferred embodiment, at least three targets are on the object. Specifically, at least three targets are attached to the object and are defined by known features of the object, i.e., by the corners of the object.

[0025] In a preferred embodiment, the camera system comprises a wide-angle lens camera and / or two or more cameras with different fields of view. This provides a wide field of view so that the camera system can record the targets.

[0026] Determining the aforementioned spatial relationship between the lidar and the camera system may involve measuring the same point or feature with both the camera system and the lidar and keeping both systems stationary. Due to the different natures of these two measurement systems, this may not be feasible, and instead, points or features in a known spatial relationship to each other can be measured.

[0027] In a preferred embodiment, the spatial relationship between the camera system and the lidar in the memory device is generated by determining the spatial relationship between the camera system and the lidar for one or more intended measurement positions of the robot, and the spatial relationship for that position can be used for subsequent measurements from the aforementioned intended measurement positions. This compensates, for example, for errors in the internal lidar coordinate system due to excessive tilt.

[0028] The spatial relationships referred to herein can refer to coordinate system transformations, particularly one or more coordinate transformation matrices. For example, when the spatial relationship between the camera system and the lidar is determined, a coordinate transformation matrix that enables the transformation of coordinates from the camera system to the coordinates of the lidar can be determined.

[0029] A reference target can be any physical entity or element suitable for accurate measurement by a measurement system.

[0030] One or more or all of the targets can include fiducial points. A fiducial point is defined as an element suitable for accurate measurement by a camera-based measurement system. The fiducial point may be an object placed within the field of view of the imaging system that appears in the created image for use as a reference point or a measurement point. This may be placed within or on the object.

[0031] Thus, the term target includes fiducial points, but some targets may not be fiducial points. Examples of targets include, for example, golf balls, nesting dolls, scribe lines, etc. Examples of fiducial points include, for example, photogrammetry markers and light-emitting diodes.

[0032] Fiducial points are suitable for measurement by a camera-based measurement system but may not be suitable for measurement by lidar. Physical targets are generally suitable for measurement by lidar but may not be suitable for measurement by a camera-based measurement system.

[0033] In a preferred embodiment, at least one of the reference targets comprises at least three fiducial points observable by a camera system and a physical element measurable by lidar, all of which are in known positions relative to each other. In an exemplary embodiment, at least one physical element may be a golf ball. Thereby, the camera system can quickly measure the reference target by observing the fiducial points, and the lidar can measure the physical element. Since the relationship between the fiducial point and the physical element is known, the aforementioned reference target is suitable for accurate measurement by both systems and enables both systems to determine the position of the center of the golf ball.

[0034] In an alternative, preferred embodiment, at least one, preferably at least three, of the targets include a sphere having a base point. The base point may be a light-emitting diode or an optical marker. The base point may be located at the center of the sphere, allowing the camera to determine the spatial orientation from one side to the aforementioned at least three reference targets. The aforementioned reference targets are used to allow the laser radar to measure its center by scanning the sphere. If the spatial position of the base point observed by the camera is known with respect to at least two other reference targets observed similarly, the spatial relationship between the camera system and the laser radar can be acquired before measurement.

[0035] In a preferred embodiment, the reference target is a nested object in which a spherically mounted base point is observable by a camera system, or a physical sphere of the same size is measurable by a laser radar.

[0036] A further aspect of the present invention relates to a method for determining measurements of an object, comprising the steps of determining the current position and / or orientation of the object as described above, and measuring one or more physical elements of the object. The physical elements may be measured after the spatial position and / or orientation of the laser radar relative to the object has been determined.

[0037] A further aspect of the present invention relates to a computer program that, when executed by a computer, includes instructions causing the computer to perform the method described above.

[0038] Further aspects of the present invention relate to a system comprising means for carrying out the above-described method. Preferably, the system comprises a camera system and a holder for mounting the camera on a laser radar. Optionally, the system also comprises a laser radar.

[0039] Non-limiting embodiments of the present invention will be described by reference to the accompanying drawings, merely as examples. [Brief explanation of the drawing]

[0040] [Figure 1] This shows a laser radar known from conventional technology. [Figure 2] This shows a laser radar equipped with a robot known from conventional technology. [Figure 3] A laser radar equipped with a camera according to a first embodiment of the present invention is shown. [Figure 4] A laser radar equipped with a camera according to a second embodiment of the present invention is shown. [Figure 5] A laser radar equipped with multiple cameras according to a third embodiment of the present invention is shown. [Modes for carrying out the invention]

[0041] Figure 3 shows one embodiment in which the camera 11 is fixedly mounted on a steering mechanism for the laser beam so that the camera moves together with the mirror or prism 4. For example, the camera 11 may be mounted on the back of the mirror 4. Figure 3 also shows a plurality of reference targets 12 positioned on or around the object to be measured 2. The object 2 may be, for example, a part of an automobile body.

[0042] When the laser radar 3 in Figure 3 is moved, the camera 11 detects the reference target 12 at a different position and orientation than before the movement, and therefore the camera's precise new position and orientation can be calculated. If the relationship between the camera's fixed internal coordinate system and the laser radar's fixed internal coordinate system is determined in advance, the laser radar's new position and orientation can also be calculated in relation to the object's coordinate system, and therefore all subsequent laser radar measurements can be converted to the same coordinate system as before the movement occurred without losing time for continuous laser radar measurements of the reference target.

[0043] In Figure 3, the relationship between the camera's fixed internal coordinate system and the laser radar's coordinate system is not constant, but it should be noted that the relationship can always be determined because the camera, and therefore its coordinate system, moves with the mirror or prism 4. Thus, the current relationship can always be determined by reading the vertical angle 8 and horizontal angle 9 of the mirror or prism relative to the body of the laser radar 3, as well as some of the internal operation of the laser radar.

[0044] Figures 4 and 5 show an alternative embodiment in which camera 11 is fixedly mounted on the main stationary body 5 of the laser radar 3. Camera 22 observes the reference target 12.

[0045] Therefore, in these embodiments, the relationship between the fixed internal coordinate system of each camera and the coordinate system of the laser radar is constant, independent of the movement of the laser radar mirror or prism 4. In some embodiments, the internal coordinate system of the laser radar can be shifted as described below.

[0046] In Figure 4, the reference target is positioned on or around the object under measurement 2. For all relevant laser radar positions and orientations, it may be necessary to use a wide-angle lens and / or several cameras 11' with different fields of view, as shown in Figure 4, to ensure that at least three reference targets 12 are within the field of view of the camera 11. In a factory production cell, the laser radar positions can be planned in advance, and the number of cameras, their required positioning, and optics can be configured as needed.

[0047] In Figure 5, the reference target is positioned to optimize camera observation, such as by ensuring a clear field of view and / or by being appropriately close to the camera and providing optimized coverage of the available camera field of view, thereby optimizing the accuracy of a given camera specification. It should also be noted that in this embodiment, the reference target 12 must be located at a known position relative to the object under measurement 2, as indicated by arrow 13.

[0048] An element of this disclosure is determining the relationship between the fixed internal coordinate system of the camera 11 and the internal coordinate system of the laser radar fixed to the laser positioning system body 5.

[0049] If the camera is fixed to the radar body 5, this relationship can be conceptually found by measuring at least three reference targets 12 that are not in a straight line, using the camera 11 and the laser radar 3. However, in practice, this may be impossible because the camera 11 may not be suitable for the precise measurement of the reference targets 12 that are suitable for the precise measurement by the laser radar 3, and vice versa. This difficulty can be overcome in various ways. Nestings that position a sphere with very high repeatability are commercially available. A special hemispherical target can have a base point suitable for measurement by a camera mounted at its center, but when rotated, at least a hemisphere exists for precise measurement by the laser radar 3. When placed in the aforementioned nesting, this alignment target allows both the camera and the laser radar to measure the same spherical center point with high precision.

[0050] Alternatively, referring to European Patent No. 0607303, a probing device for a camera can be introduced to investigate a physical reference target such as a tooling ball, which is also suitable for measurement by laser radar 3.

[0051] Furthermore, a pre-calibrated reference target consisting of 12 reference targets suitable for each system can be used, where, for example, a tooling ball suitable for laser radar measurement is placed at a known position relative to at least three non-linear reference points suitable for camera measurement.

[0052] If the camera is not fixed to the laser radar body 5 but rather moves with the beam steering mirror or prism 4, the same principle applies, but the current readings of the horizontal angle 8 and vertical angle 9 must also be taken into consideration.

[0053] This can be done by performing measurements sequentially from a single laser radar position 3, that is, by first measuring with the laser radar 3, and then locking the beam steering of the laser radar 3 while performing measurements with the camera 11, such that there is a single fixed relationship between the internal camera coordinate system and the laser radar coordinate system during camera measurement. Since the camera 11 has a relatively wide field of view, it becomes possible to measure the reference target 12 from a single camera position.

[0054] In all of these embodiments, the relationship between the internal coordinate system of the laser radar and the coordinate system of the object can be determined once and then applied to all laser radar positions.

[0055] Alternatively, noting that excessive tilting of the laser radar may cause some shift in its internal coordinate system, the relationship between the laser radar's internal coordinate system and the object's coordinate system can be determined for each intended laser radar position, and the appropriate relationship is used for subsequent object measurements at each laser radar position. This determination may be made in advance, for example, during system calibration.

Claims

1. A method for determining the current position and orientation of an object (2) relative to a laser radar (3) and for determining measurements of the object, wherein a camera system (11, 11') is attached to the laser radar (3), The position and orientation of the object relative to the camera system, To acquire one or more images from the camera system (11, 11'), To detect at least three reference targets (12) having a predetermined and fixed spatial relationship with respect to the object in one or more images, Obtaining the relative spatial relationships of the at least three reference targets from the storage device, Using the relative spatial relationships of the at least three reference targets with respect to each other and the detected at least three reference targets, the position and orientation of the at least three reference targets with respect to the camera system (11, 11') are calculated. The steps determined by, The steps include obtaining the spatial relationship between the camera system (11, 11') and the laser radar (3) from a storage device, A step of calculating the spatial position and orientation of the object with respect to the laser radar (3) by using the position and orientation of the object with respect to the camera system (11, 11') and the spatial relationship between the camera system (11, 11') and the laser radar, The steps include measuring the physical elements of the object with the laser radar, Methods that include...

2. The method according to claim 1, further comprising the step of moving the camera system from a first position to a second position by a robot, wherein the second position is the current position.

3. The method according to claim 2, wherein the current position and orientation of the laser radar are determined in response to the movement of the camera and laser radar by the robot.

4. The method according to any one of claims 1 to 3, wherein the camera system (11) is fixedly attached to the beam steering mechanism of the laser radar.

5. The method according to any one of claims 1 to 4, wherein the camera system is fixedly mounted on the main body (5) of the laser radar (3) such that the spatial relationship between the camera system and the laser radar is fixed.

6. The method according to any one of claims 1 to 5, wherein at least three targets (12) are located on the object (2).

7. The method according to claim 6, wherein the at least three targets (3) are attached to the object (2) or defined by the characteristics of the object.

8. The method according to any one of claims 1 to 7, wherein the camera system (11) comprises a wide-angle lens camera and / or two or more cameras having different fields of view.

9. The method according to any one of claims 1 to 8, wherein at least one of the reference targets (12) comprises at least three reference points observable by the camera system (11) and one physical element measurable by the laser radar, all of which are located at positions known to each other.

10. The method according to any one of claims 1 to 9, wherein at least one of the targets comprises a sphere with a base point located at its center, enabling the camera system (11) to observe the reference target and enabling the laser radar to measure the reference target by scanning the surface of the sphere.

11. The method according to any one of claims 1 to 10, wherein the at least three targets are not in a line.

12. A computer program that, when executed by a computer, includes instructions causing the computer to perform the method according to any one of claims 1 to 11.

13. A system comprising a computing means for performing the method described in any one of claims 1 to 11.

14. A system according to claim 13, comprising a camera system and a holder for mounting the camera system (11) on a laser radar (3), wherein the system comprises the laser radar.

Images