Monitoring the orientation of a laser scanner

Abstract

The invention relates to a method for monitoring the orientation of a laser scanner (3, 5). In the method, the laser scanner (3, 5) repeatedly determines distances of the laser scanner (3, 5) from a plurality of measurement points on various planar external surfaces of a measurement object (9), wherein all measurement objects (9) have the same geometric form having planar external surfaces and are arranged such that the normal vectors (11, 13) of the external surfaces of all measurement objects have defined directions in a fixed first reference system (K). From the determined distances, the directions of normal vectors (11, 13) of external surfaces of the measurement object (9) are determined, for each measurement object (9), in a second reference system (K1, K2) relating to the laser scanner (3, 5). A change to the orientation of the laser scanner (3, 5) is inferred when the direction of at least one normal vector (11, 13) in the second reference system (K1, K2) changes significantly, for example by more than a specified angle.

Description

[0001] Description

[0002] Monitoring the alignment of a laser scanner

[0003] The invention relates to a method for monitoring the alignment of a laser scanner, in particular in the vicinity of a crane.

[0004] 3D sensors are used to measure the surroundings in a variety of different automation solutions around a crane. These are typically laser scanners, in particular so-called Lidar sensors (Lidar: abbreviation for Light Detection and Ranging or Light Imaging, Detection and Ranging). A laser scanner of this type emits laser pulses that are partially reflected by objects. The distance of the laser scanner from the location reflecting the laser pulse can be determined from the propagation time of a laser pulse between the emission of the laser pulse and the reception of the reflected part of the laser pulse. By varying the direction of the emitted laser pulses, objects in the vicinity of the laser scanner can be detected ("scanned").

[0005] Due to perspective effects such as shadows or reflections, as well as for redundancy reasons to increase robustness, accuracy, or performance, sensor systems with multiple laser scanners are often used. Each laser scanner records positions in a reference system related to the laser scanner. These positions must be transformed into a common reference system ("world coordinate system"). This requires an initial calibration of the laser scanners, which records the position and orientation of the laser scanners with respect to the world coordinate system.

[0006] However, due to material deformation, as well as impacts and shocks that occur during crane operation, the alignment of a laser scanner can change, meaning that the alignment no longer matches the initially recorded calibration. This results in the measurement points recorded by the laser scanner being incorrectly transformed into the world coordinate system. Accordingly, the automation solution operates with incorrect position data, which causes objects, such as containers, to be detected in the wrong positions. As a result, the objects can no longer be stacked precisely, for example, and collisions can also occur, which can lead to operational interruptions.

[0007] Currently, the initial calibration of a laser scanner, for example, is carried out using a defined calibration object. This calibration object is measured manually with the laser scanner at a defined position and with a defined alignment. The calibration object is located in the sensor data acquired by the laser scanner, and this is used to determine the transformation between the reference system related to the laser scanner and the world coordinate system. During ongoing operation of the crane, a change in the alignment of the laser scanner is detected because the automation solution no longer functions as expected (reduced accuracy or even system failure). In this case, the alignment of the laser scanner must be checked manually, and the laser scanner must be re-calibrated if necessary.This means that the crane can no longer be used for the time required for recalibration and the sensor system must be recalibrated at great expense, for example with the aid of an external measuring system, e.g. using a theodolite.

[0008] DE 10 2008 019 373 A1 discloses a method for calibrating a measuring device of a crane, in which a container is arranged such that at least one surface of the container lies in a common field of view of at least two spaced-apart 3D sensors of the measuring device. One surface of the container is detected using the 3D sensors, and an orientation of one surface of the container is determined in the reference systems of the 3D sensors. The reference systems of the 3D sensors are adjusted based on the determined orientation of one surface of the container.

[0009] The invention is based on the object of specifying an improved method for monitoring the alignment of a laser scanner , in particular in the vicinity of a crane .

[0010] The object is achieved according to the invention by a method having the features of claim 1.

[0011] Advantageous embodiments of the invention are the subject of the dependent claims.

[0012] In the method according to the invention for monitoring the alignment of a laser scanner, distances of the laser scanner from a plurality of measuring points on different planar outer surfaces of a measuring object are repeatedly determined using the laser scanner, wherein all measuring objects have the same geometric shape with planar outer surfaces and are arranged such that the normal vectors of the outer surfaces of all measuring objects have defined directions in a specified first reference system. From the determined distances, the directions of normal vectors of outer surfaces of the measuring object are determined for each measuring object in a second reference system related to the laser scanner. A change in the alignment of the laser scanner is inferred if the direction of at least one normal vector determined in the process changes significantly in the second reference system, for example by more than a specified angle.

[0013] The measurement objects are, for example, conventional containers. The method thus makes it possible to determine the alignment of a laser scanner without a special calibration object. This particularly advantageously enables the method to shorten commissioning times for systems with laser scanners. Furthermore, the method makes it possible to monitor whether the alignment of a laser scanner has changed during operation of such a system and thus whether an initially measured calibration of the laser scanner is still valid.

[0014] In the method, the directions of the normal vectors of outer surfaces of a measurement object in the second reference system are determined from a plurality of measurement point directions, each of which is assigned to a measurement point and is each determined as an eigenvector of a scatter matrix formed from coordinates of the measurement point and of measurement points adjacent to the measurement point in the second reference system. For example, cluster centroids are determined from the measurement point directions determined for a measurement object using an agglomerative clustering method, and a cluster centroid is assigned to each outer surface of the measurement object.

[0015] In the aforementioned embodiment of the method, measurement point directions are determined for various measurement points on planar outer surfaces of a measurement object. The distribution of these measurement point directions is analyzed using a clustering method to determine the orientations of the outer surfaces of the measurement object, with the orientation of an outer surface being determined as a main orientation of measurement point directions. The cluster centroids determined in the clustering method yield the directions of the normal vectors of the outer surfaces of the respective measurement object.

[0016] In a further embodiment of the invention, the method is applied to at least two laser scanners. A faulty alignment of a laser scanner is determined if, for a measurement object, at least two corresponding cluster centroids determined with different laser scanners differ significantly from one another. This embodiment of the invention advantageously makes it possible to detect a changed alignment of a laser scanner by comparing the cluster centroids determined for different laser scanners.

[0017] In a further embodiment of the invention, if the alignment of a laser scanner changes, a transformation matrix is ​​determined which describes the change in alignment. Positions and alignments of objects determined using the laser scanner are then corrected, for example, in accordance with the transformation matrix. This enables the positions and alignments of objects determined using the laser scanner to be corrected by mathematically transforming these positions and alignments using the transformation matrix, so that the correction is carried out without recalibrating the laser scanner. Such a correction is particularly suitable for small changes in the alignment of the laser scanner. In the case of larger changes, the laser scanner is preferably re-calibrated.

[0018] In a further embodiment of the invention, the measuring objects are cuboid-shaped. Containers, for example, are particularly suitable as measuring objects.

[0019] In a further embodiment of the invention, the laser scanner is a lidar sensor. This embodiment of the invention takes into account that lidar is often used to determine the positions and orientations of objects.

[0020] In a further embodiment of the invention, the method is carried out in the vicinity of a crane. In particular, the method can be advantageously used in the vicinity of a crane used to transport containers, since containers are particularly advantageous as measurement objects.

[0021] The above-described properties, features, and advantages of this invention, as well as the manner in which they are achieved, will become clearer and more readily understood in connection with the following description of exemplary embodiments, which are explained in more detail in conjunction with the drawings.

[0022] FIG 1 shows an environment of a crane in which two laser scanners are arranged,

[0023] FIG 2 shows a flow diagram of an embodiment of the method according to the invention,

[0024] FIG 3 Transformations between reference systems.

[0025] Figure 1 (FIG 1) shows, by way of example and schematically, an environment of a crane 1 in which two laser scanners 3, 5 are arranged. The crane 1 has a gripping device 7 and is configured to transport objects. The gripping device 7 is arranged on a trolley 8 of the crane 1 and can be moved by it. The two laser scanners 3, 5 are also arranged, for example, on the trolley 8, but can also be arranged at other locations on the crane 1. Also shown are a measurement object 9, which is a cuboid-shaped container, and two normal vectors 11, 13 of outer surfaces of the measurement object 9. The measurement object 9 is arranged such that the normal vectors 11, 13 of its outer surfaces have defined directions in a fixed first reference system ("world coordinate system").

[0026] Figure 2 (FIG 2) shows a flow diagram of an embodiment of the method according to the invention with method steps 101 to 104 for monitoring the alignment of the laser scanners 3, 5.

[0027] In a first method step 101, each laser scanner 3, 5 is used to determine distances of the laser scanner 3, 5 from several measuring points on different planar outer surfaces of the measuring object 9. The scanning ranges of the laser scanners 3, 5 do not need to overlap. Only the same outer surfaces of the measuring object 9 need to be detected by both laser scanners 3, 5. Subsequently, each measuring point x is assigned a measuring point direction. For this purpose, measuring points x± are determined for the measuring point x whose distance from the measuring point x is smaller than a predetermined threshold value r. The set of these measuring points is denoted by A:

[0028] A = { AI ll ~ x|| < r}

[0029] Now the direction of a normal vector of a plane e is determined as the measuring point direction for the measuring point x , which minimizes the sum of the squared distances between the plane and the points of the set A . For this purpose, the center c of the set A is first determined according to

[0030] Here, it was assumed that the set A has cardinality n. Now, the relative position of the points x± to this center c is calculated according to yt = A - c.

[0031] The determined relative point positions y ± are now converted into a scatter matrix S according to

[0032] 5 = 2"=!^ ^, where denotes the dyadic product of the vector y± with itself . The desired direction of the measuring point to the measuring point x is the normalized eigenvector for the smallest eigenvalue of the matrix S and can be calculated in a known manner .

[0033] In a second method step 102, cluster centroids are determined for each laser scanner 3, 5 from the measurement point directions determined in the first method step 101 using an agglomerative clustering method, and a cluster centroid is assigned to each outer surface of the measurement object 9. From the cluster centroids determined for the first laser scanner 3, a first transformation TI is determined, with which positions determined in a reference system K1 related to the first laser scanner 3 are transformed into the world coordinate system K. Correspondingly, from the cluster centroids determined for the second laser scanner 5, a second transformation T2 is determined, with which positions determined in a reference system K2 related to the second laser scanner 5 are transformed into the world coordinate system K.Furthermore, a third transformation T3 is determined from this, which maps the reference systems K1 and K2 to one another or transforms positions in the reference system K1 into positions in the reference system K2. The transformations T1 and T2 form the initial calibrations of the two laser scanners 3, 5.

[0034] Figure 3 ( FIG 3 ) shows symbolically the transformations TI to T3 between the reference systems K, Kl , K2 .

[0035] In a third method step 103, the first two method steps 101, 102 are initially repeated for the two laser scanners 3, 5, but if necessary for a different measurement object 9, which, however, is geometrically designed and aligned like the measurement object 9 used in the first two method steps 101, 102. A check is then carried out to determine whether the cluster centroids determined for a laser scanner 3, 5 have changed significantly compared to the cluster centroids determined for this laser scanner 3, 5 in the second method step 102, or whether the third transformation T3 between the reference systems K1, K2 has changed significantly, for example by more than a predetermined angle. In the event of a significant change, it is concluded that the alignment of a laser scanner 3, 5 has changed, and a fourth method step 104 is carried out. Otherwise, the third method step 103 is repeated.

[0036] In the fourth method step 104, the faulty calibration detected in the third method step 103 is at least partially compensated during regular operation of the crane 1. This requires saving the cluster centers of gravity in the calibrated state. Two conditions can now be optimized using a mathematical optimization term:

[0037] Condition A:

[0038] First, the normalized directions assigned to the cluster centroids must be oriented orthogonally to each other, since the corresponding planes (container surfaces) are orthogonal to each other. For any two different orientation pairs ö^ö, the following must therefore apply:

[0039] Condition B:

[0040] In addition, a transformation matrix R is searched which minimizes the difference between the current cluster position c and the desired cluster position c according to

[0041] || / ?c — c|| = min

[0042] A solution can be determined using singular value decomposition. The transformation matrix R represents the rotation by which the calibration must be adjusted.

[0043] It must be considered that laser scanners 3 and 5, regardless of their mounting position, produce identical scaling. Therefore, the solution no longer needs to be scaled, but the case of a negative dimensional scaling (-1) must be checked. Although a mirror image represents a mathematically valid solution, it can never occur in reality due to the mounting position.

[0044] The method was described above using the example of two laser scanners 3, 5, but can be carried out analogously for any other number of laser scanners 3, 5, in particular even for only one laser scanner 3, 5. Although the invention has been illustrated and described in detail using preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived therefrom by a person skilled in the art without departing from the scope of the invention.

Claims

Patent claims 1. A method for monitoring the alignment of a laser scanner ( 3 , 5 ), wherein - distances of the laser scanner (3, 5) from a plurality of measuring points on different planar outer surfaces of a respective measuring object (9) are repeatedly determined with the laser scanner (3, 5), wherein all measuring objects (9) have the same geometric shape with planar outer surfaces and are arranged such that the normal vectors (11, 13) of the outer surfaces of all measuring objects (9) have defined directions in a fixed first reference system (K), - from the determined distances for each measurement object (9), the directions of normal vectors (11, 13) of outer surfaces of the measurement object (9) are determined in a second reference system (Kl, K2) related to the laser scanner (3, 5), wherein the directions of the normal vectors (11, 13) of outer surfaces of a measurement object (9) in the second reference system (Kl, K2) are determined from a plurality of measurement point directions, each of which is assigned to a measurement point and is each determined as an eigenvector of a scatter matrix formed from coordinates of the measurement point and of measurement points adjacent to the measurement point in the second reference system (Kl, K2), and - a change in the orientation of the laser scanner (3, 5) is inferred if the direction of at least one normal vector (11, 13) in the second reference system (Kl, K2) changes significantly, for example by more than a predetermined angle.

2. Method according to claim 1, wherein cluster centroids are determined from the measuring point directions determined for a measuring object (9) using an agglomerative clustering method and outer surfaces of the measuring object (9) are each assigned a cluster centroid in order to determine the direction of the normal vector of the respective outer surface.

3. Method according to claim 2, which is applied for at least two laser scanners (3, 5), wherein an incorrect alignment of a laser scanner (3, 5) is concluded if for a measurement object (9) at least two mutually corresponding cluster centroids determined with different laser scanners (3, 5) deviate significantly from one another.

4. Method according to one of the preceding claims, wherein in the event that the orientation of the laser scanner (3, 5) changes, a transformation matrix is ​​determined which describes the change in orientation.

5. The method according to claim 4, wherein positions and orientations of objects determined with the laser scanner (3, 5) are corrected according to the transformation matrix.

6. Method according to one of the preceding claims 1 to 3, wherein in the event that the orientation of the laser scanner (3, 5) changes, the laser scanner (3, 5) is re-calibrated.

7. Method according to one of the preceding claims, wherein the measuring objects (9) are cuboid-shaped.

8. Method according to one of the preceding claims, wherein the laser scanner (3, 5) is a lidar sensor.

9. Method according to one of the preceding claims, which is carried out in an environment of a crane (1).

Images