Monitoring orientation of laser scanner

Abstract

The invention relates to a method for monitoring the orientation of a laser scanner (3, 5). In the method, the distances of a laser scanner (3, 5) from a plurality of respective measurement points on different flat outer surfaces of individual measurement objects (9) are repeatedly determined by means of the laser scanner (3, 5), all measurement objects (9) having the same geometry with flat outer surfaces, and the measurement objects are arranged in such a way that the distance between the laser scanner (3, 5) and the respective measurement points on the different flat outer surfaces of the respective measurement objects (9) is determined. The normal vectors (11, 13) of the outer surfaces of all the measurement objects (9) have a defined direction in a fixed first reference frame (K). In a second reference frame (K1, K2) associated with the laser scanner (3, 5), the direction of the normal vector (11, 13) of the outer surface of the measurement object (9) is determined for each measurement object (9) from the determined distance. If the direction of at least one normal vector (11, 13) in the second reference frame (K1, K2) changes significantly, for example in a manner exceeding a predetermined angle, it is concluded that the orientation of the laser scanner (3, 5) changes.

Description

Technical Field

[0001] The invention relates to a method for monitoring the orientation of a laser scanner, in particular when the method is carried out in the environment of a crane. Background Art

[0002] In the context of cranes, 3D sensors are used to measure the surroundings in order to implement a variety of different automation solutions. Typically, the 3D sensor is a laser scanner, in particular a so-called Lidar sensor (Lidar: the abbreviation of Light detection and ranging or Light imaging, detection and ranging). Such laser scanners emit laser pulses, which are partially reflected by objects. From the travel time of the laser pulse between the emission of the laser pulse and the reception of the reflected part, the distance from the laser scanner to the corresponding reflection location of the laser pulse can be determined. By changing the direction of the emitted laser pulse, objects in the environment of the laser scanner can be detected ("scanned") .

[0003] Due to perspective effects, such as projection or mirroring, and for redundancy reasons in order to increase robustness, accuracy or performance, sensor systems with multiple laser scanners are often used. Each laser scanner detects positions in a reference system related to the laser scanner. These positions must be converted into a common reference system ("world coordinate system"). For this purpose, the laser scanners must initially be calibrated, whereby the position and orientation of the laser scanners relative to the world coordinate system are recorded.

[0004] However, due to material deformations and due to shocks and vibrations that occur during the operation of the crane, the orientation of the laser scanner can change so that it no longer corresponds to the calibration that was originally recorded. This can lead to an incorrect transformation of the measuring points recorded by the laser scanner into the world coordinate system. Accordingly, the automation solution can work with incorrect position data, so that objects, such as containers, are recognized at the wrong position. As a result, objects can, for example, not be stacked accurately or even cause collisions that interrupt operation.

[0005] Currently, the initial calibration of a laser scanner is performed, for example, with a defined calibration object. In a manual process, this calibration object is placed at a defined position and measured with the laser scanner in a defined orientation. The calibration object is located in the sensor data recorded by the laser scanner and the transformation between the reference system associated with the laser scanner and the world coordinate system is determined from this. During the travel operation of the crane, a change in the orientation of the laser scanner is recognized by the fact that the automation solution no longer functions as expected (reduced accuracy or even system failure). In this case, the orientation of the laser scanner must be checked manually and, if necessary, the laser scanner must be recalibrated. This results in the crane being unusable for the time of recalibration and the sensor system having to be recalibrated costly, for example by measuring with an external measuring system, such as with a theodolite.

[0006] DE 10 2008 019 373 A1 discloses a method for calibrating a measuring device for a crane, in which a container is arranged such that a surface of at least one container is located in a common field of view of 3D sensors of at least two mutually spaced measuring devices. The container surface is recorded with a 3D sensor and the orientation of the container surface in a reference system of the 3D sensor is determined. The reference system of the 3D sensor is adjusted based on the determined orientation of the container surface. Summary of the invention

[0007] It is an object of the present invention to provide an improved method for monitoring the orientation of a laser scanner, in particular to perform the method in a crane environment.

[0008] This object is achieved by the method according to the invention having the features of claim 1 .

[0009] Advantageous embodiments of the invention are the subject matter of the dependent claims.

[0010] In the method according to the invention for monitoring the orientation of a laser scanner, the distances of the laser scanner to a plurality of corresponding measuring points on different flat outer surfaces of individual measuring objects are repeatedly determined with the laser scanner, wherein all measuring objects have the same geometry with flat outer surfaces and the measuring objects are arranged such that the normal vectors of the outer surfaces of all measuring objects have a defined direction in a fixed first reference system. In a second reference system associated with the laser scanner, the direction of the normal vector of the outer surface of the measuring object is determined for each measuring object from the determined distance. If the direction of at least one of the normal vectors thus determined in the second reference system changes significantly, for example by more than a predetermined angle, a change in the orientation of the laser scanner is inferred.

[0011] The measuring object is thus, for example, a conventional container. The method thus makes it possible to determine the orientation of the laser scanner without a dedicated calibration object. The method thus makes it possible to particularly advantageously shorten the startup time of a device having a laser scanner. Furthermore, the method makes it possible to monitor whether the orientation of the laser scanner changes during operation of such a device and thus whether the calibration of the laser scanner checked initially is still valid.

[0012] In the method, in a second reference system, the direction of the normal vector of the outer surface of the measurement object is determined from a plurality of measurement point directions, a measurement point is assigned to each measurement point direction, and each measurement point direction is determined as an eigenvector of a scatter matrix formed by the coordinates of the measurement point in the second reference system and the coordinates of adjacent measurement points. For example, cluster centroids are determined from the measurement point directions determined for the measurement object using an agglomerative clustering method, and the cluster centroids are respectively assigned to the outer surface of the measurement object.

[0013] In an embodiment of the method described above, therefore, the measuring point directions are determined for different measuring points on the flat outer surface of the measuring object. The distribution of these measuring point directions is analyzed using a clustering method to determine the direction of the outer surface of the measuring object, wherein the direction of the outer surface is determined as the main direction of the measuring point directions. The cluster centroid determined in the case of the clustering method gives the direction of the normal vector of the corresponding outer surface of the measuring object.

[0014] In a further embodiment of the invention, the method is used for at least two laser scanners. If at least two mutually corresponding cluster centroids determined for the measurement object using different laser scanners differ significantly from one another, an incorrect orientation of the laser scanner is inferred. This embodiment of the invention advantageously makes it possible to detect a change in the orientation of the laser scanner by comparing the cluster centroids determined for the different laser scanners.

[0015] In a further embodiment of the invention, if the orientation of the laser scanner changes, a transformation matrix is ​​determined which describes the change in orientation. The position and orientation of the object determined by the laser scanner are then corrected, for example, according to the transformation matrix. This makes it possible to correct the position and orientation of the object determined by the laser scanner by mathematically transforming these positions and orientations with the aid of the transformation matrix, so that the correction can be achieved without recalibrating the laser scanner. This correction is particularly suitable for small changes in the orientation of the laser scanner. In the case of larger changes, the laser scanner is preferably recalibrated.

[0016] In another embodiment of the present invention, the measurement object is a rectangular parallelepiped, and in particular, a container is suitable as the measurement object.

[0017] In a further embodiment of the invention, the laser scanner is a LiDAR sensor. Embodiments of the invention take into account that LiDAR is commonly used to determine the position and orientation of an object.

[0018] In a further embodiment of the invention, the method is performed in the environment of a crane. In particular, the method can be advantageously used in the environment of a crane for transporting containers, since containers are particularly suitable as measurement objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above-mentioned properties, features and advantages of the present invention and the manner of achieving these will be further explained by the following description of embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0020] Figure 1 shows an environment of a crane with two laser scanners arranged,

[0021] Figure 2 shows a flow chart of an embodiment of a method according to the present invention,

[0022] Figure 3 The transformation between reference frames is shown. DETAILED DESCRIPTION

[0023] Figure 1 FIG1 shows, by way of example and in a schematic manner, the environment of a crane 1 in which two laser scanners 3, 5 are arranged. The crane 1 has a gripping device 7 and is designed for transporting objects. The gripping device 7 is arranged on a trolley 8 of the crane 1 and can be moved by it. The two laser scanners 3, 5 are also arranged on the trolley 8, for example, but can also be arranged at other locations on the crane 1. Also shown is a measurement object 9, which is a cuboid container, and two normal vectors 11, 13 of the outer surface of the measurement object 9. The measurement object 9 is arranged so that the normal vectors 11, 13 of its outer surface have a defined direction in a fixed first reference system ("world coordinate system").

[0024] Figure 2 FIG2 shows a flow chart of an exemplary embodiment of a method according to the invention, which has method steps 101 to 104 for monitoring the orientation of a laser scanner 3 , 5 .

[0025] In a first method step 101, the distances of the laser scanners 3, 5 to a plurality of corresponding measuring points on different planar outer surfaces of the measuring object 9 are determined with each laser scanner 3, 5. The scanning areas of the laser scanners 3, 5 do not have to overlap. It is only necessary that the same outer surface of the measuring object 9 is to be scanned by both laser scanners 3, 5. Subsequently, a measuring point direction is assigned to each measuring point x. For this purpose, the measuring points x whose distance to the measuring point x is less than a predetermined threshold value r are determined for the measuring point x.i . Let A be the set of these measurement points:

[0026] A={x i |||x i -x|| <r}

[0027] Now, the direction of the normal vector of the plane e is determined as the measurement point direction with respect to the measurement point x, which minimizes the sum of the squared distances between the points in the set A and the plane. To do this, first, according to

[0028]

[0029] Determine the centroid c of set A.

[0030] Here we assume that the set A has n powers. Now according to

[0031] y i =x i -c

[0032] Calculate point x i The relative position with respect to the center of mass c.

[0033] The determined relative point positions yi are now converted into a scatter matrix S according to

[0034]

[0035] where y i y i T Represents vector y i The sought-for measuring point direction to the measuring point x is the normalized eigenvector of the smallest eigenvalue of the matrix S and can be calculated in a known manner.

[0036] In a second method step 102, cluster centroids are determined for each laser scanner 3, 5 from the measurement point directions determined in the first method step 101 using an agglomerative clustering method and are assigned to the individual outer surfaces of the measured object 9. A first transformation T1 is determined from the cluster centroids determined for the first laser scanner 3, with which positions determined in the reference system K1 associated with the first laser scanner 3 are transformed into the world coordinate system K. Correspondingly, a second transformation T2 is determined from the cluster centroids determined for the second laser scanner 5, with which positions determined in the reference system K2 associated with the second laser scanner 5 are transformed into the world coordinate system K. Furthermore, a third transformation T3 is thereby determined, which transforms positions in the reference system K1 or, more precisely, positions mapped to one another in the reference systems K1 and K2 into positions in the reference system K2. The transformations T1 and T2 form an initial calibration of the two laser scanners 3, 5.

[0037] Figure 3 (FIG 3) symbolically shows the transformations T1 to T3 between the reference frames K, K1, K2.

[0038] In a third method step 103, the two first method steps 101, 102 are first repeated for the two laser scanners 3, 5, but where appropriate for a further measurement object 9 which is designed and configured in terms of geometry to be identical to the measurement object 9 used in the two first method steps 101, 102. Subsequently, it is checked whether the cluster centroid determined for the laser scanner 3, 5 has changed significantly relative to the cluster centroid determined for this laser scanner 3, 5 in the second method step 102, or whether the third transformation T3 between the reference systems K1, K2 has changed significantly, for example by more than a predetermined angle. If a significant change has occurred, it is inferred that the orientation of the laser scanner 3, 5 has changed, and a fourth method step 104 is performed. Otherwise, the third method step 103 is repeated.

[0039] In a fourth method step 104 , the miscalibration detected in the third method step 103 is at least partially compensated during normal operation of the crane 1 .

[0040] To do this, we need to store the cluster centroids in the calibration state. Now we can use mathematical optimization terms to optimize two conditions:

[0041] Condition A:

[0042] On the one hand, the normalized directions respectively assigned to the cluster centroids must be oriented orthogonally to each other, since the associated planes (container surfaces) are orthogonal to each other. Must meet:

[0043]

[0044] Condition B:

[0045] In addition, it is necessary to find a transformation matrix R that minimizes the current cluster position c and the desired cluster position The difference between

[0046]

[0047] For this purpose, the eigenvalue decomposition (Singular Value Decomposition) can be used to solve the problem. In this case, the transformation matrix R represents a rotation by which the calibration must be adjusted.

[0048] In doing so, it must be taken into account that the laser scanners 3, 5 (regardless of their mounting position) map the image at the same scale. Therefore, the solution no longer has to be scaled, but the possibility of negative dimensional scaling (-1) must be checked. Although reflections are mathematically valid solutions, they never occur in reality due to the mounting position.

[0049] The method has been described above using the example of two laser scanners 3 , 5 , but can be performed in a similar manner for any other number of laser scanners 3 , 5 , in particular for only one laser scanner 3 , 5 .

[0050] Although the present invention has been described in detail by way of preferred embodiments, the present invention is not limited to the disclosed examples, and those skilled in the art can derive other variants therefrom without departing from the scope of protection of the present invention.

Claims

1. A method for monitoring the orientation of a laser scanner (3, 5), wherein: - using the laser scanner (3, 5) to repeatedly determine the distance between the laser scanner (3, 5) and a plurality of corresponding measuring points on different flat outer surfaces of a corresponding measuring object (9), wherein all the measurement objects (9) have the same geometric shape with a flat outer surface and all the measurement objects are arranged in such a way that the normal vectors (11, 13) of the outer surfaces of all the measurement objects (9) have a defined direction in a fixed first reference system (K), - in a second reference system (K1, K2) associated with the laser scanner (3, 5), determining for each measured object (9) the direction of a normal vector (11, 13) of the outer surface of the measured object (9) from the determined distance, wherein the direction of the normal vector (11, 13) of the outer surface of the measured object (9) is determined in the second reference system (K1, K2) from a plurality of measuring point directions, each of which is assigned a measuring point and each of which is determined as an eigenvector of a scatter matrix formed by the coordinates of the measuring point and the measuring points adjacent to the measuring point in the second reference system (K1, K2), and If the direction of at least one normal vector (11, 13) in the second reference system (K1, K2) changes significantly, for example by more than a predetermined angle, a change in the orientation of the laser scanner (3, 5) is inferred.

2. The method according to claim 1, wherein: In order to determine the direction of the normal vector of the corresponding outer surface, an agglomerative clustering method is used to determine a cluster centroid from the measurement point directions determined for the measurement object (9) and to assign the cluster centroid to each outer surface of the measurement object (9).

3. The method according to claim 2, wherein the method is used for at least two laser scanners (3, 5), wherein: If at least two mutually corresponding cluster centroids determined for a measurement object (9) using different laser scanners (3, 5) differ significantly from one another, an orientation error of the laser scanners (3, 5) is inferred.

4. A method according to any one of the preceding claims, wherein: In the event of a change in the orientation of the laser scanner (3, 5), a transformation matrix is ​​determined which describes the change in orientation.

5. The method according to claim 4, wherein: Based on the transformation matrix, the position and orientation of the object determined using the laser scanner (3, 5) is corrected.

6. The method according to any one of the preceding claims 1 to 3, wherein: In the event of a change in the orientation of the laser scanner (3, 5), the laser scanner (3, 5) is recalibrated.

7. A method according to any one of the preceding claims, wherein: The measuring object (9) is a cuboid.

8. A method according to any one of the preceding claims, wherein: The laser scanner (3, 5) is a laser radar sensor.

9. Method according to any of the preceding claims, which is performed in the environment of a crane (1).

Images