Method for comparing two at least partially overlapping point clouds

Abstract

The invention relates to a method for comparing two at least partially overlapping point clouds, comprising the following steps: receiving a first point cloud representing a first road section, Receiving a second point cloud representing a second road section at least partially overlapping the first road section, wherein at least some of the points of the first and second point clouds each have a lane marking index as a data feature, which uniquely assigns the corresponding point of the corresponding point cloud to a lane marking of the corresponding road section, defining point cloud lane markings formed by points with the same lane marking index for the first and second point clouds, associating a point cloud lane marking of the first point cloud with a point cloud lane marking of the second point cloud, Matching the first point cloud with the second point cloud based on the associated point cloud lane markings. The invention relates to a device, a computer program and a machine-readable storage medium.

Description

The invention relates to a method for matching two at least partially overlapping point clouds, an apparatus, a computer program and a machine-readable storage medium.Prior ArtPatent document KR 102 242 653 B1 discloses a method for improving a success rate of 3D data alignment of a laser scanner in consideration of motion characteristics of an unmanned vehicle.Laid-open specification CN 1 10 832 279 A discloses a system for performing alignment of three-dimensional representations of data acquired by autonomous vehicles.Disclosure of the InventionThe object on which the invention is based is to provide a concept for matching two at least partially overlapping point clouds.This object is achieved by means of the respective subject matter of the independent claims. Advantageous embodiments of the invention are the subject matter of dependent claims.According to a first aspect, there is provided a method of matching two at least partially overlapping point clouds comprising the steps of:receiving a first point cloud representing a first road segment,receiving a second point cloud representing a second road segment at least partially overlapping the first road segment,wherein at least some of the points of the first and the second point cloud each have a lane marking index as a data feature, which uniquely assigns the corresponding point of the corresponding point cloud to a lane marking of the corresponding road segment, defining point cloud lane markings formed by points having the same lane marking index for the first and the second point cloud, associating a point cloud lane marking of the first point cloud with a point cloud lane marking of the second point cloud,matching the first point cloud with the second point cloud based on the associated point cloud trajectory markers.According to a second aspect, a device is provided which is configured to carry out all steps of the method according to the first aspect.According to a third aspect, there is provided a computer program comprising instructions which, when the computer program is executed by a computer, for example by the apparatus according to the second aspect, cause the computer program to execute a method according to the first aspect.According to a fourth aspect, a machine-readable storage medium is provided on which the computer program according to the third aspect is stored.The invention is based on and encompasses the realization that the above object is achieved by using a multi-level approach to align the two point clouds. In the present case, there are a plurality of planes or levels for the alignment: a first plane defined by the individual points of the point cloud and a second plane defined by the defined point cloud roadway markings.The points of the point clouds each have coordinates and a lane marking index as a data feature, which uniquely assigns the corresponding point of the corresponding point cloud to a lane marking of the corresponding road section. The determined point cloud trajectory markers define the second plane. This second plane is an abstract, coarser plane compared to the first plane, so that the first plane can also be referred to as a fine point plane or fine plane and wherein the second plane can also be referred to as a coarse plane or as a coarser plane.Performing association at the coarser level is easier than direct association at the fine level due to significantly lower combinatorial complexity. Moreover, a rough solution to the problem of associating at the coarser level may be used as a first estimate of a fine solution to the problem of associating at the fine level. The rough solution can thus be used, for example, as a starting value for solving the problem of associating at the fine level, which greatly simplifies this problem as a result.The point clouds thus already inherently comprise an abstraction plane, the lane marking indices, which consequently do not first have to be learned or determined from the individual points of the point clouds. The lane marking indices are thus present as a data feature of the point cloud which indicates a respective association of the points with a specific lane marking.Correspondences can be set up, for example, on the coarse plane of the point cloud roadway markings, that is to say correspondences between the point cloud roadway markings of the point clouds. On the fine plane, for example, point-to-point correspondences are then only estimated between the point cloud trajectory markings already associated with one another. Based on this estimation, the two point clouds may be matched.The concept described here therefore does not require an abstraction plane learned from the individual points of the point clouds, which can be used as a coarse plane. Instead, because the two point clouds already inherently have this abstraction plane, the complicated step for calculating learned, abstract representations of the points of the point clouds can be saved.Moreover, the concept described herein uses physically reasonable abstractions, the lane markings in the form of, for example, lines and / or lines, rather than learned which are difficult to interpret by humans.Furthermore, the concept described here may result in elimination of estimators that go beyond their lane marking, thus strengthening robustness of the matching, which may result in higher quality road maps overall.In one embodiment of the method, it is provided that, if the point cloud roadway markings each comprise a line and / or a line, the respective line and / or line are parameterized in order to ascertain line parameters and / or line parameters, wherein the roadway markings are associated based on the ascertained parameters.This brings about, for example, the technical advantage that the road markings can be efficiently associated.In one embodiment of the method, it is provided that the parameterizing comprises a respective determination of a geometric center of gravity and / or a gradient of the corresponding line and / or of the corresponding line, so that a geometric center of gravity and / or a gradient of the corresponding line and / or of the corresponding line are determined as line parameters and / or line parameters accordingly.This brings about the technical advantage, for example, that parameters which are particularly suitable for the association are determined.In one embodiment of the method, it is provided that points of associated point cloud roadway markings are associated point by point, such that a point of a point cloud roadway marking is only associated with a further point of a further point cloud roadway marking associated with the one point cloud roadway marking, wherein the first point cloud is matched to the second point cloud based on the points of the point cloud roadway markings associated point by point.This brings about the technical advantage, for example, that the associated points are physically meaningful insofar as respective points of the two point clouds that belong to one another correspondingly also belong to point cloud roadway markings associated with one another.In one embodiment of the method, it is provided that a correspondence estimation is carried out between the points of the point cloud roadway markings in order to estimate a respective correspondence between the points, wherein the points are associated point by point based on the estimated correspondences.This brings about, for example, the technical advantage that the points can be efficiently associated.In one embodiment of the method, it is provided that it is checked for one or more or each correspondence whether the mutually corresponding points respectively belong to point cloud roadway markings associated with one another, wherein if no, a new correspondence is estimated for these points, wherein if yes, these points of the correspondence are associated with one another accordingly.This brings about the technical advantage, for example, that a result of the point-by-point association is particularly reliable and trustworthy on the basis of the checking of the correspondence estimate.In one embodiment of the method, it is provided that the correspondences are estimated by means of a kernel method, entries whose associated points do not belong to point cloud roadway markings associated with one another being removed from a kernel matrix.This brings about, for example, the technical advantage that a particularly trusted kernel matrix is obtained.The kernel matrix results from the kernel function. In other words, the kernel function is a rule which determines the distances of points to one another. For a set of points, the distances of each point to each other point are measured. This creates a matrix which, for a set of N points, has N^2 entries. The entries represent the distances.Device features are obtained analogously from corresponding method features and vice versa.The method is carried out, for example, by means of the device.The method is, for example, a computer-implemented method.The device is, for example, programmatically configured to execute the computer program.Matching between the two point clouds includes, for example, determining a relative delta pose between the two point clouds.Determining a relative delta pose includes, for example, determining a relative rotation and a relative translation.A point cloud in the sense of the description has been or is generated, for example, using environment sensor data from one or more environment sensors of a motor vehicle.An environment sensor in the sense of the description is, for example, one of the following environment sensors: radar sensor, LiDAR sensor, image sensor, in particular image sensor of a video camera, ultrasonic sensor, infrared sensor and magnetic field sensor.Thus, a point cloud in the sense of the description can be generated or created, for example, using LiDAR data and / or radar data and / or video camera data and / or infrared data and / or magnetic field data and / or ultrasonic data.Environment sensor data in the sense of the description describes an environment of the motor vehicle.A roadway marking in the sense of the description is, for example, one of the following roadway markings: longitudinal marking, area marking, boundary marking, cross marking, parking area marking.A lane marking may also be referred to as a road marking.A longitudinal marking comprises, for example, one of the following longitudinal markings: guideline, warning line, roadway boundary, lane boundary, one-sided lane boundary, departure line, double lane boundary.A longitudinal mark may comprise, for example: a continuous line, a broken line, a double line both continuous and broken, a continuous double line, a broken double line.In the case of lane marking as a line, the lane marking index may also be referred to as a line index.In the case of a lane marking as a line, the lane marking index can also be referred to as a line index.If, within the scope of the description, line index is specifically written, lane marking index is always to be read along in general.The formulation that at least some of the points of the first and the second point cloud each have a lane marking index as a data feature, which uniquely associates the corresponding point of the corresponding point cloud with a lane marking of the corresponding road section, means, for example, that some or all of the points of the first and the second point cloud each have a lane marking index as a data feature, which uniquely associates the corresponding point of the corresponding point cloud with a lane marking of the corresponding road section.The exemplary embodiments and embodiments described here can be combined with one another in any desired manner, even if this is not explicitly described.The invention is explained in more detail below with reference to preferred exemplary embodiments. The following are shown here: FIG. 1 shows a flow diagram of a method according to the first aspect, FIG. 2 shows a device according to the second aspect, FIG. 3 shows a machine-readable storage medium according to the fourth aspect, FIG. 4 shows a representation of roadway markings by a point cloud with visualized line indices, FIG. 5 shows an exemplary line level correspondence estimation, FIG. 6 shows an exemplary correspondence estimation at point level; and FIG. 7 is a block diagram.FIG. 1 shows a flow diagram of a method for matching two at least partially overlapping point clouds, comprising the following steps:receiving 101 a first point cloud representing a first road segment,receiving 103 a second point cloud representing a second road section at least partially overlapping the first road section,wherein at least some of the points of the first and the second point cloud each have a lane marking index as a data feature, which uniquely assigns the corresponding point of the corresponding point cloud to a lane marking of the corresponding road segment, establishing 105 point cloud lane markings formed by points having the same lane marking index for the first and the second point cloud,associating 107 a point cloud trajectory marker of the first point cloud with a point cloud trajectory marker of the second point cloud,matching 109 the first point cloud with the second point cloud based on the associated point cloud trajectory markers.FIG. 2 shows an apparatus 201 configured to carry out all steps of the method according to the first aspect.FIG. 3 shows a machine-readable storage medium 301 on which a computer program 303 is stored. The computer program 303 comprises instructions which, when the computer program 303 is executed by a computer, cause the computer to execute a method according to the first aspect.FIG. 4 shows a graph 400 having an abscissa 401 and an ordinate 403.The x coordinates of points of a point cloud are plotted on the lower abscissa 401. The y-coordinates of the points of the point cloud are plotted on the ordinate 403. A plurality of point cloud roadway markings 407, 409, 411, 413, 415, 417, 419, 421, 423 and 425 are drawn in graph 401. These point cloud lane markings are formed by points with the same lane marking index.FIG. 5 shows an exemplary correspondence estimate at line level, i.e. at the coarser level.In detail, a first point cloud 501, also referred to as a source point cloud below, and a second point cloud 503, also referred to as a target point cloud below, is illustrated.The diagram shown in FIG. 5 is a line level diagram. This means that only the defined point cloud roadway markings are drawn in FIG. 5.This means that point cloud roadway markings have been defined for source point cloud 501 and for target point cloud 503, which are formed by points of the corresponding point cloud having the same roadway marking index.In the present case, lines are shown by way of example as an example of lane markings.In detail, two crossed lines 505, 507 are defined for the source point cloud 501, which are each lane boundary lines. A dashed line 509 runs between the two lines 505, 507, which divides the roadway, bounded by the two lane boundary lines 505, 507, into two lanes.Similarly, two crossed lines 511, 513 are defined for the target point cloud 503, which are each roadway boundary lines. Correspondingly, a dashed line 515 runs between the two lane boundary lines 511, 513, which correspondingly divides the roadway bounded by the two roadway boundary lines 511, 513 into two lanes.Line level correspondence estimation is performed. This means that the lines 505, 507, 509 of the source point cloud 501 are brought into correspondence, i.e. in agreement, with the lines 511, 513, 515 of the target point cloud 503. For this purpose, for example, a respective geometric center of gravity of the lines is determined, wherein the respective correspondence is estimated on the basis of the geometric centers of gravity.The estimated correspondences between the respective lines of the source point cloud 501 and the target point cloud 503 are represented by a line with the reference symbol 517.FIG. 6 shows an example correspondence estimation at the point level, i.e., at the fine level, using the example correspondence estimation of FIG. 5.According to the correspondence estimation at the point level or point plane, the points of the individual lines are now brought into correspondence. The correspondences between the respective points of the source point cloud and the target point cloud are represented by a line with the reference symbol 601 for some of the points.In this correspondence estimation, a point of the broken line 509 has been erroneously corresponded to a point of a lane boundary line 511. The corresponding correspondence is additionally identified by the reference numeral 603. Furthermore, the incorrect correspondence 601, 603 is also identified by a circle with the reference numeral 605.This incorrect correspondence 601, 603 can be filtered at line level and recomputed using the correspondence estimate according to FIG. 5.FIG. 7 shows a block diagram 701, which explains the concept described here by way of example.According to a function block 703, source and destination point clouds are received, which represent a first and a second road segment, respectively, wherein the second road segment overlaps the first road segment at least partially, for example completely. The points of the source and target point clouds have coordinates and a line index each as a data feature. According to a function block 705, a correspondence estimation based on line level is performed.According to a function block 707, a correspondence estimation at point level is performed with, for example, an a priori line level estimator. Here, a result of line level correspondence estimation according to the function block 705 is used.Based on a result of the correspondence estimation according to the function block 707, according to a function block 709, an matching transformation of the source and target point cloud is calculated. In other words, source and destination points are matched based on a result of the correspondence estimation according to the function block 707.In summary, the concept described here represents a simplification of the problem of correspondence estimation by using a coarse level already present in the data in order to put two point clouds on top of one another, i.e. to balance them. The data do not consist, for example, of densely populated point clouds, which represent objects or spaces, for example, but consist, for example, of sparsely populated points which represent roadway markings in the form of lines. The data, i.e. the point clouds, thus already contain this inherent abstraction level, which, in regular algorithms for "point cloud registration", must first be learned from the bare points of the point clouds in German "point cloud registration". Moreover, for example, a line index, generally a lane marking index, is present as a data feature which uniquely indicates the affiliation of the points of the point clouds to a specific line, generally to a specific lane marking.The concept uses the coarse plane of the lines to point to their correspondence. At the finer level, point-to-point correspondences are then only estimated between the lines already associated with one another.Other roadway objects can likewise be part of the point clouds, for example, but are not used further or are in particular irrelevant within the scope of the concept described here, for example. The points of the point cloud thus have coordinates and in each case a line index as data features. The line index explicitly assigns points of the cloud, i.e. of the point cloud, to a line. The lines thus form an abstract, coarser plane compared to the fine point plane.With the aid of, for example, line parameterization, such as, for example, the geometric center of gravity and / or the gradient, a correspondence estimation takes place, for example, between the lines of two point clouds to be mapped onto one another. The estimation may be performed using a kernel method, for example. For example, the estimation may be performed using an optimal transport method. The result resulting from the estimation is an association of the line indices. This means that for each point of a line, both the own and the index of the corresponding line are now present. Both features can be combined, for example, by means of a machine learning method. Subsequently, for example, a correspondence estimation is performed on the learned representation, for example, via an optimization of the Sinkhorn distance.Subsequently, for example, point correspondences are formed, wherein the line indices now present are used, for example, as a priori estimators. This can be exploited, for example, by filtering out points with incorrect correspondences. For example, the estimation is performed using a kernel method, whereby entries of the kernel matrix with false line correspondence are removed. For example, a machine learning method may also be used to use an abstract representation of the points during the estimation.References included in the specificationThis list of documents cited by the applicant has been produced in an automated manner and is only included for the better information of the reader. The list is not part of the German patent application or utility model application. The DPMA does not take any adhesion for any faults or omissions.Patent Literature citedKR 102 242 653 B1

[0002] CN 1 10 832 279 A

[0003]

Claims

Method for matching two at least partially overlapping point clouds, comprising the following steps: receiving (101) a first point cloud (501) which represents a first road section, receiving (103) a second point cloud (503) which represents a second road section which at least partially overlaps the first road section, wherein at least some of the points of the first and the second point cloud (503) each have a lane marking index as data feature which explicitly assigns the corresponding point of the corresponding point cloud to a lane marking of the corresponding road section, defining (105) point cloud lane markings (505, 507, 509, 511, 513, 515) formed by points having the same lane marking index for the first and the second point cloud (503), associating (107) a point cloud lane marking (505, 507, 509, 511, 513, 513, 515) of the first point cloud ( 501) with a point cloud trajectory marker ( 505, 507, 509, 511, 513, 515) of the second point cloud ( 503), matching ( 109) of the first point cloud ( 501) with the second point cloud ( 503) based on the associated point cloud trajectory markers ( 505, 507, 509, 511, 513, 515).The method of claim 1, wherein when the point cloud lane markings (505, 507, 509, 511, 513, 515) each comprise a line and / or a stroke, the respective line and / or stroke are parameterized to determine line parameters and / or stroke parameters, the lane markings being associated based on the determined parameters.Method according to claim 2, wherein the parameterizing comprises a respective determination of a geometric center of gravity and / or a gradient of the corresponding line and / or of the corresponding stroke, such that a geometric center of gravity and / or a gradient of the corresponding line and / or of the corresponding stroke are determined accordingly as line parameters and / or stroke parameters.The method according to any of the preceding claims, wherein points of associated point cloud trajectory markings (505, 507, 509, 511, 513, 515) are associated point by point such that a point of a point cloud trajectory marking (505, 507, 509, 511, 513, 515) is associated only with a further point of a further point cloud trajectory marking (505, 507, 509, 511, 513, 515) associated with the one point cloud trajectory marking (505, 507, 509, 511, 513, 515), wherein the first point cloud (501) is matched to the second point cloud (503) based on the point-by-associated points of the point cloud trajectory markings (505, 507, 509, 511, 513, 515).The method of claim 4, wherein a correspondence estimation is performed between the points of the point cloud trajectory markings (505, 507, 509, 511, 513, 515) to estimate a respective correspondence between the points, wherein the points are associated point by point based on the estimated correspondences.The method of claim 5, wherein for one or more or each correspondence, it is checked whether the mutually corresponding points respectively belong to point cloud trajectory markings (505, 507, 509, 511, 513, 515) associated with each other, wherein if no, a new correspondence is estimated for these points, wherein if yes, these points of the correspondence are associated with each other accordingly.The method of claim 5 or 6, wherein the correspondences are estimated using a kernel method, wherein entries are removed from a kernel matrix whose associated points do not belong to point cloud trajectory markers (505, 507, 509, 511, 513, 515) associated with each other.Device (201) which is configured to carry out all the steps of the method according to one of the preceding claims.A computer program (303) comprising instructions which, when the computer program (303) is executed by a computer, cause the computer to carry out a method according to any one of claims 1 to 7.A machine readable storage medium (301) having stored thereon the computer program (303) according to claim 9.

Images