Computer-implemented method for assigning a data point to a respective one of a plurality of objects

Abstract

The invention relates to a computer-implemented method for assigning a data point of first data (26) to a respective one of a plurality of objects (14,14 ').SOLUTION: The first data (26) comprise at least three spatial dimensions, and the method comprises the step of forming groups of projected data points (18,18 ') assigned to the respective object (14,14 '), taking into account the received information data of the two dimensional enclosing polygons (10,10 ') and the depth classes (20,20 ') of the data points, using an assignment method such that in the case of overlapping polygons (10,10 ') in which a projected data point (18,18 ') is located inside two or more two dimensional enclosing polygons (10,10 '), the groups are formed such that the depth class (20,20 ') is not assigned to a plurality of overlapping polygons (10,10 ') at the same time.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a computer-implemented method for assigning data points of a first data set to respective ones of a plurality of objects.

[0002] The invention also relates to an apparatus for processing data, including means for carrying out the method described above.

[0003] The invention further relates to a computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the above method.

[0004] The invention further relates to a computer-readable data carrier having stored thereon the computer program product described above. [Background technology]

[0005] Autonomous and semi-autonomous driving have the potential to transform mobility, for example, by reducing journey times, energy consumption, and / or emissions. 3D object recognition technology is attracting attention as a key element for autonomous driving, and in recent years, efforts to develop 3D object recognition technology based on machine learning have been gaining attention.

[0006] Existing approaches to 3D object recognition technology can be basically divided into two groups depending on whether the output data is 2D image data or a 3D point cloud, typically generated by a LiDAR sensor. For methods based on image data captured by a camera, estimating the 3D envelope of an object from 2D image data poses a major challenge. However, based on rapid developments in the field of machine learning, especially deep learning techniques, image-based 3D recognition has made remarkable progress.

[0007] However, a drawback of image-based 3D recognition techniques using machine learning is that statistical algorithms must be trained with large amounts of training data, which is laborious. Furthermore, machine learning-based algorithms for 3D object recognition can be very computationally intensive, potentially rendering these methods inefficient. This is particularly problematic when large amounts of data must be processed, as occurs in the field of data annotation. Annotated data includes bounding boxes surrounding 3D objects and labels that indicate that a 3D object is a vehicle. Annotated data is used, for example, as part of supervised learning, when algorithms such as neural networks must be trained on large datasets. Providing annotated data has traditionally been labor-intensive, sometimes manually, resulting in significant time and cost. Summary of the Invention [Problem to be solved by the invention]

[0008] Starting from this, the object of the present invention is to enable 3D object recognition in a simple, fast and / or cost-effective manner. [Means for solving the problem]

[0009] This problem is solved by the subject matter of the independent claims. Preferred developments emerge from the dependent claims.

[0010] Thus, according to the present invention, there is provided a computer-implemented method for assigning data points of a first data set to respective ones of a plurality of objects, wherein the first data set includes at least three spatial dimensions, the method comprising the steps of: - receiving information data from a two-dimensional enclosing polygon, preferably a two-dimensional enclosing rectangle, for an object represented on the second data, the second data including at least two spatial dimensions, the received information data defining the size and position of the enclosing polygon of each object in the second data; - receiving a projection matrix defining a mapping of three-dimensional data points of the first data onto two-dimensional data points in the second data; - generating projected data points by projecting three-dimensional data points of the first data using the received projection matrix; - classifying the projected data points and / or the three-dimensional data points into depth classes based on the depth values ​​of each of the three-dimensional data points; forming groups of projected data points assigned to each object using an assignment method that takes into account the received information data of the two-dimensional enclosing polygons and the depth classes of the data points, such that in the case of overlapping polygons in which the projected data points are located inside two or more two-dimensional enclosing polygons, the groups are formed in such a way that a depth class is not assigned to more than one overlapping polygon at the same time.

[0011] In other words, the method according to the invention is a method for forming clusters of data points of a first data set, where the clusters are determined based on a projection of the three-dimensional data set in two spatial dimensions, taking into account a two-dimensional enclosing polygon and a depth class of the (preferably rectangular) data points of the second data set. One aspect of the invention is that, in particular, a depth class is assigned to each polygon, and in particular, groups (so-called clusters) are found using a global finding in the case of overlapping polygons, rather than being found for each polygon individually.

[0012] According to this method, each projected data point of the first data is assigned to a cluster, where each cluster represents an object. Based on this assignment, a three-dimensional envelope, preferably a convex envelope, particularly preferably a cube, can be determined for the object represented on the first data. Since the first data has at least three spatial dimensions, the object can basically be visualized by plotting the first data on a three-dimensional grid. The respective values ​​of a data point for the three spatial dimensions are also referred to as spatial coordinates (x, y, z).

[0013] In a first step of the method, information data of a two-dimensional bounding polygon, preferably a two-dimensional bounding rectangle, is received. Preferably, the two-dimensional bounding rectangle is an axis-aligned bounding box (AABB) of the object on the second data. The second data includes at least two spatial dimensions, which means that the object represented on the second data can be represented, preferably graphically, by plotting the second data in a two-dimensional grid. The respective values ​​of the data points for the two spatial dimensions are also referred to as image coordinates (x, y).

[0014] Each two-dimensional enclosing polygon completely surrounds the object represented in the second data. In other words, for each object, no data points of the second data belonging to the object lie outside the respective polygon. The two-dimensional enclosing polygon, preferably the AABB, is defined by its information data, which are specified by its size, particularly preferably the lengths of two sides of a rectangle, and by a point in the second data, such as the position of a corner point or the position of the rectangle's center, particularly preferably by image coordinates. In particular, in the case of multiple objects, the two-dimensional enclosing polygons may overlap, or one polygon may be completely located within another polygon, for example, if the object represented in the second data is partially hidden. For example, if the second data represents a convoy, typically a portion of the rear of the leading vehicle in the convoy is hidden by the vehicle directly behind it in the technical direction of travel.

[0015] Preferably, the first data and the second data represent the same object. Preferably, the first data and the second data represent the same object at substantially the same time. That is, the first data and the second data are preferably temporally correlated with each other. In this example, substantially the same time or temporally correlated preferably means that the recording time of the second data, i.e., data having at least two spatial dimensions, is between the start and end of recording of the first data, i.e., data having at least three spatial dimensions.

[0016] In a further step of the method, a projection matrix is ​​received, which defines a mathematical mapping of three-dimensional data points of the first data to two-dimensional data points in the second data, i.e., applying the projection matrix to spatial coordinates generates image coordinates.

[0017] In a further step of the method, projected data points are generated by projecting the three-dimensional data points of the first data using the received projection matrix, which means that the projected data points are two-dimensional data points.

[0018] In a further step of the method, the projection data points and / or the three-dimensional data points are divided into depth classes, in particular based on the respective depth values ​​of the three-dimensional data points, i.e., preferably taking into account the frequency distribution of the depth values ​​of the original three-dimensional data points. In particular, a histogram is formed by dividing the projection data points and / or the three-dimensional data points into depth classes (English: bins).

[0019] In a subsequent step, groups of projected data points are formed, taking into account the received information data of the two-dimensional enclosing polygons and the depth classes of the data points, i.e., the frequency distribution of the depth values. This is done using an assignment method such that in the case of overlapping polygons in which the projected data points are located inside two or more two-dimensional enclosing polygons, groups are formed in such a way that a depth class is not assigned to more than one overlapping polygon at the same time. That is, one aspect of the invention is that each polygon is assigned a depth class, and in particular, in the case of overlapping polygons, this assignment is not found for each polygon individually, but is performed globally.

[0020] In situations where no overlapping polygons exist, it is preferably envisaged that the groups of projection data points are formed by considering a partitioning of the projection data points and / or three-dimensional data points into depth classes such that for projection data points located inside each polygon, the respective polygon is assigned the depth class with the maximum value, i.e., for projection data points located inside polygons, the frequency distribution of depth values ​​is considered and each polygon is assigned the depth class with the maximum value in the frequency distribution of the corresponding polygon.

[0021] In the case of overlapping polygons, and therefore projected data points that lie inside two or more two-dimensional enclosing polygons, the assignment method, as already mentioned, ensures that groups are formed such that a depth class is not assigned to more than one overlapping polygon at the same time, preferably by first aggregating the overlapping polygons into a global polygon, and more preferably by combining the overlapping polygons into a global polygon.

[0022] Preferably, for the projection data points located inside the overall polygon, the projection data points and / or the three-dimensional data points are then divided into depth classes such that each individual polygon of the overall polygon is assigned a depth class, with consideration given to not assigning a depth class to multiple overlapping polygons at the same time, thereby forming groups of projection data points. That is, preferably, the assignment method matches available depth classes with polygons. More preferably, a weighted assignment method is envisaged in which the assignment of depth classes to polygons maximizes the sum of the projection data points assigned to the individual polygons.

[0023] The method has the advantage in particular that it is not very computationally intensive and therefore highly resource-saving and efficient, and is therefore particularly suitable for processing large amounts of data quickly.

[0024] According to a preferred development of the invention, it is provided that the first data is a point cloud recorded by means of a lidar or radar sensor, i.e. the first data is preferably sparse data, since a lidar or radar sensor records values ​​only at a few discrete spatial coordinates.

[0025] According to a preferred embodiment of the invention, the second data is image data recorded by a camera, and the projection matrix is ​​particularly preferably defined by, inter alia, the orientation and position of the sensor used to record the first data relative to the camera and by the focal length of the camera.

[0026] The received information data of the two-dimensional enclosing polygon can basically be generated by manually annotating image data recorded by a camera. However, it is preferably envisaged that the image data recorded by the camera is evaluated using machine learning, and thus the two-dimensional enclosing polygon, preferably the AABB, is determined. In this context, according to a further preferred development of the invention, it is envisaged that the step of receiving the information data of the two-dimensional enclosing polygon comprises receiving information data of a two-dimensional minimum enclosing polygon, preferably a two-dimensional minimum enclosing rectangle, and scaling the minimum enclosing polygon by a factor greater than 1, preferably less than or equal to 1.5, and / or that the step of receiving the information data of the two-dimensional enclosing polygon comprises receiving information data of the two-dimensional minimum enclosing polygon scaled by a factor greater than 1, preferably less than or equal to 1.5. Preferably, the factor is less than or equal to 1.3, particularly preferably less than or equal to 1.2.

[0027] Preferably, the 2D minimum bounding polygon is an axis-aligned rectangle (AABB). Preferably, the object touches all four sides of the 2D minimum bounding rectangle. A compact, so-called closed and bounded 2D object has exactly one minimum AABB. This is the smallest axis-aligned rectangle that encloses the object. The minimum AABB can be determined, for example, via a minimum and maximum search through the coordinates of all corner points of the object in the second data. It has been shown that the method provides better results if the minimum bounding polygon is slightly enlarged to account for some imprecision in the projection matrix, etc., because in this way projected data points slightly outside the minimum bounding polygon are also taken into account.

[0028] Basically, the bin width of the depth classes can be different for different depth classes. However, it is preferably envisaged that the step of dividing the projection data points and / or the three-dimensional data points into depth classes based on the respective depth values ​​of the three-dimensional data points comprises dividing the projection data points and / or the three-dimensional data points into depth classes having the same class width. This makes the method particularly simple. It is further preferably envisaged that the class width depends on the number of polygons in the second data, i.e., the number of received information data of so-called two-dimensional enclosing polygons. Preferably, a small class width is selected when the number of polygons is large, and a large class width is selected when the number of polygons is small.

[0029] Basically, the step of forming a group of projection data points assigned to each object can be performed using a brute force method in which all possible assignments of depth classes to polygons are tried, although this would be computationally intensive. Alternatively, according to a preferred development of the invention, the step of forming a group of projection data points assigned to each object is performed using a Hungarian algorithm, taking into account the received information data of the two-dimensional enclosing polygon and the depth classes of the data points. This Hungarian algorithm, also known as the Kuhn-Munkres algorithm, is a very efficient algorithm for solving weighted assignment problems.

[0030] The use of the Hungarian method has the additional advantage that it can reliably find an optimal solution. Other methods exist that find equivalent or nearly equivalent solutions without requiring significantly more computational effort. For example, dynamic programming or the Ford-Fulkerson algorithm can be used to similarly find an appropriate assignment of points to depth classes.

[0031] According to a preferred development, it is also envisaged that the step of forming groups of the projection data points assigned to each object comprises forming the groups in consideration of the information data of the received two-dimensional enclosing polygon and the depth class of the data points, such that no projection data point is simultaneously assigned to more than one group, i.e. in other words, the method is preferably a strictly partitioned clustering or a strictly partitioned clustering with outliers, in which case no data point or projection data point of the first data may belong to more than one cluster.

[0032] As already mentioned, each polygon is assigned a depth class. In this connection, according to a further preferred further development, it is envisaged that the method comprises a step of assigning distance information, preferably exactly one distance information, to each formed group. The assigned distance information can preferably be used to determine the three-dimensional envelope of the object represented on the first data.

[0033] Basically, the assigned distance information can be the average value of the depth class assigned to the polygon. For example, for a depth class with a width of 5 m to 6 m, the assigned distance information would be, for example, 5.5 m. However, according to a preferred development of the invention, it is envisaged that the step of assigning distance information to each formed group comprises forming an average value over the depth values ​​of the three-dimensional data points of the group assigned to the respective object. In other words, an average value of the depth values ​​is formed over the three-dimensional data points assigned to the depth class assigned to the polygon. This average value is then preferably assigned to the polygon as distance information.

[0034] In this regard, according to a further preferred development of the invention, it is envisaged that the step of assigning distance information to each formed group comprises adding a prior knowledge term that depends on the classification result of the object, and the method preferably comprises a step of receiving the classification result of the object.

[0035] In the first data, the object is scanned by the sensor only on the surface aligned with the sensor due to its self-occlusion. In other words, in the first data, data points from objects located further away from the sensor are missing. If the distance information found based on the described method is used for the center of gravity of the three-dimensional envelope, this may lead to systematic errors due to the missing data points. To compensate for this error, it is preferably assumed that a prior knowledge term is added to the distance information. That is, preferably, in the first step of the method, not only information data of the enclosing polygon but also a classification result of the object encompassed by the polygon and represented in the second data is received.

[0036] According to a further preferred development of the invention, the method is envisaged to comprise a step of determining a three-dimensional envelope, preferably a cube, for the object represented on the first data, taking into account the formed group, preferably the assigned distance information, and the received information data of the two-dimensional enclosing polygon.

[0037] The three-dimensional envelope is preferably defined by the size, position, and orientation of the envelope in the first data. In the case of a cube, the size is preferably specified by the lengths of the three sides of the cube, the position is preferably specified by the spatial coordinates of a point, for example, a corner point or the center of gravity of the cube, and the orientation is particularly preferably specified by three angles, for example, roll-pitch-yaw angles. Particularly preferably, the distance information assigned to the group is used as the spatial coordinate for the position of the three-dimensional envelope. The two additional spatial coordinates of the position of the three-dimensional envelope preferably result from the corresponding image coordinates of the polygon in the second data.

[0038] More preferably, a predefined size distribution depending on the object class is used for the size of the three-dimensional envelope, and particularly preferably, a Gaussian distribution of sizes is used. For example, a first predefined size distribution is used for small cars and / or a second predefined size distribution is used for trucks. Basically, the size of objects, especially vehicles, can be described by a Gaussian mixture model. A mixture model is a model for representing the existence of subpopulations within an entire population.

[0039] According to a further preferred embodiment of the present invention, the step of determining a three-dimensional envelope for the object represented on the first data, taking into account the formed groups and the received information data of the two-dimensional enclosing polygon, comprises shifting and / or scaling the three-dimensional envelope within predetermined limits so that the formed groups of three-dimensional data points are completely surrounded by the three-dimensional envelope. In other words, by shifting and / or scaling the three-dimensional envelope, it is possible to achieve that as many data points of the first data assigned to a group as possible are located inside the determined envelope. Preferably, local information of the first data, such as the local point density of the first data, is also used to identify object surfaces in the first data facing the sensor, and further, this information is also used to bring the three-dimensional envelope as close as possible to these three-dimensional data points considered as surfaces.

[0040] The object of the present invention is also achieved by an apparatus for processing data, which comprises means for implementing the method described above.

[0041] The invention further relates to a computer program product comprising instructions that, when executed by a computer, cause the computer to carry out the method described above.

[0042] Furthermore, according to the present invention there is provided a computer readable data carrier, preferably a non-volatile computer readable data carrier, on which the computer program product as described above is stored.

[0043] The technical advantages of the apparatus for data processing, the computer program product and the computer-readable data carrier will be apparent to those skilled in the art from the description of the method for assigning data points of a first data to each one of a plurality of objects, as well as from the examples described below. [Brief explanation of the drawings]

[0044] [Figure 1] 1 is a schematic diagram illustrating projection data generated in a computer-implemented method for assigning data points of a first data set to respective ones of a plurality of objects, according to a preferred embodiment of the present invention; [Figure 2] 2 is a schematic diagram illustrating a histogram formed from the projection data shown in FIG. 1 in a method according to a preferred embodiment of the present invention; [Figure 3] FIG. 2 is a schematic diagram illustrating two overlapping polygons received in a computer-implemented method for assigning data points of a first data set to respective ones of a plurality of objects, according to a further preferred embodiment of the present invention; [Figure 4] FIG. 4 is a schematic diagram showing projection data for the overlapping polygon shown in FIG. 3. [Figure 5] 3 is a schematic diagram illustrating the steps used in assigning distance information to each formed group of the method according to a preferred embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0045] FIG. 1 illustrates schematically exemplary steps of a computer-implemented method for assigning data points of a first data set to respective ones of a plurality of objects, in accordance with a preferred embodiment of the present invention.

[0046] In the first step of the method, information data is received from a two-dimensional enclosing polygon 10, 16, in this example a two-dimensional enclosing rectangle 10, 16, for an object 14 represented on second data 12. As can be seen from FIG. 1, the second data 12 has two spatial dimensions and is image data 12 recorded by a camera. In this example, information data for a two-dimensional minimum enclosing polygon 16 is first received. This minimum enclosing polygon 16 is then scaled by a factor greater than 1, in this example 1.25, to obtain the enclosing polygon 10.

[0047] The received information data of polygon 10, in this example, defines the size and position of polygon 10 within image data 12. Additionally, a classification result of object 16 represented on image data 12 is received. In this example, object 16 represented on image data 12 is assigned to the class of passenger car vehicle.

[0048] Further, the method receives a projection matrix, where the projection matrix defines a mapping of three-dimensional data points of the first data 26 (not shown in Figures 1-4 but shown in Figure 5) to two-dimensional data points in the second data 12.

[0049] The first data 26, in this example, is lidar sensor data, which is time-correlated with the image data 12 recorded by the camera. The lidar sensor data has three spatial dimensions. Using the projection matrix and the lidar sensor data, a further step of the method can generate the projection data points 18 shown in FIG. 1.

[0050] In a further step of the method, the projected data points 18 are divided into depth classes 20, in particular based on the depth values ​​of each of the original three-dimensional data points 26.

[0051] In a subsequent step, groups of projected data points 18 are formed, in particular taking into account the information data of the received two-dimensional enclosing polygon 10 and the depth classes 20 of the data points 18 .

[0052] As shown in Fig. 1, in a situation where there are no overlapping polygons 10 of multiple objects 14, it is assumed that groups of projection data points 18 are formed by considering the frequency distribution 22 of depth values ​​for projection data points located inside each polygon 10, and assigning to each polygon a depth class 20 having a maximum value in the frequency distribution 22 of the corresponding polygon 10. That is, as shown in Fig. 2, figuratively speaking, a histogram 22 is formed using projection data points 18 located inside polygons 10. That is, in this embodiment, polygons 10 are assigned depth classes 20 having a depth class width of 47M to 48M.

[0053] Figures 3 and 4 illustrate the procedure for multiple objects 14, 14'. As can be seen in Figure 3, the image data 12 has, in this example, two objects 14, 14'. The car 14 here partially obscures the delivery van 14'. Also shown in Figure 3 are the two-dimensional enclosing polygons 10, 10' for the objects 14, 14'. For clarity, the projected data points 18 are not shown in Figure 3.

[0054] In such a situation involving overlapping polygons 10, 10', if histograms 22 were created separately for each polygon 10, 10' as in Figures 1 and 2, the maximum value of frequency distribution 22 would be assigned to polygon 10, 10', which would lead to the two polygons 10, 10' being assigned the same depth class 20, because the majority of the projection data points 18, 18' lying within polygon 10' originate from object 14 (the car) and not from object 14' (the delivery van).

[0055] That is, the method envisions forming groups of projected data points 18,18' using an assignment method such that, in the case of overlapping polygons 10,10' in which data points 18,18' lie inside more than one two-dimensional enclosing polygon 10,10', the groups are formed such that a depth class 20,20' is not simultaneously assigned to more than one overlapping polygon 10,10'. In this example, the Hungarian assignment method is used, which maximizes the number of projected data points 18,18' inside the polygon 10,10' during assignment.

[0056] As shown in Figure 4, this assignment is performed by combining overlapping polygons 10, 10' into a global polygon. Groups of projection data points 18, 18' are then formed by considering the frequency distribution of depth values ​​for projection data points 18, 18' located inside the global polygon.

[0057] FIG. 4 illustrates this with two overlapping polygons 10,10′ and two different depth classes 20,20′ of projected data points 18,18′, where projected data point 18 belongs to depth class 20 and projected data point 18′ belongs to depth class 20′. In this example, the assignment problem solved using the Hungarian method can be expressed in terms of a matrix. The data for the assignment problem is collected in a rectangular matrix, where each row corresponds to a source (i.e., polygon 10,10′ in this example), each column corresponds to a destination (i.e., depth class 20,20′ in this example), and each matrix element contains an estimate of the assignment (i.e., the number of projected data points 18,18′ in this example).

[0058] That is, for the assignment problem shown in Figure 4, the following matrix: [Table 1] occurs.

[0059] The entry for a location (row, column) in this matrix counts the number of points within polygon 10,10' that have depth values ​​that fall within depth class 20,20'.

[0060] The problem is to maximize the sum of the matrix entries corresponding to the assignments. In this case, to achieve this, polygon 10 is assigned depth class 20 and polygon 10' is assigned depth class 20'. The Hungarian method finds this optimal assignment; if polygons 10 and 10' were considered separately, they would both be assigned depth class 20.

[0061] The data points 18, 18' located inside the depth classes 20, 20' assigned to the polygons 10, 10' then form a group of projection data points 18, 18' assigned to the respective objects.

[0062] In a further step of the method, the groups are then assigned exactly one distance value, in this example by forming an average value over the depth values ​​of each of the original three-dimensional data points 26 of the group assigned to each object 14. The assigned distance value is used in this example to determine the three-dimensional envelope 24 (see FIG. 5) of the object 14 represented on the first data 26 as follows:

[0063] Furthermore, as shown schematically in FIG. 5, prior knowledge terms, which depend on the classification results of the object 14, are taken into account when assigning distance information to each formed group. As shown schematically in FIG. 5a), the object 14 is scanned by the sensor only on its surface aligned with the sensor direction based on its inherent shading. In other words, data points from objects 14 located further away from the sensor are largely missing. If the distance information found based on the described method is used for the center of gravity of the three-dimensional envelope 24, this can lead to a systematic error due to "missing data points." This systematic error is illustrated in FIG. 5a by the relatively large distance 28 between the front surface of the three-dimensional envelope 24 and the data points of the object 14. To compensate for this error, this embodiment provides for adding prior knowledge terms to the distance information. As shown in FIG. 5b, this reduces the distance 28 between the front surface of the three-dimensional envelope 24 and the data points of the object 14. [Explanation of symbols]

[0064] 10 2D Enclosing Polygons 12 Second data, image data 14 Object 16 Minimum Enclosing Polygon 18 projected 2D data points 20 Depth Classes 22 Histograms, frequency distributions 24 3D envelope 26 First data, Lidar sensor data 28 distance

Claims

1. 1. A computer-implemented method for assigning data points of a first data set (26) to respective ones of a plurality of objects (14, 14'), comprising: the first data (26) includes at least three spatial dimensions; The method comprises the following steps: - receiving information data from a two-dimensional enclosing polygon (10,10'), preferably a two-dimensional enclosing rectangle (10,10'), for said object (14,14') represented on second data (12), said second data (12) including at least two spatial dimensions, said received information data defining the size and position of the enclosing polygon (10,10') of each said object (14,14') within said second data (12); - receiving a projection matrix defining a mapping of three-dimensional data points of said first data (26) onto two-dimensional data points in said second data (12); - generating projection data points (18, 18') by projecting the three-dimensional data points of the first data (26) using the received projection matrix; - classifying said projection data points (18, 18') and / or said three-dimensional data points into depth classes (20, 20') based on the respective depth values ​​of said three-dimensional data points; forming groups of the projection data points (18, 18') assigned to each object (14, 14') taking into account the information data of the received two-dimensional enclosing polygons (10, 10') and the depth classes (20, 20') of the data points, using an assignment method such that, in the case of overlapping polygons (10, 10') in which the projection data points (18, 18') are located inside more than one two-dimensional enclosing polygon (10, 10'), the groups are formed in such a way that the depth class (20, 20') is not simultaneously assigned to more than one overlapping polygon (10, 10'); A method comprising:

2. the first data (26) is a point cloud recorded using a lidar or radar sensor and / or the second data (12) is image data recorded using a camera; The method of claim 1.

3. the step of receiving information data of the two-dimensional enclosing polygon (10, 10') comprises receiving information data of a two-dimensional minimum enclosing polygon (16), preferably a two-dimensional minimum enclosing rectangle (16), scaling the two-dimensional minimum enclosing polygon (16) by a factor greater than 1, preferably less than or equal to 1.5; and / or the step of receiving information data of the two-dimensional enclosing polygon (10, 10') comprises receiving information data of a two-dimensional minimum enclosing polygon (16) scaled by a factor greater than 1, preferably less than or equal to 1.

5.

3. The method according to claim 1 or 2.

4. the step of classifying the projection data points (18, 18') and / or the three-dimensional data points into depth classes (20, 20') based on the depth values ​​of each of the three-dimensional data points includes classifying the projection data points (18, 18') and / or the three-dimensional data points into depth classes (20, 20') having homogeneous class widths; 4. The method according to any one of claims 1 to 3.

5. forming groups of the projection data points (18, 18') assigned to the respective objects (14, 14') is performed using the Hungarian algorithm, taking into account the information data of the received two-dimensional enclosing polygon (10, 10') and the depth classes (20, 20') of the data points; 5. The method according to any one of claims 1 to 4.

6. The step of forming groups of the projection data points (18, 18') assigned to the respective objects (14, 14') includes forming the groups in consideration of the information data of the received two-dimensional enclosing polygon (10, 10') and the depth classes (20, 20') of the data points so that no projection data point is assigned to more than one group at the same time.

6. The method according to any one of claims 1 to 5.

7. the method comprises the step of assigning to each formed group a distance value, preferably exactly one distance value; 7. The method according to any one of claims 1 to 6.

8. the step of assigning distance information to each formed group includes forming an average value over the depth values ​​of each of the three-dimensional data points of the group assigned to each of the objects (14, 14'). The method of claim 7.

9. the step of assigning distance information to each formed group comprises adding a prior knowledge term dependent on a classification result of the object (14, 14'), and the method preferably comprises a step of receiving a classification result of the object (14, 14').

9. The method according to claim 7 or 8.

10. The method comprises the step of determining a three-dimensional envelope (24), preferably a cube (24), for the object (14, 14') represented on the first data (26) taking into account the formed group, preferably the assigned distance information, and the received information data of the two-dimensional enclosing polygon (10, 10'), 10. The method according to any one of claims 1 to 9.

11. determining a three-dimensional envelope (24) for the object (14, 14') represented on the first data (26) taking into account the formed group and the received information data of the two-dimensional enclosing polygon (10, 10') includes shifting and / or scaling the three-dimensional envelope (24) within predetermined limits so that the formed group of three-dimensional data points is completely enclosed by the three-dimensional envelope (24); The method of claim 10.

12. 1. An apparatus for data processing, comprising: The device comprises means for carrying out the method according to any one of claims 1 to 11. Device.

13. 1. A computer program product comprising: A program comprising instructions that, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 11. Computer program products.

14. A computer-readable data carrier comprising: A computer program product according to claim 13 is stored thereon. A computer-readable data carrier.

Images