Method for detecting spurious objects in sensor measurements - Patents.com

Abstract

A new method is presented for detecting spurious objects in sensor measurements around a vehicle. The present invention relates to a method for detecting spurious objects in sensor measurements around a vehicle. According to the present invention, map context information is obtained from a digital road map. Objects (O) with associated attributes are detected around the vehicle and mutual social context information of the objects (O) is generated. All available data about the traffic situation of the vehicle and the objects (O) are stored in a graph structure (GS) with nodes (K1-Km) and edges (E1-En), and relationship information (RI) is represented in the graph structure (GS) by the edges (E1-En). Taking into account the map context information and the social context information, anomalies and patterns are detected among the features of the graph structure (GS) and spurious objects are classified based on the detected anomalies and patterns.

Description

[Technical field]

[0001] The present invention relates to a method for detecting ghost objects in sensor measurements around a vehicle. [Background technology]

[0002] From the document DE 10 2004 13 2005 A1 a method for a radar system of a vehicle is known, which comprises the steps described below: - detecting two or more objects using the vehicle's radar system; initiating tracks of two or more objects in a track database, where each track stores data regarding the two or more objects and is updated based on additional detection of the two or more objects, the tracks of the two or more objects being initially unclassified tracks in the track database; using a processor to select two tracks corresponding to two of the two or more objects in the track database as a candidate pair; applying a criterion to the candidate pair using a processor to determine whether one of the two tracks of the candidate pair is a track of a false object resulting from multipath reflections and the other of the two tracks of the candidate pair is a track of a real object corresponding to the false object, where the false object is a recording of a real object at a wrong location; based on identifying that one of the two tracks of the candidate pair is a track of a false object and the other of the two tracks of the candidate pair is a track of a real object corresponding to the false object, classifying the candidate pair in the track database as tracks of a real object and a false object pair using a processor; Information from the track database is reported, including providing data only about tracks of real objects based on classification, and the information is used to control the operation of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] German Patent Application Publication No. 10 2020 124 236 A1 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to provide a new method for detecting spurious objects in sensor measurements around a vehicle. [Means for solving the problem]

[0005] This problem is solved according to the invention by a method having the features set forth in claim 1.

[0006] Advantageous embodiments of the invention are the subject of the dependent claims.

[0007] In accordance with the present invention, a method for detecting spurious objects in sensor measurements around a vehicle includes obtaining map context information from a digital road map, detecting objects with related attributes around the vehicle, and generating mutual social context information of the objects. All available data of traffic situations with vehicles and objects are stored in a graph structure including nodes and edges, and relationship information is represented in the graph structure by the edges. Taking into account the map context information and the social context information, anomalies and patterns are searched for among the features of the graph structure, and spurious objects are classified based on the detected anomalies and patterns.

[0008] Automated driving, in particular highly automated or autonomous driving, requires a precise detection of objects in the vehicle's surroundings. To meet the high requirements for safety in automated driving, e.g. according to SAE J3016 standard levels 2, 3, 4 and 5, sensor measurements from sensor modalities such as camera sensors, radar sensors and / or lidar sensors are required. In this case, the sensor modalities form in particular redundant sensor systems with several similar and / or different sensor modalities. The sensor measurements of the sensor modalities here can be used individually or in an integrated manner.

[0009] To maximize safety, the delay time of the vehicle's reaction to an object must be minimized. For this reason, even a few or even individual sensor measurements can trigger a system reaction. In addition to true positive sensor measurements, there are also false positive sensor measurements, which are the detection of an object where none actually exists. False positive measurements can arise, for example, due to undesirable sensor reflections, false positives, and / or the detection of surrounding objects, such as a vehicle depicted in a poster advertisement. Typical examples are radar reflections on guard rails or traffic signs.

[0010] In autonomous vehicles, such false positive measurements generate objects, commonly called false objects due to the fact that they do not exist in reality. If an autonomous vehicle reacts to a false object, it may trigger dangerous situations such as emergency braking, which may result in a collision.

[0011] The method allows for a particularly effective and reliable detection of spurious objects. By using information about all objects in a traffic situation or scene, the method can determine for an object whether its attributes show an anomaly compared to the surrounding objects. The method uses context information to provide a comprehensive representation of the scene, enabling a context-based classification of objects, for example using the underlying road network. The method also allows for the consideration of temporal aspects, and allows for the extraction of patterns over the temporal dimension.

[0012] The method thus allows for a highly sensitive recording of the vehicle's surroundings and then for spurious objects, i.e. false positive measurements, to be recognised and removed from the recording, thereby reducing the risk that real objects in the surroundings go undetected, i.e. the probability of false negative and false positive measurements is significantly reduced.

[0013] In a possible embodiment of the method, extensions to the learning-based classification are provided, allowing for the extraction of complex patterns that cannot be defined manually, and thus making the method and the system implementing it data-scalable.

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. [Brief description of the drawings]

[0015] [Figure 1] FIG. 2 is a schematic diagram showing a traffic situation from above. [Diagram 2] 2 is a schematic top-down view of the traffic situation according to FIG. 1, in which the nodes of the graph structure are generated; [Diagram 3] FIG. 3 is a schematic diagram of the nodes according to FIG. 2 of the graph structure and the edges connecting them. [Figure 4] FIG. 2 is a schematic diagram of a graph structure showing multiple nodes and relationships between multiple nodes. [Diagram 5]FIG. 1 is a schematic diagram illustrating the construction of a graph structure and the processing of existing information in the graph structure by a graph-based artificial neural network. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] In all the drawings, the same reference numerals are used to designate corresponding parts.

[0017] 1 shows a top-down view of the traffic situation at an intersection, which contains objects O6-O11 formed as lane segments from a digital road map. Furthermore, in the area of ​​the intersection there is an object O3 formed as a traffic light, an object O4 formed as a stop sign, an object O5 formed as a pedestrian crossing and two objects O1, O2 formed as vehicles.

[0018] For automated driving, particularly for highly automated or autonomous driving, precise detection of objects O1-O11 around the vehicle is required. For this purpose, the automated driving vehicle has a surrounding detection sensor system, for example including a lidar, radar, camera, and ultrasonic sensor. In this case, each sensor modality may be present redundantly. When recording the surroundings, the sensor measurements of the sensor modalities are used individually or in an integrated manner. Here, it is necessary to avoid false negative measurement results, i.e. the real objects O1-O11 are not detected in the surroundings, and false positive measurement results, i.e. the detection of false objects O1-O11, so-called false objects.

[0019] When recording the surroundings with low sensitivity, there is a risk that all real objects O1 to O11 are not detected in the surroundings, which may result in dangerous situations in the automatic driving of the vehicle.

[0020] Conversely, if the surroundings are recorded with high sensitivity, there is a risk that false objects that are not present in the surroundings will be detected in the surroundings, which again could lead to dangerous situations in the automated driving of the vehicle.

[0021] In order to be able to determine whether the objects O1-O11 are actually present, the method for detecting spurious objects in sensor measurements uses context information of the traffic situation or traffic scene, which makes it possible to first record the surroundings with high sensitivity and then detect spurious objects in the recorded sensor measurements and remove them from these measurements.

[0022] Here, important information is provided by the road network on which the traffic situation is based, which is taken from a digital road map as map context information, where the road network is represented by objects O6-O11 which are formed in particular as lane segments.

[0023] Further important information is provided by other vehicles around the vehicle, here shown for example by objects O1, O2. From the mutual relations of objects O1, O2, social context information is generated.

[0024] In order to achieve high sensitivity in recording the surroundings, in the method all available traffic situation data is stored in a graph structure GS, the so-called scene graph, which is shown in detail in Fig. 5. In this graph structure GS, relational information is represented by edges E1 to En connecting nodes K1 to Km.

[0025] FIG. 2 shows a top view of the traffic scene according to FIG. 1, with the generated nodes K1 to K11 shown for a possible embodiment of the graph structure GS.

[0026] To create the graph structure GS, nodes K1 to Km are formed for all available dynamic information DI, static information SI, and semantic information SEI (each of which is shown in detail in FIG. 5).

[0027] In this case, the static information SI includes objects O6 to O11 formed, for example, as lane segments, the dynamic information DI includes objects O1, O2 formed, for example, as vehicles and their trajectories, and the semantic information SEI includes object O4 formed, for example, as a stop sign, object O3 formed, for example, as an indicator light such as a traffic light, object O5 formed, for example, as a pedestrian crossing, and other traffic control facilities.

[0028] Here, for example, nodes K6 to K11 are assigned to each lane segment (objects O6 to O11), node K4 is assigned to the primary stop sign (object O4), node K5 is assigned to the pedestrian crossing (object O5), node K3 is assigned to the traffic light (object O3), and nodes K1 and K2 are assigned to objects O1 and O2 formed as vehicles, respectively.

[0029] FIG. 3 shows the nodes K1 to K11 and the edges E1 to E14 connecting them according to FIG. 2 for a possible embodiment of the graph structure GS.

[0030] In this case, edges E1 to E14 represent relationship information RI, also shown in detail in FIG. 5, ie dependencies and relationships between nodes K6 to K11.

[0031] For example, edge E1 represents, as relationship information RI, that an object O8 (node ​​K8) formed as a lane segment is connected to an object O6 (node ​​K6) formed as a lane segment. For example, when passing through an intersection, the fact that object O8 is connected to object O6 is stored as a relationship-specific attribute.

[0032] For example, edge E2 represents, as relationship information RI, that an object O10 (node ​​K10) formed as a lane segment precedes an object O8 (node ​​K8) formed as a lane segment, and that the object O10 is the first precedent of an object O6, for example, is stored as a relationship-specific attribute.

[0033] For example, edge E3 represents, as relationship information RI, that an object O6 (node ​​K6) formed as a lane segment is connected to an object O11 (node ​​K11) formed as a lane segment. For example, the fact that object O6 is adjacent to object O11 on the right side is stored as an attribute specific to the relationship.

[0034] For example, edge E4 represents, as relationship information RI, that an object O11 (node ​​K11) formed as a lane segment is connected to an object O6 (node ​​K6) formed as a lane segment. For example, the fact that object O11 is adjacent to object O6 on the left is stored as an attribute specific to the relationship.

[0035] For example, edge E5 indicates, as relationship information RI, that the regulation content of a traffic light (object O3, node K3) is applied to object O6 (node ​​K6) formed as a lane segment.

[0036] For example, edge E6 represents, as relationship information RI, that an object O11 (node ​​K11) formed as a lane segment is below an object O1 (node ​​K1) formed as a vehicle. For example, the fact that the lane segment is below the vehicle with a certain probability (e.g., 1.0) is stored as a relationship-specific attribute.

[0037] For example, edge E7 represents, as relationship information RI, that an object O7 (node ​​K7) formed as a lane segment is connected to an object O6 (node ​​K6) formed as a lane segment. For example, when passing through an intersection, the fact that object O7 is connected to object O6 is stored as a relationship-specific attribute.

[0038] For example, edge E8 represents, as relationship information RI, that an object O7 (node ​​K7) formed as a lane segment is below an object O2 (node ​​K2) formed as a vehicle. For example, the fact that the lane segment is below the vehicle with a certain probability (e.g., 0.7) is stored as a relationship-specific attribute.

[0039] For example, edge E9 represents, as relationship information RI, that object O8 (node ​​K8) formed as a lane segment is below object O2 (node ​​K2) formed as a vehicle. For example, the fact that the lane segment is below the vehicle with a certain probability (e.g., 0.3) is stored as a relationship-specific attribute.

[0040] For example, the edge E10 represents, as the relationship information RI, that an object O10 (node ​​K10) formed as a lane segment is connected to an object O9 (node ​​K9) formed as a lane segment. For example, the fact that the object O10 is connected to the object O9 is stored as an attribute specific to the relationship.

[0041] For example, edge E11 indicates, as relationship information RI, that the regulation content of a stop sign (object O4, node K4) is applied to object O9 (node ​​K9) formed as a lane segment.

[0042] For example, edge E12 indicates, as relationship information RI, that the pedestrian crossing (object O5, node K5) passes over, or is overlapped with, object O9 (node ​​K9) formed as a lane segment.

[0043] For example, edges E13 and E14 represent, as relationship information RI, that objects O1 and O2 (nodes K1 and K2) formed as vehicles interact with each other. - the geometric relationship between the objects O1, O2, e.g. the position difference between the vehicles in meters, the geometric relationship between the objects O1, O2, e.g. the difference in yaw angle between the vehicles, i.e. the difference in the orientation of the vehicles; - the kinematic relationship between objects O1 and O2, e.g. the speed difference between vehicles It is.

[0044] The above-mentioned objects O1-O11, related information RI, and attributes are possible examples. The approach described herein is not limited to the above-mentioned object types, related information RI, and attributes, but can be extended to any other possible object types, related information RI, and attributes.

[0045] Taking into account the context information, anomalies and patterns in the graph structure GS can be found (detected), which anomalies and patterns are used to classify the false objects. As features for classifying the objects O1-O11, the attributes of the objects O1-O11, detected (recorded), for example, using the respective sensor measurements, are used. These attributes can take into account, for example, the kind of the respective objects O1-O11, e.g., vehicle, road sign, pedestrian, etc., and / or the type of the respective objects O1-O11. For example, for objects O1-O11 with vehicular properties, possible object types are cars, trucks, buses, motorbikes, etc. The attributes can also include information describing further objects O1-O11, e.g., kinematic state and / or position information.

[0046] That is, the sensor measurements (measurement results) of the objects O1 to O11 to be classified each contain not only the object O1 to O11 itself but also the unique attributes of the object O1 to O11 that are useful for classification.

[0047] Furthermore, features for classifying the objects O1 to O11 include attributes of the relationships between the objects O1 to O11, such as - the geometrical relationships between the objects O1 to O11, and / or - the geometrical relationships between the objects O1 to O11 and their positions in their surroundings, and / or - the kinematic relationships between the dynamic objects O1 to O11, and / or - the semantic relations between the objects O1 to O11, and / or -Static relations between objects O1 to O11 can be used.

[0048] For example, attributes of the relationships between the objects O1 to O11 can be used as features for classifying the objects O1 to O11. These attributes can be, for example, -temporal information, the distance of the objects O1-O11 to the vehicle at the first detection of the objects O1-O11 and / or at the first sensor measurement of a series of measurements, the speed of the objects O1-O11 in comparison with the surrounding objects O1-O11, the position of the objects O1 to O11 in relation to the road network, and / or - the orientation of objects O1-O11 relative to the orientation of the lane segments below them Contains:

[0049] This means, for example, that the first sensor measurement can be classified. However, multiple "first" sensor measurements may also be classified. There is a requirement for the autonomous vehicle within what time it must react to surrounding objects O1-O11. If multiple sensor measurements of the same sensor are made within this time, multiple "first" sensor measurements can also be used for classification. Thus, the sensor measurements from this time can also be used to establish the historical context of the objects O1-O11 and provide useful patterns. This makes it possible, for example, to record the behavior of the vehicle during this time and take it into account already before the real objects O1-O11 are detected.

[0050] Furthermore, in a possible embodiment, in sensor measurements using multiple sensors, i.e. redundant sensors, the object hypotheses established from the sensor measurements of the individual sensors are taken into account and, based on at least one object hypothesis of at least one sensor measurement of one sensor, the detection of a false object in a sensor measurement performed using at least one other sensor is checked for plausibility or inaccuracy.

[0051] FIG. 4 shows a possible embodiment of a graph structure GS having a number of nodes K1 to Km and edges E1 to En representing relationships between the nodes K1 to Km.

[0052] In this case, node K1 is for example a pedestrian crossing, node K2 is a signal light, node K3 is a traffic lane, node K4 is a road user and node Km is a stop sign.

[0053] Edges E1 to En are formed between the nodes K1 to Km, and these reflect the relationships between the nodes K1 to Km, that is, relationship information RI.

[0054] For example, edge E1 indicates, as relationship information RI, that a signal light (node ​​K2) indicates or signals a pedestrian crossing (node ​​K1).

[0055] For example, edge E2 indicates, as relationship information RI, that a road user (node ​​K4) crosses the pedestrian crossing (node ​​K1).

[0056] For example, edge E3 indicates, as relationship information RI, that a pedestrian crossing (node ​​K1) passes above or overlaps a traffic lane (node ​​K3).

[0057] For example, edge E4 indicates, as relationship information RI, that a road user (node ​​K4) is on a lane (node ​​K3).

[0058] For example, edge E5 indicates, as relationship information RI, that a lane (node ​​K3) is below a road user (node ​​K4).

[0059] For example, edge E6 indicates, as relationship information RI, that a lane (node ​​K3) abuts on another node K5 to Km-1 (not shown in detail), or that node K5 to Km-1 abuts on the lane.

[0060] For example, edge E7 indicates, as relationship information RI, that a lane (node ​​K3) is connected to other nodes K5 to Km-1 (not shown in detail), or that nodes K5 to Km-1 are connected to the lane.

[0061] For example, edge E8 indicates as relationship information RI that a road user (node ​​K4) interacts with other nodes K5 to Km-1, not shown in detail.

[0062] For example, the edge E9 indicates, as relationship information RI, that the regulation content of the indicator light (node ​​K2) is applied to the lane (node ​​K3), that is, that the objects O1 to O11 on the lane are controlled.

[0063] For example, edge E10 indicates, as relationship information RI, that a lane (node ​​K3) precedes another node K5-Km-1 (not shown in detail), or that node K5-Km-1 precedes the lane.

[0064] For example, edge En indicates, as relationship information RI, that the regulation of a stop sign (node ​​Km) is applied to a lane (node ​​K3), that is, that objects O1 to O11 on this lane are to be stopped.

[0065] The depicted graph structure GS represents only one possible embodiment of such a graph structure and can be flexibly extended or restricted depending on requirements and circumstances.

[0066] FIG. 5 illustrates the construction of a graph structure GS and the processing of existing information in the graph structure GS by a graph-based artificial neural network N.

[0067] In a method for detecting spurious objects in sensor measurements of a vehicle's surroundings, classification of spurious objects is performed on a learning basis.

[0068] At this time, the static information SI, the dynamic information DI, the semantic information SEI, and the relationship information RI are transferred to the graph structure GS as input information of the traffic situation.

[0069] Subsequently, a form of learning-based approach, in particular a graph-based artificial neural network N, is used, in which the output information AI provides information on the classification of the individual objects O1-O11 as false objects and on the objects O1-O11 that are actually present.

Claims

1. 1. A method for detecting spurious objects in sensor measurements of a vehicle's surroundings, comprising: Map context information is obtained from digital road maps, Objects (O1-O11) having associated attributes are detected around the vehicle; Mutual social context information of the objects (O1-O11) is generated; all available data relating to the traffic situation of said vehicles and said objects (O1-O11) are stored in a graph structure (GS) comprising nodes (K1-Km) and edges (E1-En), and relational information (RI) is represented in said graph structure (GS) by said edges (E1-En), anomalies and patterns are detected among the features of the graph structure (GS) taking into account the map context information and the social context information; False objects are classified based on the detected anomalies and patterns. A method comprising:

2. In sensor measurements using multiple sensors, The object hypotheses established from the sensor measurements of the individual sensors are taken into account; Based on the at least one object hypothesis of the at least one sensor measurement of the sensor, a detection of a false object in a sensor measurement performed with at least one further sensor is checked for plausibility or plausibility.

2. The method according to claim 1 , characterized in that

3. The graph structure (GS) has the following characteristics: the geometrical relationships between said objects (O1-O11), and / or the geometrical relationships between said objects (O1-O11) and their positions in their surroundings, and / or the kinematic relationships between said dynamic objects (O1-O11), and / or Semantic relations between the objects (O1-O11), and / or Static relationships between the objects (O1 to O11) is used 3. The method according to claim 1 or 2.

4. The graph structure (GS) has the following characteristics: Temporal information, and / or the distance from the vehicle to at least one object (O1-O11) at the first detection of the object (O1-O11) and / or at the first sensor measurement of a series of measurements, and / or the speed of at least one object (O1-O11) in comparison with further surrounding objects (O1-O11), and / or the position of at least one object (O1-O11) in comparison with a road network determined from context information of said map, and / or the orientation of at least one object (O1-O11) in comparison with the orientation of the lane below said object (O1-O11); is used The method according to any one of claims 1 to 3.

5. The classification is performed on a learning basis by a graph-based artificial neural network (N). The method according to any one of claims 1 to 4.

Images