Multi-hypothesis object tracking for automated driving systems

Abstract

Automated driving system for a vehicle (105), wherein the system comprises the following: an electronic processor (200) configured to perform the following functions: To receive environmental information, to generate pseudo-measurement data that are associated with an object in an environment of the vehicle (105), to determine a set of linking hypotheses relating to the object based on the environmental information and the pseudo-measurement data, to determine an object status of the object based on the linking hypothesis set and to control the vehicle (105) based on the specific object status, wherein the electronic processor (200) is configured to generate multiple measurement model hypotheses for each measurement value contained in the environmental information as pseudo-measurement data, each of the multiple measurement model hypotheses being based on a different assumption, where at least one of the several measurement model hypotheses (705, 710) is based on an object classification assumption, wherein the electronic processor (200) is configured to receive environmental information from at least two sensors (115), wherein the environmental information is linked to a common field of view of the at least two sensors (115), where the pseudo-measurement data contains negative information linked to the common field of view.

Description

RELATED REGISTRATIONS

[0001] This application claims the benefit of the preliminary US patent application 62 / 854,821, filed on May 30, 2019, the entire contents of which are incorporated herein by reference. INVENTION AREA

[0002] Embodiments relate to multi-hypothesis object tracking for automated driving systems and, in particular, to robustness against errors in object tracking with multi-hypothesis approaches.

[0003] For the state of the art, reference is made to US 9,933,781 B1, which discloses an autonomous vehicle and a corresponding method for operating the vehicle. In this process, a state estimation module determines the vehicle's current location. A global route planning module determines a route to a specified destination. An action primitive planning module retrieves a driving trend from the current location, selects a sequence of action primitives based on this trend and the determined route, and generates waypoints for the vehicle based on this sequence, with each waypoint containing location coordinates and a direction. A trajectory planning module determines a trajectory for the vehicle based on the waypoints. Finally, a vehicle control module controls the vehicle's propulsion systems based on the determined trajectory.

[0004] Furthermore, DE 10 2008 013 366 A1 discloses a method for providing information to driver assistance systems. In this method, object properties relating to at least one object in the vicinity of the vehicle are provided by a measurement data processing unit of a motor vehicle equipped with a plurality of different environmental sensors to at least one driver assistance system of the motor vehicle.

[0005] Reference is also made to the publication by WYFFELS, Kevin; CAMPBELL, Mark: Negative observations for multiple hypothesis tracking of dynamic extended objects. In: 2014 American control conference, June 4-6, 2014, Portland, Oregon, USA. ACC, 2014. pp. 642-647. ISBN 978-1-4799-3274-0, as well as GRANSTRÖM, Karl; BAUM, Marcus; REUTER, Stephan: Extended object tracking: introduction, overview and applications. In: Journal of advances in information fusion, Vol. 12, 2017, No. 2, pp. 139-174. SUMMARY

[0006] Driver assistance and automated driving (collectively referred to here as "automated driving") detect one or more objects, such as another vehicle, a pedestrian, a lane marking, a curb, or similar, within an area or environment surrounding the vehicle, using a variety of sensors with numerous overlapping fields of view. Using the sensor information, an automated driving system can determine one or more parameters associated with a detected object, such as its position, direction of movement, or similar attributes (a specific object state). The automated driving system then controls the vehicle based on this determined object state. Accurate determination of the object state enables the vehicle to be controlled correctly (for example, steered to avoid a collision with the object).

[0007] However, object tracking errors are often caused by faulty measurement models, errors in sensor data preprocessing, or a combination thereof. Incorrect classification of a detected object can lead to the use of a faulty measurement model. For example, if an object is incorrectly classified as a cyclist when it is actually a vehicle, a measurement model specific to cyclists might be used to track the object. Because the object is not actually a cyclist, using the cyclist measurement model can lead to inappropriate vehicle control. Another example: During sensor data preprocessing, clustering lidar stixels to form an "L" shape can incorrectly detect one object when two objects are actually present.Such errors can lead to an incorrect object status, a tracking interruption, or a combination thereof. As a result, the vehicle may be controlled in an inappropriate manner due to the incorrect object status.

[0008] Accordingly, there is a need to improve robustness against incorrect assumptions in measurement models, data preprocessing, or a combination thereof, ultimately enhancing object tracking accuracy for use with an automated driving system. To address this and other challenges, the embodiments described here provide, among other things, methods and systems for multi-hypothesis object tracking, thus employing hypothesis generation to achieve robustness against errors in measurement models, data preprocessing, and similar processes. The embodiments described here provide multiple measurement models, preprocessing steps, or a combination thereof, which can be systematically used in parallel under different assumptions. These assumptions may relate to an object class, an object shape, diverse methods for generating clusters from raw measurement values, and similar considerations.

[0009] The invention, in a first aspect, discloses an automated driving system according to independent claim 1.

[0010] In another aspect, a method for multi-hypothesis object tracking according to independent claim 7 is disclosed.

[0011] Furthermore, a non-volatile, computer-readable medium according to independent claim 12 is disclosed.

[0012] Other aspects and embodiments will become apparent from consideration of the dependent claims, the detailed description and the associated drawings. BRIEF DESCRIPTION OF THE DRAWINGS The Fig. Figures 1A and 1B illustrate exemplary objects in an environment surrounding a vehicle, in accordance with the state of the art. The Fig. Figure 2 illustrates an exemplary environment surrounding a vehicle, in accordance with the state of the art. The Fig. Figure 3 schematically illustrates a vehicle equipped with a multi-hypothesis object tracking system according to some embodiments. The Fig. Figure 4 schematically illustrates a control of the system of Fig. 3 according to some embodiments. The Fig. Figure 5 schematically illustrates an exemplary common field of view of two sensors according to some embodiments. The Fig. Figure 6 is a flowchart illustrating a multi-hypothesis object tracking procedure implemented by the system of Fig. 3 is carried out according to some embodiments. The Fig. Figure 7 is a process model that illustrates the determination of an object based on environmental information provided by the system. Fig. 3 is implemented according to some embodiments. DETAILED DESCRIPTION

[0013] Before any embodiments are explained in detail, it is understood that the applications of these embodiments are not limited to the design details and component arrangements set forth in the following description or illustrated in the following drawings. Other embodiments are possible, and the embodiments described and / or illustrated herein can be implemented or carried out in various ways. Although the examples described herein relate, for instance, to automated driving systems, the methods described herein can be applied in other embodiments to driver assistance systems, traffic control systems, traffic management systems, safety monitoring systems, and the like. It is also understood that the term "vehicle" refers to various vehicles, including, for example, passenger cars, trucks, boats, motorcycles, drones, and others.

[0014] It should also be noted that several hardware- and software-based devices, as well as several different structural components, can be used to implement the embodiments disclosed herein. Additionally, embodiments may include hardware, software, and electronic components or modules, which, for the purposes of discussion, may be illustrated as if the majority of the components were implemented exclusively in hardware. However, those skilled in the art, even after reading this detailed description, would recognize that in at least one embodiment, the electronically based aspects can be implemented as software (for example, stored on a non-volatile, computer-readable medium) that is executable by one or more processors.Therefore, it should be noted that multiple hardware and software-based devices, as well as several different structural components, can be used to implement various embodiments. It is also understood that, although certain drawings illustrate hardware and software located in specific devices, these illustrations are for illustrative purposes only. In some embodiments, the illustrated components can be combined or separated into separate software, firmware, and / or hardware. For example, instead of logic and processing being located on and performed by a single electronic processor, they can be distributed across multiple electronic processors.Regardless of the way they are combined or divided, hardware and software components can be located on the same computing facility or distributed across different computing facilities connected by one or more networks or other suitable communication links.

[0015] In driver assistance and automated driving systems, objects in a vehicle's environment can be represented as an object list that describes them. An object can be created through sensor data fusion, object tracking over time, or a combination of these methods. Based on the object's properties, a function, determined by the vehicle's processor, decides whether a response to the object is necessary and, if so, what that response should be.In automated driving, especially in the field of sensor data fusion, it is desirable to place high demands on object quality, avoiding false negatives (such as accidentally missing / overlooking actual objects) and false positives (such as preventing faulty triggering due to incorrectly assumed objects).

[0016] As noted above, object tracking errors can be caused by faulty measurement models, errors in sensor data preprocessing, or a combination thereof. Incorrect object classification can lead to a faulty measurement model. An error in a measurement model can cause an incorrect update, which in turn can result in an incorrectly determined object state (for example, object classification, dynamic state, object extent, or a combination thereof), tracking termination, or a combination thereof.

[0017] For example: The Fig. Figures 1A and 1B illustrate the detection of a secondary vehicle 50 or a cyclist 55 using a radar sensor 60 according to the state of the art. As shown in the Fig. As can be seen in Figure 1A, a radar reflection (represented by the symbol associated with reference number 62) is located at an edge or corner of the secondary vehicle 50 that is closest to the radar sensor 60. As shown in the Fig. In contrast to the image shown in Figure 1B, a principal reflection (represented by the symbol associated with reference number 65) is located in the center of the cyclist 55 (for example, the person sitting on the bicycle). Accordingly, an incorrect classification of an object (such as classifying the secondary vehicle 50 as the cyclist 55 or vice versa) can ultimately lead to the use of an incorrect measurement model.

[0018] An error in the preprocessing of a measurement (for example, during clustering) can also produce similar undesirable results. For example: Fig. Figure 2 illustrates an exemplary environment surrounding a vehicle 70 (equipped with an automated driving system). In the illustrated example, the environment surrounding vehicle 70 contains a first object 75A and a second object 75B (both shown as secondary vehicles). As shown in the Fig. As shown in Figure 2, the clustering of lidar stixels to form an "L" shape (80) can incorrectly determine that only one object is present, instead of the two objects actually being present (in this example, the first object 75A and the second object 75B). This can lead to a pseudo-measurement (described in more detail below) resulting from the fact that the clustering used to form an object is not combined during sensor fusion processing, and therefore the object is not updated with the lidar data. This can lead to a tracking failure or an update that occurs with partially incorrect measurements, resulting in an incorrect object status. As a result of such an error, an automated driving system might perform an incorrect vehicle maneuver or react inappropriately to the object.

[0019] Measurement models and preprocessing steps for sensor data are increasingly being used with machine learning methods (for example, deep learning). These can be prone to errors in unpredictable situations. Such errors can have consequences in situations where a low false negative rate is required. This can lead to an incorrect update of an object and thus to a tracking failure. For example, an error can be caused by the object itself. For instance, it could be an object that is not present in the training data of the deep learning / neural network, which can lead to incorrect results. If such an error occurs, the assisted / automated driving system may no longer be able to react appropriately to an object, which can increase the risk of a collision.

[0020] To solve these and other problems, the embodiments described here provide, among other things, methods and systems for multi-hypothesis object tracking, thus using hypothesis generation to achieve robustness against errors in measurement models, data preprocessing, and the like. The embodiments described here provide multiple measurement models, preprocessing steps, or a combination thereof, which can be systematically used in parallel under different assumptions, such as object classification, object shape, diverse methods for generating clusters from raw measurement values, and the like.

[0021] The Fig. Figure 3 illustrates a system 100 for multi-hypothesis object tracking for an automated driving system of a vehicle 105. Although the vehicle 105 is illustrated as a four-wheeled vehicle, it can encompass various types and designs of vehicles. For example, the vehicle 105 could be a car, a motorcycle, a truck, a bus, a semi-trailer truck, or another type of vehicle.

[0022] In the illustrated example, the system 100 includes a controller 110, several sensors 115 (here collectively referred to as "the sensors 115" and individually as "the sensor 115"), a steering system 130, a braking system 135, and an acceleration control system 140. In some embodiments, the system 100, in various configurations, includes fewer, additional, or different components than those shown in the Fig. 3 are described, and can perform functions in addition to the functionality described here. For example, in some embodiments, the system 100 contains a different number of sensors 115 than the four sensors 115 described in the Fig. 3 illustrates how, for example, a single sensor 115.

[0023] As in the Fig. As illustrated in Figure 4, the controller 110 includes an electronic processor 200 (for example, a microprocessor, an application-specific integrated circuit, or other suitable electronic device), a memory 205 (for example, one or more non-volatile, computer-readable storage media), and a communication interface 210. The electronic processor 200, the memory 205, and the communication interface 210 communicate via one or more data links or buses, or a combination thereof. The information shown in the Fig. Figure 4 illustrates control 110 as an example, and in some embodiments the control 110 contains fewer, additional or different components in different configurations than shown. Fig. Figure 4 illustrates this. The controller 110 can be implemented in several independent controllers (for example, programmable electronic controllers), each configured to perform specific functions or sub-functions. Alternatively or additionally, the controller 110 can include sub-modules containing additional electronic processors, memory, or application-specific integrated circuits (ASICs) for handling input / output functions, processing signals, and applying the methods described below. In some embodiments, the controller 110 also performs functions in addition to those described here.

[0024] The electronic processor 200 is configured to access and execute computer-readable instructions (“software”) stored in memory 205. The software may contain firmware, one or more applications, program data, filters, rules, one or more program modules, and other executable instructions. For example, the software may contain instructions and associated data for performing a set of functions, including the procedures described herein. For example, in some embodiments, the electronic processor 200 executes instructions for controlling the steering system 130, the braking system 135, the acceleration control system 140, or other vehicle systems to perform an action (for example, a vehicle maneuver or vehicle behavior) in accordance with an automated driving system of the vehicle 105. As, for example, in the Fig. As illustrated in Figure 4, memory 205 can store an automated driving system 220. The automated driving system 220 controls the vehicle 105 (for example, the steering system 130, the braking system 135, the acceleration control system 140, another vehicle system, or a combination thereof) to perform a vehicle maneuver with limited input or without input from the driver of the vehicle 105.

[0025] Back to Fig. 4: The communication interface 210 allows the controller 110 to communicate with devices outside the controller 110 (for example, to receive inputs from them and provide outputs to them). As in the Fig. As shown in Figure 3, the controller 110 can be connected to one or more sensors 115, the steering system 130, the braking system 135, and the acceleration control system 140. In some embodiments, the communication interface 210 includes a connector for establishing a wired connection with devices outside the controller 110. Accordingly, in some embodiments, the controller 110 is directly coupled to one or more components of the system 100 via a dedicated wired connection. Alternatively or additionally, the communication interface 210 communicates with a communication bus (for example, a Controller Area Network (“CAN”)) to communicate indirectly with devices outside the controller 110.Accordingly, in other embodiments, the controller 110 is coupled to one or more of the components via a shared communication interface, such as a vehicle communication network or bus (for example, a CAN bus, Ethernet, or FlexRay) or a wireless connection (via a transceiver). Each of the components of the system 100 can communicate with the controller 110 using different communication types and protocols.

[0026] The sensors 115 are designed to detect or acquire measured values ​​or data associated with the environment of a vehicle 105 (“environmental information”). A sensor 115 may include, for example, a radar sensor, a lidar sensor, an ultrasonic sensor, an image sensor (or camera), or the like. In some embodiments, the system 100 includes more than one type of sensor (for example, a radar sensor and an image sensor). The environmental information may be associated with one or more objects in the environment surrounding the vehicle 105. An object may include, for example, a pedestrian, a cyclist, another vehicle, or the like.Accordingly, environmental information can include, for example, a distance between the vehicle 105 and one or more objects in the environment surrounding the vehicle 105, a position of the vehicle 105 in relation to the one or more objects in the environment surrounding the vehicle 105, or a combination thereof.

[0027] The sensors 115 can be located at different locations or positions throughout the vehicle 105 (for example, on the inside of the vehicle 105, the outside of the vehicle 105, or a combination thereof). For example, the sensors 115 can be mounted externally on a section of the vehicle 105, such as on a side mirror, the front section, the rear section, or one or more side sections of the vehicle 105. Alternatively or additionally, one or more of the sensors 115 can be located outside the vehicle or away from it, such as on infrastructure surrounding the vehicle 105. For example, the sensors 115 can be mounted on infrastructure surrounding the vehicle 105 so that the sensors 115 can transmit environmental information to the automated driving system 220 (for example, via vehicle-infrastructure communication, via backend servers, or similar means).Accordingly, any sensor that has a field of view enabling the observation of a detected object can serve as a provider of environmental information.

[0028] A sensor 115 is associated with a corresponding field of view of an area of ​​the environment surrounding the vehicle 105. One or more operating parameters, such as the extent of a sensor 115's field of view, can be based on a specific configuration of the sensor 115. Accordingly, the sensors 115 can have fields of view of different sizes (and depths). However, the sensors 115 can also have fields of view of similar (or identical) sizes (and depths). Therefore, in some embodiments, the sensors 115 can have fields of view of the same extent, different extents, or a combination thereof. In some embodiments, a sensor 115 is positioned such that it has an overlapping field of view with at least one additional sensor 115. The overlapping area in which the fields of view of each of the sensors 115 overlap is referred to here as a "common field of view." For example: Fig. Figure 5 illustrates the vehicle 105 with a first sensor 115A and a second sensor 115B. As can be seen in the illustrated example, the first sensor 115A has a first field of view 305A, and the second sensor 115B has a second field of view 305B. Fig. Figure 5 also illustrates a common field of view 310 between the first sensor 115A and the second sensor 115B.

[0029] Although the illustrated example in the Fig. If system 100 contains two sensors (the first sensor 115A and the second sensor 115B) with a common field of view (the common field of view 310), the system 100 can contain one or more additional sensors (separate from the sensors 115, but similar to them as described here), where the one or more additional sensors share a common field of view (for example, a second common field of view). For example, system 100 can contain the first sensor 115A, the second sensor 115B, and a third sensor. The third sensor can have a common field of view with the second sensor 115B (for example, a second common field of view). In this example, the second sensor 115B has a first common field of view with the first sensor 115A and a second common field of view with the third sensor.As another example: System 100 can contain the first sensor 115A, the second sensor 115B, a third sensor, and a fourth sensor. In this example, the first sensor 115A and the second sensor 115B can have a first common field of view, and the third and fourth sensors can have a second common field of view. Accordingly, System 100 can have any number of sensors 115 with any number of common fields of view shared by one or more sensors 115.

[0030] Although not illustrated, the other components of System 100 may contain similar components to the controller 110 (an electronic processor, memory, and a communication interface). However, in some embodiments, the other components of System 100 contain additional, fewer, or different components than the controller 110 in other configurations.

[0031] As stated above, the electronic processor 200 of the controller 110 executes instructions for multi-hypothesis object tracking for the system 100 to achieve robustness against errors in measurement models, data preprocessing, and the like. The electronic processor 200 executes instructions to perform the function specified in the Fig. 6. Illustrated method 600 for multi-hypothesis object tracking for the automated driving system 220. Method 600 is described as being performed by the system 100, and in particular by the automated driving system 220, as executed by the electronic processor 200. However, as stated above, the functionality described with respect to method 600 (or part thereof) can be performed by other devices, such as another controller associated with the vehicle 105, or distributed across multiple devices, such as several controllers associated with the vehicle 105. For example, in some embodiments, the functionality described with respect to method 600 (or part thereof) can be performed by a device outside of or remote from the vehicle 105, where, for example, the vehicle 105 is remotely controlled.

[0032] As in the Fig. As illustrated in Figure 6, the method 600 involves using the electronic processor 200 to receive environmental information (in block 605). As described above, the environmental information may include, for example, the distance between the vehicle 105 and one or more objects in the environment surrounding the vehicle 105, the position of the vehicle 105 relative to the one or more objects in the environment surrounding the vehicle 105, or a combination thereof. In some embodiments, the environmental information includes one or more properties relating to an object in an environment surrounding the vehicle 105. A property of an object may, for example, relate to the object's position relative to the vehicle 105, the object's velocity, the object's direction of motion, the distance between the vehicle 105 and the object, and the like.

[0033] The electronic processor 200 receives environmental information from one or more sensors 115 (via the communication interface 210). In some embodiments, the electronic processor 200 receives the environmental information continuously (for example, in real time or near real time). However, the electronic processor 200 also receives the environmental information cyclically or periodically (for example, ten or twenty times per second). In some embodiments, the environmental information is linked to a common field of view. As above with reference to the Fig. As described in section 5, in some embodiments, for example, two or more of the sensors 115 use a common field of view (for example, a common field of view 310, as in the example of the Fig. (5 is illustrated). Alternatively or additionally, in some embodiments, the sensors 115 include sensors of various types, such as radar sensors, image sensors, and the like. Accordingly, in some embodiments, the environmental information received by the electronic processor 200 can originate from variable media (for example, image-based, ultrasound-based, and the like). For example, the electronic processor 200 can receive a first set of environmental information from a radar sensor (for example, as a first sensor 115) and a second set of environmental information from a lidar sensor (for example, as a second sensor 115). According to this example, the first set of environmental information is radar-based, and the second set of environmental information is lidar-based.

[0034] The electronic processor 200 also generates pseudo-measurement data (in block 610). Pseudo-measurement data can include, for example, a pseudo-measurement value, a measurement model hypothesis, or a combination thereof. A pseudo-measurement value can, for example, be an abstraction of a set of raw sensor readings (for example, a description of a measurement cluster). Alternatively, the raw readings can be used directly as pseudo-measurements. Alternatively or additionally, a pseudo-measurement value can be generated as confirmation of an estimate, an indication of the absence of actual environmental information, or a combination thereof. For example, the pseudo-measurement value can indicate the absence of actual environmental information (for example, negative information) that can confirm an estimated or known occlusion or de-plausibilize an object if no occlusion explains the missing measurement.In some embodiments, a pseudo-measurement is linked to the object in the vicinity of the vehicle 105, such as a three-dimensional bounding box that models a lidar point cluster associated with the object. A measurement model hypothesis can be based on one or more assumptions, such as an assumed object classification or object type (an "object classification assumption"). Alternatively or additionally, a measurement model hypothesis can be a generic hypothesis without an object classification assumption. Accordingly, the electronic processor generates 200 pseudo-measurement data by generating one or more pseudo-measurements. Alternatively or additionally, the electronic processor generates 200 pseudo-measurement data by generating one or more measurement model hypotheses, each measurement model hypothesis being generated based on an object classification assumption, without an object classification assumption, or a combination thereof.For example, in the . Fig. As shown in Figure 7, the electronic processor 200 can generate a first measurement model hypothesis 705 and a second measurement model hypothesis 710 (as pseudo-measurement data). In the illustrated example, the first measurement model hypothesis 705 hypothesizes that the object is a vehicle, and the second measurement model hypothesis 710 hypothesizes that the object is a cyclist.

[0035] Accordingly, in some embodiments, the electronic processor 200 generates a pseudo-measurement data set for each sensor measurement, so that for each measurement, several alternative pseudo-measurement data hypotheses are generated using different modeling assumptions, clustering assumptions, or the like. In such embodiments, a different alternative version of pseudo-measurement data can be used for each sensor measurement.

[0036] Whether pseudo-measurements or measurement model hypotheses (as pseudo-measurement data) are generated depends, in some embodiments, on the type of sensor on which the measurement model or the pseudo-measurement is based. For example, the electronic processor 200 can generate a lidar point cluster for a lidar sensor (for example, via a deep neural network), which models the cluster as a three-dimensional bounding box. The three-dimensional bounding box can then be used as a pseudo-measurement. Alternatively or additionally, the lidar point cluster can be approximated, for example, by an L-shape, a line segment, an ellipse, or something similar. The parameters of the respective description can be used as a pseudo-measurement. Alternatively or additionally, the electronic processor 200 can generate a pseudo-measurement based on the clustering of lidar stixels, whereby the clustering does not include any object classification assumption.As another example: For a radar sensor, a reflection model (a measurement model) for radar can depend on the object type, which influences the measurement model used for object updates. Thus, the electronic processor 105 can generate multiple measurement model hypotheses with different assumptions (including hypotheses that are generic or do not include an object classification assumption).

[0037] In some embodiments, one or more of the hypotheses for the measurement model, the pseudo-measurement data, or a combination thereof, contain hypotheses that are generic, that do not contain an object classification assumption, or that contain a combination of both. These generic models generate robustness, for example, against incorrect classification information, classification assumptions, or a combination thereof. The purpose of the generic hypotheses is to become robust against incorrect assumptions by allowing for a reduced granularity of the model. These generic hypotheses can be considered as a fallback strategy for situations in which complex measurement models or pseudo-measurement hypotheses are not applicable (for example, to cover rare object shapes or measurement situations).Although the accuracy of the fallback strategy may not be sufficient to ensure comfortable vehicle responses, it can prevent the object from being lost and thus avoid dangerous situations.

[0038] As stated above, the pseudo-measurement data can contain one or more pseudo-measurements, one or more measurement model hypotheses, or a combination thereof, based on one or more different assumptions. Consequently, the decision as to which measurement model hypothesis or pseudo-measurement is ultimately used can be postponed further down the process chain as more information (from later measurement queries or from other sensors) becomes available. This is implemented in sensor data fusion, for example, by using a multi-hypothesis approach across multiple data queries, employing methods such as a labeled multi-Bernoulli ("LMB") filter.In some embodiments, one or more hypotheses in which alternative pseudo-measurements (generated, for example, by different cluster hypotheses) are used simultaneously for object updating are excluded because information from measurement data is used multiple times as a result of this, even though the independence of the measured values ​​is assumed.

[0039] As in the Fig. As shown in Figure 6, the electronic processor 200 also generates a linking hypothesis set for an object in a field of view (in block 615). In some embodiments, the field of view is a common field of view linked to two or more sensors 115, as described in more detail above. However, in other embodiments, the field of view is linked to a single sensor, such as in a single-sensor system that obtains measurement data (environmental information) over time from a single sensor 115. A linking hypothesis generally refers to a hypothesis regarding a possible object state, such as an object in a common field of view of two or more sensors 115 or an object in the field of view of a single sensor 115.When an automated driving system (such as the automated driving system 220) uses information from one or more sources (such as one or more of the sensors 115), uses one or more measurement models, or uses a combination thereof, the automated driving system 220 can determine more than one possible object state during object tracking. Each possible object state is referred to here as a linking hypothesis.

[0040] Accordingly, the electronic processor 200 determines one or more possible object states (as linking hypotheses) for an object in an environment surrounding the vehicle 105, through sensor fusion (in the Fig. 7 as shown in block 712) of the environmental information. An object state can be based on one or more properties of the object. For example, an object state can be based on the object's position relative to vehicle 105, the object's velocity, the object's direction of motion, an object classification for the object that identifies what the object is (an object classification) (for example, whether the object is another vehicle, a motorcycle, a person, a cyclist, or the like), and the like. Accordingly, determining a linkage hypothesis (or possible object state) involves determining one or more properties associated with the object. The fusion of sensor information (in block 712) involves one or more measurement models, motion models, object tracking processes, or a combination thereof (for example, the first measurement model hypothesis 705 and the second measurement model hypothesis 710 of the Fig. 7).

[0041] Accordingly, in some embodiments, the linking hypothesis set is linked to an object in a common field of view linked to two or more sensors 115, or to the field of view of a single sensor 115. The electronic processor 200 can generate the linking hypothesis set based on the environmental information received from one or more of the sensors 115, the pseudo-measurement data, or a combination thereof. The linking hypothesis set can be linked to or correspond to the pseudo-measurement data. For example, the linking hypothesis set can contain a linking hypothesis for each pseudo-measurement, each measurement model hypothesis, or a combination thereof. Returning to the one in the Fig. 7. Illustrated Example: The electronic processor 200 can generate a first linking hypothesis 715A and a second linking hypothesis 715B (as a linking hypothesis set). The first linking hypothesis 715B is linked to the second measurement model hypothesis 710, and the second linking hypothesis 715A is linked to the second measurement model hypothesis 705. As an example: The environmental information received from one or more of the sensors 115 can specify the size, speed, and direction of movement of the object. Based on the environmental information in this example, the electronic processor 200 can generate a linking hypothesis that a possible object state for the object is of the type vehicle, with the object moving at a greater speed than the vehicle 105 and in the same direction as the vehicle 105.

[0042] In some embodiments, the electronic processor 200 updates one or more linking hypotheses over time (for example, at each time cycle), combines them, and / or truncates them when additional environmental information is received from one or more sensors 115. In such embodiments, the linking hypotheses (and thus the specific object state) are monitored and periodically updated (each period being referred to as a cycle) based on new information, such as new environmental information received from one or more sensors 115. In some embodiments, the electronic processor 200 uses and manages linking hypotheses over time, for example, using multiple hypothesis tracking (“MHT”) and random finite set (“RFS”) processes, including labeled multi-Bernoulli (“LMB”) filters.

[0043] After generating the linking hypothesis set (in block 615), the electronic processor 200 then determines an object state for the object (in block 620). For example, the determined object state may include an object classification of a vehicle, where the object is (a) in the left lane, (b) traveling in the same direction as vehicle 105, and (c) traveling at 50 mph. The electronic processor 200 can determine the object state for the object based on the linking hypothesis set. Accordingly, in some embodiments, the determined object state is one of the linking hypotheses contained in the linking hypothesis set (i.e., one of the possible object states for the object). In some embodiments, the electronic processor 200 determines the object state for the object based on a probability associated with each of the linking hypotheses contained in the linking hypothesis set.For example, the electronic processor 200 can determine the object state for the object as the linking hypothesis (i.e., the possible object state) with the highest probability (for example, as the linking hypothesis that is most likely the correct object state of the object). Over time, and as more environmental information is received from one or more of the sensors 115, the electronic processor 200 can combine or merge similar linking hypotheses, while removing linking hypotheses determined to be unlikely possibilities. Accordingly, the probability associated with the merged or combined linking hypotheses increases with the number of hypotheses that are merged or combined. As another example, the probability of a linking hypothesis can increase based on the number of remaining likely hypotheses.In other words, the more linking hypotheses are removed as unlikely, the greater the probability that the remaining linking hypotheses are likely to be correct (or true).

[0044] Alternatively or additionally, in some embodiments, the electronic processor 200 forwards one or more of the linking hypotheses, along with their respective probabilities, to another component (for example, to consider which possible object state is most likely the actual object state of the detected object). In some embodiments, a preliminary selection of one or more linking hypotheses is performed prior to sensor fusion (for example, in block 712 of the Fig. 7), for example to limit computation time, restrict hypothesis growth, or a combination thereof.

[0045] As in the Fig.As shown in Figure 6, the electronic processor 200 can then control the vehicle 105 based on the specific object status (in block 625). In some embodiments, the electronic processor 200 controls the vehicle 105 by determining, based on the specific object status, whether a vehicle maneuver is to be performed. Accordingly, in some embodiments, controlling the vehicle 105 based on the specific object status does not involve performing a vehicle maneuver, or it involves performing a vehicle maneuver. Alternatively or additionally, the electronic processor 200 controls the vehicle 105 by determining, based on the specific object status, which vehicle maneuver (or reaction) is to be performed, a specified execution time for the vehicle maneuver (for example, when the vehicle maneuver is to be performed), and similar parameters.The determination of the vehicle maneuver (based on the specified object status) can be carried out using one or more automated driving techniques, which will not be discussed in detail here for the sake of brevity.

[0046] The electronic processor 200 can perform a vehicle maneuver by controlling the steering of the vehicle 105, influencing the speed of the vehicle 105 (for example, accelerating or braking the vehicle 105), or similar actions. In some embodiments, the electronic processor 200 performs the vehicle maneuver by generating one or more control signals and transmitting them to one or more components of the vehicle 105, such as the steering system 130, the braking system 135, the acceleration control system 140, and similar components. In response to receiving the control signals, the one or more components of the vehicle 105 can be controlled in accordance with the corresponding control signal.

[0047] Accordingly, the embodiments described here can achieve robustness against errors in measurement models, data preprocessing, and similar processes, for example, by filtering out erroneous hypotheses based on subsequent measurements over time. For instance, object tracking is improved by identifying and removing (or truncating) hypotheses containing erroneous tracking-measurement correlations (random errors), while retaining numerous other hypotheses over time. One method for keeping the number of hypotheses manageable might involve, for example, grouping similar hypotheses while removing (or truncating) improbable ones.

[0048] In the field of advanced object tracking, various hypotheses can also be generated using another technique – sensor data clustering. These clustering techniques minimize the effects of errors in data clustering because the hypothesis with the most consistent clustering (indicating that the hypothesis is true or correct) prevails over time. However, the rationale for these clustering techniques lies more in improving object tracking performance by enhancing the linking and clustering of measurement data than in the system's robustness against incorrect assumptions in measurement models and data preprocessing.However, with regard to the embodiments described here, specific hypotheses are determined to provide, for example, robustness against errors in measurement models and in data preprocessing steps (e.g., clustering of sensor data, object classification, and the like) that are applied during the fusion of the sensor data.

[0049] Although the examples described here relate to automated driving systems, as stated above, the methods and systems described herein can be applied in other embodiments to other systems, such as driver assistance systems, traffic control systems, traffic management systems, safety monitoring systems, and the like. For example, the sensors 115 can be installed on infrastructure. According to this example, the sensors 115 can transmit environmental information to the automated driving system (for example, transmitted via vehicle-infrastructure communication, provided by a backend server, or similar). Alternatively or additionally, the sensors 115 can acquire environmental information related to traffic control and management. For example, the sensors 115 can detect environmental information related to the observed traffic.According to this example, the electronic processor 200 can determine and control a traffic control and management action, such as opening shared lanes, controlling traffic light phases, setting prices for premium traffic, and similar functions. As another example, the sensor 115 can detect environmental information for monitoring systems, such as for monitoring the location of a vehicle, pedestrian, or similar object on a property. According to this example, the electronic processor 200 can determine and control an action of the monitoring system based on the position of a vehicle or pedestrian, such as whether an alarm should be triggered. Similarly, in some examples, the electronic processor 200 is not integrated into or related to a specific vehicle, such as vehicle 100.Instead, the electronic processor 200 can, for example, be part of a traffic control system or a monitoring system, whether local (for example, on a local traffic control or monitoring system) or remote (for example, a cloud-based application server or a remote server).

[0050] Thus, the embodiments provide, among other things, methods and systems for multi-hypothesis object tracking for automated driving systems, thereby achieving robustness against errors in measurement models, data preprocessing, and the like. Various features and advantages of certain embodiments are set forth in the following claims.

Claims

[1] Automated driving system for a vehicle (105) wherein the system comprises: an electronic processor (200) configured to perform the following functions: To receive environmental information, to generate pseudo-measurement data that are associated with an object in an environment of the vehicle (105), to determine a set of linking hypotheses relating to the object based on the environmental information and the pseudo-measurement data, to determine an object status of the object based on the linking hypothesis set and to control the vehicle (105) based on the specific object status, wherein the electronic processor (200) is configured to generate multiple measurement model hypotheses for each measurement value contained in the environmental information as pseudo-measurement data, each of the multiple measurement model hypotheses being based on a different assumption, where at least one of the several measurement model hypotheses (705, 710) is based on an object classification assumption, wherein the electronic processor (200) is configured to receive environmental information from at least two sensors (115), wherein the environmental information is linked to a common field of view of the at least two sensors (115), where the pseudo-measurement data contains negative information linked to the common field of view. [2] System according to claim 1, wherein at least one of the several measurement model hypotheses (705, 710) is a generic measurement model hypothesis. [3] System according to claim 1, wherein the electronic processor (200) is configured to update the linking hypothesis set in response to receiving new environmental information. [4] System according to claim 1, wherein each linking hypothesis (715) contained in the linking hypothesis set is linked to a possible object state of the object. [5] System according to claim 1, wherein the electronic processor (200) is configured to determine the object status of the object as one of the linking hypotheses (715) contained in the linking hypothesis set. [6] System according to claim 5, wherein the determination of the object status as one of the linking hypotheses (715) is based on one of the linking hypotheses (715) which has a higher probability than other linking hypotheses (715) included in the linking hypothesis set. [7] Method for multi-hypothesis object tracking, wherein the method comprises: to receive environmental information; to generate pseudo-measurement data with an electronic processor (200); to generate a set of linking hypotheses for an object in an environment surrounding a vehicle (105) based on the environmental information and the pseudo-measurement data; to determine an object state for the object based on the linking hypothesis set using the electronic processor (200); and to control the vehicle (105) based on the specific object status, wherein receiving the environmental information includes receiving environmental information from at least two sensors (115), wherein the environmental information is linked to a common field of view of the at least two sensors (115), where generating the pseudo-measurement data involves generating negative information linked to the common field of view, and to generate a first measurement model hypothesis (705) based on a first assumption and to generate a second measurement model based on a second assumption that differs from the first assumption. [8] The method of claim 7, further comprising: to receive new environmental information; and to update the linking hypothesis set based on the new environmental information. [9] Method according to claim 8, wherein updating the linking hypothesis set includes removing at least one linking hypothesis (715) from the linking hypothesis set. [10] Method according to claim 7, wherein determining the object status includes determining the object status as one of the linking hypothesis (715) contained in the linking hypothesis set. [11] System according to claim 7, wherein determining the object status as one of the linking hypotheses (715) includes determining one of the linking hypotheses (715) which has a higher probability than other linking hypotheses (715) included in the linking hypothesis set. [12] Non-volatile computer-readable medium storing instructions which, when executed by an electronic processor (200) of an automated driving system according to any one of claims 1 to 6, perform a set of functions.

Images