Method for determining noise statistics of object sensors

Abstract

Method for determining noise statistics of an object sensor. A method for automatically determining and using noise statistics (30) of noisy observation data (21) provided by at least one sensor (20) connected to a tracker (100) in order to improve cross-scene tracking of a time-varying object (40) using a Kalman algorithm. The Kalman algorithm allows an estimate of dynamic parameters of the object (40) to be obtained and the accuracy of the estimate is provided based on the noisy observation data (21) and based on corrected previous estimates. The method comprises the following steps: receiving reference data (51) from a data source (50) external to the Kalman algorithm and connected to the tracker (100); obtaining the noise statistics (30) reflecting the errors generated by the sensor (20) from the reference data (51); and using the noise statistics (30) as setting parameters for the Kalman algorithm.

Description

Technical Field

[0001] The present disclosure relates to the field of trackers that are commonly used for pattern recognition, such as in the automotive industry, and each tracker is connected to at least one object detector or sensor to provide noisy observation data. The tracker of the present disclosure is provided with a Kalman filter to improve cross-scene tracking of time-varying objects. Based on such a device, the present disclosure utilizes high-quality reference data to automatically collect error or noise statistics of at least one sensor, and uses the statistics to calibrate the Kalman filter of the tracker.

[0002] More specifically, the present disclosure relates to methods of automatically determining and using such noise statistics, as well as non-transitory computer-readable media and trackers that implement the methods. Background Art

[0003] Kalman filters are conventionally used in perception systems to provide stable (linear) tracking of objects across scenes by prediction based on observations from one or more sources. To this end, Kalman filters are implemented by algorithms programmed according to an internal model. In order to properly balance the Kalman filter's internal model predictions with external observations, the Kalman filter needs to estimate the observation noise of each of the sources. These sources are usually composed of sensors (such as object detectors) that are connected to the Kalman filter like peripherals. A large noise estimate will cause the tracker to give more trust to predictions issued based on its own internal state, while a low noise estimate will give more trust to the observations.

[0004] Noise estimates are typically the result of tuning and experimentation, and setting them considers a form of "black art". Kalman filter-based trackers must be manually tuned by experts, which is a labor-intensive and error-prone process. This tuning process must be repeated every time a sensor or object detector is modified, further slowing down the development cycle and increasing the risk of introducing errors. Modern vehicles are equipped with multiple sensors of each type, so the process must be repeated frequently for each of these sensors.

[0005] Modern machine learning-based object detectors typically report confidence values ​​for each detected object, which relate to the likelihood that the detection is correct / accurate. These confidence values ​​can be used by the tracker to assign higher or lower trust to individual detections, but there is currently no direct way to do this. Moreover, although confidence values ​​can be used to filter or reweight the output of a single detector, they are even more difficult to apply when the outputs of multiple detectors are combined. This is because the confidences reported by different object detectors do not necessarily lie in the same range or associate the same margin of error with each value. Therefore, generating confidences for multiple object detectors requires specially designed detectors to report similar confidence values, thereby reducing the reusability of existing algorithms.

[0006] Therefore, there is a need to improve the prior art in order to at least partially overcome the aforementioned problems and disadvantages, especially to simplify the calibration process of the tracker and to improve the reliability of the object observation, especially when multiple sensors are used with the tracker. Summary of the invention

[0007] To address this problem, the present disclosure proposes an automatic method for deriving noise calibration for a Kalman-based object detector. More specifically, a method for automatically determining and using noise statistics of noisy observation data provided by at least one sensor connected to a tracker is proposed to improve cross-scene tracking of time-varying objects using a Kalman algorithm. The Kalman algorithm can obtain an estimate of the dynamic parameters of the object and provide the accuracy of the estimate based on the noisy observation data and based on the corrected previous estimate. The aforementioned method includes the following steps:

[0008] receiving reference data from a data source external to the Kalman algorithm and connected to the tracker;

[0009] deriving the noise statistics reflecting errors produced by the sensor from the reference data; and

[0010] The noise statistics are used as setting parameters for the Kalman algorithm.

[0011] The present disclosure allows for more accurate tracking of objects and improved tracker performance. Additionally, if the tracker includes multiple sensors (i.e., multiple detectors), the present disclosure also allows for fusion of multiple detectors as a "black box" rather than assuming their internal functionality. In this way, confidence-based sensor fusion of a finished software component may be allowed.

[0012] In one embodiment, the reference data relates to a scene for which noisy observation data has been obtained by a sensor, and wherein deriving noise statistics from the reference data is obtained by comparing the reference data with the noisy observation data provided by the sensor.

[0013] Preferably, the reference data comprises annotated observation data received from a data source and is used to calibrate the tracker.

[0014] According to one embodiment, the noise statistics include the accuracy of the sensor, which defines the probability that the noisy observation data is correct.

[0015] Preferably, the accuracy of a sensor is used to determine whether the correlated noisy observation data provided by the sensor is to be considered by the tracker for tracking the object or is to be discarded.

[0016] In one embodiment, the noisy observation data relates to a plurality of measured variables of the subject, at least one pair of measured variables is identifiable from the plurality of measured variables, and the noise statistics include at least one of the following:

[0017] the expected variance of each of the measured variables, and

[0018] The expected covariance between each pair of measured variables.

[0019] Preferably, the measured variable relates to at least one of the following:

[0020] The object's dimensional data,

[0021] The object's location data within the scene,

[0022] The distance data between the object and the sensor,

[0023] The object’s absolute or relative velocity data, and

[0024] Absolute or relative acceleration data of an object.

[0025] According to a preferred embodiment, the expected covariance between each pair of measured variables is provided in the form of a covariance matrix or vector and is used as input to the Kalman algorithm to define the observation noise in the noisy observation data.

[0026] In one embodiment, the sensor reports a confidence value related to the probability that the noisy observation data is correct or accurate, and the method further comprises the steps of:

[0027] The confidence value is used as a weighting parameter which is applied to the noisy observation data via the observation noise which is further input into the Kalman algorithm.

[0028] In another embodiment, the sensor reports a confidence value related to the probability that the noisy observation data is correct or accurate, and the method further comprises the steps of:

[0029] comparing the confidence value to a plurality of predetermined confidence intervals, each of the plurality of predetermined confidence intervals being associated with an error parameter, and

[0030] Included in the noise statistics is an error parameter associated with a confidence interval including a confidence value.

[0031] Preferably, the error parameter is determined during a pre-processing phase which aims to:

[0032] Run the sensor against a set of reference data to collect multiple confidence values,

[0033] According to the confidence value, the reference data is divided into subsets,

[0034] For each subset, the noisy observation data provided by the sensor is compared with the associated reference data, and

[0035] The error covariance and accuracy of the sensor are calculated as data for forming the error parameter.

[0036] According to one embodiment, the sensor reports a confidence value quantifying the probability that the noisy observation data is correct or accurate, and the method further comprises the steps of:

[0037] using a regression model that establishes a relationship between error and confidence value, the regression model being used to determine the error corresponding to the reported confidence value;

[0038] Included in the noise statistics are the errors in the reported confidence values ​​provided by the regression model.

[0039] Preferably, the regression model is determined based on a scatter plot obtained from multiple detections of the object by the sensor, wherein each point in the scatter plot is derived based on a confidence value and an error determined for each detection.

[0040] The present disclosure also relates to a non-transitory computer-readable medium comprising program instructions, which cause a processor to execute a method according to any one of the method embodiments or according to any possible combination of the embodiments of the present disclosure.

[0041] The present disclosure also relates to a tracker for implementing the aforementioned method according to any one of the embodiments of the present disclosure or any possible combination of the embodiments of the present disclosure, the tracker comprising:

[0042] a processing unit for storing and executing the Kalman algorithm and the steps of the method;

[0043] a first communication interface for connecting at least one sensor providing noise observation data; and

[0044] A second communication interface is used to receive reference data from an external data source.

[0045] Other embodiments and advantages are disclosed in the detailed description which follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The present disclosure and the embodiments presented herein are to be considered as non-limiting examples and will be better understood with reference to the accompanying drawings, in which:

[0047] Figure 1 is a schematic representation of the main entities of an environment including a tracker suitable for implementing the method of the present disclosure,

[0048] Figure 2 is a schematic representation of a Kalman filter,

[0049] Figure 3 An illustration that helps to understand the concept of precision within statistical methods is shown.

[0050] Figure 4 is a schematic representation of a comparison between example outputs of a subject sensor and reference data to collect statistics of errors produced by the sensor,

[0051] Figure 5 is a schematic representation of the partitioning of the confidence value ranges to which the error parameters are assigned,

[0052] Figure 6 is a schematic diagram of using noise statistics as setting parameters for the Kalman algorithm based on confidence values ​​reported by the sensors, and

[0053] Figure 7 is a schematic diagram showing fused detection from multiple sources. DETAILED DESCRIPTION

[0054] Figure 1 A schematic representation of an environment including a tracker 100 suitable for implementing the method of the present disclosure is shown. The aforementioned environment also includes other main entities such as sensors 20 and objects 40 in the scene that are tracked by the tracker 100 via the sensors 20. Figure 1As an example, two sensors 20a, 20b are shown, but it should be understood that the number of sensors 20 is not limited to two, so that the tracker 100 may include or be connected to at least one sensor 20. The sensor 20 may be any type of sensor or object detector, and typically works using radar detection technology. Thus, these sensors are, for example, based on electromagnetic waves or sound waves, such as radio waves, ultrasonic waves, infrared waves, or laser waves. The object 40 in the scene may typically be an object within the road environment, such as an obstacle or a road user. The sensor 20 and the tracker 100 are preferably retracted to be mounted on a vehicle, such as an intelligent or autonomous vehicle, to obtain environmental perception on the road.

[0055] like Figure 1 Schematically shown, the tracker 100 comprises a processor or processing unit 102 for storing and running a Kalman algorithm, which may act as or be seen as a Kalman filter 101. The tracker 100 also comprises at least a first communication interface 104 for connecting at least one sensor 20, and preferably comprises a second communication interface 105 for connecting the tracker 100 to a data source 50, which may be an external data source 50. Tracking data 110 providing an estimated state of the object 40 is output from the tracker 100, for example, via a third communication interface 103.

[0056] Figure 2 A schematic illustration of a Kalman filter 101 is provided. The Kalman filter 101 can be viewed as a device hosting a mathematical tool, which is usually implemented by an algorithm 10, which is used to estimate the state of a dynamic system based on observations that are usually local and noisy. The dynamic system can usually be the movement of an object 40 within a scene. To this end, the filter 101 (more specifically the Kalman algorithm 10) takes a sequence of observation data 21 as input, so that each observation is linearly linked to an accurate unknown state. The purpose of the algorithm 10 is to estimate these states in an optimal and recursive manner. The ability of the Kalman filter 101 is its ability to predict parameters and correct errors, not only for the sensor 20 that serves as the source of the observation data, but also for the model itself. Therefore, the Kalman filter 101 is able to integrate the inaccurate terms related to the model itself, which allows the filter 101 to give a correct estimate regardless of reasonable modeling errors.

[0057] Without going into detail, Kalman algorithm 10 is based on Figure 2The first step is a prediction step 11, which predicts the state of the system, more specifically the dynamic parameters of the observed object 40. The second step is an update step 12, which uses the noisy observation data 21, i.e., the error-prone measurement results Y of the observed object 40 provided by the sensor 20, in order to improve the estimate of the system state. Since each of the sensors 20a, 20b provides noisy observations, Figure 1 In FIG. 1 , the noisy observation data 21 sent by the first sensor 20 a is identified by reference numeral 21 a , and the noisy observation data 21 sent by the second sensor 20 b is identified by reference numeral 21 b .

[0058] The Kalman algorithm 10 provides at its output output data 13 about the observed object, in particular defining the object 40 at time t or dynamic parameters associated with the object at time t. The output data 13 include an estimate 14 and the accuracy 15 of the estimate 14. Figure 2 In the output data 13, the literal value (lateral value) P t ) to identify, where is the estimate, more specifically the state vector including the estimated parameters, and P t is the estimated precision, usually the error covariance matrix.

[0059] The Kalman algorithm 10 also takes as input the corrected previous estimate 13' at time t-1. Figure 2 In , we also pass literal values ​​( P t-1 ) to identify the corrected previous estimate 13', where is the previous estimate, and P t-1 is the estimated prior precision.

[0060] Modern machine learning based object sensors (most notably neural networks) typically report confidence values ​​22 that accompany each detected object 40. The confidence values ​​22 (particularly the respective confidence values ​​22a, 22b reported by the respective sensors 20a, 20b) are generally associated with the detected object 40. Figure 1 )) is related to the likelihood that the test is correct / accurate. Figure 1 As stated, it can be viewed as the precision P of the observation Y.

[0061] It should be noted that the term "accuracy" is used here according to its statistical definition. Therefore, accuracy is considered to be the fraction of relevant instances among the instances obtained. In other words, it is the probability that the observations emitted from the sensor 20 are correct. There are two possible results regarding the output emitted by the sensor 20 according to the measurement applied to the object 40. The observation output from the sensor 20 includes some errors (for example, errors in the position or size of the detected object 40), or the sensor 20 has detected a false object because the object 40 does not actually exist in the scene. In the first case, the term "true positive" is used, while in the second case, the term "false positive" is used.

[0062] Also combined Figure 3 The concepts of "false positives" and "false negatives" are shown. For example, a model is used to predict whether a driver will stop or not stop at a traffic light. This scenario is a binary classification where the answer can be true or false. If the model must predict positive when the driver stops and negative when the driver does not stop, then a "false positive" predicts that the driver stops, when in fact the driver does not stop. A "false negative" works the other way around, such that in such a scenario, the model predicts that the driver will not stop (negative), when in fact the driver stops.

[0063] Based on the foregoing, precision can be thought of as the ratio of “true positives” to the sum of “true positives” and “false positives”. In other words, if “true positives” are recorded as TP and “false positives” are recorded as FP (e.g. Figure 3 ), then the precision P can be defined by the following expression: P = TP / (TP+FP).

[0064] exist Figure 2 In the Kalman filter shown, the errors corresponding to "true positive" cases, i.e. errors regarding the dynamics or object parameters of the object 40, are usually described by a noise covariance matrix. On the other hand, if these errors relate to "false positives" (i.e. detection of an object that is not present), they are usually described by a precision P = TP / (TP+FP). In fact, in this case, it is meaningless to determine errors in, for example, the position or size of the object 40, assuming that there is no "real" object.

[0065] In its main approach, it will be based on Figure 1 and Figure 2The method of the present disclosure is described. It relates to a method for automatically determining and using noise statistics 30 of noisy observation data 21 provided by at least one sensor 20 connected to a tracker 100 in order to improve tracking of a time-varying object 40 across a scene using a Kalman algorithm 10. The Kalman algorithm enables an estimate 14 of a dynamic parameter of the object 40 and provides an accuracy 15 of the estimate 14 based on the noisy observation data 21 and based on a corrected previous estimate 13'. The method comprises the following steps:

[0066] receiving reference data 51 from a data source 50 external to the Kalman algorithm 10 and connected to the tracker 100;

[0067] A noise statistic 30 reflecting the error caused by the sensor 20 is obtained from the reference data 51 ; and the noise statistic 30 is used as a setting parameter of the Kalman algorithm 10 .

[0068] It should be noted that the reference data 51 may generally be received at the tracker 100 (e.g. via the second communication interface 105), and that obtaining and using the noise statistics 30 may be performed by the tracker 100, in particular by the processing unit 102, e.g. by the Kalman filter 101 which may be considered as a part of the processing unit 102. It should also be noted that the noise statistics 30 should not be confused with the confidence value 22 reported by the sensor 20. In fact, the confidence value 22 is related to the data output by the sensor (e.g. Figure 1 ), while the noise statistic 30 is related to data that can generally be provided by the tracker 100, more specifically, to data provided by the processing unit 102 of the tracker, which preferably includes a Kalman filter 101. In this case, since the noise statistic 30 is provided by the processing unit 102 so as to be input as a setting parameter of the Kalman algorithm 10, the noise statistic 30 can be regarded as a setting parameter determined and used within the processing unit 102.

[0069] The data source 50 providing the reference data 51 is typically an external data source 50, in particular a data source external to the Kalman algorithm 10, so that the reference data does not come from the Kalman algorithm 10. Hence, the data source 50 is external to the Kalman filter 101 and preferably external to the processing unit 102. More preferably, the data source is external to the tracker 100. Hence, the tracker 100 may typically be connected to the data source 50 via a second communication interface 105, e.g. via a wired or wireless connection. In a preferred embodiment, the data source 50 is a remote database or server, to which the tracker 100 may be connected, e.g. via a wide area network such as the Internet.

[0070] Since the noise statistics 30 both reflect the errors produced by the sensor 20 and are derived from the reference data 51, this means that the reference data 51 can be considered as ground truth data or data reflecting absolute facts. The nature and / or format of the reference data 51 will preferably depend on the nature and / or format of the noisy observation data 21 so that they are all of the same type and can be properly compared. Therefore and according to a preferred embodiment, the reference data relates to the scene for which the noisy observation data 21 have been obtained by the sensor. In other words, the reference data 51 can be considered as a description of the scene in the same domain as the sensor 20 as input to the noisy observation data 21. For example, if the sensor outputs data about a plurality of bounding boxes (these bounding boxes can each be defined by a plurality of object parameters such as center, size and / or rotation), the reference data 51 will also relate to the bounding boxes, in particular to the same parameters as the aforementioned object parameters.

[0071] Preferably, deriving the noise statistics 30 from the reference data 51 is obtained by comparing the reference data 51 with the noisy observation data 21 provided by the sensor 20. According to the above example, the noise statistics 30 may be calculated by comparing the detected bounding box with the reference bounding box.

[0072] Recently, the use of machine learning for sensory perception has proliferated the collection and annotation of sensor data used to train machine learning models. Annotated data provides a unique opportunity to automatically measure the noise statistics 30 of the object sensor 20 and use it, for example, to calibrate the tracker 100. Therefore, in one embodiment, the reference data 51 includes annotated observation data received from the data source 50. The annotated data (also referred to as "labeled data") refers to data that is manually labeled with the desired output. For example, the annotated data can be a bounding box manually labeled by a person around an object in a scene. Therefore, the annotated data can generally be geometric data (size, distance, angle, line segment, curve, coordinate, etc.). Sometimes, the annotated data involves dynamic data (speed, acceleration, etc.) or semantic data (for example, words such as keywords). In any case, the annotated data is very useful in machine learning methods, where data pairs (each pair includes observations and annotations) are provided to algorithms that seek mappings between inputs and desired outputs. Typically, the annotated data is in the same format as the noisy observation data 21 output from the sensor 20. The only difference may concern the confidence values, which are usually omitted from the annotated data since they are considered "definitely true".

[0073] According to one embodiment, the present disclosure preferably utilizes reference data 51, preferably high quality reference data 51, in order to automatically calibrate the tracker 100. Such calibration may be performed in an automatic manner during an initialization phase or so-called calibration phase.

[0074] Figure 4 is a schematic representation of a continuous comparison between an example output of an object sensor 20 and reference data 51 in order to derive and collect therefrom the noise statistics 30 of the errors produced by the sensor 20. To this end, Figure 4 The top of Schematic depicts a sequence of such comparisons from left to right. The example depicted on the left shows a comparison that can be considered a high confidence detection, relative to the low confidence detection shown in the example depicted on the right.

[0075] To perform this determination, the noisy observation data 21 is compared to reference data 51, which generally refers to annotated data related to the same scene as the scene in which the sensor 20 was used to capture the noisy observation data 21. Such scenes can be used for validation purposes during the development of the sensor 20. Figure 4 As schematically shown, the output provided by the sensor 20 (ie, noisy observation data 21) is compared with reference data 51 in order to derive and collect noise statistics 30 reflecting the errors produced by the sensor.

[0076] The case where the detection by the sensor 20 provides a high confidence HC is illustrated by two examples where there is a slight offset between the noisy observation data 21 and the reference data 51. On the other hand, Figure 4 The right side of shows low confidence LC detections by two other examples, where the noisy observation data 21 has both size and positioning defects in the first example and significant rotation defects in the second example.

[0077] like Figure 4 As shown, the noise statistics 30 may include the accuracy P of the sensor 20, which defines the probability that the noisy observation data 21 is correct, in particular the probability that a positive detection is correct. The accuracy can be determined by combining Figure 3 The formula discussed is P=TP / (TP+FP).

[0078] Preferably, the accuracy P of the sensor 20 is used to determine whether the correlated noisy observation data 21 provided by the sensor 20 will be considered by the tracker 100 for tracking the object 40 or will be discarded.

[0079] According to one embodiment, the noisy observation data 21 relates to a plurality of measured variables 41 of the object 40, from which at least one pair of measured variables 41 is identifiable, and the noise statistics 30 include at least one of the following:

[0080] the expected variance of each of the measured variables 41, and

[0081] The expected covariance between each pair of measured variables41.

[0082] Figure 4 An embodiment is illustrated in which, in this example, the plurality of measured variables 41 are related to the values ​​or coordinates x, y and to the width, height and rotation of the object 40. If the object is, for example, Figure 4 If the rectangular box is depicted, the values ​​x, y may relate to the center of the object or to another specific point (such as a corner).

[0083] According to a general approach, the measured variable 41 may relate to at least one of the following:

[0084] Dimensional data of object 40,

[0085] Positioning data of object 40 within the scene,

[0086] distance data between the object 40 and the sensor 20,

[0087] absolute or relative velocity data of the object 40, and

[0088] Absolute or relative acceleration data of the object 40 .

[0089] Variance refers to a measure of the dispersion of sample values. The variance is determined about the mean of the measured values, in particular as the mean of the squared deviations from the aforementioned mean. Covariance provides the degree of dependence between two variables (i.e., between a pair of measured variables 41). The noise covariance is the error covariance (or in other words, the precision of the estimate), and others can be obtained by Figure 4 In V k The error covariance matrix or noise covariance matrix of the identifier is represented.

[0090] According to one embodiment, the expected covariance between each pair of measured variables 41 is given by the covariance matrix v k or vector form and is used as an input to the Kalman algorithm 10 for defining the observation noise in the noisy observation data 21. In other words, the observation noise is a noise statistic 30 that can be used as a setting parameter for the Kalman algorithm 10. Figure 2As shown, in addition to the noisy observation data 21 and the corrected prior estimate 13 ′, observation noise or noise statistics 30 are also input into the Kalman algorithm.

[0091] According to another embodiment, the sensor 20 reports a confidence value 22 ( Figure 1 In other words, the confidence value 22 depends on the aforementioned probability, or can be seen as indirectly reflecting the aforementioned probability, in the sense that a higher confidence value will be associated with a more accurate detection. In this embodiment, the method further comprises the following steps:

[0092] The confidence value 22 is used as a weighting parameter which is applied to the noisy observation data 21 via the observation noise, ie the error of the noisy observation data 21 ; the observation noise is further input into the Kalman algorithm 10 .

[0093] In this case, the confidence value 22 is used indirectly to weight the noisy observation data 21. Such an operation may be performed by any routine within the processing unit 102 or ultimately within the Kalman filter 101. Noisy observation data 21 with a high confidence value 22 will result in a low noise estimate being provided to the Kalman algorithm 10. Therefore, the tracker 100 will therefore internally give a higher weight to that detection, because the noise estimate is an important part of deciding what weight it should give to that detection.

[0094] according to Figure 5 In another embodiment schematically shown, the sensor 20 reports a confidence value 22 related to the probability that the noisy observation data 21 is correct or accurate, and the method further comprises the following steps:

[0095] comparing the comparison confidence value 22 with a plurality of predetermined confidence intervals 23, each of which is associated with an error parameter 25, and

[0096] The noise statistics 30 contain an error parameter 25 associated with a confidence interval 23 including a confidence value 22 .

[0097] Again, such operations may be performed by any routine within the processing unit 102 or ultimately within the Kalman filter 101. In addition, according to the aforementioned operations, it should be understood that the confidence 22 is assigned to one of the plurality of predetermined confidence intervals 23. Figure 5 In the embodiment, the plurality of predetermined confidence intervals are composed of a plurality of pairs of values ​​{c1, c2}, {c2, c3}, ..., {c k-1 、c k} to limit.

[0098] Preferably, the error parameter 25 is determined during a pre-processing phase which aims to:

[0099] Running the sensor 20 against a set of reference data 51 to collect a plurality of confidence values ​​22,

[0100] The reference data 51 is divided into subsets according to the confidence value 22,

[0101] For each subset, comparing the noisy observation data 21 provided by the sensor 20 with the associated reference data 51, and

[0102] The error covariance and accuracy of the sensor are calculated as data forming the error parameter 25 .

[0103] Moreover, as mentioned earlier, such operations may be performed by any routine within the processing unit 102 or ultimately within the Kalman filter 101. The calculated error covariance (noise covariance) is preferably calculated by referring to v k As Figure 4 The pre-processing stage replaces the confidence value 22 provided by the sensor 20 for an observation with the noise estimated for that particular observation. This will allow for more accurate tracking of the object 40 and for the fusion of multiple sensors as a "black box" without making assumptions about their internal functionality. In this way, tracker performance is improved while allowing confidence-based sensor fusion of a finished software component.

[0104] According to another embodiment, the sensor 20 reports a confidence value 22 quantifying the probability that the noisy observation data 21 is correct or accurate, and the method further comprises the following steps:

[0105] using a regression model that establishes a relationship between error and confidence value 22, the regression model being used to determine the error corresponding to the reported confidence value 22,

[0106] Included in the noise statistics 30 is the error provided by the regression model for the reported confidence value 22 .

[0107] As mentioned before, such an operation may be performed by any routine within the processing unit 102 or ultimately within the Kalman filter 101. In addition, for example, the aforementioned regression model may be used instead of a table providing a mapping between a first column providing confidence values ​​22 and a second column providing associated errors that may be mapped to these confidence values ​​22. By using the data in such a table, a scatter plot on a Cartesian diagram may also be obtained, wherein the values ​​of the two columns may be reported via the two axes of the diagram.

[0108] Therefore, the regression model may be determined based on a scatter plot obtained from multiple detections of the object 40 by the sensor 20 (ie, noisy observation data 21 ), wherein each point in the scatter plot is obtained based on the confidence value 22 and error determined for each detection.

[0109] As suggested in one embodiment, if the confidence values ​​22 reported by the sensors 20 are used, the noise statistics 30 can be considered as confidence-dependent noise statistics. These noise statistics 30 or confidence-dependent noise statistics have two main applications when provided as input to the tracker 100 (especially the Kalman algorithm 10 provided to the tracker 100). The first application is to use the confidence values ​​22 to improve the tracking performance of the Kalman tracker 100, and the second application is to use the confidence values ​​22 in order to fuse multiple sensors 20. In fact, it is sometimes worthwhile to subdivide the covariance estimates across different axes. For example, a LIDAR-based object sensor 20 can accurately predict the orientation of an object 40 (e.g., a vehicle), but such a sensor 20 may make more errors for pedestrians.

[0110] As a first use case aimed at using confidence information for better tracking, Figure 6 A schematic diagram of using noise statistics 30 as a setting parameter of the Kalman algorithm 10 according to the confidence value 22 reported by the sensor 20 is provided. Using the confidence value 22 in the tracker 100 allows the tracker to assign higher confidence to more accurate predictions. Instead of calibrating the tracker using a noise term based on the expected noise of the noisy observation data 21, for example during an initialization or calibration phase, the noise term can also be provided to the tracker 100 on a sample-by-sample basis. The confidence-related noise statistics 30 can be used in a variety of ways in the Kalman tracker 100 to obtain excellent performance.

[0111] In fact, as a first approach, the expected covariance matrix v can be k is fed into the tracker 100 as an "observation noise covariance" parameter. When measured correctly, the aforementioned parameter ensures that the tracker 100 optimally weights the internal state of the tracker against the external observations by the noisy observation data 21. The lower the prediction noise of a particular variable in the observation, the higher the assigned weight of the observation. In contrast to typical applications of Kalman filters, where all observations are assigned similar errors, or the errors are determined heuristically each time, in the present disclosure, the errors are predicted for each detection separately based on the actual measurements.

[0112] As a second approach, the expected accuracy P may be used to determine whether the noisy observation data 21 should be incorporated into a track, in particular into the tracking data 110 output by the tracker 100. This may be determined based on the maximum rate of false positive FPs allowed by the tracker according to predefined setting parameters.

[0113] Therefore, by providing a separate noise covariance matrix v for each noisy observation 21 k , the tracker 100 can be naturally exploited to use better or higher uncertainty estimates when incorporating more uncertain detections (i.e., more uncertain noisy observation data 21). These uncertainty estimates are also carried over into the track noise estimates maintained by the tracker 100, resulting in better decisions when a track is no longer reliable. It should be noted that the noise covariance matrix v k It can be obtained in an automatic manner during the calibration phase and therefore does not require manual tuning of the parameters.

[0114] The second use case mentioned above is intended to use the confidence information for fusing multiple object sensors 20. Figure 7 This is schematically illustrated, with the figure showing fusion of detections from multiple sources (e.g., from sensors 20a and 20b). Each of sensors 20a, 20b includes its own detection method or process, so that different object detection processes typically have different efficiencies or strengths. For example, one process may be more accurate at detecting long-distance objects, while another process may be better at separately detecting two objects that are close together.

[0115] Therefore, fusing the outputs of multiple processes together can provide excellent performance even when applied to the same noisy observation data 21. However, there are significant difficulties when trying to fuse the confidence values ​​22 provided by separate object detectors (such as sensors 20a, 20b) because there is no standard way to report confidence values ​​22, and therefore no reliable way to compare the confidences of different sensors 20a, 20b.

[0116] In practice, differences in reported confidence values ​​22 may include:

[0117] a) The confidence levels are in completely different ranges.

[0118] b) Confidence is within the same range, but based on a different scale. For example, a confidence value of 0.6 may represent a high confidence for sensor 20a, while the same value (0.6) may represent a low confidence for another sensor 20b.

[0119] c) The scales may also be exactly the same, but the sensors 20a, 20b themselves may have different prediction qualities. For example, a low confidence observation provided by a "good" sensor 20 may be comparable to a high confidence observation from a "bad" sensor 20.

[0120] Therefore, comparing the confidence values ​​22 of multiple sensors 20 would require designing the sensors to output the same "type" of confidence. However, doing so would require careful engineering and would prohibit the use of off-the-shelf software components.

[0121] Advantageously, the present disclosure also efficiently solves the problem of comparing the confidence values ​​22 emitted by these sensors 20 by transforming them into a common coordinate system that can be easily compared. In fact, for each of the sensors 20, the expected noise covariance (noise covariance matrix v ) of the sensor 20 can be determined according to the previously disclosed embodiments. k ) and precision P. More specifically, the expected noise covariance v can be determined based on the confidence value 22 of the detector based on the pre-calculated calibration data of the relevant sensor 20 k Instead of using confidence values ​​22 to evaluate the noisy observations 21, each observation (noisy observation) is now compared with its expected precision P and the expected noise covariance matrix v k Related, such as Figure 7 Depicted schematically.

[0122] By replacing the confidence value 22 with the expected noise of the sensor 20 (the noise covariance matrix v k ) and a noise term (noise statistic 30) of precision P, transforming something that only has meaning in the context of a single sensor 20 into something that has general meaning, particularly across multiple sensors 20, and particularly across any sensor 20. As a result, the uncertainties reported by the sensors 20 can be directly compared to each other without requiring any modifications to the sensors 20 themselves. Moreover, the new values ​​included in the noise statistic 30 have interpretable "real world" meaning. Advantageously, this makes it simpler to make design decisions and incorporate constraints, such as specifying a maximum tolerance for predictions of a particular variable of interest.

[0123] The present disclosure also relates to a non-transitory computer readable medium 60 including program instructions for causing a processor (such as Figure 1 The processing unit 102 executes a method according to any one of the embodiments of the present disclosure or any possible combination of the embodiments of the present disclosure.

[0124] The term "non-transitory" does not exclude legal tangible temporary storage media, such as flash drives or any rewritable storage media. In general, computer accessible media may include any tangible or non-transitory storage media or memory media, such as electronic, magnetic or optical media. Such media may be a disk, CD / DVD-ROM or any other medium, which may be connected to processing unit 102, for example, via a dedicated communication interface or one of the already disclosed communication interfaces, or the medium may be located in another device (e.g., in a permanent manner), such as processing unit 102.

[0125] As used herein, the terms "tangible" and "non-transitory" are intended to describe computer-readable storage media (or "memory") that exclude propagating electromagnetic signals, but are not intended to otherwise limit the types of physical computer-readable storage devices encompassed by the phrase computer-readable media or memory. For example, the terms "non-transitory computer-readable media" or "tangible memory" are intended to encompass types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM) and flash memory. Program instructions and data stored in non-transitory form on tangible computer-accessible storage media may also be transmitted via transmission media or signals, such as electrical, electromagnetic, or digital signals.

[0126] The present disclosure also relates to a tracker 100 that implements the aforementioned method according to any one of the embodiments of the present disclosure or any possible combination of the embodiments of the present disclosure. Figure 1 As shown, the tracker 100 includes:

[0127] a processing unit 102 which stores and runs the Kalman algorithm 10 and the steps of the method,

[0128] a first communication interface 104 to which at least one sensor 20 providing noise observation data 21 is connected, and

[0129] A second communication interface 105 receives reference data 51 from an external data source 50 .

[0130] The aforementioned steps of the method cover all necessary steps for performing any embodiment of the method or any possible combination of embodiments of the method. The estimated state of the object 40 is provided, for example, by the tracking data 110 output by the tracker 100 via the third communication interface 103. It should be noted that the number of communication interfaces may be different, for example, a single communication interface may replace a plurality of communication interfaces, in particular among the communication interfaces 103, 104, 105. Moreover, the processing unit 102 may include a Kalman filter 101 (e.g. Figure 1), or if the Kalman filter 101 is located outside the processing unit 102, the processing unit may be connected to the Kalman filter 101. The Kalman filter 101 comprises the Kalman algorithm 10 and may also directly involve the processing unit 102.

[0131] Finally, the present disclosure also relates to a vehicle 70 (preferably a motor vehicle 70) comprising at least the aforementioned tracker 100, such as Figure 1 Since data source 50 is preferably an external or remote data source, the data source is preferably not part of tracker 100 or vehicle 70 .

[0132] Although an overview of the inventive subject matter has been described with reference to specific example embodiments, various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the embodiments of the present disclosure disclosed in this specification.

Claims

1. A method for automatically determining and using noise statistics (30) of noisy observation data (21) in order to improve cross-scene tracking of a time-varying object (40) using a Kalman algorithm (10), the noise statistics (30) being provided by at least one sensor (20) connected to a tracker (100), the Kalman algorithm (10) allowing an estimate (14) of a dynamic parameter of the object (40) to be obtained and providing the accuracy of the estimate (14) based on the noisy observation data (21) and a corrected previous estimate (13'), the method comprising the following steps: receiving reference data (51) from a data source (50) external to the Kalman algorithm (10) and connected to the tracker (100); deriving the noise statistics (30) reflecting errors produced by the sensor (20) from the reference data (51); and Using the noise statistics (30) as setting parameters for the Kalman algorithm (10), wherein the noise statistics (30) include the accuracy of the sensor (20), the accuracy of the sensor (20) defining the probability that the noisy observation data (21) is correct, and wherein the sensor (20) reports a confidence value (22) related to the probability that the noisy observation data (21) is correct or accurate, and the method further comprises the following steps: comparing the confidence value (22) to a plurality of predetermined confidence intervals (23), each of which is associated with an error parameter (25); and The error parameter (25) associated with the confidence interval (23) including the confidence value (22) is included in the noise statistics (30).

2. The method according to claim 1, wherein: The reference data (51) relates to a scene for which the noisy observation data (21) has been obtained by the sensor (20), and wherein the noise statistics (30) are obtained from the reference data (51) by comparing the reference data (51) with the noisy observation data (21) provided by the sensor (20).

3. The method according to claim 2, wherein: The reference data (51) includes annotated observation data received from the data source (50), and the reference data (51) is used to calibrate the tracker (100).

4. The method according to claim 1, wherein: The accuracy of the sensor (20) is used to determine whether the relevant noisy observation data (21) provided by the sensor (20) will be considered by the tracker (100) for tracking the object (40) or will be discarded.

5. The method according to claim 1, wherein: The noisy observation data (21) relates to a plurality of measured variables (41) of the object (40), at least one pair of measured variables (41) can be identified from the plurality of measured variables, and the noise statistics (30) include at least one of the following: the expected variance of each of the measured variables (41), and The expected covariance between each pair of measured variables (41).

6. The method according to claim 5, wherein: The measured variable (41) relates to at least one of the following: dimensional data of the object (40), Positioning data of the object (40) within the scene, distance data between the object (40) and the sensor (20), absolute speed data or relative speed data of the object (40), and Absolute acceleration data or relative acceleration data of the object (40).

7. The method according to claim 5 or 6, wherein: The expected covariance between each pair of measured variables (41) is given by the covariance matrix (v k ) or in the form of a vector and is used as input to the Kalman algorithm (10) for defining the observation noise in the noisy observation data (21).

8. The method according to claim 1, wherein: The error parameter (25) is determined during a pre-processing phase which aims to: operating the sensor (20) against a set of reference data (51) to collect a plurality of confidence values ​​(22), dividing the reference data (51) into subsets according to the confidence value (22), for each subset, comparing the noisy observation data (21) provided by the sensor (20) with the associated reference data (51), and The error covariance and accuracy of the sensor (20) are calculated as data for forming the error parameter (25).

9. A non-transitory computer readable medium (60) containing program instructions, the program instructions causing a processor to execute the method according to any one of claims 1 to 8.

10. A tracker (100) implementing the method according to any one of claims 1 to 8, the tracker comprising: a processing unit storing and executing the Kalman algorithm (10) and the steps of the method; a first communication interface (104) connected to at least one sensor (20) providing noise observation data (21); as well as A second communication interface (105) receives reference data (51) from an external data source (50).

Images