Evaluation of convergence time and adjustments based on convergence time evaluation

Abstract

A convergence time of one or more parameter values tracked by a vehicle is determined and used, for example, to adjust a vehicle system or a vehicle component, or more particularly to adjust operations performed by the vehicle system or the vehicle component. The convergence time may be an amount of time that the parameter value satisfies a defined condition (e.g., converges toward a value or set of values defined as a reference value). Alternatively, the convergence time may be an amount of time to cause the error level to meet a defined condition, such as by reducing to less than or equal to a defined threshold.

Description

Technical Field

[0001] Embodiments of the disclosed subject matter are generally directed to systems and methods, including computer program products, for evaluating convergence time of predicted, measured, inferred, or otherwise estimated values ​​produced by a vehicle system or vehicle component and, for example, adjusting the vehicle system or component based on the evaluation. Background Art

[0002] Vehicles are currently designed to comply with government regulations and follow international standards, such as those provided by the International Standards Organization (ISO). These regulations and standards define requirements that vehicles must meet. For example, these regulations or standards specify that semi-autonomous control systems and autonomous control systems should meet a certain level of accuracy in order for the system to control some or all of the driving functions of the vehicle. If these accuracy levels are not achieved, the system may be prevented from controlling some or all of the driving functions of the vehicle. Therefore, the analysis and evaluation of these systems can focus on the accuracy of the results.

[0003] Analysis focused on processing time and result accuracy can be complicated by erroneous or missing data, inaccurate measurements, or unpredictable result values. This can result in incorrect adjustments of vehicle systems or vehicle components, or incorrect conclusions that a vehicle system or vehicle component meets government or standard requirements. Therefore, it is desirable to provide systems and methods for evaluating vehicle systems and vehicle components and, based on the evaluation, adjusting the vehicle systems or vehicle components in a manner that can account for erroneous or missing data, inaccurate measurements, and unpredictable result values. Summary of the invention

[0004] Exemplary embodiments relate to systems and methods for evaluating the convergence time of a series of values ​​of one or more parameters predicted, measured, estimated, or inferred by a vehicle system (or more specifically, by one or more vehicle components) and adjusting the vehicle system or the vehicle component based on the convergence time evaluation.

[0005] In one aspect, the convergence time is the time taken for a series of values ​​of a parameter output from the vehicle system or the vehicle component to satisfy a defined (e.g., predefined or dynamically defined) condition. The convergence time assessment can be based on a single series of values ​​output from the vehicle system / the vehicle component, or based on multiple series of values ​​output from the vehicle system / the vehicle component.

[0006] In another aspect, the convergence time is the time taken for a deviation between a series of values ​​of a parameter output from the vehicle system or the vehicle component and a series of reference values ​​of the parameter to satisfy a defined (e.g., predefined or dynamically defined) condition. The convergence time assessment may be based on a single series of values ​​of the parameter output from the vehicle system / the vehicle component and a single series of reference values ​​or multiple series of values ​​of the parameter output from the vehicle system / the vehicle component.

[0007] In one aspect, each value in a series of values ​​output by the vehicle system / the vehicle component comprises a time stamp. In another aspect, each value in a series of reference values ​​comprises a time stamp.

[0008] The vehicle system or the vehicle component may be, for example, an electronic control unit, an integrated device controller or a sensor. In one aspect, the vehicle system or the vehicle component may be, for example, a vehicle safety system or an object recognition component.

[0009] [Claims will be inserted here once finalized] BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more embodiments and, together with the description, serve to explain these embodiments. In the drawings:

[0011] Figures 1A to 1C is a schematic diagram of a vehicle according to an embodiment;

[0012] Figure 2A and Figure 2B is a flow chart of an exemplary method according to an embodiment;

[0013] Figure 3 is a graph illustrating a series of values ​​of a parameter output from a vehicle system / vehicle component and a series of reference values ​​according to an embodiment;

[0014] Figure 4 is a graph illustrating convergence time for a series of estimated speed values ​​and a series of reference speed values ​​according to an embodiment;

[0015] Figure 5 including a graph illustrating an error function versus sample convergence time versus defined conditions according to an embodiment;

[0016] Figure 6 including a graph illustrating an error function versus sample convergence time versus defined conditions according to an embodiment;

[0017] Figure 7 is a graph illustrating a first divergent sample event according to an embodiment;

[0018] Figure 8 is a graph illustrating convergence time based on average divergent samples according to an embodiment;

[0019] Fig. 9 and Fig.10 is a graph illustrating relative convergence times for the same data but with different defined conditions according to an embodiment; and

[0020] Fig.11 is a graph illustrating relative convergence time curves for different defined conditions according to an embodiment. DETAILED DESCRIPTION

[0021] The following description of exemplary embodiments is made with reference to the accompanying drawings. The same figure numbers in different drawings identify the same or similar elements. The following detailed description does not limit the present invention. Instead, the scope of the present invention is defined by the appended claims.

[0022] References throughout the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the disclosed subject matter. Thus, the phrases "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Additionally, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Additionally, the term "or" in this specification means "and / or".

[0023] Exemplary embodiments relate to systems and methods for determining convergence time of a series of values ​​of a parameter tracked by a vehicle (e.g., by a sensing system of the vehicle). One or more parameter values ​​may each be an estimated value predicted, measured, estimated, or inferred by a vehicle system or vehicle component (e.g., a sensing system that processes outputs from one or more sensors). In some implementations, convergence time may be used to evaluate operations performed by a vehicle system and / or to adjust operations performed by a vehicle system or vehicle component. Non-limiting examples of parameters include perceived object speed, object position, intersection over union (IoU) scores between two objects, Mahalanobis distance, Kalman filter innovation, Manhattan distance, cosine dissimilarity distance, loss function of a machine learning component, number of objects in a scene, state of a vehicle ahead, different values ​​describing the state of the surrounding environment of a vehicle while driving or while performing a parking function, etc.

[0024] Figures 1A to 1C is a schematic diagram of a vehicle system / vehicle component according to an embodiment. Figure 1AA vehicle 100A is illustrated that includes a vehicle system / vehicle component 102 coupled to a processor 104A of the vehicle. The vehicle system / vehicle component 102 may also be referred to as a sensing system. In some examples, the processor may be configured to further adjust the operation of the vehicle system / vehicle component 102 based on the convergence time (details of which are described in more detail below).

[0025] Figure 1B A vehicle 100B is illustrated that includes a vehicle system / vehicle component 102 that is coupled to a processor 104B that is configured to execute modules that evaluate convergence time and adjust the vehicle system / vehicle component 102 based on the evaluation (details of which are described in more detail below).

[0026] Figure 1C A vehicle 100C is illustrated that includes a vehicle system / vehicle component 102 that is coupled to a processor 104B via a processor 106 that includes dedicated hardware or software for evaluating convergence time and adjusting the vehicle system / vehicle component 102 based on the evaluation (details of which are described in more detail below). Figure 1A In a vehicle of the type shown in FIG. 1 , processor 104A is a processor that performs relative convergence time processing in addition to performing other types of processing, and Figure 1B and Figure 1C The processor 104B in the embodiment is a processor dedicated to performing relative convergence time processing. Thus, for example, the processor 104A can be a main processor of the vehicle. As another example, the processor 104A can be a sensor processor that processes sensor signals and performs relative convergence time processing. Additionally, the processor 106 can be a main processor of the vehicle, or can be another processor that couples the relative time convergence processor 104B with the vehicle system / vehicle component 102.

[0027] Processor 104A and processor 104B may include hardware configured to execute software, or more generally to execute the steps of a method (such as a method for determining convergence time). In one embodiment, the processor described herein may include at least one of the following: a microprocessor, a system on a chip (SoC), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a microcontroller, etc. Although not specifically described, processor 104A, processor 104B and / or processor 106 may include a memory storing processor executable code to perform the functions disclosed herein and other functions. The memory may be any type of non-transitory memory.

[0028] exist Figures 1A to 1CIn the illustrated vehicle, the vehicle system / vehicle component 102, the processor 104A or the processor 106, and the relative convergence time processor 104B can be appropriately connected to each other by direct connection via a system bus (such as a CAN bus commonly used in vehicles). The vehicle system / vehicle component 102 can be any system or component that predicts, measures, estimates, or infers parameters. Non-limiting examples of vehicle systems / vehicle components 102 include electronic control units (ECUs), integrated device controllers (IDCs), sensors (e.g., radars, laser radars (LIDAR), image sensors, etc.), object recognition systems, automatic parking systems, systems for preventing collisions during parking, crossing traffic warning systems, collision avoidance systems, driving systems (e.g., adaptive cruise control, automatic lane keeping and / or control, emergency brake assist systems, semi-autonomous driving systems, autonomous driving systems), occupant safety systems (e.g., seat belts and / or airbag deployment systems), pedestrian safety systems, etc., which can be implemented by hardware or as software executed on hardware.

[0029] It should be recognized that Figures 1A to 1C The three vehicle configurations illustrated in FIG. 1 are non-limiting examples, and the systems and methods described below may be implemented in a variety of different vehicle configurations without departing from the present invention.

[0030] Figure 2A and Figure 2B Example of Figures 1A to 1C The method performed by the vehicle illustrated in FIG. Figure 2A, processor 104A or processor 104B receives information (step 202) defining the conditions used as a part of the evaluation. Vehicle system / vehicle component 102 outputs a series of values ​​(hereinafter referred to as parameter values) of the parameters received by processor 104A or processor 104B and the time (hereinafter referred to as time value) associated with each value of the series of parameter values ​​(step 204). For example, the time value can be a timestamp. If the timestamp is not associated with these values, these values ​​can be organized by indexing. The series of parameter values ​​are predicted, measured, estimated, inferred or otherwise determined by vehicle system / vehicle component 102. Processor 104A or processor 104B uses the series of parameter values ​​and the associated time value to calculate the time period (also referred to as time amount or time passed) that these values ​​meet the conditions of definition (step 206). In one embodiment, the time period can measure the time or how much time is needed for the series of parameter values ​​to converge to meet the conditions of definition, and therefore can be referred to as relative convergence time. In this example, the conditions of definition can also be referred to as acceptance criteria in this article. In some implementations, the processor 104A or the processor 104B then adjusts the vehicle system / vehicle component 102 based on the time period (step 208). It should be appreciated that in some instances, the calculated time period is acceptable, in which case step 208 may be omitted.

[0031] Now go to Figure 2B , processor 104A or processor 104B receives information describing defined conditions used as part of the evaluation (step 210). The vehicle system / vehicle component 102 outputs a series of parameter values ​​and associated time values ​​received by processor 104A or processor 104B (step 212A). The parameter is predicted, measured, estimated, or inferred by the vehicle system / vehicle component 102. Processor 104A or processor 104B also receives a series of reference values ​​with associated time values ​​(step 212B). These reference values ​​are also referred to herein as ground truth values. Similarly, the time values ​​associated with the parameter values ​​are also referred to herein as timestamps. If a timestamp is not associated with the parameter value, the parameter values ​​can be organized by indexing. Although Figure 2B Step 212A and step 212B are illustrated as being performed in parallel, but these values ​​and associated times may be received sequentially by processor 104A or processor 104B. In addition, these values ​​and associated times may be received as a batch, or when they are generated by vehicle system / vehicle component 102.

[0032] Processor 104A or processor 104B correlates the series of parameter values ​​output by the vehicle system / vehicle component with the series of reference values ​​based on the associated time values ​​(step 214). This allows the series of parameter values ​​output by the vehicle system / vehicle component 102 to be aligned with the corresponding series of reference values ​​in time. Processor 104A or processor 104B determines the deviation between the time alignment values ​​and determines the time period in which the deviation meets the defined condition (step 216). The deviation is also referred to as an error value in this article. Processor 104A or processor 104B then adjusts the vehicle system / vehicle component 102 based on the time period (step 218). It should be appreciated that in some instances, the calculated time period is acceptable, in which case step 218 can be omitted.

[0033] Now that an overview of the exemplary aspects has been provided, Figures 3 to 11 A more detailed discussion of the system and method is provided.

[0034] Figure 3 1 illustrates a graph of convergence time-related values ​​in a sequence of estimated values ​​and reference values. The leftmost point on the x-axis is the sample event time, and the rightmost point on the x-axis is the convergence event time, i.e., the time point at which the difference between the output of the vehicle system / vehicle component 102 (labeled "resulting estimated value") and the reference value meets the acceptance criteria, which is the time point at which the difference between the output of the vehicle system / vehicle component 102 (labeled "resulting estimated value") and the reference value meets the acceptance criteria. Figure 3 The curve of is the time point when the error amount is lower than or equal to the defined error threshold. The zigzag line 302 represents the output of the vehicle system / vehicle component 102, and the zigzag line 304 represents the series of reference values. The relative convergence time (sample-RCT) of the sample is Figure 3 It is represented as the leftmost vertical line, whose length is proportional to the distance (elapsed time) between the sample event (the leftmost point on the x-axis) and the convergence point (the rightmost point on the x-axis).

[0035] Each parameter value provided by vehicle system / vehicle component 102 is received by processor 104A or processor 104B and processed as a Si occurs and is given by the value x i Describes a single sample event S i :

[0036] S i = {x i , t si} (1)

[0037] For i=1..N S , where N S is the number of samples available for computing the relative convergence time.

[0038] Sample S iAcceptance criteria A i describes a time-invariant function with a binary outcome (true or false). In this non-limiting example, if the sample value S i Satisfies the given Boolean expression P(S i ), then function A i Returns true, otherwise returns false:

[0039]

[0040] An example of a Boolean expression that accepts the value x i The given error Err(x i ) is lower than a given threshold T err :

[0041] P(S i ): Err(x i ) ≤ T err (3)

[0042] In the above formula, given value x i The error function Err(x i ) can be defined as the estimated value x i With the ground truth x gt,i The absolute value of the difference between the (reference) values:

[0043] Err(x i )=|x gt,i -x i | (4)

[0044] To simplify the notation, x gt,i will be referred to as x gt .

[0045] The use of errors and Boolean expressions is an exemplary implementation, and various other Boolean expressions P(S) may be used. i ) to specify the exact acceptance criterion A i As will be appreciated from the discussion herein, the disclosed systems and methods may involve other types of Boolean expressions and acceptance criteria, and may operate in the presence of different information types, missing data, or noisy results.

[0046] Additionally, the sampling rate of the reference value may be different from the sampling rate of the parameter value output by the vehicle system / vehicle component 102. This discussion assumes that all of these specific specifications are defined and addressed by the operator based on specific use cases. In other words, it is assumed that for each sample event, an acceptance criterion can be calculated (i.e., in the above example, for each sample event, there is a reference value available).

[0047] The convergence point describes the "converged" sample event for which the acceptance criterion A i (S i ) is true, and for its previous sample events, the acceptance criterion A i-1 (S i-1 ) is false. In other words, for example, when x i The error is lower than the threshold T err When x i Towards its predefined acceptance criteria A i (S i )convergence.

[0048] For a given sample S i , sample event relative convergence time sample-RCT(S i ) indicates the current S i The time that elapsed between an event and its closest convergence point event T(i) (which is in the future).

[0049]

[0050] where a describes any sample event S that occurs after the current sample event Si a Or accept standard A a The index of , and d describes the index of i Any sample event S occurring between its corresponding convergence point at index c d Or accept standard A d The index of .

[0051] Considering the sample event S i Occurs at Si and the next closest convergence point event occurs at time T(i), sample-RCT(S i ) can be expressed as (see Figure 3 ):

[0052] Sample - RCT (S i )=Δt i =T(i)-t Si (6)

[0053] When convergence points are not available (e.g., when all future sample events never converge toward the “acceptance criterion”), the sample-RCT (S i ) is Δt max and can be chosen to be zero or non-zero.

[0054] A "batch of samples-RCT values" (or simply "a batch") describes the number of samples before a given convergence point (see Figure 5) or the set of all consecutive non-zero samples - RCTs computed before the end of the sequence (if no convergence point is found).

[0055] Taking into account the above notation, we can define a sequence relative convergence time, which will be referred to as RCT (omitting the world "sequence"). In one non-limiting embodiment, RCT is defined as the sample average and is calculated as all samples - RCT (S i )’s average value:

[0056]

[0057] The unbiased RCT sample variance represents a measure of the uncertainty of the RCT and can be calculated as:

[0058]

[0059] The RCT sample variance indicates how far the sample RCT values ​​spread out from their mean RCT. The lower the variance, the higher the confidence that can be found in the provided RCT.

[0060] To calculate the RCT, assume that the sample events S i An independent event.

[0061] If the sample-RCT (S i ) values ​​are normally distributed, then given a sample event sequence, it has mean μ = RCT and standard deviation σ RCT The probability distribution of the normal distribution N(μ=RCT,σ RCT ) is maximized.

[0062] Figure 4 The example shows a sequence-RCT calculated using parameter values, which are the estimated speed (vel) sequence of the object provided by the vehicle ECU. If the ECU is used to perform perception functions, the ECU can be called a "perception ECU". Figure 4, speed estimate 404 is for a single object provided by the vehicle perception ECU. These parameter values ​​are input to processor 104A or processor 104B. Zigzag line 402 is the reference speed. The absolute difference between zigzag line 402 and zigzag line 404 provides the speed error. The acceptance criterion in this example is whether the speed error is less than 0.2m / s. All vertical lines are the relative convergence (RCT) values ​​estimated in time for each sample event. It can be seen, for example, that it takes 1.2 seconds for the estimation component to converge when the object is first identified (first vertical line). Afterwards, if time 64 is used as the reference point, the sample event RCT value is reduced to 0.6. Finally, after convergence, the RCT time is 0, which means that the value output by the processor 104A or processor 104B algorithm has converged with the reference value within the speed error and there is no estimation time delay. It can also be seen that around time 68 or 71.5, due to the high estimation error, the sample event RCT calculation returns a positive RCT value, which is interpreted as the time required for the parameter value output by processor 104A or processor 104B to return to its expected acceptance criteria (approximately 0.1 seconds to 0.2 seconds).

[0063] In summary, the above use cases provide concrete examples for calculating RCT values, the details are as follows:

[0064] - Sample event value x i : Described by the detected object speed: x i =vel i

[0065] - for accepting standard boolean expressions:

[0066] P(S i ):Error(x i )≤T err |vel i -vel gt，i |≤T err (9)

[0067] Where T err is the error threshold, set to 0.2 (m / s), vel i is the estimated velocity, and vel gt,i is the ground truth velocity.

[0068] -The acceptance criterion function is described as:

[0069]

[0070] We will now discuss the differences between ideal / theoretical use cases for calculating RCT and conventional practical use cases, and will demonstrate that the calculation of RCT can be applied to any of these use cases.

[0071] Although the above use case uses error (or error function) as a defined condition (i.e., acceptance criterion), in other use cases, different defined conditions may be used. In a non-limiting example where the vehicle system / vehicle component 102 implements computer vision functionality, the defined condition may be the number of edges of an object identified in a captured image.

[0072] To better express, Figure 5 This depicts the situation when the acceptance criterion is based on the error function:

[0073] Error(i) = x gt，i -x i (11)

[0074] And when the error function is below a given threshold, the parameter value of a given component converges to the acceptance criterion:

[0075] P(S i ): Error(i)<Threshold(12)

[0076] Therefore, instead of displaying the raw result values, the error function (the difference between the reference value and the parameter value output by the vehicle system / vehicle component 102) is plotted directly, as shown in FIG. Figure 3 or Figure 4 The resulting values ​​are shown in .

[0077] Figure 5 The error function versus sample convergence time (vertical bars) versus acceptance criteria is illustrated. This example can be called an "ideal" example, where the error function is described by a monotonically decreasing function, where the sample times are equidistant and have only one single convergence point. In such a case, the RCT analysis can be simplified to the following result:

[0078]

[0079] where Δt1 is the sample-RCT of the first available sample event (at time t C1 The first received result value at t S1 is the convergence time step. The above relationship can be derived from interpreting the distribution of vertical RCT bars. This can be performed in two different ways:

[0080] (1) The average sample-RCT is the middle of the segment connecting the first sample-RCT (S1) and the convergence point.

[0081] (2) The distribution of sample-RCT describes a right triangle with two orthogonal sides of length Δt1.

[0082] Unlike the ideal use case, the actual result data used as input to processor 104A or processor 104B looks similar to Figure 6 , which illustrates the error function versus sample convergence time (vertical bars) versus acceptance criteria.

[0083] The values ​​associated with the computational convergence may be affected by:

[0084] - Missing data: Events may be accompanied by missing information and not all error / data samples may be available.

[0085] - Unordered sample events: Sample events may not be time-ordered (may be provided out of sequence), so RCT calculation should be able to handle "unordered" sequences, for example by sorting sample events by timestamp / index.

[0086] - Arbitrary data or error functions: The actual data (i.e., the parameter values ​​output by the vehicle system / vehicle component 102) may be corrupted or inaccurate. The output of the processor 104A or processor 104B that must be evaluated may provide unpredictable results. Thus, as an example, Figure 5 Unlike, the actual error function can be depicted by an arbitrary shape (not necessarily a nice monotonically decreasing function). Therefore, multiple convergence points may be provided. In this case, the simple, closed-form solution of computing the RCT is not suitable.

[0087] - Unknown convergence point: The convergence point may be unknown (it is not known whether the output value of processor 104A or processor 104B will converge with the reference value). This is especially common for the last sequence data samples, for which the convergence point is not available (no information) in the future.

[0088] - Overlapping data: Multiple events (ie, parameter values ​​output by vehicle systems / vehicle components 102 ) may be acquired / received simultaneously.

[0089] - Non-uniform / multiple "acceptance criteria": Multiple sample events (i.e., parameter values ​​output by vehicle systems / vehicle components 102) that may be acquired simultaneously may be subject to different "acceptance criteria"

[0090] This precludes the possibility of simple reasoning about, for example, an "ideal" use case.

[0091] As will be appreciated from the above discussion, processor 104A or processor 104B is able to handle all of the above constraints and challenges, thereby providing reliable information that is consistent and proportional to the convergence time of the component being evaluated.

[0092] Furthermore, the disclosed systems and methods may work with components having different automotive safety integrity level (ASIL) capabilities.

[0093] The calculation of the RCT may be performed by a dedicated component for a single vehicle system / vehicle component 102 (e.g., Figure 1B and Figure 1C ) or it may be performed by a common component for multiple vehicle systems / vehicle components 102 (e.g., Figure 1A Processor 104A or Figure 1B and Figure 1C This reduces costs by avoiding the need to implement and rebuild specific evaluators for specific components.

[0094] As will be appreciated from the above discussion, the RCT provides important information about the performance of the vehicle system / vehicle component 102, and this information can be used to adjust the operation of the vehicle system / vehicle component 102 to improve the performance of the vehicle system / vehicle component 102. Variations of the RCT can also provide important information for evaluating and adjusting the vehicle system / vehicle component 102, including RCT based on first divergent sample events (RCT-FDS), RCT based on mean divergent samples (RCT-MDS), RCT curves, and total RCT convergence sensitivity, each of which will now be described in detail.

[0095] Figure 7 The first divergent sample-RCT value is illustrated, which is the first sample in a group of consecutive data samples (this example shows 3 different sample-RCT groups). The approximation of the RCT calculation only considers these samples. The RCT based on the first divergent sample is shown as an approximation. Considering that the error function may converge / diverge several times, the RCT-FDS only illustrates the first sample-RCT (S) in a batch of consecutive sample-RCT values ​​before a given convergence point. i ). These first data samples describe events when the resulting data exceeds the acceptance criteria, or in other words, events where the resulting data diverges (as opposed to a convergence point).

[0096] Because only the sample-RCT values ​​with the largest convergence times are considered, the first divergent sample-RCT value is biased towards the worst case. The advantage of this technique is faster processing time, at the expense of being less accurate for use cases containing missing data samples (gaps in the result data).

[0097] Will combine Figure 8 The RCT based on mean divergence samples (RCT-MDS) is described. The RCT based on mean divergence samples is calculated similarly to the previously described RCT-FDS. Figure 7 and Figure 8 It will be appreciated that the difference is that for each batch of consecutive RCT values, the mean sample-RCT (S i) for processing. RCT based on average divergent samples reduces complexity and simplifies processing (i.e., fewer samples to process, to memorize, etc.) at the expense of being less accurate on sequences with missing data or in use cases with multiple acceptance criteria.

[0098] Now combine Fig. 9 and Fig.10 Describe the RCT curve. In the RCT calculation process, the acceptance criterion function can use a specific threshold. The calculated RCT value depends on the value of the set threshold. For example, Figure 4 As shown, for the speed-RCT, the following acceptance criterion function is used:

[0099]

[0100] Where T err Indicates the speed error threshold.

[0101] Fig. 9 and Fig.10 This paper illustrates how RCT results can be affected by the error threshold. Fig. 9 , the zigzag line 902 represents the reference speed, the zigzag line 904 represents the speed output of the vehicle system / vehicle component 102, the zigzag line 906 represents the convergence time, and the vertical line represents the RCT. Fig.10 , zigzag line 1002 represents the reference speed, zigzag line 1004 represents the speed (ie, parameter value) output by the vehicle system / vehicle component 102, zigzag line 1006 represents the convergence time, and the vertical line represents the RCT.

[0102] For the speed error threshold of 1.5 m / s (see Fig. 9 , marked as "speed threshold"), the RCT obtained is 0.65s (marked as "norct" in the figure). For the speed error threshold of 0.2m / s (see Fig.10 ), the RCT value obtained is 8.15 s. It should be understood that the distribution of sample-RCT values ​​(vertical bars) is different in each of these cases, i.e., if a smaller threshold value (0.2 m / s, see Fig.10 ), then for most of the series, the RCT does not converge towards the “acceptance criterion” used.

[0103] The RCT curve provides a better understanding of how the RCT evolves based on the different constraints adopted for the acceptance criteria. Therefore, the RCT calculation is repeated for the entire data set with different acceptance criteria sampled from a set of possibilities. For example, in the case of speed-RCT illustrated in the figure above, the RCT curve is calculated by plotting multiple RCT values ​​calculated with different speed error thresholds.

[0104] Now combine Fig.11 To describe the total convergence sensitivity (RCT sensitivity), and its calculation can be summarized by the following steps:

[0105] 1. Calculate the RCT curve for a series of sampling acceptance criteria - RCT.

[0106] 2. Calculate the sum of all RCTs calculated in the previous steps:

[0107]

[0108] in:

[0109] RCT (A j )-describes the acceptance criteria function A by using a specific j The RCT is calculated at iteration j; and

[0110] N k is the number of iterations, and each iteration j represents the use of a specific acceptance criterion A j RCT (A j ) are calculated separately.

[0111] 3. Calculate the integral of the worst-case curve. In the worst case, for each iteration of step 2,

[0112] We will get Δt max , which is the time allocated for the case where the estimate never converges:

[0113] Normality factor = N K Δt max (15)

[0114] 4. Calculate the convergence sensitivity:

[0115]

[0116] It should be understood that if the algorithm converges, then its SUM RCT approaches 0, and then the RCT probability approaches 1, and if the algorithm never converges, its SUM RCT =N K Δt max , and therefore the convergence probability RCT probability is close to 0.

[0117] The above discussion in conjunction with speed as a parameter value output by the vehicle system / vehicle component 102 is merely exemplary, and any other parameter value output by the vehicle system / vehicle component 102 may be employed. For example, the image recognition output of the vehicle system / vehicle component may have an acceptance criterion that at least five lines of an object must be identified before the object is considered. Thus, the output from the vehicle system / vehicle component 102 may be a series of parameter values ​​for the number of lines of the object identified, and the RCT value would be the time it took for the vehicle system / vehicle component output to indicate that five lines of the object were identified.

[0118] Furthermore, while the discussion relates to a single vehicle system / vehicle component and a single parameter, processor 104A or processor 104B may process outputs from multiple vehicle systems / vehicle components, each vehicle system / vehicle component having one or more parameters analyzed for relative convergence time, which relative convergence time may then be used to adjust corresponding ones of the vehicle systems / vehicle components.

[0119] The disclosed embodiments provide systems and methods for evaluating convergence time and adjusting vehicle systems / vehicle components based on the evaluation of convergence time of the output of the vehicle system / vehicle component. It should be understood that this description is not intended to limit the present invention. On the contrary, the exemplary embodiments are intended to cover replacements, modifications and equivalents included within the spirit and scope of the present invention as defined by the appended claims. In addition, in the specific implementation of the exemplary embodiments, many specific details are set forth in order to provide a comprehensive understanding of the claimed invention. However, those skilled in the art will understand that the various embodiments can be practiced without such specific details.

[0120] Although features and elements of the exemplary embodiments are described in particular combinations in the embodiments, each feature or element can be used alone without the other features and elements of the embodiments, or in various combinations with or without the other features and elements disclosed herein.

[0121] This written description uses examples of the disclosed subject matter to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the subject matter is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are within the scope of the claims.

[0122] Without in any way limiting the scope, interpretation, or application of the claims appearing below, the technical effect of one or more of the exemplary embodiments disclosed herein is the ability to determine whether the temporal convergence of a series of values ​​output by a vehicle system / vehicle component complies with government regulations and / or international standards, thereby enabling such vehicle system / vehicle component to operate on public roads.

[0123] If desired, the different functions discussed herein may be performed in different orders and / or simultaneously with each other. In addition, if desired, one or more of the above functions may be optional or may be combined. Although various aspects of some embodiments in the embodiments are set forth in the independent claims, other aspects of some embodiments in the embodiments include other combinations of features from the embodiments and / or dependent claims with features of the independent claims, rather than just the combinations explicitly set forth in the claims. It should also be noted that, in this article, although exemplary embodiments are described above, these descriptions should not be considered as limiting. On the contrary, several changes and modifications may be made without departing from the scope of some embodiments in the embodiments as defined in the appended claims.

Claims

1. A method, comprising: Outputting, by a vehicle system (102) or a vehicle component (102) of a vehicle, a series of parameter values ​​and a series of time values ​​associated with the series of parameter values, wherein the series of parameter values ​​are estimated values ​​predicted, measured, estimated, inferred, or otherwise determined by the vehicle system (102) or the vehicle component (102); Receiving (202) the series of parameter values ​​and the associated series of time values ​​from the vehicle system (102) or the vehicle component (102) by a processor (104) of the vehicle; determining (206), by the processor (104), a time period during which the series of parameter values ​​satisfies a defined condition using at least one time value from the series of time values; Outputting the time period by the processor (104); as well as The vehicle system (102) or the vehicle component (102) is adjusted (208) based on the output time period.

2. A method according to claim 1, wherein the series of parameter values ​​includes a first series of parameter values ​​and a second series of parameter values, and wherein the time period determined by the processor (104) includes a first time period in which the first series of parameter values ​​meets the defined condition, and includes a second time period in which the second series of parameter values ​​meets the defined condition.

3. The method according to claim 1 or 2, wherein the vehicle system (102) or the vehicle component (102) is or includes at least one of the following: an electronic control unit, an integrated device controller, or a sensor.

4. The method according to any one of claims 1 to 3, wherein the vehicle system (102) is a safety system of the vehicle, or wherein the vehicle component (102) is a component of the safety system of the vehicle.

5. A method according to any one of claims 1 to 4, wherein the series of parameter values ​​is a series of speed values.

6. The method according to any one of claims 1 to 4, wherein the vehicle system (102) or the vehicle component (102) includes an object recognition system, and wherein the series of parameter values ​​are the number of edges of the object recognized by the object recognition system.

7. The method according to any one of claims 1 to 6, further comprising: Receiving (212B), by the processor (104), a series of reference values ​​and a series of time values, wherein each time value in the series of time values ​​is respectively associated with a value in the series of reference values; correlating, by the processor, each of the values ​​of the first series of parameter values ​​with a corresponding value of the values ​​of the series of reference values ​​using the series of time values ​​associated with each of the parameter values ​​of the first series of parameter values ​​and the series of time values ​​associated with the values ​​of the series of reference values ​​(214) to produce a series of sets of correlated values ​​and associated correlated times; as well as determining, by the processor (104), a difference between each value in each set of correlation values ​​in the series of sets of correlation values, The time period for determining that the series of parameter values ​​satisfy the defined condition includes The time values ​​associated with the first series of parameter values ​​and the associated correlation times are used by the processor (104) to determine (216) a time period over which the series of sets of correlation values ​​meet a defined condition.

8. A vehicle, comprising: A vehicle system (102) or a vehicle component (102) configured to output a series of parameter values ​​and a series of time values ​​associated with the series of parameter values, wherein the parameter values ​​are predicted, measured, estimated or inferred by the vehicle system (102); and a processor (104) configured to receive the series of parameter values ​​and the associated series of time values ​​from the vehicle system (102) or the vehicle component (102); using at least one time value in the series of time values ​​to determine a time period during which the series of parameter values ​​satisfies a defined condition; as well as Output the time period, The vehicle system (102) or the vehicle component (102) is adjusted based on the output time period.

9. The vehicle of claim 8, wherein the series of parameter values ​​comprises a first series of parameter values ​​and a second series of parameter values, and the time period determined by the processor (104) comprises a first time period during which the first series of parameter values ​​satisfy the defined condition and a second time period during which the second series of parameter values ​​satisfy the defined condition.

10. The vehicle according to claim 8 or 9, wherein the vehicle system (102) or the vehicle component (102) is an electronic control unit, an integrated device controller, or a sensor.

11. The vehicle according to claim 8 or 9, wherein the vehicle system (102) is a safety system of the vehicle, or the vehicle component is a component of the safety system of the vehicle.

12. A vehicle as claimed in any one of claims 8 to 11, wherein the series of parameter values ​​are speed values.

13. The vehicle according to claim 8 or 9, wherein the vehicle system (102) or the vehicle component (102) includes an object recognition component, and the series of parameter values ​​are the number of edges of the object recognized by the object recognition system component.

14. The vehicle according to any one of claims 8 to 13, wherein the processor (104) is configured to perform the receiving, the determining, the processing, the outputting and the adjusting, and is configured to process other data for operating the vehicle.

15. The vehicle of any one of claims 8 to 14, wherein the processor is further configured to: Receiving (212B) a series of reference values ​​and a series of time values, wherein each time value in the series of time values ​​is respectively associated with one value in the series of reference values; correlating (214) each of the values ​​of the first series of parameter values ​​with a corresponding value of the values ​​of the series of reference values ​​using the series of time values ​​associated with each of the parameter values ​​of the first series of parameter values ​​and the series of time values ​​associated with the values ​​of the series of reference values ​​to produce a series of sets of correlated values ​​and associated correlated times; and determining a difference between each value in each set of correlation values ​​in the series of sets of correlation values, The time period for determining that the series of parameter values ​​satisfy the defined condition includes The time values ​​associated with the first series of parameter values ​​and the associated correlation times are used to determine (216) a time period during which the series of sets of correlation values ​​meet a defined condition.

16. A method comprising: Outputting, by a vehicle system (102) or a vehicle component (102) of a vehicle, a first series of parameter values ​​and a series of time values, wherein each time value in the series of time values ​​is respectively associated with a parameter value in the first series of parameter values, wherein the parameter values ​​in the series of parameter values ​​are predicted, measured, estimated or inferred by the vehicle system (102) or the vehicle component (102); Receiving (212A), by a processor (104) of the vehicle, the first series of parameter values ​​and the associated series of time values ​​from the vehicle system (102) or the vehicle component (102); Receiving (212B), by the processor (104), a series of reference values ​​and a series of time values, wherein each time value in the series of time values ​​is respectively associated with a value in the series of reference values; correlating, by the processor, each of the values ​​of the first series of parameter values ​​with a corresponding value of the values ​​of the series of reference values ​​using the series of time values ​​associated with each of the parameter values ​​of the first series of parameter values ​​and the series of time values ​​associated with the values ​​of the series of reference values ​​(214) to produce a series of sets of correlated values ​​and associated correlated times; determining, by the processor (104), a difference between each value in each set of correlation values ​​in the series of sets of correlation values; determining (216), by the processor (104), using the time values ​​associated with the first series of parameter values ​​and the associated correlation times, a time period over which the series of sets of correlation values ​​satisfy a defined condition; Outputting the time period by the processor (104); as well as The vehicle system (102) or the vehicle component (102) is adjusted (218) based on the output time period.

17. A method according to claim 16, wherein the first series of parameter values ​​and the series of reference values ​​each include an initial series of values ​​and a subsequent series of values, and the time period determined by the processor (104) includes a first time period in which the initial series values ​​satisfy the defined condition and a second time period in which the subsequent series values ​​satisfy the defined condition.

18. The method of claim 16 or 17, wherein the vehicle system (102) or the vehicle component (102) is an electronic control unit, an integrated device controller, or a sensor.

19. The method according to claim 16 or 17, wherein the vehicle system (102) is a safety system of the vehicle, or the vehicle component is a component of the safety system of the vehicle.

20. A method according to any one of claims 16 to 19, wherein the parameter of the first series of parameter values ​​is speed.

21. The method of claim 16 or 17, wherein the vehicle system (102) or the vehicle component (102) comprises an object recognition component, and the parameter of the first series of parameter values ​​is the number of edges of the object recognized by the object recognition component.

22. A vehicle, comprising: A vehicle system (102) or a vehicle component (102) configured to output a first series of parameter values ​​and a series of time values, wherein each time value in the series of time values ​​is respectively associated with a value in the first series of parameter values, wherein the parameter values ​​in the series of parameter values ​​are predicted, measured, estimated or inferred by the vehicle system (102) or the vehicle component (102); and a processor (104) configured to receive (212A) from the vehicle system (102) or the vehicle component (102) the first series of parameter values ​​and the series of time values ​​respectively associated with one of the parameter values ​​in the first series of parameter values; Receiving (212B) a series of reference values ​​and a series of time values, wherein each time value in the series of time values ​​is respectively associated with one value in the series of reference values; correlating (214) each of the values ​​of the first series of values ​​with a corresponding value of the values ​​of the series of reference values ​​using the series of time values ​​associated with each of the parameter values ​​of the first series of parameter values ​​and the series of times associated with the values ​​of the series of reference values ​​to produce a series of sets of correlated values ​​and associated correlated times; determining a difference between each value in each set of correlation values ​​in the series of sets of correlation values; using the time values ​​associated with the first series of parameter values ​​and the associated correlation times to determine (216) a time period over which the series of sets of correlation values ​​satisfies a defined condition; and Output the time period, The vehicle system (102) or the vehicle component (102) is adjusted based on the output time period.

23. A vehicle according to claim 22, wherein the first series of parameter values ​​and the series of reference values ​​each include an initial series value and a subsequent series value, and the time period determined by the processor (104) includes a first time period in which the initial series value satisfies the defined condition and a second time period in which the subsequent series value satisfies the defined condition.

24. The vehicle of claim 22 or 23, wherein the vehicle system (102) or the vehicle component (102) is an electronic control unit, an integrated device controller, or a sensor.

25. The vehicle according to claim 22 or 23, wherein the vehicle system (102) is a safety system of the vehicle, or the vehicle component is a component of the safety system of the vehicle.

26. A vehicle as claimed in any one of claims 22 to 25, wherein the series of parameter values ​​are speed values.

27. The vehicle of claim 22 or 23, wherein the vehicle system (102) or the vehicle component (102) comprises an object recognition component, and the parameter of the first series of parameter values ​​is the number of edges of the object recognized by the object recognition component.

28. The vehicle of any one of claims 22 to 27, wherein the processor (104) is configured to perform the receiving, the determining, the processing, the outputting and the adjusting, and is configured to process other data for operating the vehicle.

29. A non-transitory computer readable medium storing processor-executable instructions which, when executed by the processor, cause the processor to perform the method of any one of claims 1 to 7.

30. A non-transitory computer readable medium storing processor executable instructions which, when executed by the processor, cause the processor to perform the method of any one of claims 16 to 21.

Images