Credibility evaluation method of whole vehicle in-the-loop simulation system based on site

Abstract

The invention provides a site-based credibility evaluation method for a whole vehicle in-the-loop simulation system, and relates to the technical field of vehicle testing. The method comprises the following steps: constructing a credibility evaluation index covering two dimensions of virtual-real interaction fidelity and system real-time synchronization performance; performing consistency check on the virtual-real interaction fidelity evaluation index and the system real-time synchronization performance evaluation index to obtain a first check result aiming at the virtual-real interaction fidelity evaluation index and a second check result aiming at the system real-time synchronization performance evaluation index; performing dimensionless conversion on the first test result and the second test result to obtain a first dimensionless credibility score for the virtual-real interaction fidelity evaluation index and a second dimensionless credibility score for the system real-time synchronization performance evaluation index; and based on the first dimensionless credibility score and the second dimensionless credibility score, calculating the comprehensive credibility of the site-based whole vehicle in-the-loop simulation system, and outputting a credibility evaluation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobile testing, in particular to a credibility evaluation method of a field-based vehicle-in-the-loop simulation system. BACKGROUND

[0002] With the continuous improvement of the intelligent level of automobiles, the functional complexity of Advanced Driver Assistance Systems (ADAS) and Autonomous Driving (AD) systems continues to improve, and higher requirements are put forward for the comprehensiveness, safety and efficiency of testing and verification technology. Vehicle-in-the-Loop (VIL) simulation testing technology, as a key bridge connecting virtual simulation and real vehicle testing, has become a key means for ADAS / AD system development and verification. Among them, the field-based vehicle-in-the-loop simulation system, as a kind of VIL system with unique structure and characteristics, combines the dynamic characteristics of real vehicles with virtual scene simulation technology, not only retains the high-fidelity dynamic response of real vehicle testing, but also has the diversification, repeatability and high safety of virtual testing scenes, effectively overcoming the shortcomings of limited scene coverage, high risk and high cost of pure real vehicle testing, and making up for the shortcomings of pure virtual simulation in reflecting the behavior of real vehicles.

[0003] Credibility, as an important indicator of the field-based vehicle-in-the-loop simulation system, is used to represent the accuracy and reliability of the simulation test results, and directly affects the effectiveness of ADAS / AD system development and verification. At present, there is no special credibility evaluation method for the field-based vehicle-in-the-loop simulation system. Due to the essential differences in system architecture and operation mechanism, the existing evaluation methods of other in-the-loop simulation systems cannot be directly applied to the credibility evaluation of this type of system. That is, the existing evaluation methods cannot scientifically, accurately and comprehensively evaluate the credibility of the field-based vehicle-in-the-loop simulation system.

[0004] Therefore, it is urgent to establish a multi-dimensional credibility evaluation method specially for the core characteristics of the field-based vehicle-in-the-loop simulation system, to solve the problems of insufficient adaptability and one-sided evaluation of existing evaluation methods, and to provide technical support for the reliable application of this type of system. SUMMARY

[0005] Therefore, the present application provides a credibility evaluation method of a field-based vehicle-in-the-loop simulation system, to solve the problems of insufficient adaptability and one-sided evaluation of existing evaluation methods.

[0006] The credibility evaluation method of the field-based vehicle-in-the-loop simulation system provided by the embodiments of the present application comprises: An evaluation index covering two dimensions of virtual-real interaction fidelity and system real-time synchronization performance is constructed. The virtual-real interaction fidelity evaluation index and the system real-time synchronization performance evaluation index are subjected to consistency test by using a gray correlation degree analysis method and an interval construction method respectively to obtain a first test result and a second test result, the first test result being a test result for the virtual-real interaction fidelity evaluation index, and the second test result being a test result for the system real-time synchronization performance evaluation index. The first test result and the second test result are subjected to non-dimensional conversion to obtain a first non-dimensionalized credibility score and a second non-dimensionalized credibility score in the interval [0, 1], the first non-dimensionalized credibility score being a non-dimensionalized credibility score for the virtual-real interaction fidelity evaluation index, and the second non-dimensionalized credibility score being a non-dimensionalized credibility score for the system real-time synchronization performance evaluation index. Based on the first non-dimensionalized credibility score and the second non-dimensionalized credibility score, a comprehensive credibility of the site-based whole vehicle-in-the-loop simulation system is calculated, and a credibility evaluation result is output.

[0007] The credibility evaluation method of the site-based whole vehicle-in-the-loop simulation system according to the embodiments of the present application introduces multi-dimensional evaluation indexes of two core dimensions of virtual-real interaction fidelity and system real-time and synchronization performance, can completely reflect key influencing factors of system credibility, solves the problem of one-sidedness of existing methods, and realizes all-around characterization of system credibility. Moreover, it can realize consistency verification with pertinence and unified quantitative standard. The present application can systematically solve the problem of insufficient adaptability of existing credibility evaluation methods, realizes comprehensive, accurate and objective evaluation of credibility of the site-based whole vehicle-in-the-loop simulation system, and provides a scientific basis for reliable test and verification of assisted driving and automatic driving systems.

[0008] The above description is only a summary of the technical solutions of the present application, in order to enable the technical means of the present application to be more clearly understood, the present application can be implemented in accordance with the content of the description, and in order to enable the above and other purposes, features and advantages of the present application to be more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0009] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of the credibility evaluation method of the site-based whole vehicle-in-the-loop simulation system according to the embodiments of the present application is shown. DETAILED DESCRIPTION

[0010] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly described. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0011] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / " generally indicates that the objects before and after are in an "or" relationship.

[0012] In the related art, the evaluation method for the hardware-in-the-loop simulation system mainly focuses on the consistency of the hardware interface and the model accuracy, but lacks evaluation dimensions for the real vehicle actuator response characteristics, the vehicle dynamics behavior and the virtual-real environment interaction fidelity. The evaluation method for the bench-type vehicle-in-the-loop simulation system focuses on the matching degree of the bench and the model, ignoring the motion state measurement of the real vehicle in the field environment, the real-time synchronization of the virtual scene and the real vehicle position and other characteristics in the field vehicle-in-the-loop simulation system. Other in-the-loop simulation system credibility evaluation methods also have the problems of single evaluation dimension, and fail to fully consider the virtual sensor model accuracy, the environment model physical reality, the virtual-real state synchronization accuracy and other field VIL system specific key indicators.

[0013] The embodiments of the present application propose a credibility multi-dimensional evaluation method specially for the core characteristics of the field-based vehicle-in-the-loop simulation system, which solves the problems of single evaluation dimension, insufficient test pertinence, non-uniform quantitative standard and lack of application guide of the existing evaluation methods. The credibility evaluation method of the field-based vehicle-in-the-loop simulation system provided by the embodiments of the present application will be described in detail below in combination with the drawings and specific embodiments and application scenarios. In the case of no conflict, the embodiments and features in the embodiments described below can be combined with each other.

[0014] The embodiments of the present application provide a credibility evaluation method for a field-based vehicle-in-the-loop simulation system, as shown in Figure 1 The method comprises: Step 101, constructing credibility evaluation indexes covering two dimensions of virtual-real interaction fidelity and system real-time synchronization performance; Step 102, the virtual-real interaction fidelity evaluation index and the system real-time synchronization performance evaluation index are subjected to consistency test by using the grey correlation degree analysis method and the interval construction method respectively, to obtain a first test result and a second test result, the first test result is the test result for the virtual-real interaction fidelity evaluation index, and the second test result is the test result for the system real-time synchronization performance evaluation index; Step 103, the first test result and the second test result are subjected to non-dimensional conversion to obtain a first non-dimensionalized credibility score and a second non-dimensionalized credibility score in the interval [0, 1], the first non-dimensionalized credibility score is the non-dimensionalized credibility score for the virtual-real interaction fidelity evaluation index, and the second non-dimensionalized credibility score is the non-dimensionalized credibility score for the system real-time synchronization performance evaluation index; Step 104, based on the first non-dimensionalized credibility score and the second non-dimensionalized credibility score, the comprehensive credibility of the site-based whole vehicle-in-the-loop simulation system is calculated, and a credibility evaluation result is output.

[0015] In this embodiment, first, the credibility evaluation index covering the virtual-real interaction fidelity and the system real-time synchronization performance is constructed; second, according to the attribute of the evaluation index, the grey correlation degree analysis method and the interval construction method are used for targeted consistency test respectively; then, the test result is subjected to non-dimensional conversion to obtain a non-dimensionalized credibility score in the interval [0, 1]; finally, the comprehensive credibility of the site-based whole vehicle-in-the-loop simulation system is calculated, and a determination result of high, medium and low credibility levels is output.

[0016] The embodiment of the present application introduces multi-dimensional evaluation indexes of virtual-real interaction fidelity and system real-time and synchronization performance, which can completely reflect the key influencing factors of system credibility, solve the problem of one-sided evaluation of existing methods, and realize all-around characterization of system credibility. Moreover, it can realize targeted consistency verification and unified quantitative standard. The present application can systematically solve the problem of insufficient adaptability of existing credibility evaluation methods, realize comprehensive, accurate and objective evaluation of the credibility of the site-based whole vehicle-in-the-loop simulation system, and provide a scientific basis for reliable testing and verification of assisted driving and automatic driving systems.

[0017] The credibility evaluation method of the site-based whole vehicle-in-the-loop simulation system according to the embodiment of the present application specifically includes the following steps: Step one: constructing a multi-dimensional credibility evaluation index system (1) Virtual-Real Interaction Realism The virtual-real interaction realism dimension mainly includes three evaluation indexes: sensor model accuracy, environment model physical accuracy, and vehicle dynamics interface consistency. Among them, the sensor model accuracy is used to represent the statistical agreement degree of the virtual sensor output data and the real sensor measured data in the same scene. The environment model physical accuracy is used to represent the nominal error of the dynamics parameters of the road and target objects in the virtual scene and the corresponding parameters in the real world. The vehicle dynamics interface consistency is used to represent the matching degree of the real vehicle actuator response characteristics and the preset dynamics model of the simulation system.

[0018] (2) System Real-Time and Synchronization Performance The system real-time and synchronization performance mainly includes three evaluation indexes: end-to-end closed-loop delay, sensor synchronization accuracy, and virtual-real state synchronization accuracy. Among them, the end-to-end closed-loop delay is used to represent the maximum delay from the virtual scene triggering event to the real vehicle actuator producing effective action. The sensor synchronization accuracy is used to represent the timestamp deviation of different types of virtual sensor output data. The virtual-real state synchronization accuracy is used to represent the alignment error of the real vehicle actual state and the virtual scene host vehicle simulation state.

[0019] Step two: perform consistency test on the credibility evaluation indexes of each dimension Perform consistency test calculation on each credibility evaluation index established in step one, output quantitative consistency test results, which include the following two sub-steps: Sub-step 2.1: use the grey correlation degree analysis method to perform consistency test on the virtual-real interaction realism evaluation indexes one by one, which includes the following five sub-steps: Sub-step 2.1.1: determine the reference sequence as , and the comparison sequence as . Among them, n represents the sequence length, , represents m comparison sequences. represents the reference sequence, which is usually selected as the real vehicle field test result, the theoretical optimal value or the standard value as the reference sequence, represents the comparison sequence, which is based on the field VIL system test data.

[0020] Sub-step 2.1.2: data standardization processing​ Considering that the dimensions of different indicators may be different, the sequence needs to be standardized to eliminate the influence of dimension. The mean value method is used in this application to standardize the sequence, which can be expressed as:

[0021] In the formula, represents the kth data point of the ith sequence in the original sequence, represents the standardized data, represents the mean value of the ith sequence.

[0022] Substep 2.1.3: Calculate the sequence difference For each data point , calculate the absolute difference between the comparison sequence and the reference sequence: . Wherein, represents the absolute difference of the kth data point, represents the kth data point of the reference sequence after standardization, represents the kth data point of the comparison sequence after standardization.

[0023] Further determine the extreme value, the two-level minimum difference can be expressed as , and the two-level maximum difference can be expressed as .

[0024] Substep 2.1.4: Calculate the correlation coefficient Use the following formula to calculate the correlation coefficient :

[0025] In the formula, is the resolution coefficient, usually taking , used to adjust the difference degree of the correlation coefficient.

[0026] Substep 2.1.5: Calculate the grey correlation degree Average the correlation coefficients output by substep 2.1.4 to determine the grey correlation degree, which can be expressed as:

[0027] In the formula, represents the grey correlation degree, that is, the consistency test result-output, the first test result, the value range is [0, 1], the closer the correlation degree is to 1, the better the consistency.

[0028] Substep 2.2: Use the interval construction method to perform consistency test on the system real-time and synchronization performance evaluation indicators, which includes the following three substeps: Substep 2.2.1: Construct the standard allowable interval as:

[0029] In the formula, S represents a reference value, and T represents an allowed absolute error limit value, both of which are mainly determined by the design requirements of each reliability evaluation index or the physical true value.

[0030] Sub-step 2.2.2: Obtain the measured value of the reliability evaluation index and calculate the absolute deviation of the measured value from the standard value wherein M represents the actual output value of the system on the index obtained through measurement or experiment.

[0031] Sub-step 2.2.3: Calculate the deviation rate, which can be expressed as:

[0032] In the formula, represents the deviation rate, which is the direct output result of the consistency test - the second test result.

[0033] Step three: Dimensionless conversion of the consistency test result The dimensionless conversion process adopts differentiated conversion rules for the consistency test results of different types of evaluation indexes. For the consistency test results of the virtual-real interactive fidelity evaluation index, dimensionless conversion is performed to obtain the dimensionless reliability score. The calculation formula is:

[0034] In the formula, represents the dimensionless reliability score of the virtual-real interactive fidelity evaluation index.

[0035] For the consistency test results of the real-time and synchronization performance evaluation index, dimensionless conversion is performed to obtain the dimensionless reliability score. The calculation formula is:

[0036] In the formula, represents the dimensionless reliability score of the real-time and synchronization performance evaluation index.

[0037] Step four: Comprehensive reliability assessment The comprehensive reliability of the venue-based VIL system is assessed, which includes the following three sub-steps: Sub-step 4.1: Calculate the comprehensive reliability of the venue-based VIL system, which can be expressed as:

[0038] In the formula, R is the comprehensive reliability score, and the value range is [0, 1].

[0039] Sub-step 4.2: Reliability level division criteria The credibility level is divided into high, medium and low three levels, wherein, represents high credibility, that is, the field-based VIL system has high credibility and can be directly used for key function test verification of the ADAS / AD system; represents medium credibility, , that is, it is recommended to optimize the field-based VIL system before putting it into key function or performance test; represents low credibility, that is, the field-based VIL system has low credibility, and it is recommended to comprehensively check the problems of virtual scene simulation, data injection, state measurement and other core modules and complete rectification before re-executing the credibility evaluation.

[0040] Sub-step 4.3: According to the comprehensive credibility calculated in sub-step 4.1 and the credibility level division criterion constructed in sub-step 4.2, the credibility evaluation result of the field-based VIL system is obtained.

[0041] It should be noted that in this paper, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of another identical element in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiments of the present application is not limited to the order of functions shown or discussed, but can also include functions performed in a substantially simultaneous manner or in the opposite order, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0042] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative, not restrictive, and those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A credibility evaluation method for a site-based vehicle-in-the-loop simulation system, characterized in that, include: Construct a credibility evaluation index covering two dimensions: the realism of virtual-real interaction and the real-time synchronization performance of the system; The consistency of the virtual-real interaction realism evaluation index and the system real-time synchronization performance evaluation index is tested by the grey relational analysis method and the interval construction method, respectively, to obtain the first test result and the second test result. The first test result is the test result for the virtual-real interaction realism evaluation index, and the second test result is the test result for the system real-time synchronization performance evaluation index. The first and second test results are transformed into dimensionless form to obtain the first dimensionless confidence score and the second dimensionless confidence score located in the interval [0,1]. The first dimensionless confidence score is the dimensionless confidence score for the evaluation index of the realism of virtual-real interaction, and the second dimensionless confidence score is the dimensionless confidence score for the evaluation index of the real-time synchronization performance of the system. Based on the first dimensionless credibility score and the second dimensionless credibility score, the overall credibility of the site-based vehicle-in-the-loop simulation system is calculated, and the credibility evaluation result is output.

2. The method according to claim 1, characterized in that, Construct a credibility evaluation index covering two dimensions: the realism of virtual-real interaction and the real-time synchronization performance of the system, including: A credibility evaluation index with two dimensions is constructed, namely the realism of virtual-real interaction and the real-time synchronization performance of the system. Several exclusive evaluation indicators are set under each dimension. The evaluation metrics for the realism of virtual-real interaction include: sensor model accuracy, environmental model physical accuracy, and vehicle dynamics interface consistency. Among them, sensor model accuracy is used to characterize the statistical consistency between the output data of virtual sensors and the measured data of real sensors in the same scene; environmental model physical accuracy is used to characterize the nominal error between the dynamic parameters of roads and target objects in the virtual scene and the corresponding parameters in the real world; and vehicle dynamics interface consistency is used to characterize the degree of matching between the response characteristics of the real vehicle actuators and the preset dynamic model of the simulation system. The evaluation metrics for the system's real-time and synchronization performance include: end-to-end closed-loop delay, inter-sensor synchronization accuracy, and virtual-real state synchronization accuracy. Among them, end-to-end closed-loop delay is used to characterize the maximum delay from the virtual scene triggering an event to the actual vehicle actuator producing an effective action; inter-sensor synchronization accuracy is used to characterize the timestamp deviation of the output data of different types of virtual sensors; and virtual-real state synchronization accuracy is used to characterize the alignment error between the actual state of the actual vehicle and the simulated state of the virtual scene's main vehicle.

3. The method according to claim 1, characterized in that, The consistency of the evaluation index for the realism of virtual-real interaction and the evaluation index for the real-time synchronization performance of the system were tested using grey relational analysis and interval construction method, respectively, yielding the first test result and the second test result, including: Sub-step 2.1: Use grey relational analysis to perform a consistency test on the evaluation index of the realism of virtual-real interaction, and obtain the first test result; wherein, sub-step 2.1 includes: Sub-step 2.1.1: Determine the reference sequence as The comparison sequence is Where n represents the sequence length, , indicating that there are m comparison sequences; Sub-step 2.1.2: Standardize the sequence using the mean method. The standardization formula is as follows: In the formula, This represents the k-th data point of the i-th sequence in the original sequence. This represents the k-th data point after standardization. Let represent the mean of the i-th sequence; Sub-step 2.1.3: For each data point Calculate the absolute difference between the comparison sequence and the reference sequence. ;in, This represents the absolute difference of the k-th data point. This represents the k-th data point after standardization of the reference sequence. This represents the k-th data point after the comparison sequence has been standardized; the extreme values ​​are determined, and the minimum difference between the two levels is expressed as... The maximum difference between the two levels is expressed as ; Sub-step 2.1.4: Calculate the correlation coefficient Correlation coefficient The calculation formula is: In the formula, The discrimination coefficient is used to adjust for the degree of difference in the correlation coefficient. ; Sub-step 2.1.5: Calculate the average correlation coefficient to determine the grey relational degree. Grey relational degree The calculation formula is: In the formula, This represents the grey relational degree, i.e., the result of the first test, with a value range of [0, 1]. Sub-step 2.2: Use the interval construction method to perform a consistency check on the system's real-time synchronization performance evaluation indicators, and obtain the second check result; wherein, sub-step 2.2 includes: Sub-step 2.2.1: Construct the standard allowable interval as follows: In the formula, S represents the reference value, and T represents the allowable absolute error limit; Sub-step 2.2.2: Obtain the measured value M of the system's real-time synchronization performance evaluation index, and calculate its absolute deviation from the reference value S. ; Sub-step 2.2.3: Calculate the deviation rate Deviation rate The calculation formula is: In the formula, This represents the deviation rate, i.e., the result of the second test.

4. The method according to claim 1, characterized in that, The first and second test results are transformed into dimensionless values ​​to obtain the first and second dimensionless confidence scores located in the interval [0,1], including: The first test result is transformed into a dimensionless form to obtain the first dimensionless confidence score. The formula for the dimensionless transformation is as follows: In the formula, This represents the first dimensionless confidence score. This indicates the result of the first test; The second test result is then transformed into a dimensionless form to obtain the second dimensionless confidence score. The formula for the dimensionless transformation is as follows: In the formula, This represents the second dimensionless credibility score. This indicates the result of the second test.

5. The method according to claim 1, characterized in that, Based on the first and second dimensionless confidence scores, the overall confidence of the site-based vehicle-in-the-loop simulation system is calculated, and the confidence assessment results are output, including: Calculate the overall reliability of the site-based vehicle-in-the-loop simulation system. The calculation formula is: In the formula, R represents the overall credibility. This represents the first dimensionless confidence score. This represents the second dimensionless credibility score; Based on comprehensive credibility and credibility level classification criteria, the credibility assessment results of the site-based vehicle-in-the-loop simulation system are obtained; among which, the credibility level classification criteria include: It has been determined to be of high credibility level. This indicates a medium confidence level. It was determined to be at a low credibility level.

Images