Credibility evaluation method for site whole vehicle in-the-loop simulation test system
By constructing a multi-layered credibility evaluation index framework and a subjective-objective integrated weighting method, the scientific and accuracy issues of credibility assessment of the whole vehicle in-loop simulation test system were solved, and credibility quantification and level assessment were realized, thereby improving the scientific nature and interpretability of the assessment results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-02
- Publication Date
- 2026-04-07
AI Technical Summary
Existing technologies lack scientific credibility assessment schemes, making it difficult to quantify the credibility of on-site vehicle-in-the-loop simulation testing systems, resulting in unconvincing simulation results.
A multi-layered credibility evaluation index framework is constructed, which quantifies the core influencing factors of credibility from multiple dimensions at each subsystem level. The subjective and objective fusion weighting method is used to calculate the fusion weight of the credibility quantification index, and the weighted aggregation method is used to map it into the credibility semantic level.
It enables a scientific and quantitative assessment of the credibility of the whole vehicle in-loop simulation test system, improves the scientificity and accuracy of the assessment results, lowers the interpretation threshold, and facilitates quick understanding of the credibility level.
Smart Images
Figure CN121807693A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent driving vehicle simulation testing, and in particular to a reliability evaluation method for a site-based whole vehicle-in-the-loop simulation testing system. Background Technology
[0002] With the rapid development of intelligent connected vehicle technology, testing and evaluation have become crucial for ensuring vehicle safety and reliability. Currently, the "three-pillar method" is widely used for testing and evaluation of intelligent connected vehicles, encompassing three main categories: simulation testing, track testing, and road testing. Among these, simulation testing uses computer modeling to simulate vehicles, sensors, and driving scenarios in a virtual environment, thereby verifying autonomous driving systems. It offers advantages such as efficient coverage of both conventional and edge scenarios, effective avoidance of safety risks, test repeatability, and parameter controllability.
[0003] In the simulation testing system, the whole-vehicle-in-the-loop simulation test takes the whole vehicle as the core. It generates diverse virtual test scenarios through simulation software, injects data collected by virtual sensors into the algorithm controller, and then tests the motion state of the actual vehicle in the simulation environment. The whole-vehicle-in-the-loop simulation test not only retains the advantages of simulation testing in terms of scenario coverage, safety risks, repeatability, and controllability, but also enhances the realism of the simulation through a combination of virtual and real methods.
[0004] The reliability of a vehicle-in-the-loop (VIN) simulation testing system is an indicator describing the consistency between simulation and real-world test results, and a prerequisite for judging the effective application of test results. Without a scientific reliability assessment, test data and conclusions generated based on the VIN simulation testing system will lack technical persuasiveness. Therefore, establishing a scientific reliability assessment scheme is the cornerstone supporting the engineering application of VIN simulation testing and a key to promoting the continuous development of the entire simulation testing system's capabilities.
[0005] Current research largely focuses on the credibility assessment of traditional software simulation or hardware-in-the-loop simulation, and a systematic evaluation scheme for on-site vehicle-in-the-loop simulation has not yet been developed. Compared to other simulation testing systems, on-site vehicle-in-the-loop simulation testing systems introduce real vehicles and actual roads into the test loop, which is fundamentally different from other systems in structure. Therefore, it is difficult to refer to the credibility assessment methods of other simulation systems to conduct credibility assessments of on-site vehicle-in-the-loop simulation testing systems.
[0006] In summary, how to construct a scientific credibility assessment scheme for a vehicle-in-the-loop simulation test system, quantify the credibility of the system, and thus ensure the accuracy and credibility of the simulation results has become a key technical challenge that urgently needs to be overcome in the field of intelligent driving simulation testing. Summary of the Invention
[0007] In view of this, this application provides a reliability evaluation method for a track-based vehicle-in-the-loop simulation test system, which solves the problem of the lack of a reliability evaluation scheme for track-based vehicle-in-the-loop simulation test systems.
[0008] This application provides a reliability evaluation method for a site-based vehicle-in-the-loop simulation test system, including: From the perspective of each subsystem level of the on-site vehicle-in-the-loop simulation test system, multiple dimensions are used to determine the quantifiable indicators of credibility. Based on the weighting method that integrates subjective and objective factors, the fusion weights of the credibility quantification indicators are calculated. By using a weighted aggregation method, the credibility quantification index and fusion weight are calculated to obtain the credibility value of the whole vehicle in-loop simulation test system in the field, and the credibility value is mapped to the credibility semantic level.
[0009] The method described in the embodiments of this application may also have the following additional technical features: In the above technical solution, optionally, the site vehicle-in-the-loop simulation test system includes a virtual scene simulation subsystem, a scene data injection subsystem, an ADAS / AD algorithm software and hardware platform and a real vehicle subsystem, and a real vehicle motion state measurement subsystem; The determination of reliability metrics from multiple dimensions at each subsystem level of the on-site vehicle-in-the-loop simulation test system includes: A first credibility metric is constructed for the virtual scene simulation subsystem. The first credibility metric includes a static scene element simulation consistency evaluation index, a dynamic scene element simulation consistency evaluation index, a virtual camera detection position error evaluation index, and a virtual radar ranging error evaluation index. A second reliable metric is constructed for the data injection subsystem of the aforementioned scenario. The second reliable metric includes a bus message packet loss rate evaluation metric, a message data accuracy evaluation metric, and a data injection synchronization accuracy evaluation metric. A third reliable metric is constructed for the ADAS / AD algorithm hardware and software platform and the actual vehicle subsystem. The third reliable metric includes the control decision response delay compliance rate evaluation index. A fourth reliable quantitative index is constructed for the actual vehicle motion state measurement subsystem. The fourth reliable quantitative index includes a longitudinal velocity correlation evaluation index, a lateral velocity correlation coefficient evaluation index, a vehicle position error evaluation index, and a vehicle attitude error evaluation index.
[0010] In any of the above technical solutions, optionally, the weighting method based on the fusion of subjective and objective factors, for calculating the fusion weight of the credibility quantification index, includes: The subjective weights of each credibility quantification index are calculated based on the subjective weighting method, and the objective weights of each credibility quantification index are calculated based on the objective weighting method. The subjective weights and objective weights of the indicators are weighted and fused to obtain the fusion weights of the corresponding credible quantification indicators. The process involves calculating a credibility metric and fusion weights using a weighted aggregation method to obtain the credibility value of the on-site vehicle-in-the-loop simulation test system, and mapping the credibility value to a credibility semantic level, including: The reliability metrics are normalized to obtain normalized data. , Indicates a reliable metric The normalized value; The reliability value of the on-site vehicle-in-the-loop simulation test system is calculated using an aggregation algorithm; the formula for calculating the reliability value is as follows:
[0011] Where D represents the confidence value of the on-site vehicle-in-the-loop simulation test system. This represents the normalized value of the j-th credibility metric. Let be the fusion weight corresponding to the j-th credibility metric.
[0012] In any of the above technical solutions, optionally, the subjective weighting of each credibility quantification index based on the subjective weighting method includes: Based on prior experience, various credibility metrics are quantified. The indicators are sorted from highest to lowest importance to obtain the importance order relationship, which is as follows:
[0013] in, The ordinal indicator represents the highest level of importance. The next most important indicator is indicated by its order of importance, and so on. The ordinal indicator representing the least important value. It is the k-th ordinal indicator after the indicators are sorted by importance, where k represents the importance ordinal number, k=1,2,...,12; Determine the relative importance of adjacent ordinal indicators in the importance ranking relationship, and calculate the ordinal indicator weight of the least important ordinal indicator; where the relative importance of adjacent ordinal indicators is the ratio of their importance, and the formula for calculating the ordinal indicator weight of the least important ordinal indicator is:
[0014] in, Indicator of adjacent order and The ratio of importance The ordinal indicator weight represents the ordinal indicator with the lowest importance. Following the reverse order of indicator importance, and based on the relative importance of adjacent indicators, the weights of other indicators are calculated sequentially. The formula for calculating the weights of other indicators is as follows:
[0015] in, This represents the weight of the k-th ordered indicator in the importance order relationship. This represents the weight of the (k+1)th ordered indicator in the importance order relationship. Based on the mapping relationship between credible quantification metrics and ordinal metrics in the importance ranking relationship, the weights of ordinal metrics are determined. This is reduced to the subjective weights of each reliable metric. ,in, Let g be the subjective weight of the g-th credibility quantification index, where g = 1, 2, ..., 12.
[0016] The objective weights of each credible quantification index calculated based on the objective weighting method include: An indicator data matrix is constructed based on historical test data from m samples; the indicator data matrix is as follows:
[0017] Where X represents the indicator data matrix, Let represent the value of the j-th confidence metric for the i-th sample, where i = 1, 2, ..., m, j = 1, 2, ..., 12; Based on indicator attributes Dimensionless processing is performed, where I1, I2, I6, I7, I8, I9, I 10 These are positive indicators: I3, I4, I5, I 11 I 12 It belongs to the negative index; the formula for dimensionless processing is:
[0018] in, Indicator data matrix The value after dimensionless processing; Calculate the correlation coefficient of information overlap, the degree of information difference, and the information content parameter among the credibility metrics; the formula for calculating the correlation coefficient of information overlap is:
[0019] in, Let represent the correlation coefficient between the j-th credibility metric and the h-th credibility metric. This represents the dimensionless mean of the j-th indicator. This represents the dimensionless mean of the h-th indicator. This represents the dimensionless value of the i-th sample of the h-th confidence metric; The formula for calculating the degree of information discrepancy is:
[0020] in, This represents the sum of the information differences between the j-th credibility metric and the other 11 metrics; The formula for calculating the information content parameter is:
[0021] in, This represents the sum of information content of the j-th credibility metric; The objective weights of the credibility quantification index are calculated based on the information content parameter; the formula for calculating the objective weights is:
[0022] in, This represents the objective weight of the j-th credibility metric.
[0023] In any of the above technical solutions, optionally, the step of weighting and fusing the subjective weights and objective weights of the indicators to obtain the fusion weight of the corresponding credible quantification indicator includes: The subjective weights and objective weights of the indicators are weighted and fused to obtain the fused weight of the corresponding credibility quantification indicator; the formula for weighted fusion is:
[0024] in, This represents the fusion weight of the j-th credibility metric. This represents the objective weight of the j-th credibility metric. This represents the subjective weight of the j-th credibility metric. and These represent the allocation coefficients for objective weights and subjective weights, respectively. After calculating the fusion weights of each credibility quantification index, the range method is used to transform the values of each fusion weight to the [0,1] interval to obtain the transformed fusion weights.
[0025] The credibility assessment method for the on-site vehicle-in-the-loop simulation test system in this application constructs a multi-layered credibility evaluation index framework, quantifying the core influencing factors of system credibility from multiple dimensions at each subsystem level. Then, a weighting method based on subjective and objective fusion is employed, integrating expert experience and test data to calculate index weights. Finally, the credibility value of the on-site vehicle-in-the-loop simulation test system is calculated through weighted aggregation, and further mapped to an intuitive credibility semantic level.
[0026] Compared to credibility assessment methods in related technologies, the method proposed in this application achieves a scientific and quantitative assessment of credibility, specifically reflected in: (1) This application proposes a credibility evaluation method for a whole vehicle in-the-loop simulation test system, which realizes the quantification of the credibility of the whole vehicle in-the-loop simulation test system and the scientific evaluation of its credibility level.
[0027] (2) This application adopts the subjective and objective fusion weighting method, which integrates the experience of experts in the field with the statistical characteristics of measured data, effectively avoids the arbitrariness of purely subjective judgment and the "mechanical" defects of pure data dependence, and ensures the improvement of the scientificity and accuracy of the evaluation results.
[0028] (3) This application transforms complex and multidimensional quantitative indicators into intuitive comprehensive scores and semantic levels by weighted aggregation and credibility level mapping, which greatly reduces the threshold for interpreting the evaluation results and makes it easier for relevant personnel to quickly grasp the credibility level of the whole vehicle in-loop simulation test system.
[0029] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0030] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating the credibility evaluation method of the site-based vehicle-in-the-loop simulation test system according to an embodiment of this application is shown. Detailed Implementation
[0031] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.
[0032] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0033] This application provides a reliability assessment method for a vehicle-in-the-loop (VIN) simulation testing system. It constructs a multi-layered reliability evaluation index framework, quantifying the core influencing factors of system reliability from multiple dimensions at each subsystem level. Then, it employs a subjective-objective fusion weighting method to determine index weights, balancing expert experience with data objectivity and effectively avoiding the limitations of relying solely on subjective judgment or single data-driven approaches. Finally, a weighted aggregation method is used to fuse the complex multi-dimensional evaluation results into a single reliability value, mapping it to an intuitive reliability semantic level, thereby clarifying the evaluation conclusions. The assessment method proposed in this application can scientifically quantify and evaluate the reliability of a VIN simulation testing system, filling a research gap in reliability assessment methods in this field.
[0034] The reliability evaluation method of the on-site vehicle-in-the-loop simulation test system provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0035] This application provides a reliability evaluation method for a site-based vehicle-in-the-loop simulation test system, such as... Figure 1 As shown, the method includes: Step 101: Determine the quantifiable metrics of credibility from multiple dimensions at each subsystem level of the on-site vehicle-in-the-loop simulation test system.
[0036] In this step, a multi-layered credibility evaluation index framework is constructed, which quantifies the core influencing factors of system credibility from multiple dimensions at each subsystem level, namely, credibility quantification indicators.
[0037] The on-site vehicle-in-the-loop simulation test system comprises four components: a virtual scene simulation subsystem, a scene data injection subsystem, an ADAS / AD algorithm hardware and software platform and real vehicle subsystem, and a real vehicle motion state measurement subsystem. Around these subsystems and the entire test system, four subsystem levels are constructed for virtual scene simulation, scene data injection, ADAS / AD algorithm hardware and software platform and real vehicle, and real vehicle motion state measurement, respectively.
[0038] In one embodiment of this application, step 101 includes: Sub-step 1.1: Construct a reliable metric for the site vehicle-in-the-loop simulation test system for the virtual scene simulation subsystem layer, which is also the first reliable metric.
[0039] The reliability metrics of the on-site vehicle-in-the-loop simulation test system for the virtual scene simulation subsystem layer include static scene element simulation consistency evaluation index, dynamic scene element simulation consistency evaluation index, virtual camera detection position error evaluation index, and virtual radar ranging error evaluation index.
[0040] The static scene element simulation consistency evaluation index I1 represents the consistency between the static scene elements generated by the virtual scene simulation subsystem and the static scene elements in the real world. The calculation formula is shown in Equation (1): (1) In equation (1), This indicates the consistency of static scene element simulation. , Represents static scene elements in a simulation scenario. This represents a mapping function based on expert prior experience, which subjectively quantifies the simulation consistency of static scene elements into scalar values within the interval [0,1], and the values are positively correlated with the consistency level.
[0041] The dynamic scene element simulation consistency evaluation index I2 represents the consistency between the dynamic scene elements generated by the virtual scene simulation subsystem and the dynamic scene elements in the real world. The calculation formula is shown in equation (2): (2) In equation (2), This indicates the consistency of simulation for dynamic scene elements. Represents dynamic scene elements in a simulation scenario. This represents a mapping function based on expert prior experience, which subjectively quantifies the simulation consistency of dynamic scene elements into scalar values within the interval [0,1], and the values are positively correlated with the consistency level.
[0042] The virtual camera detection position error evaluation index I3 represents the deviation between the center position of the target object detected by the virtual camera in the virtual scene simulation subsystem and its actual center position. The calculation formula is shown in equation (3): (3) In equation (3), This indicates the positional error detected by the virtual camera. This represents the coordinates of the actual position of the target object's center projected onto the image plane. This indicates the coordinates of the center position of the target object as perceived by the virtual camera.
[0043] The virtual radar ranging error evaluation index I4 represents the difference between the measured distance and the actual distance of the virtual radar in the virtual sensor simulation subsystem. The calculation formula is shown in equation (4): (4) In equation (4), Indicates the virtual radar ranging error. This represents the actual distance value. This represents the distance value measured by the virtual radar.
[0044] Sub-step 1.2: Construct a reliable metric for the site vehicle-in-the-loop simulation test system oriented towards the scenario data injection subsystem layer, which is also the second reliable metric.
[0045] The reliability evaluation indicators of the site vehicle-in-the-loop simulation test system of the scenario-oriented data injection subsystem layer include bus message packet loss rate evaluation indicator, message data accuracy evaluation indicator, and data injection synchronization accuracy evaluation indicator.
[0046] The bus message packet loss rate evaluation index I5 represents the proportion of messages that the scenario data injection subsystem failed to inject into the actual vehicle bus during the data injection process. The calculation formula is shown in equation (5): (5) In equation (5), Indicates the bus message packet loss rate. This indicates the number of packets lost during the data injection process. This indicates the total number of messages.
[0047] The message data accuracy evaluation index I6 represents the consistency ratio between the message data injected by the scene data injection subsystem and the virtual sensor detection data during the data injection process. The calculation formula is shown in equation (6): (6) In equation (6), Indicates the accuracy of message data. This indicates the number of packets whose injected message data matches the virtual sensor detection data. This indicates the total number of messages.
[0048] The data injection synchronization accuracy evaluation index I7 represents the synchronization error between the timestamp of the injected message and the corresponding time in the virtual scene. The calculation formula is shown in equation (7): (7) In equation (7), Indicates the precision of data injection synchronization. This represents the actual synchronization error, which is the difference between the message timestamp and the virtual scene time. This indicates the maximum permissible synchronization error.
[0049] Sub-step 1.3: Construct a reliable metric for the field vehicle-in-the-loop simulation test system for ADAS / AD algorithm software and hardware platform and real vehicle subsystem layer, also known as the third reliable metric.
[0050] The reliability evaluation index of the on-site vehicle-in-the-loop simulation test system for ADAS / AD algorithm hardware and software platforms and actual vehicle subsystems includes the control decision response delay compliance rate evaluation index.
[0051] The control decision response delay compliance rate evaluation index I8 represents the proportion of controllers whose delay from receiving sensor data to outputting control commands is within the allowable range. The calculation formula is shown in equation (8): (8) In equation (8), This indicates the rate of achieving the target for control decision response delay. This indicates the number of decisions with a response latency of less than 30 ms. Indicates the total number of decisions.
[0052] Sub-step 1.4: Construct a reliable metric for the site vehicle-in-the-loop simulation test system oriented towards the actual vehicle motion state measurement subsystem layer, which is also the fourth reliable metric.
[0053] The reliability evaluation indicators of the site-based vehicle-in-the-loop simulation test system for the actual vehicle motion state measurement subsystem layer include longitudinal velocity correlation evaluation indicators, lateral velocity correlation coefficient evaluation indicators, vehicle position error evaluation indicators, and vehicle attitude error evaluation indicators.
[0054] The longitudinal velocity correlation evaluation index I9 represents the consistency between the longitudinal velocity output by the simulation model and the longitudinal velocity of the actual vehicle. The calculation formula is as follows: As shown: (9) Mode middle, Represents the longitudinal velocity correlation coefficient. This represents the longitudinal velocity of the vehicle output by the i-th simulation model. This represents the longitudinal speed of the i-th actual vehicle. This represents the average longitudinal velocity of the vehicle output by the simulation model. This represents the average longitudinal speed of the actual vehicle. This represents the number of longitudinal velocities, i=1,2,..., .
[0055] Lateral velocity correlation coefficient evaluation index I 10 This indicates the consistency between the trend of the vehicle's lateral velocity output by the simulation model and the actual vehicle's lateral velocity. The calculation formula is shown in equation (10): (10) In equation (10), This represents the correlation coefficient of lateral velocity. This represents the lateral velocity of the vehicle output by the i-th simulation model. This represents the lateral velocity of the i-th actual vehicle. This represents the average lateral velocity of the vehicle output by the simulation model. This represents the average lateral speed of the actual vehicle. This represents the number of lateral velocities, i=1,2,..., .
[0056] Vehicle position error evaluation index I 11 , representing the deviation between the vehicle position output by the simulation model and the actual vehicle position, is calculated using the formula shown in equation (11): (11) In equation (11), Indicates the vehicle position error. This represents the distance between the actual vehicle's coordinates and the origin of the real-world coordinate system. This represents the distance between the vehicle coordinate points output by the simulation model and the origin of the virtual scene coordinate system.
[0057] Vehicle attitude error evaluation index I 12 , representing the deviation between the vehicle yaw angle output by the simulation model and the actual yaw angle of the real vehicle, is calculated using the formula shown in equation (12): (12) In equation (12), This indicates the vehicle's attitude error. Indicates the actual vehicle yaw angle. This represents the vehicle yaw angle output by the simulation model.
[0058] Step 102: Calculate the fusion weight of the credibility quantification index based on the subjective and objective fusion weighting method.
[0059] In one embodiment of this application, step 102 includes: Sub-step 2.1: Calculate the subjective weights of the indicators based on the subjective weighting method. First, based on prior experience, the indicators The indicators are sorted from highest to lowest importance to obtain the order of importance, as shown in formula (13): (13) In equation (13), The first ordinal indicator indicates the most important indicator. The second ordinal indicator indicates the next most important indicator. The 12th ordinal indicator represents the indicator with the lowest importance. It is the kth ordinal indicator after the indicators are sorted by importance, where k represents the importance index, k=1,2,...,12.
[0060] Secondly, determine the relative importance of adjacent order indicators in importance order relations.
[0061] Adjacent order indices in importance order relations and The ratio of importance is denoted as Used to indicate relative importance , Based on expert prior experience Assignment, The larger the value, the more likely it is to represent The greater the importance than .
[0062] Then, the ordinal weight of the least important ordinal indicator is calculated according to formula (14): (14) In equation (14), The ordinal indicator weight represents the ordinal indicator with the lowest importance.
[0063] Furthermore, the weights of the other ordinal indicators are calculated in reverse order, as shown in formula (15): (15) In equation (15), This represents the weight of the k-th ordered indicator in the importance order relationship.
[0064] Finally, based on the mapping relationship between the indicators and the ordinal indicators ranked by importance, the weights of the ordinal indicators are determined. Reverting to subjective weights of indicators ,in, Let g be the subjective weight of the indicator, where g = 1, 2, ..., 12.
[0065] Sub-step 2.2: Calculate the objective weights of the indicators based on the objective weighting method. First, an indicator data matrix is constructed based on historical test data of m samples, as shown in formula (16); (16) In equation (16), X represents the index data matrix. Let represent the value of the j-th confidence metric for the i-th sample, where i = 1, 2, ..., m, j = 1, 2, ..., 12.
[0066] Furthermore, based on the indicator attributes, for Dimensionless processing is performed. Among them, I1, I2, I6, I7, I8, I9, I... 10 These are positive indicators: I3, I4, I5, I 11 I 12 It belongs to the negative index, and the dimensionless processing formula is shown in equation (17): (17) In equation (17), Indicator data matrix The value after dimensionless processing.
[0067] Then, the correlation coefficient of information overlap, the degree of information difference, and the information content parameter are calculated among the credibility metrics.
[0068] The formula for calculating the correlation coefficient of information overlap between the credibility metrics is shown in Equation (18). The larger the absolute value of the correlation coefficient, the higher the information overlap between the two metrics.
[0069] (18) In equation (18), Let represent the correlation coefficient indicating the information overlap between the j-th credibility metric and the h-th credibility metric. This represents the dimensionless mean of the j-th credibility metric. This represents the dimensionless mean of the h-th confidence metric. This represents the dimensionless value of the i-th historical test data for the j-th credibility metric. This represents the dimensionless value of the i-th historical test data for the h-th confidence metric.
[0070] The formula for calculating the degree of information difference between the credibility metrics is shown in equation (19): (19) In equation (19), This represents the sum of the information differences between the j-th indicator and the other 11 indicators.
[0071] The formula for calculating the information content parameter between the credibility metrics is shown in equation (20): (20) In equation (20), This represents the sum of information content of the j-th indicator.
[0072] Finally, the objective weight of the indicator is calculated based on the information content parameter of the credibility quantification indicator. The more effective information an indicator contains, the higher its weight should be. The objective weight of the indicator can be obtained by normalizing the information content, as shown in formula (21). (twenty one) In equation (21), This represents the objective weight of the j-th credibility metric.
[0073] Sub-step 2.3: The subjective weights and objective weights are weighted and fused together. The weighted fusion formula is shown in equation (22): (twenty two) In equation (22), This represents the fusion weight of the j-th credibility metric. and Let represent the allocation coefficients of objective weight and subjective weight, respectively, and the calculation formula for the allocation coefficient is shown in equation (23): (twenty three) After calculating the fusion weights of each credibility metric, the range method is used to transform the values of each fusion weight to the [0,1] interval. The transformed values are the final fusion weights of the metrics. ,and .
[0074] Step 103: Calculate the credibility quantification index and fusion weight through weighted aggregation to obtain the credibility value of the whole vehicle in-loop simulation test system, and map the credibility value to the credibility semantic level.
[0075] In one embodiment of this application, step 103 includes: First, the raw measurement data of each collected reliable quantitative indicator are normalized to obtain normalized data. , Indicators The normalized values of the original measurement data.
[0076] Then, the confidence value is calculated using an aggregation algorithm, as shown in equation (24): (twenty four) In equation (24), D represents the confidence value of the on-site vehicle-in-the-loop simulation system. This represents the normalized value of the j-th credibility metric. Let be the fusion weight corresponding to the j-th credibility metric.
[0077] Finally, the credibility semantic level of the vehicle-in-the-loop simulation test system is evaluated. Based on the mapping relationship between the credibility value and the credibility semantic level of the vehicle-in-the-loop simulation test system in Table 1, the credibility semantic level of the vehicle-in-the-loop simulation system is evaluated.
[0078] Table 1. Mapping Relationship between Credibility Score and Credibility Level
[0079] This application addresses the lack of a credibility assessment scheme for vehicle-in-the-loop (VIN) simulation testing systems at test sites by proposing a credibility assessment method for such systems. The method first constructs a multi-layered credibility evaluation index framework, quantifying the core influencing factors of system credibility from multiple dimensions at each subsystem level. Then, it employs a subjective-objective fusion weighting method to determine index weights, fully integrating domain expert experience while accurately reflecting the objective information inherent in the data, effectively overcoming the bias caused by purely subjective judgments or single data-driven approaches. Finally, based on a weighted aggregation method, the complex multi-index evaluation results are aggregated into a single credibility score, which is further mapped to an intuitive credibility semantic level, thus clarifying the evaluation conclusions.
[0080] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0081] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A reliability evaluation method for a vehicle-in-the-loop simulation test system, characterized in that, include: From the perspective of each subsystem level of the on-site vehicle-in-the-loop simulation test system, multiple dimensions are used to determine the quantifiable indicators of credibility. Based on the weighting method that integrates subjective and objective factors, the fusion weights of the credibility quantification indicators are calculated. By using a weighted aggregation method, the credibility quantification index and fusion weight are calculated to obtain the credibility value of the whole vehicle in-loop simulation test system in the field, and the credibility value is mapped to the credibility semantic level.
2. The method according to claim 1, characterized in that, The on-site vehicle-in-the-loop simulation test system includes a virtual scene simulation subsystem, a scene data injection subsystem, an ADAS / AD algorithm hardware and software platform and a real vehicle subsystem, as well as a real vehicle motion state measurement subsystem; The determination of reliability metrics from multiple dimensions at each subsystem level of the on-site vehicle-in-the-loop simulation test system includes: A first credibility metric is constructed for the virtual scene simulation subsystem. The first credibility metric includes a static scene element simulation consistency evaluation index, a dynamic scene element simulation consistency evaluation index, a virtual camera detection position error evaluation index, and a virtual radar ranging error evaluation index. A second reliable metric is constructed for the data injection subsystem of the aforementioned scenario. The second reliable metric includes a bus message packet loss rate evaluation metric, a message data accuracy evaluation metric, and a data injection synchronization accuracy evaluation metric. A third reliable metric is constructed for the ADAS / AD algorithm hardware and software platform and the actual vehicle subsystem. The third reliable metric includes the control decision response delay compliance rate evaluation index. A fourth reliable quantitative index is constructed for the actual vehicle motion state measurement subsystem. The fourth reliable quantitative index includes a longitudinal velocity correlation evaluation index, a lateral velocity correlation coefficient evaluation index, a vehicle position error evaluation index, and a vehicle attitude error evaluation index.
3. The method according to claim 1, characterized in that, The weighting method based on the fusion of subjective and objective factors calculates the fusion weights of the credibility quantification indicators, including: The subjective weights of each credibility quantification index are calculated based on the subjective weighting method, and the objective weights of each credibility quantification index are calculated based on the objective weighting method. The subjective weights and objective weights of the indicators are weighted and fused to obtain the fusion weights of the corresponding credible quantification indicators. The process involves calculating a credibility metric and fusion weights using a weighted aggregation method to obtain the credibility value of the on-site vehicle-in-the-loop simulation test system, and mapping the credibility value to a credibility semantic level, including: The reliability metrics are normalized to obtain normalized data. , Indicates a reliable metric The normalized value; The reliability value of the on-site vehicle-in-the-loop simulation test system is calculated using an aggregation algorithm; the formula for calculating the reliability value is as follows: Where D represents the confidence value of the on-site vehicle-in-the-loop simulation test system. This represents the normalized value of the j-th credibility metric. Let be the fusion weight corresponding to the j-th credibility metric.
4. The method according to claim 3, characterized in that, The subjective weights of each credibility quantification index calculated based on the subjective weighting method include: Based on prior experience, various credibility metrics are quantified. The indicators are sorted from highest to lowest importance to obtain the importance order relationship, which is as follows: in, The ordinal indicator represents the highest level of importance. The next most important indicator is indicated by its order of importance, and so on. The ordinal indicator representing the least important value. It is the k-th ordinal indicator after the indicators are sorted by importance, where k represents the importance ordinal number, k=1,2,...,12; Determine the relative importance of adjacent ordinal indicators in the importance ranking relationship, and calculate the ordinal indicator weight of the least important ordinal indicator; where the relative importance of adjacent ordinal indicators is the ratio of their importance, and the formula for calculating the ordinal indicator weight of the least important ordinal indicator is: in, Indicator of adjacent order and The ratio of importance The ordinal indicator weight represents the ordinal indicator with the lowest importance. Following the reverse order of indicator importance, and based on the relative importance of adjacent indicators, the weights of other indicators are calculated sequentially. The formula for calculating the weights of other indicators is as follows: in, This represents the weight of the k-th ordered indicator in the importance order relationship. This represents the weight of the (k+1)th ordered indicator in the importance order relationship. Based on the mapping relationship between credible quantification metrics and ordinal metrics in the importance ranking relationship, the weights of ordinal metrics are determined. This is reduced to the subjective weights of each reliable metric. ,in, Let g be the subjective weight of the g-th credibility quantification index, where g = 1, 2, ..., 12. The objective weights of each credible quantification index calculated based on the objective weighting method include: An indicator data matrix is constructed based on historical test data from m samples; the indicator data matrix is as follows: Where X represents the indicator data matrix, Let represent the value of the j-th confidence metric for the i-th sample, where i = 1, 2, ..., m, j = 1, 2, ..., 12; Based on indicator attributes Dimensionless processing is performed, where I1, I2, I6, I7, I8, I9, I 10 These are positive indicators: I3, I4, I5, I 11 I 12 It belongs to the negative index; the formula for dimensionless processing is: in, Indicator data matrix The value after dimensionless processing; Calculate the correlation coefficient of information overlap, the degree of information difference, and the information content parameter among the credibility metrics; the formula for calculating the correlation coefficient of information overlap is: in, Let represent the correlation coefficient between the j-th credibility metric and the h-th credibility metric. This represents the dimensionless mean of the j-th indicator. This represents the dimensionless mean of the h-th indicator. This represents the dimensionless value of the i-th sample of the h-th confidence metric; The formula for calculating the degree of information discrepancy is: in, This represents the sum of the information differences between the j-th credibility metric and the other 11 metrics; The formula for calculating the information content parameter is: in, This represents the sum of information content of the j-th credibility metric; The objective weights of the credibility quantification index are calculated based on the information content parameter; the formula for calculating the objective weights is: in, This represents the objective weight of the j-th credibility metric.
5. The method according to claim 4, characterized in that, The step of weighting and fusing the subjective weights and objective weights of the indicators to obtain the fusion weights of the corresponding credible quantification indicators includes: The subjective weights and objective weights of the indicators are weighted and fused to obtain the fused weight of the corresponding credibility quantification indicator; the formula for weighted fusion is: in, This represents the fusion weight of the j-th credibility metric. This represents the objective weight of the j-th credibility metric. This represents the subjective weight of the j-th credibility metric. and These represent the allocation coefficients for objective weights and subjective weights, respectively. After calculating the fusion weights of each credibility quantification index, the range method is used to transform the values of each fusion weight to the [0,1] interval to obtain the transformed fusion weights.