Performance evaluation method, device and equipment of drive-by-wire chassis and storage medium

Abstract

The application relates to the field of automatic driving, and proposes a performance evaluation method, device and equipment of a drive-by-wire chassis and a storage medium. The method comprises the following steps: collecting test data under a predetermined test scene; determining key index data of the drive-by-wire chassis under the set test scene according to the collected test data; and evaluating the performance of the drive-by-wire chassis through the key index data according to a pre-set performance evaluation model. Compared with a subjective evaluation method, the performance evaluation method greatly reduces the influence of human subjective errors, makes the evaluated performance of the drive-by-wire chassis more objective, reduces the probability of problems occurring during automatic driving control, and improves the safety of an automatic driving vehicle.

Description

Technical Field

[0001] This application belongs to the field of autonomous driving, and in particular relates to a method, apparatus, device and storage medium for evaluating the performance of a drive-by-wire chassis. Background Technology

[0002] Drive-by-wire chassis are crucial in the current deployment of autonomous driving, and their performance directly impacts the deployment process. Having a high-performance drive-by-wire chassis is essential for achieving high-quality autonomous driving.

[0003] Currently, there are no complete acceptance standards for drive-by-wire chassis in the automotive industry, and even factory standards are lacking. Autonomous driving implementation departments often only conduct functional acceptance tests on drive-by-wire chassis, unable to determine their performance in achieving autonomous driving on that chassis. Furthermore, subjective evaluations of drive-by-wire chassis are not applicable to autonomous driving control. Often, even when subjective feelings are positive, using autonomous driving control can lead to unforeseen control problems, potentially causing financial losses to passengers or even endangering their lives. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, apparatus, device, and storage medium for evaluating the performance of a drive-by-wire chassis, in order to solve the problem that inaccurate subjective evaluations of drive-by-wire chassis in the prior art may cause economic losses to passengers of autonomous vehicles, or even endanger their lives.

[0005] A first aspect of this application provides a performance evaluation method for a drive-by-wire chassis, the method comprising:

[0006] Collect test data under predetermined test scenarios;

[0007] Based on the collected test data, the key performance indicators of the drive-by-wire chassis under the set test scenario are determined.

[0008] The performance of the drive-by-wire chassis is evaluated based on the key indicator data according to the pre-set performance evaluation model.

[0009] In conjunction with the first aspect, in a first possible implementation of the first aspect, the performance of the drive-by-wire chassis is evaluated using the key indicator data based on a pre-defined performance evaluation model, including:

[0010] Obtain the estimated bias of the pre-trained machine learning model under the test scenario;

[0011] When the estimated deviation is less than a preset deviation threshold, the performance of the drive-by-wire chassis is evaluated using the machine learning model.

[0012] When the estimated deviation is greater than or equal to a preset deviation threshold, the performance of the drive-by-wire chassis is evaluated using a pre-set hierarchical analysis model.

[0013] In conjunction with the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the machine learning model includes a scoring and evaluation model, a man-hour evaluation model for autonomous driving deployment, and a difficulty grading model for autonomous driving deployment. When the estimated deviation is less than a preset deviation threshold, the performance of the drive-by-wire chassis is evaluated through the machine learning model, including:

[0014] The key indicator data are input into the scoring and evaluation model to evaluate the rating of the drive-by-wire chassis.

[0015] The key indicator data are input into the evaluation model for the time required for autonomous driving deployment to evaluate the time required for autonomous driving deployment of the drive-by-wire chassis.

[0016] The key indicator data are input into the autonomous driving deployment difficulty grading model to evaluate the autonomous driving deployment difficulty level of the drive-by-wire chassis.

[0017] In conjunction with the first possible implementation of the first aspect, in the third possible implementation of the first aspect, the performance of the drive-by-wire chassis is evaluated through a pre-defined hierarchical analysis model, including:

[0018] The score corresponding to the key indicator data is determined based on the pre-defined correspondence between the range of key indicator data and the score.

[0019] The evaluation score of the drive-by-wire chassis is determined based on the determined score and the corresponding weight.

[0020] The required man-hours for autonomous driving deployment of the drive-by-wire chassis and the level of difficulty in autonomous driving deployment are determined based on the evaluation score of the drive-by-wire chassis.

[0021] In conjunction with the first aspect, in the fourth possible implementation of the first aspect, the test scenario includes one or more of the following: driving test scenario, braking test scenario, and steering test scenario;

[0022] When the predetermined test scenario is a driver test scenario, the key indicator data includes one or more of the following: driver execution curve similarity, maximum instruction response time, and average instruction response time.

[0023] When the predetermined test scenario is a braking test scenario, the key indicator data includes one or more of the following: maximum difference in average deceleration, average difference between average deceleration and actual command issuance, maximum time-related deceleration difference, maximum speed-related deceleration difference, maximum difference in braking distance, similarity of braking execution curve, maximum braking response time, average braking response time, maximum braking execution time, and average braking execution time.

[0024] When the predetermined test scenario is a steering test scenario, the key indicator data includes one or more of the following: maximum step steady-state error, average step steady-state error, maximum step overshoot, average step overshoot, maximum step response time, average step response time, maximum step execution time, average step execution time, maximum response time of sine test, average response time of sine test, maximum execution error of sine test, average execution error of sine test, and steering execution curve similarity.

[0025] In conjunction with the first aspect, in the fifth possible implementation of the first aspect, before collecting test data in the predetermined test scenario, the method further includes:

[0026] If it is determined that there is no manual intervention in the vehicle's control mechanism, then the vehicle will enter the predetermined test scenario according to the control instructions of the test scenario.

[0027] In conjunction with the first aspect, in the sixth possible implementation of the first aspect, test data is collected under a predetermined test scenario, including:

[0028] The collected scene segment test data is determined based on the message identifier of the scene segment test data included in the collected data.

[0029] The scenario type of the scenario segment test data is determined based on the message identifier of the scenario number included in the collected data.

[0030] A second aspect of this application provides a performance evaluation device for a drive-by-wire chassis, the device comprising:

[0031] The test data acquisition unit is used to collect test data under a predetermined test scenario.

[0032] The key indicator data determination unit is used to determine the key indicator data of the drive-by-wire chassis under the set test scenario based on the collected test data.

[0033] The performance evaluation unit is used to evaluate the performance of the drive-by-wire chassis based on the key indicator data according to a pre-set performance evaluation model.

[0034] A third aspect of this application provides a performance evaluation device for a drive-by-wire chassis, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in any of the first aspects.

[0035] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any of the first aspects.

[0036] The beneficial effects of this application embodiment compared with the prior art are as follows: This application collects test data in the evaluation test scenario, determines the key indicator data of the drive-by-wire chassis in the set test scenario based on the test data, and evaluates the performance of the drive-by-wire chassis through the key indicator data according to the performance evaluation model set in the evaluation. Compared with subjective evaluation methods, the performance evaluation method described in this application greatly reduces the influence of human subjective error, makes the evaluated performance of the drive-by-wire chassis more objective, reduces the probability of problems occurring during autonomous driving control, and improves the safety of autonomous vehicles. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram illustrating the implementation process of a performance evaluation method for a drive-by-wire chassis provided in an embodiment of this application;

[0039] Figure 2 This is a schematic diagram illustrating the implementation process of test data acquisition provided in an embodiment of this application;

[0040] Figure 3 This is a schematic diagram illustrating the implementation process of obtaining key indicator data provided in an embodiment of this application;

[0041] Figure 4 This is a schematic diagram illustrating the implementation process of key indicator data processing provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of a performance evaluation device for a drive-by-wire chassis provided in an embodiment of this application;

[0043] Figure 6 This is a schematic diagram of a performance evaluation device for a wire-controlled chassis provided in an embodiment of this application. Detailed Implementation

[0044] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0045] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0046] Figure 1 A schematic diagram illustrating the implementation process of a performance evaluation method for a drive-by-wire chassis provided in this application embodiment is described in detail below:

[0047] In S101, test data is collected under the predetermined test scenario.

[0048] The test scenarios in this application embodiment may include driving test scenarios, braking test scenarios and steering test scenarios. The steering test scenarios may be further subdivided into steering step test scenarios, steering sine test scenarios and steering center test scenarios.

[0049] Specifically, the drive test scenario can be used to test the vehicle's drive performance, examining the accuracy and consistency of drive execution, as well as the maximum speed the vehicle can reach under low drive torque. Different amounts of drive torque (0%-100%) can be applied to accelerate the vehicle to a set maximum value before stopping. The acquired signals include vehicle speed and torque execution feedback.

[0050] Braking test scenarios can be used to measure a vehicle's braking performance, examining braking accuracy, consistency, and minimum braking duration. During a braking test, the vehicle is first accelerated to a specified speed with a set throttle opening, then different braking openings (0%-100%) are applied until the vehicle comes to a stop. Signals acquired during the test include vehicle speed and longitudinal acceleration feedback signals.

[0051] The steering step test scenario can test a vehicle's steering performance, primarily examining steering accuracy, consistency, response time, and overshoot. During the steering step test, the vehicle is stationary and steers from the center position to the left and right at its rated steering angular velocity, respectively, by 0%-100% of its maximum steering angle, maintaining this position for a period of time before returning to the center position. Signals acquired during the test include the actual steering angle feedback value and the actual steering angular velocity feedback value.

[0052] The steering sine wave test scenario can test a vehicle's steering performance, primarily examining steering accuracy, consistency, response time, and overshoot. During the test, the vehicle, while stationary, oscillates sinusoidally from the center position with different periods at various amplitudes (0%-100% of the maximum steering angle) using the rated steering angular velocity, finally returning to the center position. Signals acquired during the test include actual steering angle feedback values ​​and actual steering angular velocity feedback values.

[0053] The steering centering test scenario can test the vehicle's steering zero-point offset. The vehicle's steering angle can be controlled to the center position, and the vehicle travels a certain distance forward from a standstill. Signals to be acquired during the test include the vehicle's lateral offset, etc.

[0054] Before collecting the test data, this application may include a test mode determination process. The test data collection process may also include checks on the validity and integrity of the test data, thereby increasing the reliability of the acquired test data. The specific data collection process is as follows: Figure 2 As shown, the details are as follows:

[0055] In S010, it is determined whether the conditions for autonomous driving access are met.

[0056] The requirement for autonomous driving access is used to test the driver's ability to take over the tested drive-by-wire chassis. It can detect whether any takeover behavior exists in various vehicle control mechanisms (including pedals and steering wheel), such as the accelerator pedal having travel, the brake pedal having travel, or the steering wheel torque exceeding the EPS (Electric Power Steering) torque limit. If takeover behavior is found, S011 is executed to check and repair the access conditions and related communications. If repair is not possible, the current test can be exited, or it can be determined that entry into the test scenario is not possible.

[0057] If no takeover attempt is detected, proceed to S020 to switch to autonomous driving mode. In other words, by determining the driving mode, the test vehicle can automatically switch to autonomous driving mode to facilitate the collection of test data.

[0058] In autonomous driving mode, step S030 can be executed to select a test scenario. The test scenarios in this embodiment may include multiple scenarios, such as braking test scenarios, driving test scenarios, and steering test scenarios. The system can be programmed to automatically select test scenarios in a predetermined order, or it can receive test scenarios selected by the tester.

[0059] Depending on the selected test scenario, the vehicle can enter either the S040 driving test scenario, the S041 braking test scenario, or the S042 steering test scenario. In each scenario, it executes the corresponding scenario commands sent by the device command sending module. For example, in the driving test scenario, different amounts (0%-100%) of driving torque are applied to accelerate the vehicle to a set maximum value before stopping. The vehicle performs corresponding actions based on the scenario commands, including driving, braking, or steering. During the execution of these scenario commands, the system records real-time timing data. During the data collection process, manual intervention is detected; specifically, in S050, it is checked whether manual intervention occurs during the test. If manual intervention occurs, the system returns to S020 and switches to autonomous driving mode. If no manual intervention occurs, the system proceeds to S060 to determine if data collection for all three scenario types has been completed. If not, the system returns to S030 to select another test scenario; if completed, the process ends.

[0060] In S102, based on the collected test data, the key indicator data of the drive-by-wire chassis under the set test scenario are determined.

[0061] In this embodiment, key indicator data for different test scenarios can be calculated based on the test data. To improve the calculation efficiency of key indicator data, the validity of the data can be judged before calculating the key indicator data, so that valid test data can be extracted for the calculation of key indicator data. The key indicator data extraction process from the test data is as follows: Figure 3 As shown, it includes:

[0062] In S070, the scene data is initially extracted.

[0063] Specifically, when the device command sending module sends a scene command corresponding to a fixed scene, it can simultaneously issue two message identifiers, one to record the scene segment test data and the other to record the scene number of the scene segment test data. From the collected time-series data, the scene segment test data can be initially extracted from the time-series data based on the message identifiers of the scene segment test data.

[0064] In S080, the scene to which the data belongs is determined based on the scene number.

[0065] The test scenario corresponding to the test data of the scenario segment can be determined from the message identifier of the scenario number.

[0066] Based on the scenario number, the test data of the scenario segment may belong to the S090 driving test scenario, the S091 braking test scenario, or the S092 steering test scenario.

[0067] After determining the test scenario to which the scene segment test data belongs, the validity of the scene segment test data can be determined based on the judgment indicators corresponding to the test scenario. The validity of the curve can be judged by comparing the data curve characteristics corresponding to the test scenario with the curve characteristics of the scene segment test data, including the actual length of the curve and whether there is manual intervention during the time period of the curve. This determines whether the scene segment test data is valid, and invalid test data is discarded in S101.

[0068] In S110, key indicator data is extracted, and in S111, the key indicator data is archived and saved.

[0069] The key performance indicators (KPIs) differ across different testing scenarios. For driver-driven testing scenarios, the corresponding KPIs may include:

[0070] (1) Driving execution curve similarity (overall differences between curves).

[0071] Calculate the RMSE (Root-Mean-Square Error) between speed curves or torque feedback curves during the drive process, and normalize the RMSE using the value issued by the current command to calculate the percentage. The formula can be as follows:

[0072]

[0073]

[0074] Among them, y ji Let y be the i-th value in the j-th curve. ki Let y be the i-th value in the k-th curve. aim The target value issued in the current test is n, the total amount of data being compared is n, and NRMSE is the curve similarity index.

[0075] (2) Maximum response time of instructions.

[0076] The maximum response time for the instruction can be calculated as the time difference between the instruction being issued and its actual execution, extracted from all repeated data, and the maximum value is determined. The formula is as follows:

[0077] T RMaxi =T Ai -T Si T RMax =max(T) RMaxi ), where T RMaxi T represents the response time of the current data segment. Si The timestamp for issuing the instruction for the current data segment, T Ai T is the actual execution timestamp of the current data segment.RMax The maximum response time across all duplicate data.

[0078] (3) Average response time of instructions.

[0079] The average response time of the instruction is calculated by averaging all repeated response time data. The formula is as follows:

[0080] T RMean =mean(T) RMaxi (i = 1, 2, 3, ..., n), where T RMean This represents the average response time across all duplicate data.

[0081] In braking test scenarios, the key performance indicators (KPIs) may include:

[0082] (1) The maximum difference in average deceleration.

[0083] The maximum difference in average deceleration can be calculated by taking the average of the vehicle's actual braking deceleration after each repeated data point is issued, finding the maximum difference among these values, and then comparing this difference (in percentage form) with the current command value. The formula is as follows:

[0084] Among them, A avg A is the set of the mean braking decelerations of all repeated data. aim Decelerate to the target.

[0085] (2) The average difference between the average deceleration and the actual command issued.

[0086] The average difference between the average deceleration and the actual braking deceleration after the braking command is issued can be calculated by taking the average of the actual braking deceleration of the vehicle after the braking command is issued in each repeated data point, and then calculating the average of this value across all repeated data points. The difference (in percentage form) is then compared to the current command value. The formula is as follows:

[0087]

[0088] (3) Time-dependent maximum deceleration difference.

[0089] The time-related maximum deceleration difference can be calculated by taking the time-series-based deceleration difference between repeated data points at the same starting point, and comparing its maximum value with the current instruction value to calculate the difference (in percentage form). The formula is as follows:

[0090] E i =A ji -A ki , Among them, A ji Let A be the deceleration corresponding to the i-th time point in the j-th repetition of data.ki Let E be the deceleration corresponding to the i-th time point in the k-th repetition of data. i The set of . Where deceleration is the rate of change of velocity in the opposite direction to acceleration.

[0091] (4) The difference in maximum deceleration related to velocity.

[0092] The velocity-related maximum deceleration difference can be calculated based on the correspondence between velocity and deceleration in each repeated data point, obtained from time-series data. The velocity-based deceleration difference between each repeated data point is then calculated, and its maximum value is determined. The formula is as follows:

[0093] E i =A ji -A ki , Among them, A ji Let A be the deceleration corresponding to the i-th velocity point in the j-th repetition of data. ki Let E be the deceleration corresponding to the i-th velocity point in the k-th repetition of data. i A set of.

[0094] (5) Maximum difference in braking distance.

[0095] The maximum difference in braking distance can be calculated as the maximum difference between the actual braking distances of each repeated data point. The formula is as follows:

[0096] Where D is the set of braking distances for each repeated data point.

[0097] (6) Similarity of braking execution curves.

[0098] The similarity of the braking execution curves can be calculated by determining the RMSE (Root-Mean-Square Error) between the curves, normalizing the RMSE using the value issued by the current command, and then calculating the percentage. The formula is as follows:

[0099] Among them, y ji Let y be the i-th value in the j-th curve. ki Let y be the i-th value in the k-th curve. aim The target value issued in the current test is n, the total amount of data being compared is n, and NRMSE is the curve similarity index.

[0100] (7) Maximum braking response time.

[0101] The maximum braking response time can be calculated by taking the time difference between the issuance of the instruction and its actual execution, and extracting the maximum value from all repeated data. The formula is as follows:

[0102] T RMaxi =T Ai -T Si T RMax =max(T) RMaxi ), where T RMaxi T represents the response time of the current data segment. Si The timestamp for issuing the instruction for the current data segment, T Ai T is the actual execution timestamp of the current data segment. RMax The maximum response time across all duplicate data.

[0103] (8) Braking average response time.

[0104] The average braking response time can be calculated by averaging all repeated response time data. The formula is as follows:

[0105] T RMean =mean(T) RMaxi ), where T RMean This represents the average response time across all duplicate data.

[0106] (9) Maximum braking execution time.

[0107] The maximum braking execution time can be calculated by taking the time difference between the issuance of the instruction and the actual execution of the instruction until 80% of the required braking time, and extracting the maximum value from all repeated data. The formula is as follows:

[0108] T EMaxi =T Ai -T Si T EMax =max(T) EMaxi ), where T EMaxi T represents the execution time of the current data segment. Si The timestamp for issuing the instruction for the current data segment, T Ai T is the timestamp when the actual feedback of the current data segment is executed up to 80% of the demand braking. EMax This represents the average execution time across all duplicate data.

[0109] (10) Average braking execution time.

[0110] The average braking execution time can be calculated by averaging all repeated execution time data. The formula is as follows:

[0111] T EMean =mean(T) EMaxi ), where T EMean This represents the average response time across all duplicate data.

[0112] For steering test scenarios, the key performance indicators (KPIs) may include:

[0113] (1) Maximum steady-state error value of step jump.

[0114] The maximum steady-state error value of the step can be obtained by calculating the steady-state error of each repeated data point and finding the maximum value among the repeated data points.

[0115] (2) Step average steady-state error value.

[0116] The step average steady-state error value can be obtained by calculating the steady-state error of each repeated data point and then averaging the values ​​among the repeated data points.

[0117] (3) Maximum step overshoot.

[0118] The maximum step overshoot can be obtained by calculating the overshoot of each repeated data point and finding the maximum value among the repeated data points.

[0119] (4) Step average overshoot.

[0120] The step average overshoot can be obtained by calculating the overshoot of each repeated data point and then averaging the results among the repeated data points.

[0121] (5) Maximum step response time.

[0122] The maximum step response time can be obtained by calculating the response time of each repeated data point and finding the maximum value among the repeated data points.

[0123] (6) Step average response time.

[0124] The step average response time can be obtained by calculating the response time of each repeated data point and then averaging the response times of each repeated data point.

[0125] (7) Maximum execution time of step jump.

[0126] The maximum execution time of the step jump can be obtained by calculating the execution time of each repeated data and finding the maximum value among the repeated data.

[0127] (8) Average execution time of step jump.

[0128] The average execution time of the step jump can be obtained by calculating the execution time of each repeated data point and then averaging the execution times of each repeated data point.

[0129] (9) Maximum response time of sine wave test.

[0130] The maximum response time of the sine test can be obtained by calculating the response time of each repeated data point and finding the maximum value among the repeated data points.

[0131] (10) Average response time of sine test.

[0132] The average response time of the sinusoidal test can be obtained by calculating the response time of each repeated data point and then averaging the responses of each repeated data point.

[0133] (11) Maximum execution error of sine test.

[0134] The maximum execution error of the sine test is calculated by determining the execution error of each repeated data point and finding the maximum value among the repeated data points.

[0135] (12) Average execution error of sine test.

[0136] The average execution error of the sinusoidal test can be obtained by calculating the execution error of each repeated data and then averaging the results among the repeated data.

[0137] (13) Steering execution curve similarity.

[0138] The similarity of the steering execution curves can be calculated by taking the RMSE (Root-Mean-Square Error) between the curves, normalizing the RMSE using the value of the current command, and then calculating the percentage. The formula is as follows:

[0139] Among them, y ji Let y be the i-th value in the j-th curve. ki Let y be the i-th value in the k-th curve. aim The target value issued in the current test is n, the total amount of data being compared is n, and NRMSE is the curve similarity index.

[0140] Based on the three types of test scenarios mentioned above, key indicator data corresponding to each test scenario can be calculated to obtain a key indicator data table. S121 can be used to determine whether all data extraction has been completed, that is, whether the number of data in the obtained key indicator data table matches the preset number of data. If there is unaccepted data, S120 is used to supplement the scenario data for acceptance, and the process returns to the initial scenario data acceptance step in S070.

[0141] If all key indicator data has been extracted, then the key indicator data extraction is complete.

[0142] In S103, the performance of the drive-by-wire chassis is evaluated using the key indicator data based on a pre-set performance evaluation model.

[0143] Key indicator data for different test scenarios were extracted in S102, and the performance of the drive-by-wire chassis can be evaluated based on the key indicator data according to the pre-set performance evaluation model.

[0144] The performance of the drive-by-wire chassis can include its evaluation score, the man-hours required for autonomous driving deployment, and the difficulty level of autonomous driving deployment.

[0145] Among the possible implementations, it is possible to do the following: Figure 4 As shown, when evaluating the performance of a drive-by-wire chassis using key performance indicators (KPIs), the following can be included:

[0146] In S130, it is determined whether the scoring deviation of the machine learning model is less than a predetermined deviation threshold.

[0147] The scoring deviation can be determined using test case data. A machine learning model can be used to calculate the scores of existing vehicle test cases, which are then compared to the actual scores of the test cases to determine the scoring deviation of the machine learning model, such as less than 5%.

[0148] If the score deviation is less than a predetermined deviation threshold, S141 can be executed to evaluate the performance of the drive-by-wire chassis using a machine learning model.

[0149] The machine learning model may include a scoring and evaluation model, a man-hour evaluation model for autonomous driving deployment, and an autonomous driving deployment difficulty grading model. Through the machine learning model, the key indicator data can be input into the scoring and evaluation model to evaluate the evaluation score of the drive-by-wire chassis; the key indicator data can be input into the autonomous driving deployment man-hour evaluation model to evaluate the man-hours required for autonomous driving deployment of the drive-by-wire chassis; and the key indicator data can be input into the autonomous driving deployment difficulty grading model to evaluate the autonomous driving deployment difficulty level of the drive-by-wire chassis.

[0150] In S183, the current chassis evaluation score, the estimated man-hours required for autonomous driving deployment, and the estimated difficulty level of autonomous driving deployment are obtained.

[0151] The machine learning model in this application embodiment can be continuously iterated and updated as the database is continuously expanded.

[0152] When the scoring deviation of the machine learning model is greater than the preset deviation threshold, the hierarchical analysis model from S140 to S180 is executed to evaluate the performance of the drive-by-wire chassis.

[0153] In S140, the dividing line corresponding to the key indicator data is defined, and the key indicator data is scored.

[0154] For example, a passing score 'a' can be set for each indicator. Within the interval [0, a] between the passing score 'a' and the infinite ideal value 0 (the smaller the indicator, the better), the score is evenly distributed in the interval [50, 100].

[0155] The score is evenly distributed within the range of [a, 5a], which is four times below the passing grade 'a'. If the test value exceeds the range of [0, 5a], the score is 0.

[0156] Using the above strategy for the same indicator can provide an objective horizontal comparison.

[0157] After scoring is completed, a comprehensive evaluation can be performed using the Analytic Hierarchy Process (AHP). In S150, the weights of all indicator items can be derived based on the AHP method.

[0158] The judgment matrix is ​​formed by pairwise comparison and selection to calculate the weights of each level. The weights of each level are then combined to obtain the final index weights. The judgment matrix can be checked for consistency, which can avoid logical problems that may occur in subjective judgment.

[0159] In S160, the current overall chassis score is calculated through weighted averages. This allows the determination of the evaluation score for the drive-by-wire chassis.

[0160] In S170, the current chassis is sorted among all known drive-by-wire chassis.

[0161] Based on the scores of each key indicator and the weights corresponding to those scores, the current ranking of the drive-by-wire chassis among known drive-by-wire chassis can be determined.

[0162] In S180, the estimated man-hours and the estimated difficulty level of autonomous driving deployment are included.

[0163] Based on the man-hours required for autonomous driving deployment and the autonomous driving deployment difficulty level of adjacent known drive-by-wire chassis in the sorting, the man-hours required for autonomous driving deployment and the autonomous driving deployment difficulty level of the current drive-by-wire chassis can be determined.

[0164] The system can execute S190, autonomous driving deployment plan design, S191, autonomous driving deployment cost analysis, and S192, generate a drive-by-wire chassis performance test report on the current chassis data archive.

[0165] The performance evaluation method for the drive-by-wire chassis described in this application can achieve the following effects:

[0166] The performance evaluation method described in this application can effectively evaluate the performance of each drive-by-wire chassis, and based on the accumulation of relevant data on drive-by-wire chassis, it can design relevant models to design autonomous driving deployment plans, analyze autonomous driving deployment costs, etc.

[0167] The performance evaluation method described in this application can propose different system requirements for autonomous driving systems based on the performance classification of drive-by-wire chassis. Different levels of drive-by-wire chassis matched with different levels of autonomous driving systems (here, level refers to the quality of the autonomous driving control system rather than the level of autonomous driving) can achieve the same autonomous driving goal.

[0168] Based on this, the performance evaluation method described in this application can evaluate the performance of autonomous driving systems deployed on different drive-by-wire chassis. With an objective and detailed performance evaluation and score of the drive-by-wire chassis, the influence of the chassis on the autonomous driving performance can be largely shielded, making the evaluation of the autonomous driving system performance more fair.

[0169] Based on the performance evaluation method described in this application, more stringent and specific requirements can be imposed on drive-by-wire chassis suppliers, making calibration and parameter adjustment in the later stages of autonomous driving deployment simpler.

[0170] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0171] Figure 5 This is a schematic diagram of a performance evaluation device for a drive-by-wire chassis provided in an embodiment of this application. The device includes:

[0172] The test data acquisition unit 501 is used to collect test data under a predetermined test scenario;

[0173] The key indicator data determination unit 502 is used to determine the key indicator data of the drive-by-wire chassis under the set test scenario based on the collected test data.

[0174] The performance evaluation unit 503 is used to evaluate the performance of the drive-by-wire chassis based on the key indicator data according to the pre-set performance evaluation model.

[0175] Figure 5 The performance evaluation device for the drive-by-wire chassis shown is, and Figure 1 The performance evaluation method for the drive-by-wire chassis shown corresponds to this.

[0176] Figure 6 This is a schematic diagram of a performance evaluation device for a drive-by-wire chassis provided in one embodiment of this application. Figure 6As shown, the performance evaluation device 6 for a drive-by-wire chassis in this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as a drive-by-wire chassis performance evaluation program. When the processor 60 executes the computer program 62, it implements the steps in the various drive-by-wire chassis performance evaluation method embodiments described above. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the various device embodiments described above.

[0177] For example, the computer program 62 can be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 62 in the performance evaluation device 6 of the drive-by-wire chassis.

[0178] The performance evaluation device for the drive-by-wire chassis may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of the performance evaluation device 6 for the drive-by-wire chassis and does not constitute a limitation on the performance evaluation device 6 for the drive-by-wire chassis. It may include more or fewer components than shown, or combine certain components, or different components. For example, the performance evaluation device for the drive-by-wire chassis may also include input / output devices, network access devices, buses, etc.

[0179] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0180] The memory 61 can be an internal storage unit of the performance evaluation device 6 of the wire-controlled chassis, such as a hard disk or memory of the performance evaluation device 6 of the wire-controlled chassis. The memory 61 can also be an external storage device of the performance evaluation device 6 of the wire-controlled chassis, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the performance evaluation device 6 of the wire-controlled chassis. Furthermore, the memory 61 can include both internal storage units and external storage devices of the performance evaluation device 6 of the wire-controlled chassis. The memory 61 is used to store the computer program and other programs and data required by the performance evaluation device of the wire-controlled chassis. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0181] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0182] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0183] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0184] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0185] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0186] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0187] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0188] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method of evaluating the performance of a drive-by-wire chassis, characterized by, The method includes: Collect test data under predetermined test scenarios; Based on the collected test data, the key performance indicators of the drive-by-wire chassis under the set test scenario are determined. According to a pre-set performance evaluation model, the performance of the drive-by-wire chassis is evaluated using the key indicator data, including: obtaining the estimation deviation of a pre-trained machine learning model in the test scenario; when the estimation deviation is less than a preset deviation threshold, the performance of the drive-by-wire chassis is evaluated using the machine learning model, including: inputting the key indicator data into a scoring evaluation model to obtain an evaluation score for the drive-by-wire chassis; inputting the key indicator data into an autonomous driving deployment time evaluation model to evaluate the time required for autonomous driving deployment of the drive-by-wire chassis; and inputting the key indicator data into an autonomous driving deployment difficulty evaluation model. A tiered model is used to evaluate the autonomous driving deployment difficulty level of the drive-by-wire chassis. When the estimated deviation is greater than or equal to a preset deviation threshold, an evaluation score for the drive-by-wire chassis is obtained through a pre-set hierarchical analysis model. Based on the evaluation score, the current chassis's ranking among all known drive-by-wire chassis is determined. The required man-hours and autonomous driving deployment difficulty level are estimated based on the ranking, including: determining the required man-hours and autonomous driving deployment difficulty level of the current drive-by-wire chassis based on the man-hours and autonomous driving deployment difficulty levels of adjacent known drive-by-wire chassis in the ranking.

2. The method of claim 1, wherein, The performance of the drive-by-wire chassis is evaluated using a pre-defined hierarchical analysis model, including: The score corresponding to the key indicator data is determined based on the pre-defined correspondence between the range of key indicator data and the score. The evaluation score of the drive-by-wire chassis is determined based on the determined score and the corresponding weight.

3. The method of claim 1, wherein, The test scenarios include one or more of the following: driving test scenarios, braking test scenarios, and steering test scenarios; When the predetermined test scenario is a driver test scenario, the key indicator data includes one or more of the following: driver execution curve similarity, maximum instruction response time, and average instruction response time. When the predetermined test scenario is a braking test scenario, the key indicator data includes one or more of the following: maximum difference in average deceleration, average difference between average deceleration and actual command issuance, maximum time-related deceleration difference, maximum speed-related deceleration difference, maximum difference in braking distance, similarity of braking execution curve, maximum braking response time, average braking response time, maximum braking execution time, and average braking execution time. When the predetermined test scenario is a steering test scenario, the key indicator data includes one or more of the following: maximum step steady-state error, average step steady-state error, maximum step overshoot, average step overshoot, maximum step response time, average step response time, maximum step execution time, average step execution time, maximum response time of sine test, average response time of sine test, maximum execution error of sine test, average execution error of sine test, and steering execution curve similarity.

4. The method of claim 1, wherein, Before collecting test data in the predetermined test scenario, the method further includes: If it is determined that there is no manual intervention in the vehicle's control mechanism, then the vehicle will enter the predetermined test scenario according to the control instructions of the test scenario.

5. The method of claim 1, wherein, Collect test data under predetermined test scenarios, including: The collected scene segment test data is determined based on the message identifier of the scene segment test data included in the collected data. The scenario type of the scenario segment test data is determined based on the message identifier of the scenario number included in the collected data.

6. A device for evaluating performance of a by-wire chassis, characterized by The device includes: The test data acquisition unit is used to collect test data under a predetermined test scenario. The key indicator data determination unit is used to determine the key indicator data of the drive-by-wire chassis under the set test scenario based on the collected test data. The performance evaluation unit is used to evaluate the performance of the drive-by-wire chassis based on the key indicator data according to a pre-set performance evaluation model. This includes: obtaining the estimation deviation of a pre-trained machine learning model under the test scenario; when the estimation deviation is less than a preset deviation threshold, evaluating the performance of the drive-by-wire chassis using the machine learning model, including: inputting the key indicator data into a scoring evaluation model to obtain an evaluation score for the drive-by-wire chassis; inputting the key indicator data into an autonomous driving deployment time evaluation model to evaluate the time required for autonomous driving deployment of the drive-by-wire chassis; and inputting the key indicator data into an autonomous driving deployment time evaluation model. A deployment difficulty grading model is used to evaluate the autonomous driving deployment difficulty level of the drive-by-wire chassis. When the estimated deviation is greater than or equal to a preset deviation threshold, an evaluation score for the drive-by-wire chassis is obtained through a pre-set hierarchical analysis model. Based on the evaluation score, the current chassis's ranking among all known drive-by-wire chassis is determined. The required man-hours and autonomous driving deployment difficulty level are estimated based on the ranking, including: determining the required man-hours and autonomous driving deployment difficulty level of the current drive-by-wire chassis based on the man-hours and autonomous driving deployment difficulty levels of adjacent known drive-by-wire chassis in the ranking.

7. A device for evaluating performance of a drive-by-wire chassis, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images