Whole vehicle evaluation method and system for intelligent driving vehicle navigation driving assistance system

Abstract

The invention relates to the technical field of intelligent driving evaluation, and discloses a vehicle evaluation method and system for an intelligent driving vehicle navigation driving assistance system, and the method comprises a test procedure and an evaluation method. The test regulation is used for setting different test items according to the performance of the whole vehicle and acquiring test results of the tested vehicle in real time under the different test items; the evaluation method is used for scoring the whole NOA system according to a test result, and obtaining a performance grade according to the score. The test items comprise an NOA whole vehicle test item and a trafficability requirement test item; the NOA whole vehicle test item is used for performing scene construction and test on a plurality of performance items of a test vehicle according to requirements; the trafficability requirement test item is used for judging the trafficability of the test vehicle test according to a set judgment rule; the evaluation method comprises the step of scoring according to the performance of the whole vehicle in the test process and a set scoring standard. According to the invention, the performance of the whole NOA system is evaluated, a complete, standard and stable evaluation mode is realized, and the test efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving system testing and evaluation, and in particular to a whole-vehicle testing and evaluation method and system for an intelligent driving vehicle pilot driving assistance system. Background Art

[0002] In the context of the rapid development of intelligent driving technology, the Navigation on Autopilot (NOA) system, as an important component to improve the vehicle's autonomous driving capabilities, is receiving extensive attention and in-depth research in the industry. In order to ensure the stability and safety of the NOA system, various road tests need to be conducted on the NOA system to ensure that the NOA system performance meets the requirements of intelligent driving.

[0003] The existing road testing method is to conduct generalized tests on social roads and conduct random scenario tests. Due to the fragmentation of the test scenarios, it is impossible to fully verify the performance of the NOA system in complex and changeable real-world scenarios such as different road conditions, weather, and traffic flow. This testing gap makes the test results lack wide applicability and it is difficult to accurately reflect the actual performance and potential problems of the vehicle in actual driving. In order to fully obtain the performance of the NOA system, the current focus is on performance testing of a single function. In this single function test, problems are often analyzed and solved in isolation, making it difficult to fully evaluate the overall performance and stability of the system in the actual driving environment, resulting in a lack of completeness in the test results.

[0004] In addition, due to the lack of test coverage and the limitations of test scenarios, testers need to spend a lot of time and energy to build and simulate various test scenarios in the hope of capturing possible problems, which makes the test efficiency low and the test cost high. Therefore, it is necessary to build a professional, complete, comprehensive and standardized vehicle testing method. Summary of the invention

[0005] The present invention aims to provide a whole-vehicle evaluation method and system for a smart driving car pilot driving assistance system, so as to solve the problem that the existing testing method can only test a single performance, resulting in incomplete test results, low test efficiency, and difficulty in ensuring test stability.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme, a whole vehicle evaluation method for a pilot driving assistance system of an intelligent driving car, this scheme sets the test method into two parts: a test procedure and an evaluation method, and sets test items through the pilot driving assistance system (NOA) on the performance of the whole vehicle, and evaluates the performance of the whole vehicle of the NOA system to achieve a complete, standard and stable evaluation method, improve the test coverage, and improve the overall test efficiency. The evaluation method specifically includes a test procedure and an evaluation method; the test procedure is used to set different test items according to the performance of the whole vehicle, and obtain the test results of the test vehicle in real time under different test items; the evaluation method is used to score the whole vehicle of the NOA system according to the test results, and obtain the performance level according to the score;

[0007] The test items include NOA vehicle test items and passability test items; the NOA vehicle test items are used to construct scenarios and conduct tests on multiple performance items of the test vehicle according to requirements; the passability test items are used to judge the passability of the test vehicle test according to the set judging rules;

[0008] The evaluation method includes scoring according to the performance of the whole vehicle during the test according to the set scoring criteria.

[0009] At the same time, this solution also provides a whole vehicle evaluation system for an intelligent driving car pilot driving assistance system, which is applied to the above-mentioned whole vehicle evaluation method for an intelligent driving car pilot driving assistance system, including a test procedure unit and an evaluation method unit; the test procedure unit includes a test vehicle, a scene module, a positioning module, a true value system module and a data acquisition module; the evaluation method unit includes a scoring module and a judgment module.

[0010] Furthermore, the performance items include NOA function realization capability, ODD recognition capability, human-computer interaction rationality, complex scene recognition and response, and avoidance capability. The evaluation rules include closed-field tests and open-road tests, covering functional logic and functional performance, which are more comprehensive than conventional tests. The test scenarios are more representative, more versatile, and highly consistent with users' daily driving scenarios, providing an important reference for the optimization and improvement of the pilot driving assistance system.

[0011] Furthermore, it also includes setting test items for each performance item, and the test items include test scenarios, site types and test cases.

[0012] Furthermore, the evaluation rule is to calculate the total pass score of each performance item according to the basic score of each test item and the weight of the corresponding performance item, which can be expressed as T n =x1*λ n +x2*λ n +...+x m *λn ; In the formula, n represents the corresponding performance item; m represents the corresponding test item; T n is the total pass score of the corresponding performance item; x m is the basic score of the corresponding test item; n is the weight of the corresponding performance item.

[0013] Furthermore, the evaluation criteria are to conduct 3 tests according to the set test items, and if all 3 tests pass the requirements, the score is calculated. If the number of passes is less than 3, the score is 0. By adopting 3 tests, the accuracy and reliability of the test results can be ensured, and accidental phenomena that affect the test results can be avoided. Passing 3 times can reflect the consistency and stability of the functional performance.

[0014] Furthermore, before the test begins, the initial speed error of the test vehicle is ±2km / h. Since the evaluation method mainly focuses on the tested scenario and system performance, the speed accuracy required is not high. Secondly, the speed of the vehicle is controlled by the vehicle's own control system, and the control accuracy of each vehicle is different. By controlling the speed error to ±2km / h, a larger speed error can meet the test requirements and increase the versatility of this method.

[0015] Furthermore, the test may be conducted in a closed area or on a public road.

[0016] Furthermore, the scenario module is used to construct a test intersection scenario and test conditions; the positioning module is used to obtain the location information of the test vehicle; the truth value system is used to collect the relationship information of the target object relative to the test vehicle; and the data acquisition module is used to collect and store the test data of the test vehicle in real time.

[0017] Furthermore, the position information includes the orientation, heading angle, longitude and latitude, speed and acceleration of the test vehicle; and the relationship information includes position relationship, relative speed and relative acceleration.

[0018] The principles and advantages of this solution are:

[0019] The NOA system is a complex integrated system that involves the collaborative work of multiple modules such as environmental perception, decision-making planning, and vehicle control. In order to ensure the effectiveness of the NOA system test, the existing technology often uses generalized tests on social roads and random scene tests. Although this method can ensure the authenticity and effectiveness of the test scene, it is limited by the test road environment. The road scenes that can be used for testing are variable and unstable, making it difficult to reproduce the tested scenes and the test problems obtained, resulting in the lack of wide applicability of this test method. Due to the large number of variables and instability in different scenarios, it is also difficult to accurately locate the performance problems based on the overall performance of the test vehicle, which makes the test results lack stability.

[0020] In order to accurately test the performance of the NOA system, the existing technology mainly tests a single function, which sets a special test scenario and test conditions according to the performance to be tested, and also assumes that other parameters of the test vehicle remain unchanged. This results in the problems being analyzed and solved in isolation, ignoring the interaction with other system components, resulting in deviations between the test results and the actual response.

[0021] This solution has formulated test procedures and evaluation methods for the performance of the NOA system on the whole vehicle, in order to standardize the test scenarios and test cases, and to score and evaluate in a quantifiable way to form a standard test process. At the same time, test items and performance items are formulated in a targeted manner according to the performance of the whole vehicle, which can ensure the integrity of the test; at the same time, the performance of the system in different scenarios can be obtained, and the problem can be reproduced; the stability of the system performance in the same scenario can be guaranteed; it is more widely applicable, the test is more professional, and the test efficiency is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a process flow of a whole vehicle evaluation method of a pilot driving assistance system for an intelligent driving vehicle according to the present invention;

[0023] Figure 2 The present invention is a schematic structural diagram of a vehicle evaluation system for a pilot driving assistance system for an intelligent driving vehicle. DETAILED DESCRIPTION

[0024] The following is further described in detail through specific implementation methods:

[0025] Embodiment 1

[0026] As attached Figure 1 As shown, a vehicle evaluation method for a pilot driving assistance system of an intelligent driving car in this embodiment is designed to test the performance of the pilot driving assistance system (NOA) on the whole vehicle, and a complete set of testing methods for the whole vehicle test of the NOA system is designed. The performance in different scenarios can ensure the integrity and stability of the test, and at the same time, the problem can be reproduced to improve the test efficiency. In this embodiment, the test method specifically includes two parts: a test procedure and an evaluation method.

[0027] In this embodiment, the test is conducted in a closed area or on a public road. It should be noted that when the test vehicle collides with a curb or other obstacles, or cannot find the correct path, or there is a risk of collision, the tester needs to take over in time, and the test ends at this time.

[0028] The test procedure is used to set different test items according to the performance of the whole vehicle, and obtain the test results of the test vehicle in real time under different test items.

[0029] In this embodiment, the test items include NOA vehicle test items and passability requirement test items. NOA vehicle test items are used to construct scenarios and conduct tests on multiple performance items of the test vehicle according to requirements; passability requirement test items are used to judge the passability of the test vehicle test according to the set judgment rules.

[0030] In this embodiment, according to the performance of the NOA system on the whole vehicle, the performance items are set as NOA function realization capability items, ODD recognition capability items, reasonable human-computer interaction items, complex scene recognition and response items, and avoidance capability items. The evaluation rules include closed field tests and open road tests, and specific analysis and setting of performance items are carried out to ensure coverage of functional logic and functional performance, which is more comprehensive than conventional test content. At the same time, the test scenarios are more typical and highly consistent with the user's daily driving scenarios, providing an important reference for the optimization and improvement of the pilot driving assistance system.

[0031] At the same time, test items are set for each performance item, and the test items include test scenarios, site types and test cases. The following is a detailed description of the test item settings for each performance item.

[0032] (1) NOA function realization capability item, mainly for testing the basic function performance of high-speed NOA to ensure the availability of the function. According to the test requirements, the test is constructed according to the test items in Table 1 below, including multiple test scenarios such as lane change, avoidance, speed change, merging, special roads, etc., and different test cases and site types are set to ensure that the performance of the test vehicle in different scenarios can be obtained, ensuring the integrity and accuracy of the test.

[0033] Table 1 NOA functional realization capability test items

[0034]

[0035] (2) ODD identification capability items are mainly used to verify the system design and operation boundaries through various boundary tests to ensure the performance and safety of the system under different conditions. Test construction is carried out according to the test items in Table 2 below based on the boundary type, including map boundaries, positioning boundaries, lane width boundaries and other scenarios, and different test cases and site types are set to ensure that the performance of the test vehicle in different scenarios can be obtained, complete test results can be obtained, and the integrity and accuracy of the test can be ensured.

[0036] Table 2 ODD recognition ability test items

[0037]

[0038] (3) Reasonable items of human-computer interaction. The rationality test of NOA human-computer interaction is to ensure that users can effectively interact with the system by verifying the rationality of human-computer interaction, and obtain accurate, clear and consistent feedback, so as to ensure the interactive experience and safety between users and the system. In a closed venue, the test is constructed according to the test items in Table 3 below, including scenarios such as inability to enter, avoidance, function exit, occlusion, and failure, and corresponding test cases are set to ensure that the performance of the test vehicle in different scenarios can be obtained, and complete test results can be obtained to ensure the integrity and accuracy of the test.

[0039] Table 3 Test items for reasonable human-computer interaction

[0040]

[0041] (4) Complex scene recognition and response items, by verifying the system's recognition and response capabilities in complex scenes, to ensure the system's performance and safety in complex scenes. According to the scene requirements, the test is constructed according to the test items in Table 4 below, including various complex scenes such as night environment, branch ramps, confluence ramps, line pressure, complex ramps, congestion, faults, etc., and different site types and test cases are set to ensure that the performance of the test vehicle in different scenes can be obtained, ensuring the integrity and accuracy of the test.

[0042] Table 4 Complex scene recognition and response test items

[0043]

[0044] (5) Avoidance capability item: By verifying the avoidance capability of the system in extreme situations, the safety and reliability of the system in extreme situations are ensured, and the system performance and limitations are clarified. In a closed area, the test is constructed according to the test items in Table 5 below, including scenarios such as lane change, speed change, cut-in, accident, lane line recognition, traffic sign recognition, etc., and corresponding test cases are set to ensure that the performance of the test vehicle in different scenarios can be obtained, complete test results can be obtained, and the integrity and accuracy of the test can be ensured.

[0045] Table 5 Avoidance ability test items

[0046]

[0047]

[0048] The test vehicles are tested according to the test items set above, the test results are obtained and recorded, and the test results are scored and evaluated according to the set evaluation method. In this embodiment, before the test starts, the initial displayed speed of the test vehicle should be maintained within the error range of ±2km / h, while ensuring that the NOA system of the test vehicle is normal and available. Because the evaluation method mainly focuses on the tested scenario and system performance, the speed requirement is not high. Secondly, the speed of the vehicle is controlled by the vehicle's own control system, and the control accuracy of each vehicle is different. Therefore, controlling the speed error within a larger range of ±2km / h can not only meet the test requirements, but also increase the versatility of this method, with a wider range of application and more flexible operation.

[0049] In this embodiment, the evaluation method is to score the NOA system vehicle according to the test results, and obtain the corresponding performance level according to the score. The evaluation method includes scoring according to the performance of the vehicle during the test according to the set scoring criteria.

[0050] In this embodiment, the evaluation criteria are to conduct three tests according to the set test items. If all three tests pass the requirements, the scores are calculated according to the set evaluation rules. If the number of passes is less than 3, the score is 0, which means it fails. After the total score is calculated according to the evaluation rules, the total score is graded to obtain the performance score of the NOA system vehicle. In this embodiment, three tests are used to ensure the accuracy and reliability of the test results, avoid the occurrence of occasional phenomena that affect the test results, and will not increase too much testing workload, effectively reduce the workload, and ensure the reliability and effectiveness of the test results. At the same time, it is determined that the three tests are passed before the score is given, which can ensure the consistency and stability of the vehicle's functional performance and improve the credibility of the test results.

[0051] In this embodiment, the judging rule is to calculate the total pass score of each performance item according to the basic score of each test item and the weight of the corresponding performance item, and obtain the final score of the NOA vehicle performance, which can be expressed as T n =x1*λ n +x2*λ n +...+x m *λ n ; In the formula, n represents the corresponding performance item; m represents the corresponding test item; T n is the total pass score of the corresponding performance item; x m is the basic score of the corresponding test item; n is the weight of the corresponding performance item.

[0052] In this embodiment, starting from the performance of the whole vehicle, the test results of the NOA whole vehicle test are evaluated in a quantitative manner. According to the difficulty of each test item and the amount of test cases, the corresponding basic score is set, combined with the most direct sensory experience and the importance of safety relevance during the actual use of the user, the weight and total score of each performance item are set accordingly, among which the weight of the NOA function realization capability item is set to 0.2, and the total score is 1.8; the weight of the ODD recognition capability item is set to 0.15, and the total score is 1.35; the weight of the reasonable human-computer interaction item is set to 0.15, and the total score is 1.35; the weight of the complex scene recognition and response item is set to 0.25, and the total score is 2.25; the weight of the avoidance capability item is set to 0.25, and the total score is 2.25. The total score of passability is equal to the basic score multiplied by the weight, and the proportion is converted to obtain the final score of the NOA whole vehicle performance. Finally, the total score of passability is graded to obtain the evaluation result. In this embodiment, the evaluation levels include 4 levels, namely excellent, good, fair and poor. When 8<total pass score ≤9, it is excellent; when 6<total pass score ≤7, it is good; when 3<total pass score ≤5, it is fair; when 0≤total pass score ≤2, it is poor.

[0053] In this embodiment, since the prior art mainly adopts generalization test on social roads, random scene test is carried out, which causes the test scene to be affected by the changes in the real environment, making it difficult for the test scene to meet both authenticity and comprehensive coverage, so that the test scene has many limitations, and the test scene and test problems cannot be reproduced, and it is difficult to ensure the stability of the test results. Therefore, in order to fully test the performance of the NOA system in the prior art, a customized test is carried out for a single function. Although this method can ensure the fixity of the test scene, the measured problems are mostly independent, ignoring the influence relationship with other systems, and it is difficult to comprehensively evaluate the overall performance and stability of the system in the actual driving environment. Therefore, most of the test standards in the prior art are standards for single function testing, and there is no standard for vehicle testing. And due to the differences between automobile manufacturers, the production standards adopted are different, resulting in differences in performance between vehicles, and it is difficult to achieve a unified vehicle testing method that can meet the performance of all vehicles.

[0054] This solution, through the performance of the NOA system on the whole vehicle, has formulated performance items such as function realization capability, ODD recognition capability, rationality of human-computer interaction, recognition and response of complex scenarios, and ability to avoid accidents in extreme situations. By setting fixed test items, a limited number of tests are conducted from the performance direction to achieve the repeatability of scenarios and problems. At the same time, from the perspective of performance, it can be applied to a variety of test vehicles and conduct overall testing of the whole vehicle without considering the impact of each model on the NOA system separately, realizing an efficient and complete testing method, ensuring the integrity of the system testing in different scenarios, and the stability in the same scenario, effectively improving the overall testing efficiency.

[0055] Embodiment 2

[0056] As attached Figure 2 As shown, in this embodiment, a whole vehicle evaluation system for a pilot driving assistance system for an intelligent driving car is provided, which is applied to a whole vehicle evaluation method for a pilot driving assistance system for an intelligent driving car in Embodiment 1, and includes a test procedure unit and an evaluation method unit. The test procedure unit is used to set specific test items and test items, and the evaluation method unit is used to score the obtained test results according to the set evaluation rules based on the performance of the whole vehicle.

[0057] The test procedure unit includes a test vehicle, a scenario module, a positioning module, a truth system module and a data acquisition module.

[0058] The scenario module is used to construct the test intersection scenario and the test conditions. In this embodiment, the test intersection scenario includes the test scenario and the test site type, and the test conditions include the set test cases, as described in the first embodiment, which will not be repeated here.

[0059] The positioning module is used to obtain the position information of the test vehicle. In this embodiment, the position information includes signals such as the position, heading angle, longitude and latitude, speed and acceleration of the test vehicle.

[0060] The truth value system is used to collect the relationship information of the target object relative to the test vehicle. The relationship information includes the position relationship, relative speed, relative acceleration and other information between the target object and the test vehicle.

[0061] The data acquisition module is used to collect and store the test data of the test vehicle in real time, including spatial information such as position, speed, acceleration, video information, GPS positioning information, etc., and save functional test data.

[0062] The evaluation method unit includes a scoring module and a judgment module. The scoring module is used to calculate the total pass score of the test vehicle after determining that the test vehicle meets the passing requirements for three times, and obtain the final performance score of the NOA vehicle. The judgment module is used to judge the three tests of the test vehicle. If all three times meet the requirements, the scoring module is started, otherwise it is not satisfied and is directly recorded as 0 points. At the same time, the judgment module is also used to grade the calculated total pass score after the scoring module calculates the final score, and obtain the final performance evaluation of the NOA vehicle.

[0063] In this embodiment, the test is conducted in a closed area or on a public road. If the vehicle collides with the curb or other obstacles, or cannot find the correct path, or there is a risk of collision, the tester needs to take over in time and the test ends. By accurately acquiring the real-time information of the test vehicle, the stability and timeliness of the test process data are ensured, while ensuring the integrity of the test, the reliability and accuracy of the data information are ensured, the test efficiency is effectively improved, and a professional, complete, and standard vehicle testing method is achieved.

[0064] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A vehicle evaluation method for a pilot driving assistance system of an intelligent driving vehicle, characterized by: Used to test the performance of the NOA system on the whole vehicle, the evaluation method includes a test procedure and an evaluation method; the test procedure is used to set different test items according to the performance of the whole vehicle, and obtain the test results of the test vehicle in real time under different test items; the evaluation method is used to score the NOA system whole vehicle according to the test results, and obtain the performance level according to the score; The test items include NOA vehicle test items and passability test items; the NOA vehicle test items are used to construct scenarios and conduct tests on multiple performance items of the test vehicle according to requirements; the passability test items are used to judge the passability of the test vehicle test according to the set judging rules; The evaluation method includes scoring according to the performance of the whole vehicle during the test according to the set scoring criteria.

2. The whole vehicle evaluation method of the intelligent driving vehicle pilot driving assistance system according to claim 1 is characterized by: The performance items include NOA function realization capability item, ODD recognition capability item, human-computer interaction rationality item, complex scene recognition and response item and avoidance capability item.

3. The whole vehicle evaluation method of the intelligent driving vehicle pilot driving assistance system according to claim 1 is characterized by: It also includes setting test items for each performance item, and the test items include test scenarios, site types and test cases.

4. The whole vehicle evaluation method of the intelligent driving vehicle pilot driving assistance system according to claim 3 is characterized by: The evaluation rule is to calculate the total pass score of each performance item according to the basic score of each test item and the weight of the corresponding performance item, which can be expressed as T n =x1*λ n +x2*λ n +...+x m *λ n ; In the formula, n represents the corresponding performance item; m represents the corresponding test item; T n is the total pass score of the corresponding performance item; x m is the basic score of the corresponding test item; n is the weight of the corresponding performance item.

5. The vehicle evaluation method of the intelligent driving vehicle pilot driving assistance system according to claim 1 is characterized by: The evaluation standard is to conduct three tests according to the set test items. If all three tests pass the requirements, the score is calculated. If the number of passes is less than 3, the score is 0.

6. The whole vehicle evaluation method of the intelligent driving vehicle pilot driving assistance system according to claim 1 is characterized by: It also includes that before the test begins, the initial displayed speed error of the test vehicle is ±2km / h.

7. The whole vehicle evaluation method of the intelligent driving vehicle pilot driving assistance system according to claim 1 is characterized by: It also includes testing in closed areas or on public roads.

8. A vehicle evaluation system for a smart car pilot driving assistance system, characterized by: A whole-vehicle evaluation method for an intelligent driving car pilot driving assistance system as described in claims 1-7 includes a test procedure unit and an evaluation method unit; the test procedure unit includes a test vehicle, a scenario module, a positioning module, a true value system module and a data acquisition module; the evaluation method unit includes a scoring module and a judgment module.

9. The whole vehicle evaluation system of the intelligent driving car pilot driving assistance system according to claim 8 is characterized by: The scenario module is used to construct the test intersection scenario and test conditions; the positioning module is used to obtain the location information of the test vehicle; the truth value system is used to collect the relationship information of the target object relative to the test vehicle; and the data acquisition module is used to collect and store the test data of the test vehicle in real time.

10. The whole vehicle evaluation system of the intelligent driving car pilot driving assistance system according to claim 9 is characterized by: The position information includes the orientation, heading angle, longitude and latitude, speed and acceleration of the test vehicle; the relationship information includes position relationship, relative speed and relative acceleration.

Images