Performance evaluation method and device for intelligent driving simulation test scene and vehicle

Abstract

The embodiment of the invention provides a performance evaluation method and device for an intelligent driving simulation test scene and a vehicle, and belongs to the technical field of automatic driving. A simulation test platform is established; designing an evaluation index system according to the simulation test platform; setting a simulation test scene according to the evaluation index system; obtaining simulation test data according to the simulation test scene; and analyzing the simulation test data to obtain a performance evaluation result. According to the method, an evaluation index system of an intelligent driving simulation test scene is perfected, algorithm optimization is facilitated, system performance is improved, and safety and reliability of an automatic driving system are improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a performance evaluation method, device and vehicle for an intelligent driving simulation test scenario. Background Art

[0002] The differences between intelligent driving simulation test scenarios and actual vehicle testing may lead to the following adverse effects and disadvantages:

[0003] 1. Insufficient environmental complexity and authenticity:

[0004] It is often difficult for simulation tests to fully simulate the complex traffic and environmental conditions in the real world, such as weather changes, road conditions, the behavior of other traffic participants, etc. These factors may have a significant impact on the autonomous driving system in actual driving, and simulation scenarios may not be able to accurately simulate these complexities.

[0005] 2. Limitations of sensor models:

[0006] The sensor models used in the simulation platform may be different from the sensors equipped on the actual vehicle, or the accuracy and real-time performance of the sensor data in the simulation may not be as good as that of the real vehicle. This may affect the evaluation of the system's ability to perceive the surrounding environment.

[0007] 3. Performance of the algorithm under real-time conditions:

[0008] The algorithms of the autonomous driving system may perform differently in simulation and in actual driving conditions. Real vehicle testing can demonstrate the system's reaction speed, decision-making ability, and ability to handle unforeseen events in a real-time environment, which may not be fully captured or captured with insufficient accuracy in simulation.

[0009] 4. Incompleteness of safety assessment:

[0010] Although simulation testing can evaluate the performance of autonomous driving systems in a controlled environment, it has limitations in fully evaluating safety. Certain security vulnerabilities or behaviors in specific scenarios may need to be discovered and verified through actual vehicle testing, because safety risk simulation in a simulation environment may not fully cover actual potential risks.

[0011] 5. Verification of human-computer interaction and user experience:

[0012] The intelligent experience part, especially the verification of human-computer interaction and user interface, may not be able to truly simulate the user's experience and feedback in actual driving in the simulation environment. For example, the actual response speed and accuracy of voice recognition and interface operation may deviate due to the virtual nature of the simulation environment. Summary of the invention

[0013] The main purpose of the embodiments of the present application is to provide a performance evaluation method, device and vehicle for an intelligent driving simulation test scenario.

[0014] The technical solution adopted by the present invention is:

[0015] In one aspect, an embodiment of the present invention provides a method for evaluating the performance of an intelligent driving simulation test scenario, the method comprising the following steps:

[0016] Establish a simulation test platform;

[0017] According to the simulation test platform, design an evaluation index system;

[0018] According to the evaluation index system, setting up a simulation test scenario;

[0019] According to the simulation test scenario, obtaining simulation test data;

[0020] The simulation test data is analyzed to obtain a performance evaluation result.

[0021] Furthermore, the establishment of the simulation test platform includes the following steps:

[0022] Use the cloud simulation platform to establish a simulation test platform for intelligent connected vehicles.

[0023] Furthermore, the design of an evaluation index system based on the simulation test platform includes the following steps:

[0024] Obtain information about intelligent connected vehicles;

[0025] According to the information of the intelligent networked vehicle and the simulation test platform, an evaluation index system of the intelligent networked vehicle is designed;

[0026] The evaluation index system includes the first-level index of intelligent driving, the second-level index of intelligent driving, the first-level index of intelligent experience, and the second-level index of intelligent experience;

[0027] The first-level intelligent driving indicators include vehicle following capability information, obstacle recognition and response information;

[0028] The secondary indicators of intelligent driving include line following capability information, intelligent parking information, system prompt information, driver monitoring information, lane change assistance and remote upgrade capability information;

[0029] The first-level indicators of intelligent experience include intelligent prompt information, intelligent driving information, and intelligent interaction information;

[0030] The secondary indicators of intelligent experience include forward collision warning information, lane departure warning information, blind spot monitoring information, automatic emergency braking information, voice interaction information, and LCD instrument display information.

[0031] Furthermore, setting a simulation test scenario according to the evaluation index system includes the following steps:

[0032] According to the evaluation index system, setting safety and intelligent autonomous driving simulation scenario evaluation indicators on the simulation test platform;

[0033] According to the safety and intelligence automatic driving simulation scenario evaluation indicators, a simulation test scenario is set;

[0034] The simulation test scenarios include a target vehicle stationary scenario, a low-speed driving scenario, a deceleration scenario, a lane cutting scenario, a lane centering scenario, and a blind spot detection scenario.

[0035] Further, obtaining simulation test data according to the simulation test scenario includes the following steps:

[0036] Running the simulation test scenario on the simulation test platform, and collecting parameters of the main vehicle and surrounding vehicles through the sensor model;

[0037] The parameters include vehicle status, lane status, and signal light status;

[0038] According to the parameters, simulation test data is obtained.

[0039] Furthermore, analyzing the simulation test data to obtain a performance evaluation result includes the following steps:

[0040] Define indicator weights;

[0041] Calculate the overall intelligence score of the simulation test scenario according to the indicator weights and the simulation test data;

[0042] A performance evaluation result is obtained according to the overall intelligence score.

[0043] Furthermore, the performance evaluation method of the intelligent driving simulation test scenario further includes the following steps:

[0044] According to the performance evaluation results, optimization and improvement information of the intelligent connected vehicle is obtained.

[0045] On the other hand, an embodiment of the present invention also provides a performance evaluation device for an intelligent driving simulation test scenario, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the performance evaluation method for the intelligent driving simulation test scenario as described above is implemented.

[0046] On the other hand, an embodiment of the present invention further provides a vehicle, which includes a performance evaluation device for the intelligent driving simulation test scenario as described above.

[0047] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method described above.

[0048] The embodiments of the present application include at least the following beneficial effects: The present application provides a performance evaluation method, device and vehicle for an intelligent driving simulation test scenario. The present invention establishes a simulation test platform; designs an evaluation index system based on the simulation test platform; sets a simulation test scenario based on the evaluation index system; obtains simulation test data based on the simulation test scenario; and analyzes the simulation test data to obtain a performance evaluation result. The present invention improves the evaluation index system for intelligent driving simulation test scenarios, helps optimize algorithms, improve system performance, and enhance the safety and reliability of autonomous driving systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a performance evaluation method for an intelligent driving simulation test scenario provided by an embodiment of the present invention;

[0050] Figure 2 is a flow chart of a performance evaluation method for an intelligent driving simulation test scenario provided by an embodiment of the present invention;

[0051] Figure 3 is a flow chart of a performance evaluation method for an intelligent driving simulation test scenario provided by an embodiment of the present invention;

[0052] Figure 4 is a flow chart of a performance evaluation method for an intelligent driving simulation test scenario provided by an embodiment of the present invention;

[0053] Figure 5 is a flow chart of a performance evaluation method for an intelligent driving simulation test scenario provided by an embodiment of the present invention;

[0054] Figure 6 is a flow chart of a performance evaluation method for an intelligent driving simulation test scenario provided by an embodiment of the present invention;

[0055] Figure 7is a flow chart of a performance evaluation method for an intelligent driving simulation test scenario provided by an embodiment of the present invention;

[0056] Figure 8 It is a schematic diagram of a technical route for evaluating the comprehensive capabilities of an intelligent connected vehicle provided by an embodiment of the present invention;

[0057] Fig. 9 is a schematic diagram of the intelligent driving indicators provided by an embodiment of the present invention;

[0058] Fig.10 It is a structural schematic diagram of a performance evaluation device for an intelligent driving simulation test scenario provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0060] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0061] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0063] The present invention takes into account that due to the differences between simulation tests and actual vehicles, there are limitations in evaluating them using only pass or fail indicators, and there are also certain safety risks. Therefore, it is crucial to establish a complete and reasonable evaluation system to quantitatively evaluate various parameter indicators.

[0064] First, because the traffic conditions and environmental factors in real scenes are complex and changeable, if you want to measure the quality of a simulation scene, you need to consider it comprehensively from multiple aspects. It is far from sufficient to consider safety alone. Secondly, safety can only evaluate the feasibility of the autonomous driving function. In this era of vehicle intelligence, if you want the autonomous driving function to be more complete and comprehensive, and enhance the decision-making ability of autonomous driving vehicles as intelligent systems for tasks and the cognitive understanding of complex environments, you also need to establish a relatively comprehensive intelligent evaluation standard for autonomous driving vehicles, so as to promote a virtuous cycle of automotive industry design, production and sales, stimulate automobile manufacturers to improve the level of automobile intelligence, and provide a reference for consumers when buying cars.

[0065] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0066] On the one hand, an embodiment of the present invention provides a performance evaluation method for an intelligent driving simulation test scenario. Specifically, referring to Figure 1 , the method comprises the following steps:

[0067] S100, establishing a simulation test platform;

[0068] S200. Design an evaluation index system based on the simulation test platform;

[0069] S300, setting a simulation test scenario according to the evaluation index system;

[0070] S400, obtaining simulation test data according to the simulation test scenario;

[0071] S500: Analyze the simulation test data to obtain performance evaluation results.

[0072] As an optional implementation, step S100 of the present invention to establish a simulation test platform involves building a dedicated cloud simulation platform project for simulating the autonomous driving system of intelligent networked vehicles. The platform sets up a simulation environment based on different vehicle models and autonomous driving performance, and conducts comprehensive performance testing and algorithm verification through software-in-the-loop testing. This platform not only supports simulation of static and dynamic scenarios, but also includes testing of vehicle perception, positioning, decision-making, planning, control and network functions.

[0073] In the step of designing the evaluation index system based on the simulation test platform of S200, the evaluation index system of intelligence is designed, which is divided into two first-level indicators: intelligent driving and intelligent experience. Intelligent driving evaluates the vehicle's detection, recognition and response capabilities to traffic signs, lane lines, signal lights, obstacles, pedestrians, vehicles, etc. on different road types (such as general roads, urban expressways, and highways). Intelligent experience evaluates whether the system can quickly activate vehicle functions through voice recognition commands, the response speed of the human-computer interaction system, and the vehicle's automatic adjustment of driving modes under different weather conditions.

[0074] In the S300 stage, specific simulation test scenarios are set according to the designed evaluation index system. These scenarios include but are not limited to vehicle stationary, low speed, deceleration, lane cutting, lane centering, blind spot detection, etc., so as to comprehensively evaluate the autonomous driving capabilities and intelligent experience of intelligent connected vehicles.

[0075] In the step of obtaining simulation test data according to the simulation test scenario in S400, the simulation test platform collects relevant parameters of the main vehicle and surrounding vehicles, lane status, signal light status and other data through the sensor model. These data will serve as the basis for subsequent analysis and be used for quantitative evaluation of various capabilities of the intelligent driving system.

[0076] S500 analyzes the simulation test data and obtains the performance evaluation results. This step is to analyze the collected simulation test data. Based on the calculation formula and weight defined in the evaluation index system, each simulation test scenario is comprehensively scored. These scores will directly reflect the performance of the intelligent connected vehicle system in various capabilities and the overall intelligence level, providing users and developers with quantitative intelligence evaluation results.

[0077] As an optional implementation mode, the present invention targets the complex and changeable driving environment of traffic conditions and environmental factors in real scenarios. The present invention aims to improve the evaluation index system of intelligent driving simulation test scenarios, starting from establishing intelligent indicators of intelligent driving simulation test scenarios, adding more evaluation parameters, and quantitatively evaluating the task decision-making ability of intelligent vehicles and their cognitive understanding ability of complex environments, so that the characteristics of intelligent vehicles as intelligent systems can also be quantitatively evaluated.

[0078] The establishment of intelligent indicators helps to quantitatively evaluate the performance of autonomous driving simulation test scenarios by converting test results into specific values. In this way, the comparison and pros and cons analysis between different environmental scenarios become more intuitive and objective. At the same time, the direction of improvement of intelligent driving test scenarios can be clarified. Through the evaluation results, it can be clarified in which aspects the scenario is lacking, thereby guiding R&D personnel to improve these problems. This helps to optimize algorithms, improve system performance, and ultimately improve the safety and reliability of autonomous driving systems.

[0079] refer to Figure 2 The embodiment of the present invention discloses step S100 of establishing a simulation test platform, including the following steps:

[0080] S110. Use the cloud simulation platform to establish a simulation test platform for intelligent connected vehicles.

[0081] As an optional implementation, the present invention S110: using a cloud simulation platform to establish a simulation test platform for intelligent networked vehicles. In this step, by using cloud computing and simulation technology, a platform capable of simulating the driving conditions of intelligent networked vehicles is built to provide the necessary infrastructure and environment for subsequent testing and evaluation.

[0082] refer to Figure 3 The embodiment of the present invention discloses that step S200 designs an evaluation index system according to a simulation test platform, including the following steps:

[0083] S210, obtaining information of the intelligent network-connected vehicle;

[0084] S220. Design an evaluation index system for the intelligent networked vehicle based on the information of the intelligent networked vehicle and the simulation test platform;

[0085] S230, the evaluation index system includes the first-level index of intelligent driving, the second-level index of intelligent driving, the first-level index of intelligent experience, and the second-level index of intelligent experience;

[0086] S240, first-level indicators of intelligent driving include vehicle following capability information, obstacle recognition and response information;

[0087] S250, intelligent driving secondary indicators include line following capability information, intelligent parking information, system prompt information, driver monitoring information, lane change assistance and remote upgrade capability information;

[0088] S260, first-level indicators of intelligent experience include intelligent prompt information, intelligent driving information, and intelligent interaction information;

[0089] S270, intelligent experience secondary indicators include forward collision warning information, lane departure warning information, blind spot monitoring information, automatic emergency braking information, voice interaction information, and LCD instrument display information.

[0090] As an optional implementation, the present invention S210: obtains information of the intelligent networked vehicle. This step involves collecting various data and information related to the intelligent networked vehicle, including technical specifications, software and hardware configurations, and other related parameters of the vehicle.

[0091] S220: Design an evaluation index system for intelligent connected vehicles based on the information and simulation test platform of intelligent connected vehicles. In this process, a comprehensive evaluation system is developed based on the collected data and the characteristics of the simulation platform to ensure that the performance and functions of intelligent connected vehicles can be comprehensively and systematically evaluated.

[0092] S230: The evaluation index system includes the following aspects:

[0093] Level 1 intelligent driving indicators: covers vehicle following capability information and obstacle recognition and response information, and is used to evaluate the driving safety and efficiency of vehicles in complex traffic environments.

[0094] Secondary indicators of intelligent driving: including line-following capability, intelligent parking, system prompts, driver monitoring, lane change assistance and remote upgrade capability information, which are used to deeply evaluate the vehicle's driving assistance functions and the technological advancement of the system.

[0095] First-level indicator of intelligent experience: involves intelligent prompts, intelligent driving and intelligent interaction information, and evaluates the vehicle user experience and the friendliness of human-computer interaction.

[0096] Secondary indicators of intelligent experience: including forward collision warning, lane departure warning, blind spot monitoring, automatic emergency braking, voice interaction and LCD instrument display information, which are used to evaluate the vehicle's responsiveness in emergency situations and the clarity of information display.

[0097] refer to Figure 4 The embodiment of the present invention discloses that step S300 sets a simulation test scenario according to an evaluation index system, including the following steps:

[0098] S310. According to the evaluation index system, set safety and intelligent autonomous driving simulation scenario evaluation indicators on the simulation test platform;

[0099] S320, setting a simulation test scenario according to safety and intelligent autonomous driving simulation scenario evaluation indicators;

[0100] S330, the simulation test scenarios include a target vehicle stationary scenario, a low-speed driving scenario, a deceleration scenario, a lane cutting scenario, a lane centering scenario, and a blind spot detection scenario.

[0101] As an optional implementation, in the embodiment of the present invention:

[0102] S310. According to the evaluation index system, set the safety and intelligence of the autonomous driving simulation scenario evaluation index on the simulation test platform. This includes defining key indicators to ensure the safety and intelligence of the system, such as obstacle avoidance capability, traffic rules compliance, and intersection handling capability.

[0103] S320. Accurately set simulation test scenarios based on safety and intelligence autonomous driving simulation scenario evaluation indicators. This step ensures that the simulation environment can simulate a variety of complex traffic scenarios, such as city streets, highways, and complex intersections, to comprehensively evaluate the performance of the autonomous driving system.

[0104] S330, simulation test scenarios include target vehicle stationary scenario, low-speed driving scenario, deceleration scenario, lane cutting scenario, lane centering scenario and blind spot detection scenario. These scenarios cover various common situations in daily driving and are designed to test the response capability and performance of the autonomous driving system in different scenarios.

[0105] refer to Figure 5 The embodiment of the present invention discloses that step S400 obtains simulation test data according to the simulation test scenario, including the following steps:

[0106] S410, running a simulation test scenario on a simulation test platform, and collecting parameters of the host vehicle and surrounding vehicles through a sensor model;

[0107] S420, the parameters include vehicle status, lane status, and signal light status;

[0108] S430: Obtain simulation test data according to the parameters.

[0109] As an optional implementation, in the embodiment of the present invention:

[0110] S410: Run the set simulation test scenario on the simulation test platform, and collect parameters of the main vehicle and surrounding vehicles through the sensor model. These parameters cover key information such as the dynamic state of the vehicle, the current lane situation, and the state of the traffic light.

[0111] S420, the collected parameters include data on vehicle status, lane status, signal light status, etc. These data are crucial for evaluating the perception and decision-making capabilities of the autonomous driving system and can help analyze the behavior and response of the system in a simulation environment.

[0112] S430: Generate detailed simulation test data based on the collected parameters and data. Such data is not only used to evaluate the performance of the system in a specific scenario, but also to identify potential improvement points and optimization strategies, thereby improving the overall performance and safety of the autonomous driving system.

[0113] refer to Figure 6The embodiment of the present invention discloses that step S500 analyzes the simulation test data to obtain the performance evaluation result, including the following steps:

[0114] S510, define indicator weights;

[0115] S520, calculating the overall intelligence score of the simulation test scenario according to the indicator weights and the simulation test data;

[0116] S530. Obtain a performance evaluation result according to the overall intelligence score.

[0117] As an optional implementation, in the embodiment of the present invention:

[0118] S510. Defining indicator weights includes determining the importance of each evaluation indicator for the overall intelligence assessment. These weights reflect the relative importance of different aspects in the autonomous driving system. For example, for the first-level and second-level indicators of intelligent driving and intelligent experience, the weight of each indicator will be determined based on its contribution to system functionality and user experience.

[0119] S520: Calculate the overall intelligence score of the simulation test scenario based on the indicator weights and the simulation test data. This step involves combining the specific data of each test scenario with the pre-defined indicator weights, and performing a weighted summation of the scores of each indicator through an appropriate calculation formula to obtain a single intelligence score for each scenario.

[0120] S530: Obtain performance evaluation results based on the overall intelligence score. In this stage, the system will evaluate and classify the overall performance of the autonomous driving system in a specific simulation scenario based on the obtained intelligence score. These results can help developers understand the strengths and improvements of the system under different conditions, and guide system optimization and improvement to improve its safety, efficiency and user experience in actual road environments.

[0121] refer to Figure 7 The performance evaluation method of the intelligent driving simulation test scenario disclosed in the embodiment of the present invention further includes the following steps:

[0122] S600. Obtain optimization and improvement information of the intelligent connected vehicle based on the performance evaluation result.

[0123] As intelligent driving technology continues to evolve, the performance evaluation of simulation test scenarios is particularly critical. The embodiment of the present invention discloses a performance evaluation method for intelligent driving simulation test scenarios, which is not limited to verifying the functionality of the system, but also focuses on evaluating its stability, adaptability and safety under various circumstances. The following will describe step S600 in detail and its importance.

[0124] Step S600 is one of the key links of the method. It not only involves the acquisition of performance evaluation results, but also further provides important information for optimizing and improving intelligent connected vehicles. In the development of intelligent driving technology, simulation test scenarios have become one of the main means of evaluating and verifying algorithms, systems and overall solutions. Compared with field tests, simulation tests have the advantages of low cost, high safety, and easy environmental control. They can simulate a variety of complex situations in different virtual scenarios to more comprehensively and systematically evaluate the performance of intelligent driving systems.

[0125] First, step S600 requires obtaining information based on the performance evaluation results. These results are usually provided by the simulation platform, including data on vehicle dynamic response, path planning accuracy, reaction speed of sensing devices, etc. By analyzing this data, we can fully understand the performance of the system in different scenarios and identify potential problems or performance bottlenecks. For example, if the system performs well in adaptive cruise control on highways, but has difficulty in path planning in complex urban road scenarios, this information will help to optimize and improve it in a targeted manner.

[0126] Secondly, step S600 emphasizes the acquisition of optimization and improvement information. This information is not limited to finding problems, but more importantly, it provides improvement directions and optimization suggestions for the system. Based on the performance evaluation results, algorithms, sensor settings or control strategies that need to be improved can be identified to improve the overall performance and user experience of smart connected vehicles. For example, through simulation tests, it was found that in rainy or nighttime environments, the system's perception of surrounding vehicles is insufficient. At this time, it can be recommended to optimize the sensor configuration or improve the deep learning model to improve perception accuracy.

[0127] The key point is that step S600 is not only a process of data collection and analysis, but also a key part of system optimization. In the field of intelligent driving, the pace of performance optimization is fast, requiring continuous iteration and improvement. Through performance evaluation of simulation test scenarios, the frequency and risk of field tests can be effectively reduced, and the R&D efficiency and cost-effectiveness of the system can be improved.

[0128] In summary, step S600 plays a key role as part of the intelligent driving simulation test scenario method. It provides valuable information and suggestions for the optimization and improvement of intelligent connected vehicles by obtaining and analyzing the performance evaluation results. In the future, with the continuous advancement of intelligent driving technology and the expansion of application scenarios, this simulation-based performance evaluation method will play an increasingly important role and promote the development of intelligent transportation systems in a safer and more efficient direction.

[0129] The present invention proposes intelligent index parameters to quantitatively evaluate the task decision-making ability and the cognition and understanding of complex environments in autonomous driving simulation test scenarios.

[0130] As an optional implementation mode, the evaluation system constructed by the present invention, through the intelligent connected vehicle simulation test verification public service platform built by the cloud simulation platform project, is based on software-in-the-loop testing. While comprehensively testing the perception, positioning, decision-making, planning, control, networking and other algorithms and functions of the intelligent connected vehicle's automatic driving system performance, it evaluates its simulation test scenarios.

[0131] The simulation test platform conducts software-in-the-loop testing of intelligent connected vehicles based on their models and autonomous driving performance. At the same time, the platform adds designed safety, intelligence and other autonomous driving simulation scenario evaluation indicators to build an autonomous driving system evaluation system. The indicator parameters of this system are more flexible and reasonable, and have the comprehensive ability to evaluate intelligent connected vehicles. For specific technical routes, see Figure 8 .

[0132] In the present invention, the evaluation index system of intelligence is set to include two first-level indicators, namely intelligent driving and intelligent experience. They all evaluate the scene through subjective evaluation. Intelligent driving refers to the ability to detect, identify and respond to traffic signs, logos, lane lines, signal lights, obstacles, pedestrians, vehicles and two-wheeled vehicles in general roads, urban expressways and highways. Whether it has the functions of multi-lane or single-lane lateral lane switching and full-speed longitudinal following, the scene of target vehicle stationary, low speed, deceleration, lane cutting, lane centering, blind spot detection, etc. is evaluated. Intelligent experience refers to whether the automatic driving system can activate vehicle-related functions through rapid voice recognition commands, the response speed of the human-computer interaction system, the interconnection between the car system and the mobile phone, the identification of dangerous vehicle situations and emergency rescue assistance, the detection of life in the car after the vehicle is locked, and the vehicle automatically switches to different driving modes according to meteorological related parameters to cope with weather changes, etc., which provide an evaluation basis for the evaluation of intelligent experience.

[0133] As an optional implementation, the present invention sets 7 secondary indicators under intelligent driving: vehicle following capability, other obstacle recognition and response capability, line following capability, intelligent parking capability, system prompt capability, driver monitoring capability, lane change assistance capability and remote upgrade capability. The indicators under intelligent driving are as follows: Fig. 9 shown.

[0134] There are three secondary indicators under the intelligent experience: intelligent prompts, intelligent driving and intelligent interaction. Intelligent prompts are to pay attention to safety through display and alarm information to improve the performance and satisfaction of the entire system, including forward collision warning system, lane departure warning, blind spot monitoring warning, forward passing object warning, four-door and two-lid status warning and other contents. Intelligent driving includes automatic emergency braking performance, lane assist system, highway parallel avoidance system, intelligent parking assistance, driving combination control and other contents. Intelligent interaction is to realize the multi-screen integrated interactive experience of central control, LCD instrument, head-up display, rear seat entertainment and other screens in the intelligent cockpit, as well as more intelligent interaction methods such as voice recognition and gesture control, including voice interaction human-computer interaction, voice recognition ability, voice control, semantic association, mobile terminal interaction, mobile terminal display, vehicle control, mobile terminal warning, central control screen interaction, interface information display, central control screen operability, entertainment system operation and extended application and other contents. The indicators under the intelligent experience are as follows Fig. 9 shown.

[0135] After running the simulation test scenario, the sensor model will send back the collected parameters of the main vehicle, nearby vehicles, target vehicles, lane status, signal light status, etc. to the platform for analysis. The platform will score the test scenario results according to the calculation formula of each indicator score, and finally calculate the overall intelligence score of the scenario according to the defined weights of each indicator, so that users and developers can have a more intuitive and quantitative experience of the system intelligence.

[0136] The present invention relates to the field of autonomous driving technology, and in particular to a method system for evaluating the intelligence of autonomous driving simulation test scenarios. Intelligent networked vehicles that support autonomous driving functions must undergo a large number of tests and verifications before mass production to ensure the safety of vehicle functions and performance, thereby ensuring the safety of drivers and other traffic participants. Among them, simulation test software-in-the-loop testing can use virtual simulation software to build scenarios in computers, and simulate various static environments and dynamic scenarios to achieve simulation testing of autonomous driving vehicles. It is an important means of developing and verifying autonomous driving systems.

[0137] refer to Fig.10 On the other hand, an embodiment of the present invention also provides a performance evaluation device for an intelligent driving simulation test scenario, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a performance evaluation method for the intelligent driving simulation test scenario as described above is implemented.

[0138] The performance evaluation device of the intelligent driving simulation test scenario in an embodiment of the present invention includes a memory and a processor.

[0139] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0140] The memory may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. Among them, ROM can store static data or instructions required by the processor or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0141] The memory stores executable codes, and when the executable codes are processed by the processor, the processor can execute part or all of the methods described above.

[0142] On the other hand, an embodiment of the present invention further provides a vehicle, which includes a performance evaluation device as described above in the intelligent driving simulation test scenario.

[0143] The vehicle of the embodiment of the present invention includes an electric drive assembly of the performance evaluation device of the above-mentioned intelligent driving simulation test scenario. Specifically, the vehicle can be a private car, such as a sedan, SUV, MPV or pickup truck. The vehicle can also be an operating vehicle, such as a van, bus, small truck or large trailer. The vehicle can be a gasoline vehicle or a new energy vehicle. When the vehicle is a new energy vehicle, it can be a hybrid vehicle or a pure electric vehicle.

[0144] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above method.

[0145] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0146] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A performance evaluation method for intelligent driving simulation test scenarios, characterized in that: The method comprises the following steps: Establish a simulation test platform; According to the simulation test platform, design an evaluation index system; According to the evaluation index system, setting up a simulation test scenario; According to the simulation test scenario, obtaining simulation test data; The simulation test data is analyzed to obtain a performance evaluation result.

2. The method according to claim 1, characterized in that: The establishment of the simulation test platform comprises the following steps: Use the cloud simulation platform to establish a simulation test platform for intelligent connected vehicles.

3. The method according to claim 2, characterized in that The design of the evaluation index system according to the simulation test platform comprises the following steps: Obtain information about intelligent connected vehicles; According to the information of the intelligent networked vehicle and the simulation test platform, an evaluation index system of the intelligent networked vehicle is designed; The evaluation index system includes the first-level index of intelligent driving, the second-level index of intelligent driving, the first-level index of intelligent experience, and the second-level index of intelligent experience; The first-level intelligent driving indicators include vehicle following capability information, obstacle recognition and response information; The secondary indicators of intelligent driving include line following capability information, intelligent parking information, system prompt information, driver monitoring information, lane change assistance and remote upgrade capability information; The first-level indicators of intelligent experience include intelligent prompt information, intelligent driving information, and intelligent interaction information; The secondary indicators of intelligent experience include forward collision warning information, lane departure warning information, blind spot monitoring information, automatic emergency braking information, voice interaction information, and LCD instrument display information.

4. The method according to claim 1, characterized in that: The step of setting a simulation test scenario according to the evaluation index system comprises the following steps: According to the evaluation index system, setting safety and intelligent autonomous driving simulation scenario evaluation indicators on the simulation test platform; According to the safety and intelligence automatic driving simulation scenario evaluation indicators, a simulation test scenario is set; The simulation test scenarios include a target vehicle stationary scenario, a low-speed driving scenario, a deceleration scenario, a lane cutting scenario, a lane centering scenario, and a blind spot detection scenario.

5. The method according to claim 1, characterized in that The step of obtaining simulation test data according to the simulation test scenario comprises the following steps: Running the simulation test scenario on the simulation test platform, and collecting parameters of the main vehicle and surrounding vehicles through the sensor model; The parameters include vehicle status, lane status, and signal light status; According to the parameters, simulation test data is obtained.

6. The method according to claim 1, characterized in that The step of analyzing the simulation test data to obtain a performance evaluation result comprises the following steps: Define indicator weights; Calculate the overall intelligence score of the simulation test scenario according to the indicator weights and the simulation test data; A performance evaluation result is obtained according to the overall intelligence score.

7. The method according to claim 1, characterized in that The performance evaluation method of the intelligent driving simulation test scenario also includes the following steps: According to the performance evaluation results, optimization and improvement information of the intelligent connected vehicle is obtained.

8. A performance evaluation device for an intelligent driving simulation test scenario, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the performance evaluation method for the intelligent driving simulation test scenario as described in any one of claims 1 to 7 is implemented.

9. A vehicle, characterized in that: The vehicle includes a performance evaluation device for the intelligent driving simulation test scenario as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images