Scene extraction generation and evaluation method for vehicle safety test

Abstract

The invention relates to the technical field of automatic driving automobiles, in particular to a scene extraction, generation and evaluation method for a vehicle safety test, which adopts a semantic segmentation network to keep scene element features in original natural driving data to the greatest extent. Based on a high-level decision deep reinforcement learning algorithm, a rough set is used to extract an association generation rule of a typical dangerous scene, the typical dangerous scene can be automatically generated in combination with simulation software, and a certain design thought is provided for automatically extracting and generating the characteristics of the scene by referring to natural driving data; meanwhile, the overall intelligence of the self-driving automobile is evaluated in combination with two dimensions of natural driving evaluation and test scene complexity evaluation, and key index advantages in different aspects are combined, so that the multi-dimensionality of evaluation of the self-driving automobile is improved; and the scene generation efficiency and the intelligent level are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving vehicles, and in particular to a scene extraction, generation and evaluation method for vehicle safety testing. Background Art

[0002] Traditional automobile testing considers a single interaction mode between the driver and the car. In the autonomous driving test scenario, the complex interaction between people, cars, and the environment makes the test scenarios and tasks more complex; in the simulation scenario test, there are too many test cases, and automation is needed to help improve the efficiency of scenario generation. With the advancement of artificial intelligence, the decision-making method of deep reinforcement learning can help discover the information shortage and insufficiency problems in the construction of scenario conditions. For example, the invention patent with publication number CN111144015A discloses a method for constructing a virtual scene library for autonomous driving vehicles. The method obtains logical scene data by clustering through an unsupervised learning method; builds a natural driving data acquisition system to collect relevant data, and extracts lane change and entry scene data from the collected natural driving data; then annotates the scene library elements to form a lane change and entry scene data set; uses the K-means algorithm to obtain K lane change and entry logical scenes; and constructs a virtual scene library based on the Prescan software. For example, the invention patent with publication number CN118067417A discloses a method and system for evaluating driving comfort based on an autonomous driving vehicle. The system method determines the basic information of the test scene, allowing the tester to build a test scene based on the basic information of the test scene; in the process of testing the comfort of the test scene, the tester's comfort data is collected based on the detection equipment, and then the factors affecting the driving comfort are determined by analyzing the comfort data.

[0003] At present, the existing test scene extraction and generation methods based on natural driving data mostly use manual annotation methods for element extraction, and mostly adopt manual construction in scene construction. The common clustering scene construction method based on feature indicators is highly abstract, and the overly direct discretization assignment loses too much information and cannot well reflect the coupling relationship between people, vehicles, and the environment. In summary, the existing evaluation system only evaluates the performance of autonomous driving vehicles themselves, and does not propose corresponding evaluation methods and indicators for scene generation methods; therefore, automated scene extraction, generation, and evaluation methods are extremely necessary and urgently needed. Summary of the invention

[0004] The purpose of the present invention is to provide a scene extraction, generation and evaluation method for vehicle safety testing. The scene extraction, generation and evaluation method adopts a semantic segmentation network to retain the scene element characteristics in the original natural driving data to the greatest extent, uses rough sets based on a high-level decision-making deep reinforcement learning algorithm to extract the association generation rules of typical dangerous scenes, and can automatically generate typical dangerous scenes in combination with simulation software.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: A scene extraction, generation and evaluation method for vehicle safety testing comprises the following steps: Step 1: Data processing module: The original data used is the natural driving data set; the video data in the natural driving data set is divided into frame-by-frame images; the road, traffic facilities, and traffic participant scene elements are extracted from each frame of the image based on the semantic segmentation network; the same element is tracked according to the timestamp, and the number of traffic participants, the spatial position relationship between traffic participants, and the mutual motion relationship feature indicators are extracted, and the working condition scenes are clustered based on the above feature indicators; Step 2: High-level decision module: The traffic rule monitor provides priority information about the priority relationship between the self-vehicle and other vehicles. The safety checker based on the responsibility-sensitive safety model further checks whether the self-vehicle and other vehicles violate the rules. Finally, the reward threshold and the corresponding dangerous action cluster are defined in the high-level decision module, and the association generation rules of typical dangerous scenarios are extracted using rough sets. Step 3: Scenario generation module: The scenario generation module includes parameterization of key elements of logical scenarios based on functional scenario description and joint generation of specific scenarios based on Prescan-based front-end interactive interface; Step 4: Scenario evaluation module: Based on the processed data and scenario variables, the autonomous driving evaluation dimensions include: safety, comfort, and expected functional safety indicators; the test scenario complexity evaluation dimensions include: scenario complexity and task complexity indicators; the specific scenarios generated above are evaluated in combination with various evaluation indicators and the evaluation results are generated.

[0006] The step 1 comprises the following sub-steps: Step 11: Scene element extraction based on semantic segmentation; Step 12: Feature extraction: Track the same element according to the timestamp, and extract the feature of the element from the spatial position change of the same element in the pictures of adjacent frames; Step 13: Clustering of working scene: Use the tomographic clustering method to cluster the scenes with timestamps and motion information to obtain working scenes; connect the image clusters of the tomographic clustering method into video blocks according to the timestamps.

[0007] The step 2 comprises the following sub-steps: Step 21: Constructing a traffic rule monitor: The introduced traffic rule monitor provides priority information including the priority relationship between the ego vehicle and other vehicles; the traffic rule monitor further checks whether the ego vehicle violates the rule; Step 22: Build a security checker model: Verify whether the current state is safe according to the RSS model; Step 23: Build a deep reinforcement learning DRL model: This mainly includes the following sub-steps: define the state space, action space and reward model, and perform the deep reinforcement learning process; Step 24: Generate a model based on threshold-based rules.

[0008] The step 3 comprises the following sub-steps: Step 31: Parameterization of logical scenarios based on functional scenario description: The clustered working scenario descriptions are used to obtain key elements or key categories at different levels; based on the understanding and description of the functional scenarios, each element in the logical scenario is characterized by assigning values ​​and types in a parameterized form, so as to realize the mapping of the elements and features of natural driving data to the simulation test scenario; Step 32: Based on the method of jointly generating scenes with the front-end interactive interface built on the Prescan platform, Vue.js (a JavaScript framework for building user interfaces) is used in JavaScript to build a front-end interface for joint simulation with Prescan; the front-end interface operation box mainly includes four parts: scene selection, element selection, parameter setting, action control and visualization interface.

[0009] The step 11 comprises the following sub-steps: Step 111: When extracting features, the semantic segmentation network first annotates different categories and elements in the image to generate data, and records the features in the image as one of the predefined categories, such as roads, traffic facilities, traffic participants, and weather, to generate a training data set; Step 112: Build a semantic segmentation model based on data, select the BiseNetV2 bilateral segmentation network as the neural network in the model, deploy the network, and use the generated training data set to train the model; Step 113: Output prediction results for each frame of the natural driving data through the deployed semantic segmentation model.

[0010] The step 24 comprises the following sub-steps: Step 241: define the scenes with the top 15% of the total reward value as typical dangerous scenes, output multiple action clusters within the threshold range as the operation rules of the typical dangerous scenes, and simultaneously retain the corresponding state space to generate a state-action rough set, thereby forming an association generation rule for the typical dangerous scenes; M=(U,At,{V a |a∈At},{I a |a∈At}) In the formula, U={S a ,a a}(a∈At) is the domain consisting of a finite set of objects, and is the input sample that meets the threshold; At is a finite non-empty attribute set consisting of state and action element features that meet the threshold conditions; V a Represents attribute a∈A t The attribute value range is the value domain of attribute a. a :U→V a It is an information function and an association rule.

[0011] The step 32 comprises the following sub-steps: Step 321: Scenario selection includes logic scheme selection and logic sub-scenario setting; the logic scheme selection is function-oriented, such as the intersection traffic function; the logic sub-scenario setting is based on the clustering results obtained from natural driving data, such as the traffic scene of the main road-branch road intersection without a signal light; Step 322: Element selection is to select important elements extracted based on natural driving data, including static scene elements, dynamic scene elements and meteorological elements; static scene elements can be quickly constructed using the element library of the Prescan scene editing interface; dynamic scene elements can be designed based on the JAVA language framework as the API (Application Programming Interface) of the front-end operation interface, so as to combine it with the API of Prescan to construct static scene elements; Step 323: The parameter setting and action control part is driven by the logical scenario parameterization; for the components that can be directly assigned, the specific parameter value can be directly entered in the input box to achieve accurate setting of the numerical parameter; for the elements that cannot be directly assigned, the category parameters can be set in a single selection; based on the high-level decision module, the scenario generation rules are obtained, the scenarios in the natural driving data are reproduced, and the edge danger scenarios that do not appear in the natural driving data are generated; Step 324: The visualization interface part uses visualization scenes and charts to demonstrate the automatic scene extraction and generation results; based on the parameterization of the logical scene, after the parameters are set in the front-end interface, Prescan will pop up and run the simulation to display the typical scene generated by the construction.

[0012] The beneficial effects of the present invention are as follows: compared with the existing vehicle test scene extraction and generation methods, the scene extraction, generation and evaluation method for vehicle safety testing provided by the present invention has the following advantages: The scene extraction, generation and evaluation method for vehicle safety testing of the present invention adopts a semantic segmentation network to retain the scene element features in the original natural driving data to the greatest extent, uses rough sets based on a high-level decision-making deep reinforcement learning algorithm to extract the association generation rules of typical dangerous scenes, and can automatically generate typical dangerous scenes in combination with simulation software, and proposes a certain design idea for automatically extracting scene features and generating them with reference to natural driving data; at the same time, it evaluates the overall intelligence of the autonomous driving car from two dimensions of natural driving evaluation and test scene complexity evaluation, and combines the advantages of key indicators in different aspects to improve the multidimensionality of the evaluation of the autonomous driving car; greatly improves the efficiency and intelligence level of scene generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a general framework diagram of the scene extraction, generation and evaluation method for vehicle safety testing of the present invention; Figure 2 This is the process of processing scene natural driving data by a semantic segmentation model in an embodiment of the present invention; Figure 3 This is a working condition scene diagram obtained by clustering in an embodiment of the present invention; Figure 4 It is a safety and rule perception framework of the DRL decision module of the autonomous driving vehicle in an embodiment of the present invention; Figure 5 Generating a test scenario for a scenario generation module in an embodiment of the present invention; Figure 6 An example scenario generated in an embodiment of the present invention; Figure 7 Framework diagram of the multi-dimensional evaluation module for autonomous driving scenarios in an embodiment of the present invention. DETAILED DESCRIPTION

[0014] Specific embodiment 1: The technical scheme of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention. It should be noted that: in the present invention, if there is no special description, all the embodiments and preferred implementation methods mentioned herein can be combined with each other to form a new technical scheme. In the present invention, if there is no special description, all the technical features and preferred features mentioned herein can be combined with each other to form a new technical scheme. The "scope" disclosed in the present invention is in the form of lower limit and upper limit, which can be one or more lower limits, and one or more upper limits, respectively. Preferably, the reaction method herein is carried out sequentially. Unless otherwise specified, the professional and scientific terms used herein have the same meaning as those familiar to those skilled in the art.

[0015] like Figure 1 As shown, the overall block diagram of a scene extraction, generation and evaluation method for vehicle safety testing provided by the present invention includes the following steps: Data processing module: The original data used is the natural driving data set; the video data in the natural driving data set is divided into frame-by-frame images; based on the semantic segmentation network, scene elements such as roads, traffic facilities, traffic participants (motor vehicles, non-motor vehicles, pedestrians, etc.) are extracted from each frame of the image; the same element is tracked according to the timestamp, and characteristic indicators such as the number of traffic participants, the spatial position relationship between traffic participants, and the mutual motion relationship are extracted, and the working condition scenes are clustered based on the above characteristic indicators; High-level decision-making module: A safety rule-aware deep reinforcement learning (DRL) method is proposed. The traffic rule monitor provides priority information including the priority relationship between the self-vehicle and other vehicles. The safety checker based on the responsibility-sensitive safety model further checks whether the self-vehicle and other vehicles violate the rule. Finally, the reward threshold and the corresponding dangerous action cluster are defined in the high-level decision-making module, and the association generation rules of typical dangerous scenarios are extracted using rough sets; Scenario generation module: This module includes parameterization of key elements of logical scenarios based on functional scenario descriptions and joint generation of specific scenarios based on the front-end interactive interface built based on Prescan; Scenario evaluation module: Based on the processed data and scenario variables mentioned above, the autonomous driving evaluation dimensions include: safety, comfort, and expected functional safety indicators; the test scenario complexity evaluation dimensions include: scenario complexity and task complexity indicators; the specific scenarios generated above are evaluated in combination with various evaluation indicators and evaluation results are generated.

[0016] Furthermore, the above data processing module includes the following sub-steps: scene element extraction, feature extraction and working condition scene clustering based on semantic segmentation; Scene feature extraction based on semantic segmentation: When extracting features, the semantic segmentation network first annotates the different categories and elements in the image to generate data, and records the features in the image as one of the predefined categories, such as roads, traffic facilities, traffic participants (motor vehicles (including other vehicles and the vehicle itself), non-motor vehicles, pedestrians, etc.), weather, etc., to generate a training data set. Build a semantic segmentation model with data as the guide, select the BiseNetV2 bilateral segmentation network as the neural network in the model, deploy the network, and use the generated training data set to train the model. Finally, the frame-by-frame images in the natural driving data are output with the prediction results of the deployed semantic segmentation model; in the feature extraction part, the trained neural network model is used to identify and extract the semantic labels and image data of common categories in the autonomous driving scene data set.

[0017] The process of configuring the network training model for semantic segmentation is as follows Figure 2 As shown, the programming software is used to configure the network model. The main focus in the yml file is first train_dataset: #Configuration of the training data set, where type selects the format of the data set, and dataset_root is the path to the data set; Tramsforms is a strategy for data processing, which can be changed here according to actual needs. The core is the model type, which refers to the neural network used. Num_classes is the number of types of training segmentation and recognition, and Iters is the number of training iterations, which can be changed according to the training effect; according to the above precautions, use the programming software to create a my yml data sequence file. When configuring the neural network model, select 1000 for the Iters training iteration number, select the created Dataset data file path for data_root, and select the BiseNetV2 bilateral segmentation network for the model neural network model; After the neural network model configuration is completed, start training. Run the python language in the folder and execute the code python train.py--config my.yml--do_eval\--use_vdl--save_interval 200--save_dir output The neural network model configured in the above code is in the config my.yml file. use_vdl is the use of the training detection tool, which can view indicators such as accuracy and number of iterations in training. save_dir output means saving the trained parameters in the created output file; this completes the training of the neural network model. Then, the data processed in the dataset is input into the trained neural network model for semantic segmentation recognition, and the scene elements are recognized and extracted by using the semantic segmentation network; feature extraction: the same element is tracked according to the timestamp, and the feature of the element is extracted from the spatial position change of the same element in the picture of the adjacent frame. The features of road elements include road grade, road intersection mode, number of lanes, and lane width. The features of traffic facilities include whether there are traffic lights, traffic light cycle, traffic light phase, whether there are traffic signs, traffic sign types, whether there are traffic markings, and traffic marking types. Other vehicle characteristics include other vehicle speed, other vehicle acceleration / deceleration, distance between other vehicle and main vehicle, other vehicle type, and other vehicle driving direction. Main vehicle characteristics include main vehicle speed, main vehicle acceleration / deceleration, main vehicle driving direction, driving lane position, and distance between main vehicle and starting point. Non-motor vehicle / pedestrian characteristics include non-motor vehicle / pedestrian speed, non-motor vehicle / pedestrian acceleration / deceleration, non-motor vehicle / pedestrian stay time at a certain point, and pedestrian type. Meteorological information includes weather; Working condition scene clustering: The scene is clustered using the tomographic clustering method to ensure that the timestamps in each category of images are continuous after clustering. At the beginning, each frame of the image is considered as a category as a sample. If there are n samples, then n categories are considered; then, two categories with similar distances are combined into one category, so that n-1 categories are obtained; then the previous method is repeated, and the distance between the newly generated category and another similar category is recalculated, and the two categories with similar distances are combined into one category, so that n-2 categories are obtained; then the previous method is repeated, and the distance between the newly generated n-2 category and another category is recalculated... This algorithm will continuously calculate the distance between categories and combine them until all samples with similar features are combined into one category. Euclidean distance is used in the calculation of sample distance. Because the features used for clustering are derived from feature recognition based on semantic segmentation, with timestamps and working condition information, the clustering results can obtain working condition scenes, as shown in the attached manual. Figure 3 The clustering results of the tomographic clustering are connected into video blocks according to the timestamps. The scenes in the video blocks are simple and the functions are single, which is convenient for subsequent rule generation and extraction.

[0018] Furthermore, the high-level decision-making module includes the following sub-steps: building a traffic rule monitor, a safety checker, a deep reinforcement model, and a threshold-based rule generation model.

[0019] As the instruction manual Figure 4 As shown in the figure, it is the framework of the DRL decision module of the autonomous vehicle. The task of the DRL agent is to learn the correct behavior actions through the vehicle state input of natural driving data and the feedback of the reward mechanism, and output them as driving rules, which are subsequently introduced into the scene generation.

[0020] Constructing a traffic rule monitor: The introduced traffic rule monitor provides priority information including the priority relationship between the ego vehicle and other vehicles; the traffic rule monitor further checks whether the ego vehicle violates the rule. priority and monitoring traffic regulations violations vi The results of olation are further merged into the state-state space (s extend,t ) and the reward function for training the DRL agent. For each intersection, a conflict region C is first defined as the area surrounded by the stop line. In order to determine whether the ego vehicle violates the priority rule, only vehicles that meet the following priority conditions are considered: Where d c,i is the Euclidean distance from vehicle i to the conflict area, v i is its speed. Define the arrival time threshold as T a =3s, monitoring range is D m =30m; If the ego vehicle has geometric overlap with the conflict region, the traffic rule monitor returns y violation , and if at least one vehicle i meets the condition, then condition y priority =True is true.

[0021] Construct a safety checker model: The safety checker verifies whether the current state is safe according to the RSS (Responsibility Sensitive Safety) model; given the agent's operation a agent , the security checker returns a tuple (y unsafe ,a safe ), where y unsafe is a Boolean value that indicates whether the current state is unsafe, and a safe is a safe fallback action. If the DRL agent does not choose a braking action in an unsafe situation, the safety checker interferes with the DRL agent's decision and provides a safe fallback action a safe .

[0022] Building a deep reinforcement learning (DRL) model: This mainly includes the following sub-steps: defining the state space, action space, and reward model, and executing the deep reinforcement learning process. Define the state space in deep reinforcement learning. The state representation of the DRL agent is a list of features containing information about the ego vehicle and other road users, including the five other vehicles closest to the intersection conflict area in terms of Euclidean distance. The state (state space) representation with priority information at time step t is defined as: s extend,t =[s ego ,s veh,1 ,s veh,2 ,s veh,3 ,s veh,4 ,s veh,5 ] The state space of the ego vehicle. The state of the ego vehicle is defined as s ego =[v ego ,d goal ,d conflict ], including the vehicle's speed v ego , the distance to the target d goal , the distance to the conflict area d conflict . ; Other car state space: s veh,i =[v i ,d c,i ,y priority,i ](i=1,2,3,4,5) includes the vehicle's speed v i , the distance to the conflict area d c,i , and priority information y priority,iWhen the vehicle leaves the center of the intersection along the reference driving path, give d c,i If vehicle i has a higher priority than the host vehicle, the priority information y priority,i is set to True. Defining the action space of DRL: The task of the DRL agent is to make high-level decisions to drive the autonomous vehicle through an intersection or driving scenario. There are three discrete actions: driving a drive , be cautious cautious and brakes brake Represents a high-level decision action.

[0023] Define the reward model: It includes the collision penalty for whether the vehicle collides, the penalty for unsafe situations identified by the safety checker, the efficiency positive and negative rewards for completing a training cycle, and the rule reward from the traffic rule monitor for violating the above right-of-way rules; R=R collision +R unsafe +R efficiency +R rule ; R collision is the collision penalty. If a collision is detected, the maximum allowed episode length is reached (timeout), or the ego vehicle reaches the goal, the training of each episode ends. If the ego vehicle collides with other vehicles, a penalty is imposed based on the speed of the ego vehicle, which is assigned a value of R collision =-2v ego -5; R unsafe In a given situation, if the ego vehicle is in an unsafe situation y identified by the safety checker unsafe = True, the penalty assigned at each time step is R unsafe =-0.1; efficiency R efficiency is used to encourage the ego carrier to complete each episode as quickly as possible. Therefore, for each time step of the training set, a negative reward R is given efficiency =-0.1. rule is based on the inspection results from the traffic regulations monitor violation For each time step, when the ego vehicle is waiting behind the stop line to give way to a vehicle with higher priority, it receives a positive reward R rule,co = 0.1. In this case, i.e. when the ego vehicle complies with the driving rules, there is no penalty for efficiency and the efficiency reward is reset to R efficiency =0.

[0024] The decision-making process of deep reinforcement learning: can be modeled as a Markov decision process defined by a tuple (γS, a, T, R, γ), where S is the state space, a is the action space, T is the state transition model that describes the probability of transition from one state to another, R is the reward model, and γ∈[0,1] is the discount factor for calculating the cumulative expected reward. According to the transition model T, the environment transitions to a new state S0∈S with probability T(S, a) = Pr(S, a) and assigns a reward r = r(S, a) to the agent. The goal of the agent is to find a strategy π: S→A that maximizes the cumulative expected reward by determining the action A∈A that should be taken given the state S∈S. The model-free reinforcement learning algorithm Q-learning uses the state-action value function Q(s, a) to represent the strategy π, where the optimal Q function Q * (s,a) satisfies the Bellman equation: In DRL, the state-action value function is represented by a neural network with weights θ. The approximate solution to the above equation can be obtained by minimizing the following loss function: L(θ)=E s′ [(r+γ a′ axQ(s′,a′;θ)-Q(s,a;θ)) 2 ] Threshold-based rule generation model: define the scenarios with the top 15% of total reward values ​​as typical dangerous scenarios, and output multiple action clusters a within the threshold range final As the operation rules of typical dangerous scenarios, the corresponding state space is retained synchronously to generate state-action rough sets, and then form the association generation rules of typical dangerous scenarios; M=(U,At,{V a |a∈At},{I a |a∈At}); where U={s a ,a a}(a∈At) is the domain composed of a finite set of objects, which is the input sample that meets the threshold; At is a finite non-empty attribute set composed of state and action element features that meet the threshold conditions; V a Represents attribute a∈A t The attribute value range is the value domain of attribute a. a :U→V a is an information function and an association rule.

[0025] As the instruction manual Figure 5 As shown, it is a specific scene generation block diagram of the scene generation module, and the scene generation module includes the following sub-steps: logical scene parameterization based on functional scene description; front-end interface interacting with Prescan; Logical scenario parameterization based on functional scenario description. The description of the functional scenario can be understood as a combination of different positions and behaviors of the vehicle and other traffic participants under certain road topology and traffic signal conditions. In the construction of the logical scenario based on the functional scenario description, the above clustered working condition scenarios are described in the functional scenario to obtain key elements or key categories at different levels in the scenario. Based on the understanding and description of the functional scenario, the key information of each element class in the logical scenario is then mapped to the corresponding parameters or parameter option types, so as to represent each logical scenario in a parameterized form. This step realizes the mapping of the elements and features of natural driving data to the simulation test scenario. The information that can be directly assigned to each element is directly expressed in digital form, such as the speed, acceleration, deceleration, distance, etc. in the vehicle; the speed, acceleration, or the time that the pedestrian stays at a certain location. The information elements that cannot be assigned can be classified according to their definition and function, such as the vehicle type in the vehicle category can be divided into: sedan, bus or truck; the steering action of the vehicle can be divided into three directions: left turn, straight, and right turn; the pedestrian type can be: adult, child; the meteorological category can be divided into three categories: sunny, rainy, and snowy.

[0026] The parameterization of the key elements of the logical scenario based on the functional scenario is shown in the following table. The above method is used to represent the element behavior actions in the scenario in a parameterized form. When generating a specific scenario next, you only need to determine the specific parameters of the scenario to generate a complete specific scenario; Based on the method of jointly generating scenes of the front-end interactive interface built on the Prescan platform, an application program interface of the operation interface is built through Java programming software, so that it can be jointly simulated with the Prescan software. The core is to use Vue.js (JavaScript framework for building user interfaces) in JavaScript to build the front-end interface; its front-end interface operation box mainly includes four parts: scene selection, element selection, parameter setting, action control and visualization interface.

[0027] Scenario selection includes logical scheme selection and logical sub-scenario setting: logical scheme selection is function-oriented, such as intersection traffic function; logical sub-scenario setting is based on clustering results obtained from natural driving data, such as the traffic scenario of main road-branch road intersection without signal lights; Element selection is to select important elements extracted from natural driving data, including static scene elements, dynamic scene elements and meteorological elements. For static scene elements, you can use the element library of the Prescan scene editing interface to quickly build static scenes of different needs. For dynamic scene elements, you can use the web interface designed based on the JAVA language framework as the API (Application Programming Interface) of the front-end operation interface, that is, the application programming interface, to combine it with the API of prescan to build static scene elements.

[0028] The parameter setting and action control parts are driven by logical scenario parameterization: for components that can be directly assigned values, specific parameter values ​​can be directly entered in the input box to achieve accurate setting of numerical parameters; for elements that cannot be directly assigned values, category parameters can be set with a single selection; based on the high-level decision-making module, the scene generation rules are obtained, which makes it possible not only to reproduce the scenes in natural driving data, but also to generate marginal danger scenes that do not appear in natural driving data.

[0029] The visualization interface uses visualization scenes and charts to demonstrate the results of automatic scene extraction: Based on the parameterization of the logical scene, after setting the parameters in the front-end interface, Prescan will pop up and run the simulation to display the typical scene generated. Figure 6 As shown: For example, a turning scenario is generated in a sunny day at a crossroad without a traffic light. The main vehicle turns left through the intersection, and other vehicles go straight or turn left through the intersection. The logical scenario can be represented by the following parameters: In the formula, HV is the main vehicle, and RV is the two participating vehicles: α speed and α acceleration velocity and acceleration numerical variables respectively; α turning is the direction of travel, with values ​​of 1 / 2 / 3 representing right turn, straight ahead, and left turn respectively; β intersection Indicates the type of intersection, with a value of 1 representing a cross intersection; β light Indicates the type of traffic light, with values ​​of 1 / 2 indicating whether there is a traffic light or not; β weather Indicates the weather, with values ​​of 1 / 2 / 3 representing sunny, rainy, and snowy days respectively; Constructing the intersection turning function scenario After parameterizing the above logical scenario, you only need to select the corresponding scenario parameters on the constructed front-end interface: intersection and traffic light type, main vehicle / other vehicle speed, acceleration, forward direction and weather conditions to generate a complete specific scenario.

[0030] As the instruction manual Figure 7As shown, it is a multi-dimensional evaluation module for autonomous driving scenarios, and the multi-dimensional evaluation module for scenarios includes the following sub-steps: autonomous driving driving evaluation and test scenario complexity evaluation; The evaluation indicators of the autonomous driving driving dimension include: safety, comfort, and expected functional safety, all of which are selected from specific scenarios generated by simulation; Safety indicators can be quantified by referring to some indicators between the autonomous driving target vehicle and other traffic participants, non-motor vehicles and pedestrians. For example, the above-mentioned dangerous scenarios can be characterized by indicators such as headway (THW) and time to collision (TTC). Safety indicators are selected according to the specific scenarios of the test case, as shown in the following table: Scenario Use Case Safety indicators Following car scene Headway THWW Cut to the scene Time to Collision TTC Lane change scenario Time to Collision TTC Pedestrians crossing <![CDATA[Collision probability or safety Field indicators > Taking the intersection turning scenario after the above scenario cluster analysis and screening as an example, the safety indicator collision time TTC=0s is selected; the comfort index is a factor that affects the driver's viewing and feeling, and the lateral acceleration and longitudinal acceleration of the car and the set of their derivatives can be considered.

[0031] The indicator of expected functional safety is the relative driving safety index of the autonomous driving car during driving; therefore, combining the three evaluation indicators of the autonomous driving driving dimension, there is a calculation formula: In the formula, AD is the quantitative result of the evaluation index of the autonomous driving dimension (safety, comfort, and expected functional safety), and C is the weight coefficient of the autonomous driving evaluation, which takes values ​​of 0 and 1. x (x=1, 2, 3) are the evaluation results of the three dimensions in the autonomous driving evaluation, N Mx (x=1, 2, 3) is the weight coefficient of the three dimensions of driving quality, ranging from 0 to 1; the test scenario complexity evaluation indicators include scenario complexity and task complexity. In actual scenario testing and evaluation, the test scenario affects most driving tasks. Therefore, before the final evaluation of the intelligence of the autonomous driving vehicle, scenario complexity and task complexity are also used as the main indicators to measure the intelligence of the former.

[0032] In the quantification of scene complexity, the scene complexity calculation of the above hazardous conditions is considered to be divided into static complexity and dynamic complexity. In the calculation of static complexity, the type and weight of static scene elements are considered. It is mainly to determine the static scene elements in the scene and give corresponding preset scores to different elements. According to the weights and preset scores of different static environmental elements, the static scene complexity formula is: S0=θ1×(γ1∑K1+γ2∑K2+γ3∑K3); where: S0 is the complexity of the static scene; γ n The weight corresponding to the nth group of static scene components; K nIt is the sum of the preset scores corresponding to the nth group of scene components in the static scene components. Where θ1 is the static complexity coefficient, and its calculation formula is: Where: m is the total number of types of grouping labels corresponding to each scene component; p x It is the ratio of the same type of feature variables to the total number of feature variables in the static scene complexity.

[0033] The complexity of dynamic scenes is related to the dynamic factors in the scene: the main dynamic factors include the ego vehicle and other traffic participants. The influence of dynamic factors is regarded as one of the important influencing mechanisms of environmental complexity. The distance and relative speed between the ego vehicle and other traffic participants are used as indicators affecting vehicle attributes. The formula for dynamic complexity is: S1=s·A1·A2; where A1 is the attribute value of the test vehicle (the vehicle itself); A2 is the attribute value of other traffic participants; s is a constant. The calculation formula for the attribute value of the test vehicle (the vehicle itself) is: Where: c is a constant; t1 is the reaction time of the test vehicle (ego vehicle); v1 is the driving speed of the test vehicle; d1 is the distance between the ego vehicle and other traffic participants; θ is the angle between the ego vehicle and other traffic participants; the calculation formula for the attribute values ​​of other traffic participants is: Where: k is a constant; t2 is the time when other traffic participants are perceived; v2 is the speed of other traffic participants; d2 is the movement distance of other traffic participants. Therefore, according to the above static complexity and dynamic complexity, the test scene complexity is calculated as follows: S = S0·S1; where S is the complexity of the test scenario. The complexity of the driving task is mainly to quantify the indicators of the three aspects of the perception detection task, the driving planning task, and the driving action control task. The calculation formula of the task complexity is: TC=P·D P=[i1,...,i n1 ,K1,...,K n2 ,Gr1,...,Gr n3 ] D=[D1,...,D (n1+n2+n3) ] T Where TC is the task complexity result, P is the characteristic vector of the three indicators of task complexity mentioned above, and D is the weight of the driving task element.

[0034] Constructing a scene evaluation module: Based on the above-mentioned autonomous driving evaluation and test scene complexity evaluation, the evaluation calculation method of each index is written into the Java language, and then an evaluation front-end interface is constructed through Vue.js. The front-end interface mainly includes the selection of the above-mentioned evaluation indexes. After the specific scene is generated, when the test evaluation is carried out, the data source and the selection of the evaluation index are completed in sequence. In the scene evaluation module, the back end calls the data according to the result selected by the front end to perform the performance evaluation analysis of the autonomous driving scene. After completion, the scene evaluation module will transmit the result to the front-end display interface, and can choose to store the result in the local corresponding generated scene file. The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes by using the above-disclosed technical content without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still belongs to the scope of the technical solution of the present invention.

Claims

1. A scene extraction, generation and evaluation method for vehicle safety testing, characterized in that: The following steps are involved: Step 1: Data processing module: The raw data used is the natural driving dataset; The video data in the natural driving dataset is divided into frame-by-frame images; Based on the semantic segmentation network, the scene elements of roads, traffic facilities, and traffic participants are extracted from each frame of the image; the same element is tracked according to the timestamp, and the characteristic indicators of the number of traffic participants, the spatial position relationship between traffic participants, and the mutual motion relationship are extracted, and the working condition scenes are clustered based on the above characteristic indicators; Step 2: High-level decision module: The traffic rule monitor provides priority information about the priority relationship between the self-vehicle and other vehicles. The safety checker based on the responsibility-sensitive safety model further checks whether the self-vehicle and other vehicles violate the rules. Finally, the reward threshold and the corresponding dangerous action cluster are defined in the high-level decision module, and the association generation rules of typical dangerous scenarios are extracted using rough sets. Step 3: Scenario generation module: The scenario generation module includes parameterization of key elements of logical scenarios based on functional scenario description and joint generation of specific scenarios based on Prescan-based front-end interactive interface; Step 4: Scenario evaluation module: Based on the processed data and scenario variables, the autonomous driving evaluation dimensions include: safety, comfort, and expected functional safety indicators; the test scenario complexity evaluation dimensions include: scenario complexity and task complexity indicators; the specific scenarios generated above are evaluated in combination with various evaluation indicators and the evaluation results are generated.

2. A scene extraction, generation and evaluation method for vehicle safety testing according to claim 1, characterized in that: The step 1 comprises the following sub-steps: Step 11: Scene element extraction based on semantic segmentation; Step 12: Feature extraction: Track the same element according to the timestamp, and extract the feature of the element from the spatial position change of the same element in the pictures of adjacent frames; Step 13: Clustering of working scene: Use the tomographic clustering method to cluster the scenes with timestamps and motion information to obtain working scenes; connect the image clusters of the tomographic clustering method into video blocks according to the timestamps.

3. A scene extraction, generation and evaluation method for vehicle safety testing according to claim 2, characterized in that: The step 2 comprises the following sub-steps: Step 21: Constructing a traffic rule monitor: The introduced traffic rule monitor provides priority information including the priority relationship between the ego vehicle and other vehicles; the traffic rule monitor further checks whether the ego vehicle violates the rule; Step 22: Build a security checker model: Verify whether the current state is safe according to the RSS model; Step 23: Build a deep reinforcement learning DRL model: This mainly includes the following sub-steps: define the state space, action space and reward model, and perform the deep reinforcement learning process; Step 24: Generate a model based on threshold-based rules.

4. A scene extraction, generation and evaluation method for vehicle safety testing according to claim 3, characterized in that: The step 3 comprises the following sub-steps: Step 31: Logical scenario parameterization based on functional scenario description: The clustered working scenario descriptions are used to obtain key elements or key categories at different levels; Based on the understanding and description of functional scenarios, each element in the logical scenario is represented by assigning values ​​and types in a parameterized form, so as to map the elements and features of natural driving data to simulation test scenarios; Step 32: Based on the method of jointly generating scenes with the front-end interactive interface built on the Prescan platform, Vue.js (a JavaScript framework for building user interfaces) is used in JavaScript to build a front-end interface for joint simulation with Prescan; the front-end interface operation box mainly includes four parts: scene selection, element selection, parameter setting, action control and visualization interface.

5. A scene extraction, generation and evaluation method for vehicle safety testing according to claim 2, characterized in that: The step 11 comprises the following sub-steps: Step 111: When extracting features, the semantic segmentation network first annotates different categories and elements in the image to generate data, and records the features in the image as one of the predefined categories, such as roads, traffic facilities, traffic participants, and weather, to generate a training data set; Step 112: Build a semantic segmentation model based on data, select the BiseNetV2 bilateral segmentation network as the neural network in the model, deploy the network, and use the generated training data set to train the model; Step 113: Output prediction results for each frame of the natural driving data through the deployed semantic segmentation model.

6. A scene extraction, generation and evaluation method for vehicle safety testing according to claim 3, characterized in that: The step 24 comprises the following sub-steps: Step 241: define the scenes with the top 15% of the total reward value as typical dangerous scenes, output multiple action clusters within the threshold range as the operation rules of the typical dangerous scenes, and simultaneously retain the corresponding state space to generate a state-action rough set, thereby forming an association generation rule for the typical dangerous scenes; M=(U,At,{V a |a∈At},{I a |a∈At}) In the formula, U={S a ,a a }(a∈At) is the domain consisting of a finite set of objects, and is the input sample that meets the threshold; A t is a finite non-empty attribute set consisting of state and action element features that meet the threshold conditions; V a Represents attribute a∈A t The attribute value range is the value domain of attribute a. a :U→V a is an information function and an association rule.

7. A scene extraction, generation and evaluation method for vehicle safety testing according to claim 4, characterized in that: The step 32 comprises the following sub-steps: Step 321: Scenario selection includes logic scheme selection and logic sub-scenario setting; the logic scheme selection is function-oriented, such as the intersection traffic function; the logic sub-scenario setting is based on the clustering results obtained from natural driving data, such as the traffic scene of the main road-branch road intersection without a signal light; Step 322: element selection is to select important elements extracted based on natural driving data, including static scene elements, dynamic scene elements and meteorological elements; Static scene elements can be quickly constructed using the element library in the Prescan scene editing interface; Dynamic scene elements can be designed based on the JAVA language framework web interface as the front-end operation interface API (Application Programming Interface), that is, the application programming interface, so that it can be combined with the API of prescan to build static scene elements; Step 323: The parameter setting and action control part is driven by the logic scene parameterization; for the components that can be directly assigned, the specific parameter value can be directly entered in the input box to achieve accurate setting of the numerical parameter; for the elements that cannot be directly assigned, the category parameters can be set by a single selection; Based on the high-level decision module, the scene generation rules are obtained to reproduce the scenes in the natural driving data and generate edge danger scenes that do not appear in the natural driving data; Step 324: The visualization interface part displays the scene automatic extraction generation result with visualization scenes and chart demonstrations; Based on the parameterization of the logical scenario, after setting the parameters in the front-end interface, Prescan will pop up and run the simulation to display the typical scenario generated by the build.

Images