Evaluation method for automatic driving road passing regulation conformity test scene

Abstract

The invention discloses an evaluation method for an automatic driving road passing regulation conformance test scene. The evaluation method comprises the steps that a seven-layer logic scene framework with scene elements as minimum units is constructed; obtaining a scene set based on the seven-layer logic scene architecture; wherein the seven-layer logic scene architecture comprises a road layer, a traffic infrastructure layer, a road and traffic facility temporary event layer, a traffic participant layer, an environment layer, a communication state layer and a self-vehicle state layer; and constructing a multi-dimensional test scene evaluation model, extracting scene elements of each test scene, and inputting the scene elements into the multi-dimensional test scene evaluation model to obtain a comprehensive evaluation result of each test scene. According to the method, scene elements are taken as units, four dimensions including true rationality, law and regulation coverage, test suitability and danger complexity and 15 indexes are covered, a comprehensive and effective test scene can be screened out from massive scenes, compliance and safety of an automatic driving system are ensured, and development of the technology in a legal framework is promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and particularly relates to an evaluation method for a compliance test scenario of autonomous driving road traffic regulations. Background Art

[0002] With the development of technology, more and more autonomous driving vehicles are being tested and even put into trial operation on open roads. However, the road traffic environment is complex and changeable, with pedestrians, non-motor vehicles, and traditional motor vehicles mixed together, and the road conditions include various scenarios such as urban congested roads, highways, and rural paths. For autonomous driving vehicles to safely and smoothly integrate into it, they must strictly follow the existing road traffic regulations. When conducting compliance tests on road traffic regulations, there are problems such as the long-tail effect of test scenarios. In addition to common traffic scenarios, there are also a large number of low-frequency, rare but extremely dangerous situations, and it is difficult to discover and cover all scenarios through traditional test methods. In addition, the number of autonomous driving test scenarios will increase exponentially with the combination of various factors, which requires scheduling a large amount of test resources, undoubtedly exacerbating the test difficulty. To achieve a comprehensive evaluation of autonomous driving vehicles in terms of compliance tests for road traffic regulations, it is necessary to construct an evaluation method for relevant test scenarios to ensure that the test scenarios comply with regulations, are effective, and truly reflect the ability of autonomous driving to abide by traffic rules.

[0003] In actual tests, too many scenarios will lead to problems such as low test efficiency and long test cycles. Currently, for different autonomous driving vehicle test scenarios, especially compliance test scenarios for road traffic regulations, there is a lack of a clear evaluation method. The existing scenario evaluation methods mainly focus on simulation tests and still have deficiencies in terms of authenticity evaluation; second, the industry's evaluation of scenarios mainly focuses on complexity and risk, lacking consideration for compliance verification; third, the existing evaluation methods tend to be subjective evaluations, lacking specific objective evaluation methods and evaluation systems. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes an evaluation method for a compliance test scenario of autonomous driving road traffic regulations to solve the problems existing in the above prior art.

[0005] To achieve the above object, the present invention provides an evaluation method for a compliance test scenario of autonomous driving road traffic regulations, including:

[0006] Constructing a seven-layer logical scenario architecture with scenario elements as the smallest unit; obtaining a scenario set based on the seven-layer logical scenario architecture; wherein, the seven-layer logical scenario architecture includes a road layer, a traffic infrastructure layer, a temporary event layer of roads and traffic facilities, a traffic participant layer, an environment layer, a communication status layer, and a self-vehicle status layer;

[0007] Build a multi-dimensional test scenario evaluation model, extract the scenario elements of each test scenario, and input them into the multi-dimensional test scenario evaluation model to obtain the comprehensive evaluation results of each test scenario.

[0008] Optionally, each layer of the seven-layer logic scenario architecture includes several types of attributes, and each type of attribute includes several scenario elements.

[0009] Optionally, the scenario set is the Cartesian product of different scenario elements in different attributes.

[0010] Optionally, the multi-dimensional test scenario evaluation model includes an evaluation index value calculation layer and a comprehensive calculation layer.

[0011] Optionally, the evaluation index value calculation layer calculates the evaluation index values from four dimensions: real rationality, regulation coverage, test adaptability, and danger complexity;

[0012] The real rationality dimension includes the real environment restoration degree, scenario parameter rationality degree, driving behavior simulation accuracy, and meteorological simulation accuracy;

[0013] The regulation coverage dimension includes the violation degree of traffic participants, regulation clause coverage, driving task matching degree, and regulation test frequency;

[0014] The test adaptability dimension includes the test target achievement degree, function test pertinence, and boundary condition coverage;

[0015] The danger complexity dimension includes the richness of scenario elements, the change degree of scenario elements, the traffic flow speed change rate, and the traffic flow complexity.

[0016] Optionally, the comprehensive calculation layer assigns weights to each evaluation index, and performs weighted summation based on each evaluation index value and the corresponding weight value to obtain the comprehensive evaluation result of the test scenario, where the weighted summation calculation is performed after data standardization of each evaluation index value.

[0017] Optionally, the process of assigning weights to each evaluation index includes:

[0018] Obtain the average expert score of each evaluation index, perform normalization processing on the average expert scores of all evaluation indexes to obtain the subjective weight of each evaluation index; use the principal component analysis method to obtain the objective weight of each evaluation index; based on the subjective weight and the objective weight, use the Lagrangian function to obtain the combined weight of each evaluation index.

[0019] Optionally, the process of obtaining the objective weight of each evaluation index includes:

[0020] Standardize the numerical values of each evaluation index for several scenarios; calculate the covariance matrix between the evaluation indexes based on the standardized numerical values, perform eigenvalue decomposition on the covariance matrix, and calculate the eigenvalues and corresponding eigenvectors; select the eigenvectors corresponding to the top k largest eigenvalues as the principal components; regress the principal components onto the original evaluation indexes to obtain the objective weights of each evaluation index.

[0021] Compared with the prior art, the present invention has the following advantages and technical effects:

[0022] The present invention proposes a comprehensive model and evaluation method constructed with scenario elements as the smallest unit, including 4 dimensions and 15 indicators, for quantitatively analyzing and evaluating the compliance test scenarios of autonomous driving vehicle road traffic regulations. For the exploding scenario library, it helps to screen out test scenarios that take into account four aspects: real rationality, regulation coverage, test adaptability, and danger complexity, and can comprehensively and effectively evaluate the compliance and safety of autonomous driving systems. The present invention comprehensively considers multiple key fields from actual driving environment simulation to laws and regulations, then to test resource utilization efficiency and system safety performance limit testing, avoiding the limitations of single-dimensional evaluation, and thus more accurately evaluating whether the test scenarios can comprehensively cover various situations that the autonomous driving system may face. In addition, fully considering the requirements of laws and regulations in the test scenarios can make the autonomous driving system pay more attention to compliance verification during the development process, and promote the healthy development of autonomous driving technology within the legal framework. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0024] Figure 1 It is a schematic diagram of the scenario architecture of an embodiment of the present invention;

[0025] Figure 2 It is a schematic diagram of four dimensions for evaluating the road traffic regulation compliance test scenarios of an embodiment of the present invention;

[0026] Figure 3 It is a schematic diagram of the application process of the road traffic regulation compliance test scenario evaluation method of an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will refer to the drawings and combine the embodiments to detail this application.

[0028] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0029] Embodiment 1

[0030] As Figures 1-3 shown, in this embodiment, an evaluation method for an autonomous driving road traffic regulation compliance test scenario is provided, including:

[0031] Construct a seven-layer logical scenario architecture with scenario elements as the smallest unit; obtain a scenario set based on the seven-layer logical scenario architecture; wherein, the seven-layer logical scenario architecture includes a Road Layer Layer1, a Traffic Infrastructure Layer Layer2, a Road and Traffic Facility Temporary Event Layer Layer3, a Traffic Participant Layer Layer4, an Environment Layer Layer5, a Communication Status Layer Layer6, and a Self-Vehicle Status Layer Layer7;

[0032] Furthermore, each layer architecture in the seven-layer logical scenario architecture includes several types of attributes, and each type of attribute includes several types of scenario elements.

[0033] Furthermore, the scenario set is the Cartesian product of different scenario elements in different attributes.

[0034] Layer1 includes 3 types of attributes: road attributes, lane attributes, and appurtenance attributes. Road attributes include road grade (expressway, first-class and second-class highways, etc.), road status (whether there is ice or water accumulation, whether there are potholes or faults, etc.), road type (main road, secondary road, access road), road geometry (curvature, length, slope). Lane attributes include the number of lanes (two lanes, one lane, etc.), lane type (motor vehicle lane, non-motor vehicle lane, etc.), lane topology (parallel lanes, turning lanes, and intersection lanes, etc.), lane geometry (width, direction, length, etc.), lane line color (yellow, white), lane line type (solid line, dotted line, solid and dotted line, etc.). Appurtenance attributes include appurtenance type (central median, speed bump, etc.), appurtenance location (starting position, ending position).

[0035] Layer 2 includes three types of attributes: signal light attributes, sign attributes, and road marking attributes. Signal light attributes include signal light type (motor vehicle signal light, lane direction indicator, crosswalk signal light, etc.), signal light geometry (position, height, orientation), signal light status (constant on, flashing, off), and signal light phase (red, yellow, green). Sign attributes include sign type (warning, prohibition, indication, etc.), sign geometry (position, height, orientation), and sign content (such as speed limit 60, 70, 80). Road marking attributes include road marking type (lane line, crosswalk line, guiding line, etc.), road marking geometry (position, size, orientation), and road marking form (whether it is blurred, color, etc.).

[0036] Layer 3 includes three types of attributes: road change attributes, lane change attributes, and environmental change attributes. Road change attributes include change type (road curvature change, road slope change, road covering change, etc.), change location (starting point, ending point), change status (curvature value, slope value, covering type), and change frequency (curvature change frequency, slope change frequency, covering change frequency, etc.). Lane change attributes include change type (lane type change, lane color change, lane width change, lane number change, lane center line offset, occupation of lane, etc.), change location (starting point, ending point), change status (type, color, width, number, offset amount, etc. after change), and change frequency (type change frequency, color change frequency, number change frequency, etc.). Environmental change attributes include change type (weather change, visibility change, temperature change, light change, etc.), change location (starting point, ending point), and change status (sunny or rainy, high or low temperature, strong or weak light, etc.).

[0037] Layer 4 includes three types of attributes: vehicle attributes, person attributes, and other attributes. Vehicle attributes include single vehicle information (type, spacing, speed) and traffic flow information (congestion level, traffic flow range). Person attributes include pedestrian information (spacing, speed, orientation), crowd information (position, density), and traffic police information (position, gesture). Other attributes include living beings (animal movement state) and inanimate objects (obstacle position).

[0038] Layer 5 includes three types of attributes: meteorological attributes, lighting attributes, and other attributes. Meteorological attributes include weather conditions (rain, snow, fog), temperature, wind speed, and visibility. Lighting attributes include lighting type (natural light, artificial light), lighting position, and lighting intensity. Other attributes include shadow interference (type, position) and electromagnetic interference (source, position).

[0039] Layer 6 includes three types of attributes: roadside unit, edge computing unit, and positioning unit. The roadside unit includes location, communication type (DSRC, 4G, etc.), sensor type (camera, lidar, etc.), and parameters (radius, angle, direction, intensity, etc.). The edge computing unit includes location, communication type, and parameters. The positioning unit includes location, communication type, device type (GPS, high-precision map, etc.), and parameters (computing speed, signal strength, transmission rate, etc.).

[0040] Layer 7 includes two types of attributes: dynamic driving task and component information. The dynamic driving task includes type (lane change, overtaking, avoidance, etc.), completion time, and completion quality (number of attempts, number of stops). The component information includes component type (headlight, windshield wiper, etc.) and component status (on, off).

[0041] Filter the attributes property (m and scene elements element (mn , which can form a set S of scenes. Each scene in the set is represented as S + , which is composed of different attributes of the seven-layer architecture and scene elements combined.

[0042] S = {S +};

[0043] S + = {∑Layer ( , 1 ≤ j ≤ 7};

[0044] Layer ( = {∑property (m , 1 ≤ m ≤ M};

[0045] property (m = {∑element (mn , 1 ≤ n ≤ N};

[0046] where m is the number of attributes in each layer, and n is the number of scene elements in each attribute. The scene set S can be expressed as the Cartesian product of different scene elements in different attributes of different architectures,

[0047] S = element 111 × element 112 ×... × element (mn ×... × element 7MN .

[0048] Build a multi-dimensional test scenario evaluation model, extract the scene elements of each test scenario, input them into the multi-dimensional test scenario evaluation model, and obtain the comprehensive evaluation results of each test scenario.

[0049] Furthermore, the multi-dimensional test scenario evaluation model includes an evaluation index value calculation layer and a comprehensive calculation layer.

[0050] Furthermore, the evaluation index value calculation layer calculates the evaluation index values from four dimensions: real rationality, regulation coverage, test adaptability, and danger complexity;

[0051] The real rationality dimension includes the real environment restoration degree, scene parameter rationality degree, driving behavior simulation accuracy, and meteorological simulation accuracy;

[0052] (1) Real environment restoration degree E Jaccard

[0053] The scenarios for conducting compliance tests of road traffic regulations need to highly restore the real traffic environment, including the attributes and scene elements of each layer such as the road layer, traffic participant layer, and environment layer mentioned above. The set of scene elements of the test scenario {element (mn} test and the set of scene elements of the real collected data {element (mn} real The larger the intersection ratio, the higher the repetition rate, and the closer the test scenario is to the real scenario.

[0054]

[0055] E Jaccard The range is between 0 and 1. The closer it is to 1, the higher the real environment restoration degree of the test scenario.

[0056] (2) Scene parameter rationality degree E cos

[0057] The various parameters of the test scenario are the change values of the scene elements and need to be set reasonably. For example, the setting of traffic flow should be based on actual traffic data, and parameters such as the density and speed of vehicles and pedestrians should conform to the actual situation. If the traffic flow in the test scenario is too high or too low and far from the actual situation, it will affect the test rationality. The vector of a certain scene parameter parm of the test scenario changing with time is

[0058]

[0059] The vector corresponding to the change of this parameter with time in the scene of real collected data is

[0060]

[0061] E cos (parm) is the dot product of the two vectors divided by the norm, and the range is between -1 and 1. The closer it is to 1, the more in line with the actual situation the setting of this scene parameter is.

[0062] (3) Driving behavior simulation accuracy E hausdorff

[0063] The test scenario needs to consider whether the behavior logics of traffic participants such as motor vehicles, non-motor vehicles, and pedestrians conform to reality, and whether driving behaviors such as lane changes and overtaking are natural. The set of behavior trajectory points of a certain traffic participant in the test scenario is {path(t p )} test . To identify the situation where its unit time and movement speed are inconsistent with those of traffic participants in the real environment, consider calculating the hausdorff distance from the trajectory point set {path(t q )} real in the real scenario, which is used to evaluate the driving behavior simulation accuracy of this traffic participant. For each trajectory point path(t p )} test in {path(t p )}, find the trajectory point path(t q )} real in {path(t q )} p that is closest to it, and the distance between the two points is d(path(t q ),path(t

[0064] hausdorff = max{d(path(t p ),path(t q ))};

[0065] E hausdorff = ∑max{d(path(t p ),path(t q ))};

[0066] hausdorff is the maximum value of the closest point distance, and summing up the hausdorff of all traffic participants gives E hausdorff . The smaller it is, the more in line with reality the driving behavior is.

[0067] (4) Meteorological simulation accuracy E KL

[0068] Since different meteorological conditions have different degrees of influence on the perception system of autonomous driving, the test scenario needs to consider the accuracy of simulating traffic environmental conditions such as weather, temperature, visibility, and lighting. Regarding the scenario elements of a certain meteorological condition, such as temperature, as a discrete random variable, by comparing the probability distribution A of this scenario element in the test scenario (such as dividing the temperature into multiple intervals and the occurrence probabilities of different temperature intervals) with the probability distribution B of the corresponding scenario element in the real environment, A = {a1, a2, ···, a n}, B = {b1, b2, ···, b n} and calculate the relative entropy to measure the difference between the two probability distributions.

[0069]

[0070] E KL The smaller it is, the smaller the difference between the probability distribution of meteorological scene elements in the test scenario and the probability distribution of meteorological scene elements in the real environment, indicating that the meteorological simulation accuracy of this test scenario is higher.

[0071] The regulation coverage dimension includes the degree of violation of traffic participants, the coverage of regulation clauses, the matching degree of driving tasks, and the regulation test frequency;

[0072] (1) The degree of violation of traffic participants E +llegal

[0073] When an autonomous vehicle conducts a compliance test for road traffic regulations, the test should be carried out on the premise that the driving behaviors of other traffic participants except the vehicle itself comply with the regulations. For each traffic participant, traverse all the regulation clause sets L = {l1, l2, ···, l k}, where k is the total number of regulation clauses. Record the number of violated regulations. The number of regulations violated by the i-th violating traffic participant is c + , the total number of violating traffic participants is c, and the total number of traffic participants is C.

[0074]

[0075] E +llegal Comprehensively considering the number of violated regulations of each violating traffic participant and the proportion of traffic participants' violations, it intuitively reflects the overall degree of violation of traffic participants in a test scenario. E +llegal The higher it is, the more serious the degree of violation. When E +llegal is 0, it meets the requirements of the compliance test scenario for road traffic regulations.

[0076] (2) The coverage of regulation clauses E coverage

[0077] All the regulation clause sets of the Road Traffic Safety Law and its implementing regulations L = {l1, l2, ···, l k}, where k is the total number of regulation clauses. For each regulation l + , according to the above 7-layer scenario architecture, layer by layer record the scenario elements involved in this regulation (such as for maintaining a safe speed, it is necessary to check the road type and lane number on the road layer, whether there is a speed limit sign on the traffic infrastructure layer, etc.; for bad weather, it is necessary to check whether there are weather changes such as rain, snow, and fog on the environment layer, etc.), l +The set of scenario elements involved is {element (mn} li 。For a test scenario, it is judged one by one whether each element in the set of scenario elements (mn is included in it. If all are included, then this regulation l + is marked l + = 1; as long as there is one scenario element that is not included, the mark l + = 0.

[0078]

[0079] E coverage is the ratio of the number of regulation clauses involved in the test scenario to the total number of regulation clauses. The larger it is, the more regulation clauses are covered. E coverage Intuitively reflects the breadth of the regulation coverage of this scenario and helps to evaluate whether the scenario comprehensively covers the compliance test requirements of road traffic regulations.

[0080] (3) Driving task matching degree E match

[0081] According to the Road Traffic Safety Law and its implementing regulations, driving tasks are divided into 12 categories: starting and stopping, safe speed, safe distance, lane passing, lane changing, overtaking, meeting vehicles, reversing, intersection passing, avoidance, U-turn, and obeying traffic police commands, ensuring that all driving behaviors compliant with regulations can be covered. Each category of driving task may include multiple driving behaviors and involve multiple regulations (for example, intersection passing includes driving behaviors such as going straight, turning left, turning right, and stopping, and involves multiple regulations such as obeying traffic lights, driving in the specified lane, and yielding to pedestrians). For a test scenario, it includes driving tasks {d1, d2, ···, d D}, where D is the number of driving tasks involved. For each driving task d + a G×H matrix is constructed. The rows are the G driving behaviors included in d + , and the columns are the H regulations involved in d + . If the driving behavior in the i-th row has an associated relationship with the regulation in the j-th column, then g + ×h ( = 1, otherwise g + ×h ( = 0.

[0082]

[0083] E match The larger it is, the more driving behaviors in the test scenario can establish an association with the regulation clauses, and the higher the matching degree between the driving task and the regulation.

[0084] (4) Regulation test frequency E frequency

[0085] For real-world violations, calculate the violation rate of each regulation, that is, the percentage of the number of road traffic participants who violate this regulation among the total number of all traffic participants who violate regulations. Since a single traffic participant may have multiple violations, the total number needs to be calculated cumulatively. According to the violation rate, the set of regulation violation rates Lf = {lf1, lf2, ···, lf k}. The set of all regulation clauses L = {l1, l2, ···, l k}, where k is the total number of regulation clauses. According to the above regulation clause coverage, the regulations involved in the test scenario are obtained, that is

[0086]

[0087] The higher the violation rate of a certain regulation in the real world, the higher its importance and the degree of attention. The larger E frequency , the higher the test frequency of the regulations that reflect a high degree of attention. E frequency can represent the frequencies of different regulations in the test scenario, which helps to discover whether some regulations are over-tested or under-tested.

[0088] The test adaptability dimension includes the achievement degree of the test objective, the pertinence of function testing, and the coverage of boundary conditions;

[0089] (1) The achievement degree of the test objective E target

[0090] The test scenario should be able to effectively test specific functions of the autonomous driving system, and clarify the set of functions J = {j1, j2, ···, j J} to be tested in the system under test, such as emergency braking, lane keeping, autonomous lane change, etc. Count the number of times r that the function J is effectively tested (triggered and correctly executed) and the total number of tests R in the test scenario, and calculate the function test achievement ratio

[0091]

[0092] (2) The pertinence of function testing E tr+gger

[0093] It is very necessary to evaluate whether the scenario can cover the main functions of the system. For each function in the set of functions J = {j1, j2, ···, j J} of the system under test, it should be checked whether the scenario contains a variety of different test cases. The set of driving behaviors of traffic participants included in the test scenario G = {g1, g2, ···, g G}, such as the automatic emergency braking function test corresponding to the sudden braking of the vehicle ahead, the autonomous lane-changing function test corresponding to the number of lanes, the vehicle speed difference, the position and speed changes of the vehicles in adjacent lanes, etc. Construct a matrix of J×G, where the rows are driving behaviors g + , and the columns are functions j ( , if g + can trigger function j ( , that is, g + can test j ( , j ( ×g + = 1, if it cannot be triggered, then j ( ×g + = 0.

[0094]

[0095] E tr+gger The larger it is, the more targeted the scenario is for the function test of the autonomous driving system, and the more it can fully test whether the autonomous driving system can accurately and safely complete driving operations under various complex conditions.

[0096] (3) Boundary condition coverage E border

[0097] The test scenario needs to cover the boundary conditions that the autonomous driving system may face, and check whether the scenario can effectively test the system's response ability under the boundary conditions. For example, when the operating speed of the autonomous driving system is 0 - 80 km / h, when testing the speed control function, not only the normal range of conditions needs to be tested, but also the boundary conditions of the highest and lowest limits need to be considered. When the vehicle is approaching the boundary speed, whether the system's stability and safety control can be tested, and whether the system's response to acceleration and deceleration instructions is correct under these boundary conditions. Therefore, based on the designed operating range of the autonomous driving system, all boundary conditions are clarified, including but not limited to road conditions (road damage, missing traffic signs), environmental factors (bad weather, low temperature, low visibility), traffic participant conditions (target vehicle traveling at high speed, non-motor vehicle), etc. Considering the boundary condition set O = {o1, o2, ···, o O} and the scenario element set {element (mn}, determine the truth function σ(o + ),

[0098]

[0099] E border = σ(o1 ∧ o2 ∧ ··· ∧ o O );

[0100] When E borderWhen E = 1, the test scenario can cover all boundary conditions of the autonomous driving system. When E border = 0, there are uncovered boundary conditions.

[0101] The dimension of danger complexity includes the richness of scenario elements, the degree of change of scenario elements, the change rate of traffic flow speed, and the complexity of traffic flow.

[0102] (1) Richness of scenario elements E num

[0103] Rich traffic elements can better simulate the real and complex traffic environment. Therefore, the integrity of traffic participants, traffic infrastructure, and temporary events in the test scenario should be considered. For a scenario, the more the number of scenario elements it contains, the richer the scenario represents. Different scenario elements such as roads (straight roads, curves, ramps, etc.), environments (sunny days, rainy days, foggy days, as well as day and night, etc.), traffic participants (different types of motor vehicles and non-motor vehicles, pedestrians with different walking speeds and behavior patterns, and the interaction between traffic participants) will all affect the vehicle's perception and decision-making, and can better test the ability of the autonomous driving system to comply with traffic rules.

[0104] E num = |{element (mn} test |;

[0105] {element (mn} test is the set of scenario elements, and E num is the cardinality of the set, and the larger it is, the higher the richness.

[0106] (2) Degree of change of scenario elements E change

[0107] The autonomous driving system needs to operate stably in a complex and changeable environment. A higher degree of change of scenario element attributes can test whether the autonomous driving system can correctly, quickly perceive, make decisions, and control the vehicle when facing frequently changing scenarios (such as when a special vehicle suddenly appears on the road, the autonomous driving system needs to quickly make reactions such as braking or avoidance). If the attributes of scenario elements hardly change in a test scenario, then this test scenario may not be able to fully test the performance of the autonomous driving system, and the test scenario needs to be optimized. The time change sequence of the set of scenario elements {element (mn} test is {S(t + ) (mn}, that is, the attribute value of a certain scenario element element (mn at time t + is S(t + )(mn , where \(i\in[0,T]\). The number of changes in the attributes of each scenario element is calculated using the indicator function \(1(x)\).

[0108]

[0109] E change The larger the value, the higher the degree of change in the scenario elements, and the more reliable and safe the response ability of the autonomous driving system can be evaluated.

[0110] (3) Rate of change of traffic flow speed \(E\) T[VC

[0111] The amplitude and frequency of speed changes of each traffic participant in the traffic flow of the statistical test scenario are counted. The standard deviation of speed is \(\sigma\) veloc+ty , and the frequency of speed change is \(f\) veloc+ty ,

[0112] E T[VC =\(\sigma\) veloc+ty \(\times f\) veloc+ty ;

[0113] E T[VC The larger the value, the stronger the traffic flow dynamics, and the higher the difficulty for the autonomous driving system to cope with.

[0114] (4) Traffic flow complexity \(E\) T[C

[0115] The traffic flow density \(TFD\) is defined as the number of traffic participants in the traffic flow per unit road length of the test scenario. \(U\) T] is the total number of traffic participants, and \(length\) is the road length.

[0116]

[0117] The traffic flow behavior diversity considers the types of driving tasks of traffic participants in the test scenario. For a test scenario, it contains driving tasks \(\{d_1, d_2, \cdots, d\) D}\), where \(D\) is the number of driving tasks.

[0118]

[0119] E T[C The higher the value, the more information about traffic participants of more quantities and more types the autonomous driving system needs to identify, process, and predict, indicating that the scenario is more complex.

[0120] Furthermore, the comprehensive calculation layer assigns weights to each evaluation index and performs weighted summation based on the values of each evaluation index and the corresponding weight values to obtain the comprehensive evaluation result of the test scenario. Among them, the weighted summation calculation is performed after data standardization of the values of each evaluation index.

[0121] Further, the process of assigning weights to each evaluation index includes:

[0122] Obtain the average scores of experts for each evaluation index, normalize the average scores of experts for all evaluation indexes to obtain the subjective weights of each evaluation index; use the principal component analysis method to obtain the objective weights of each evaluation index; based on the subjective weights and objective weights, use the Lagrangian function to obtain the combined weights of each evaluation index.

[0123] Further, the process of obtaining the objective weights of each evaluation index includes:

[0124] Standardize the values of each evaluation index for several scenarios; calculate the covariance matrix between the evaluation indexes based on the standardized values, perform eigenvalue decomposition on the covariance matrix, calculate the eigenvalues and corresponding eigenvectors; select the eigenvectors corresponding to the top k largest eigenvalues as the principal components; regress the principal components onto the original evaluation indexes to obtain the objective weights of each evaluation index.

[0125] Specifically, the subjective evaluation method uses the expert scoring method. Z experts score 15 evaluation indexes to obtain an evaluation matrix of Z×15. For the v-th evaluation index (v = 1, 2,... 15), the score given by the z-th expert is e zv (z = 1, 2,... Z), then the average score of this index is

[0126]

[0127] Normalize the average scores of the 15 evaluation indexes, θ v is the subjective weight of the v-th evaluation index,

[0128]

[0129] The objective evaluation method uses the principal component analysis method. Calculate the values of each evaluation index for U scenarios respectively. For the u-th scenario (u = 1, 2,... U), e uv is the calculated value of the v-th evaluation index, is the mean value of the v-th evaluation index. Standardize the data,

[0130]

[0131] Calculate the covariance matrix between the indexes according to the standardized data, that is, a 15×15 matrix. The element in the i-th row and j-th column is the covariance between the i-th evaluation index and the j-th evaluation index. Since the data has been standardized, the mean values of the i-th evaluation index and the j-th evaluation index are both 0,

[0132]

[0133] Perform eigen decomposition on the covariance matrix [related +( , calculate the eigenvalues λ v and the corresponding eigenvectors u v . Sort according to the magnitude of λ v , and select the first k largest λ v and the corresponding u v as the principal components, so that the cumulative variance contribution rate reaches more than 90%,

[0134]

[0135] Regress the principal components onto the original evaluation indicators to obtain the weights of each indicator. The objective weight ρ v of the v-th evaluation indicator can be expressed as

[0136]

[0137] u v is the eigenvector of the j-th principal component, and [sta v is the v-th column vector of the standardized data matrix [sta uv .

[0138] According to the subjective weight θ v and the objective weight ρ v of the v-th evaluation indicator, the combined weight ω v can be obtained. Construct the Lagrangian function to obtain the optimal solution ω v ,

[0139] ω v =αθ v +(1 - α)ρ v ;

[0140] α is the subjective preference coefficient, 1 - α is the objective preference coefficient. If there is no preference, α = 0.5 can be set. After determining the weights of each evaluation indicator in the test scenario, according to the combined weight ω v and the calculation results sta v of the data standardization indicators, the comprehensive evaluation result evaluation is obtained,

[0141]

[0142] The above is only the preferred specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for evaluating the compliance test scenario of an autonomous driving road traffic regulation, characterized in that: The following steps are involved: Construct a seven-layer logical scene architecture with scene elements as the smallest unit; A scenario set is obtained based on the seven-layer logical scenario architecture; wherein the seven-layer logical scenario architecture includes a road layer, a traffic infrastructure layer, a road and traffic facility temporary event layer, a traffic participant layer, an environment layer, a communication status layer, and a vehicle status layer; A multi-dimensional test scenario evaluation model is constructed, and scenario elements of each test scenario are extracted and input into the multi-dimensional test scenario evaluation model to obtain comprehensive evaluation results of each test scenario.

2. The method for evaluating the compliance test scenario of autonomous driving road traffic regulations according to claim 1, characterized in that: The multi-dimensional test scenario evaluation model includes an evaluation index value calculation layer and a comprehensive calculation layer.

3. The method for evaluating the compliance test scenario of the autonomous driving road traffic regulations according to claim 2, characterized in that: The evaluation index value calculation layer calculates the evaluation index value from four dimensions: authenticity and rationality, regulatory coverage, test adaptability and hazard complexity.

4. The method for evaluating the compliance test scenario of autonomous driving road traffic regulations according to claim 3, characterized in that: The dimensions of authenticity and rationality include the degree of restoration of the real environment, the rationality of scene parameters, the accuracy of driving behavior simulation and the accuracy of weather simulation; The regulatory coverage dimensions include the degree of violation of traffic participants, coverage of regulatory provisions, driving task matching and regulatory testing frequency; The test adaptability dimensions include test goal achievement, functional test targeting, and boundary condition coverage; The hazard complexity dimensions include the richness of scene elements, the degree of change of scene elements, the rate of change of traffic flow speed, and the complexity of traffic flow.

5. The method for evaluating the compliance test scenario of autonomous driving road traffic regulations according to claim 4, characterized in that: The comprehensive calculation layer assigns weights to each evaluation index, and performs weighted summation based on each evaluation index value and the corresponding weight value to obtain a comprehensive evaluation result of the test scenario, wherein each evaluation index value is data standardized and then weighted summation is performed.

6. The method for evaluating the compliance test scenario of autonomous driving road traffic regulations according to claim 4, characterized in that: The process of assigning weights to each evaluation indicator includes: The average expert score of each evaluation indicator is obtained, and the average expert score of all evaluation indicators is normalized to obtain the subjective weight of each evaluation indicator; the principal component analysis method is used to obtain the objective weight of each evaluation indicator; based on the subjective weight and the objective weight, the Lagrangian function is used to obtain the combined weight of each evaluation indicator.

7. The method for evaluating the compliance test scenario of the autonomous driving road traffic regulations according to claim 4, characterized in that: The process of obtaining an objective weight for each evaluation indicator includes: The numerical values ​​of various evaluation indicators of several scenes are standardized; the covariance matrix between the evaluation indicators is calculated based on the standardized numerical values, the covariance matrix is ​​eigendecomposed, and the eigenvalues ​​and corresponding eigenvectors are calculated; the eigenvectors are selected as principal components according to the eigenvalues; the principal components are regressed to the original evaluation indicators to obtain the objective weights of each evaluation indicator.

Images