An evaluation method of an autonomous driving road traffic regulation compliance test scene

Abstract

The application discloses an evaluation method for automatic driving road traffic regulation compliance test scene, comprising: constructing a seven-layer logical scene architecture taking scene elements as the minimum unit; obtaining a scene set based on the seven-layer logical scene architecture; wherein the seven-layer logical scene architecture comprises a road layer, a traffic infrastructure layer, a road and traffic infrastructure temporary event layer, a traffic participant layer, an environment layer, a communication state layer and a self-vehicle state layer; constructing a multi-dimensional test scene evaluation model, extracting scene elements of each test scene, inputting into the multi-dimensional test scene evaluation model, and obtaining comprehensive evaluation results of each test scene. The application takes scene elements as units, covers four dimensions of real reasonability, regulation coverage, test adaptability and danger complexity and 15 indexes, can filter out comprehensive and effective test scenes from a large number of scenes, ensures compliance and safety of the automatic driving system, and promotes the development of the technology within the legal framework.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, and in particular relates to an evaluation method for autonomous driving road traffic regulation compliance test scenarios. Background Technology

[0002] With technological advancements, an increasing number of autonomous vehicles are being tested and even piloted on open roads. However, road traffic environments are complex and ever-changing, with pedestrians, non-motorized vehicles, and traditional motorized vehicles sharing the road. Road conditions encompass various scenarios, including congested urban roads, highways, and rural trails. For autonomous vehicles to integrate safely and smoothly into these environments, they must strictly adhere to existing road traffic regulations. When conducting road traffic regulation compliance tests, issues such as the long-tail effect of test scenarios arise. Besides common traffic scenarios, there are numerous low-frequency, rare, but highly dangerous situations that are difficult to identify and cover using traditional testing methods. Furthermore, the number of autonomous driving test scenarios grows exponentially with various factors, requiring the allocation of substantial testing resources and undoubtedly increasing the testing difficulty. To achieve a comprehensive evaluation of autonomous vehicles' road traffic regulation compliance testing, it is necessary to construct evaluation methods for relevant test scenarios to ensure that the test scenarios compliantly, effectively, and realistically reflect the autonomous driving's ability to comply with traffic rules.

[0003] In actual testing, too many scenarios can lead to problems such as low testing efficiency and lengthy testing cycles. Currently, there is a lack of clear evaluation methods for different autonomous vehicle testing scenarios, especially road traffic regulation compliance testing scenarios. Existing scenario evaluation methods are: firstly, mainly focused on simulation testing, which is still insufficient in terms of realism evaluation; secondly, industry evaluation of scenarios focuses more on complexity and risk, with insufficient consideration for compliance verification; and thirdly, existing evaluation methods are mostly biased towards subjective evaluation, lacking specific objective evaluation methods and evaluation systems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes an evaluation method for autonomous driving road traffic regulation compliance testing scenarios, thereby resolving the issues present in the prior art.

[0005] To achieve the above objectives, the present invention provides an evaluation method for an autonomous driving road traffic regulation compliance test scenario, comprising:

[0006] A seven-layer logical scene architecture is constructed with scene elements as the smallest unit; a scene set is obtained based on the seven-layer logical scene architecture; wherein, the seven-layer logical scene architecture includes a road layer, a traffic infrastructure layer, a road and traffic facility temporary event layer, a traffic participant layer, an environment layer, a communication state layer, and a vehicle state layer;

[0007] A multi-dimensional test scenario evaluation model is constructed, and the scenario elements of each test scenario are extracted and input into the multi-dimensional test scenario evaluation model to obtain the comprehensive evaluation results of each test scenario.

[0008] Optionally, each layer of the seven-layer logical scene architecture includes several types of attributes, and each type of attribute includes several types of scene elements.

[0009] Optionally, the scene set is the Cartesian product of different scene elements in different attributes.

[0010] Optionally, the multi-dimensional test scenario evaluation model includes an evaluation index value calculation layer and a comprehensive calculation layer.

[0011] Optionally, the evaluation index value calculation layer calculates the evaluation index value from four dimensions: authenticity and rationality, regulatory coverage, test adaptability, and hazard complexity.

[0012] The dimensions of realism and rationality include the degree of realism in environmental reproduction, the rationality of scene parameters, the accuracy of driving behavior simulation, and the accuracy of weather simulation.

[0013] The dimensions of regulatory coverage include the degree of violation by traffic participants, coverage of regulatory provisions, matching degree of driving tasks, and frequency of regulatory testing;

[0014] The test adaptability dimensions include the degree of achievement of test objectives, the relevance of functional tests, and the coverage of boundary conditions.

[0015] The dimension of hazard complexity includes the richness of scene elements, the degree of change of scene elements, the rate of change of traffic flow speed, and the complexity of traffic flow.

[0016] Optionally, the comprehensive calculation layer assigns weights to each evaluation indicator and performs a weighted summation based on each evaluation indicator value and its corresponding weight value to obtain a comprehensive evaluation result of the test scenario. The weighted summation calculation is performed after standardizing the data of each evaluation indicator value.

[0017] Optionally, the process of assigning weights to each evaluation indicator includes:

[0018] The average expert score for each evaluation indicator is obtained, and the average expert score for all evaluation indicators is normalized to obtain the subjective weight of each evaluation indicator. Principal component analysis is used to obtain the objective weight of each evaluation indicator. Based on the subjective and objective weights, the combined weight of each evaluation indicator is obtained using the Lagrange function.

[0019] Optionally, the process of obtaining the objective weight of each evaluation indicator includes:

[0020] The values ​​of various evaluation indicators for several scenarios are standardized; the covariance matrix between the evaluation indicators is calculated based on the standardized values; the covariance matrix is ​​decomposed into eigenvalues ​​and corresponding eigenvectors; the eigenvectors corresponding to the top k largest eigenvalues ​​are selected as principal components; the principal components are regressed to the original evaluation indicators to obtain the objective weights of each evaluation indicator.

[0021] Compared with the prior art, the present invention has the following advantages and technical effects:

[0022] This invention proposes a comprehensive model and evaluation method, constructed with scenario elements as the smallest unit, encompassing 4 dimensions and 15 indicators. This model is used to quantitatively analyze and evaluate test scenarios for the road traffic compliance of autonomous vehicles. For an overwhelmingly large scenario library, it helps to select test scenarios that comprehensively and effectively assess the compliance and safety of autonomous driving systems, considering factors such as realism, regulatory coverage, test adaptability, and the complexity of hazards. This invention comprehensively considers multiple key areas, from simulating actual driving environments to laws and regulations, to the efficiency of test resource utilization and the limits of system safety performance testing, avoiding the limitations of single-dimensional evaluation. This allows for a more accurate assessment of whether test scenarios can comprehensively cover all situations that autonomous driving systems may face. Furthermore, fully considering legal and regulatory requirements in test scenarios enables autonomous driving systems to focus more on compliance verification during development, promoting the healthy development of autonomous driving technology within the legal framework. Attached Figure Description

[0023] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0024] Figure 1 This is a schematic diagram of the scenario architecture according to an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram illustrating four dimensions of a test scenario for evaluating compliance with road traffic regulations, as described in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram illustrating the application process of the road traffic regulation compliance test scenario evaluation method according to an embodiment of the present invention. Detailed Implementation

[0027] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0028] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0029] Example 1

[0030] like Figure 1-3 As shown, this embodiment provides an evaluation method for a test scenario of compliance with autonomous driving road traffic regulations, including:

[0031] Construct a seven-layer logical scene architecture with scene elements as the smallest unit; obtain a scene set based on the seven-layer logical scene architecture; the seven-layer logical scene architecture includes the road layer (Layer 1), traffic infrastructure layer (Layer 2), temporary events layer for roads and traffic facilities layer (Layer 3), traffic participants layer (Layer 4), environment layer (Layer 5), communication state layer (Layer 6), and vehicle state layer (Layer 7).

[0032] Furthermore, each layer of the seven-layer logical scenario architecture includes several types of attributes, and each type of attribute includes several types of scenario elements.

[0033] Furthermore, the scene set is the Cartesian product of different scene elements in different attributes.

[0034] Layer 1 includes three categories of attributes: road attributes, lane attributes, and appurtenance attributes. Road attributes include road grade (expressway, Class I / II highway, etc.), road condition (whether there is ice or water accumulation, potholes or faults, etc.), road type (main road, secondary road, auxiliary road), and road geometry (curvature, length, gradient). Lane attributes include the number of lanes (two lanes, single lane, etc.), lane type (motor vehicle lanes, non-motor vehicle lanes, etc.), lane topology (parallel lanes, turning lanes, and intersecting lanes, etc.), lane geometry (width, direction, length, etc.), lane line color (yellow, white), and lane line type (solid line, dashed line, mixed solid and dashed line, etc.). Appurtenance attributes include appurtenance type (median strip, speed bump, etc.) and appurtenance location (starting position, ending position).

[0035] Layer 2 includes three categories of attributes: traffic light attributes, sign attributes, and road marking attributes. Traffic light attributes include traffic light type (vehicle traffic lights, lane direction indicators, pedestrian crossing lights, etc.), traffic light geometry (position, height, orientation), traffic light status (constant on, flashing, off), and traffic light phase (red, yellow, green). Sign attributes include sign type (warning, prohibitory, instruction, etc.), sign geometry (position, height, orientation), and sign content (e.g., speed limit 60, 70, 80). Road marking attributes include road marking type (lane lines, pedestrian crossing lines, guide lines, etc.), road marking geometry (position, size, orientation), and road marking appearance (whether it's blurred, color, etc.).

[0036] Layer 3 includes three types of attributes: road change attributes, lane change attributes, and environmental change attributes. Road change attributes include change type (road curvature change, road slope change, road cover change, etc.), change location (start point, end point), change state (curvature value, slope value, cover type), and change frequency (number of curvature changes, number of slope changes, number of cover changes, etc.). Lane change attributes include change type (lane type change, lane color change, lane width change, lane number change, lane centerline offset, lane occupancy, etc.), change location (start point, end point), change state (changed type, color, width, number, offset, etc.), and change frequency (number of type changes, number of color changes, number of number changes, etc.). Environmental change attributes include change type (weather change, visibility change, temperature change, light change, etc.), change location (start point, end point), and change state (sunny / rainy weather, high / low temperature, strong / weak light, etc.).

[0037] Layer 4 includes three categories of attributes: vehicle attributes, pedestrian attributes, and other attributes. Vehicle attributes include single-vehicle information (type, spacing, speed) and traffic flow information (congestion level, traffic flow range). Pedestrian attributes include pedestrian information (spacing, speed, direction), crowd information (location, density), and traffic police information (location, gestures). Other attributes include animal attributes (animal movement status) and inanimate attribute attributes (obstacle location).

[0038] Layer 5 includes three categories of attributes: meteorological attributes, lighting attributes, and other attributes. Meteorological attributes include weather conditions (rain, snow, fog), temperature, wind speed, and visibility. Lighting attributes include lighting type (natural light, artificial light), lighting location, and lighting intensity. Other attributes include shadow interference (type and location) and electromagnetic interference (source and location).

[0039] Layer 6 includes three types of attributes: roadside units, edge computing units, and positioning units. Roadside units include location, communication type (DSRC, 4G, etc.), sensor type (camera, LiDAR, etc.), and parameters (radius, angle, direction, intensity, etc.). Edge computing units include location, communication type, and parameters. Positioning units include location, communication type, device type (GPS, high-precision map, etc.), and parameters (computing speed, signal strength, transmission rate, etc.).

[0040] Layer 7 includes two types of attributes: dynamic driving tasks and component information. Dynamic driving tasks include type (lane change, overtaking, avoidance, etc.), completion time, and completion quality (number of attempts, number of stops). Component information includes component type (lights, wipers, etc.) and component status (on, off).

[0041] Filter by category property (m and scene elements (mn Let S be a set of scenes that can be formed. Each scene in the set is represented as S. + It is composed of different attributes and scene elements in a 7-layer architecture.

[0042] S={S +};

[0043] S + ={∑Layer ( ,1≤j≤7};

[0044] Layer ( ={∑property (m ,1≤m≤M};

[0045] property (m ={∑element (mn ,1≤n≤N};

[0046] Where m is the number of attributes in each layer, and n is the number of scene elements in each attribute. The scene set S can be represented as the Cartesian product of different scene elements in different attributes of different architectures.

[0047] S = element 111 ×element 112 ×...×element (mn ×...×element 7MN .

[0048] A multi-dimensional test scenario evaluation model is constructed, and the scenario elements of each test scenario are extracted and input into the multi-dimensional test scenario evaluation model to obtain the comprehensive evaluation results of each test scenario.

[0049] Furthermore, the multi-dimensional test scenario evaluation model includes an evaluation index value calculation layer and a comprehensive calculation layer.

[0050] Furthermore, the evaluation index value calculation layer calculates the evaluation index value from four dimensions: authenticity and rationality, regulatory coverage, test adaptability, and hazard complexity.

[0051] The dimensions of realism and reasonableness include the degree of realism in environmental simulation, the reasonableness of scene parameters, the accuracy of driving behavior simulation, and the accuracy of weather simulation.

[0052] (1) Realistic environmental fidelity E Jaccard

[0053] The scenario for conducting road traffic regulation compliance tests needs to highly replicate the real traffic environment, including the attributes and scenario elements of each layer mentioned above, such as the road layer, traffic participant layer, and environment layer. The set of scenario elements for the test scenario {element} (mn} test The set of scene elements with real collected data {element (mn} real The higher the intersection-union ratio, the higher the repetition rate, and the closer the test scenario is to the real scenario.

[0054]

[0055] E Jaccard The range is between 0 and 1. The closer it is to 1, the higher the degree of realism of the test scenario.

[0056] (2) Scene parameter rationality E cos

[0057] The various parameters of the test scenario represent the changing values ​​of scenario elements and must be set reasonably. For example, traffic flow settings should be based on actual traffic data; parameters such as vehicle and pedestrian density and speed must conform to reality. If the traffic flow in the test scenario is too high or too low, deviating significantly from reality, it will affect the rationality of the test. The vector representing the change of a certain scenario parameter `parm` over time is...

[0058]

[0059] The vector representing the change of this parameter over time in a real-world data collection scenario is:

[0060]

[0061] E cos (parm) is the dot product of two vectors divided by the norm, ranging from -1 to 1. The closer it is to 1, the more realistic the parameter settings for the scenario are.

[0062] (3) Driving behavior simulation accuracy E hausdorff

[0063] The test scenario needs to consider whether the behavioral logic of traffic participants such as motor vehicles, non-motor vehicles, and pedestrians conforms to reality, and whether the corresponding driving behaviors such as lane changing and overtaking are natural. The set of behavioral trajectory points of a certain traffic participant in the test scenario is {path(t p )} test To identify discrepancies between the actual time and speed of traffic participants in real-world scenarios, we consider calculating the trajectory point set {path(t)} that corresponds to the real-world scene. q )} real The Hausdorff distance is used to evaluate the accuracy of the driving behavior simulation for this traffic participant. For {path(t p )} test Each trajectory point path(t) p ), find {path(t q )} real The path(t) is the closest point in the trajectory. q The distance between the two points is d(path(t)). p ),path(t q )),

[0064] hausdorff=max{d(path(t p ),path(t q ))};

[0065] E hausdorff =∑max{d(path(t) p ),path(t q ))};

[0066] Hausdorff is the maximum distance to the nearest point. E is obtained by summing the Hausdorff values ​​for all traffic participants. hausdorff The smaller the value, the more realistic the driving behavior is.

[0067] (4) Accuracy of meteorological simulation E KL

[0068] Because different weather conditions have varying degrees of impact on autonomous driving perception systems, test scenarios need to consider the accuracy of simulating traffic environmental conditions such as weather, temperature, visibility, and lighting. For a specific weather-related scenario element, such as temperature, it is treated as a discrete random variable. The probability distribution A of this scenario element in the test scenario (e.g., the probability of occurrence of different temperature intervals divided into multiple intervals) is compared with the corresponding probability distribution B in the real environment, where A = {a1, a2, ..., a...}. n}, B={b1,b2,···,b n The relative entropy is calculated to measure the difference between two probability distributions.

[0069]

[0070] E KL The smaller the value, the smaller the difference between the probability distribution of meteorological scene elements in the test scenario and the probability distribution of meteorological scene elements in the real environment, indicating that the meteorological simulation accuracy of the test scenario is higher.

[0071] The dimensions of regulatory coverage include the degree of violation by traffic participants, coverage of regulatory provisions, matching degree of driving tasks, and frequency of regulatory testing;

[0072] (1) Degree of violation by traffic participants E +llegal

[0073] When autonomous vehicles undergo road traffic compliance testing, the testing should be conducted only after all other road users, excluding the vehicle itself, have complied with the regulations. For each road user, the set of all legal provisions of the Road Traffic Law and its implementing regulations is L = {l1, l2, ..., l...}. k}, where k is the total number of legal provisions. Record the number of legal violations; the number of legal violations committed by the i-th traffic participant is c. + The total number of traffic violators is c, and the total number of traffic participants is C.

[0074]

[0075] E +llegal It comprehensively considers the number of traffic violations committed by each participant and the proportion of violations committed by each participant, thus intuitively reflecting the overall level of violations committed by traffic participants in a test scenario. +llegal The higher the value, the more serious the violation. +llegal A value of 0 indicates compliance with road traffic regulations and the required test scenario.

[0076] (2) Coverage of regulatory provisions E coverage

[0077] The set of all legal provisions of the Road Traffic Safety Law and its implementing regulations is L = {l1, l2, ..., l...} k}, where k is the total number of legal provisions. For each legal provision l + Based on the aforementioned 7-layer scenario architecture, each layer records the scenario elements involved in this regulation (such as the road type and number of lanes at the road layer for maintaining a safe speed, the presence of speed limit signs at the traffic infrastructure layer, and the presence of weather changes such as rain, snow, and fog at the environmental layer for severe weather). +The set of scene elements involved is {element} (mn} li For a test scenario, evaluate each element in the set of scenario elements one by one. (mn Whether it is included in it, and if it is fully included, then this regulation is... + Mark l + =1; if there is even one scene element that is not included, mark l. + =0.

[0078]

[0079] E coverage This is the ratio of the number of regulatory provisions involved in the test scenario to the total number of regulatory provisions; a higher ratio indicates that more regulatory provisions are covered. coverage This visually reflects the breadth of the scenario's regulatory coverage and helps assess whether the scenario fully covers the road traffic regulation compliance test requirements.

[0080] (3) Driving task matching degree E match

[0081] According to the Road Traffic Safety Law and its implementing regulations, driving tasks are divided into 12 categories: stopping and starting, safe speed, safe distance, lane keeping, lane changing, overtaking, meeting oncoming traffic, reversing, intersection driving, yielding, U-turns, and obeying traffic police commands. This ensures coverage of all compliant driving behaviors. Each category of driving task may include multiple driving behaviors and involve multiple regulations (e.g., intersection driving includes driving straight, turning left, turning right, and stopping, involving multiple regulations such as obeying traffic lights, driving in designated lanes, and yielding to pedestrians). For a test scenario, there are driving tasks {d1, d2, ..., d...} D}, where D is the number of driving tasks involved. For each driving task d + Construct a G×H matrix with d columns. + The included G driving behaviors are listed as d + If the driving behavior in row i is related to the regulation in column j, then g + ×h ( =1, otherwise g + ×h ( =0.

[0082]

[0083] E match The larger the value, the more driving behaviors in the test scenario can be associated with the regulations, and the higher the degree of matching between the driving task and the regulations.

[0084] (4) Regulatory testing frequency E frequency

[0085] To calculate the violation rate for each regulation in the real world, this is the percentage of road users who violated that regulation out of the total number of all traffic users who violated the regulation. Since a single traffic user may have multiple violations, the total number needs to be accumulated. Based on the violation rates, the set of regulation violation rates, Lf = {lf1, lf2, ..., lf...}, is calculated. k The set of all legal provisions L = {l1, l2, ..., l...} k}, where k is the total number of regulatory clauses. Based on the above regulatory clause coverage, the regulations involved in the test scenario are obtained, i.e.

[0086]

[0087] The higher the real-world violation rate of a regulation, the greater its importance and the level of attention it receives. frequency The larger the value, the higher the frequency of regulatory testing, reflecting the level of attention it receives. frequency It can represent the frequency with which different regulations appear in test scenarios, which helps to discover whether certain regulations are overtested or undertested.

[0088] Test adaptability dimensions include the degree to which test objectives are achieved, the relevance of functional tests, and the coverage of boundary conditions;

[0089] (1) Test target achievement E target

[0090] The test scenario should be able to effectively test specific functions of the autonomous driving system, and clearly define the set of functions that the system under test needs to test, J = {j1,j2,...,j...} J For functions such as emergency braking, lane keeping, and autonomous lane changing, the number of times (r) function J is effectively tested (triggered and correctly executed) and the total number of tests (R) are counted in the test scenario. The functional test achievement rate is then calculated.

[0091]

[0092] (2) Functional testing targeting E tr+gger

[0093] It is essential to assess whether the scenario can cover the main functions of the system. For the set of functions of the system under test, J = {j1,j2,...,j...} J For each function, the scenario should be checked to ensure it includes multiple different test cases. The test scenario includes a set of driving behaviors of traffic participants, G = {g1, g2, ..., g...}. GFor example, a sudden braking by the vehicle in front can be tested to correspond to the automatic emergency braking function; changes in the number of lanes, vehicle speed differences, and the position and speed changes of vehicles in adjacent lanes can be tested to correspond to the autonomous lane changing function. A J×G matrix is ​​constructed, and the driving behavior is represented by g. + Listed as function j ( If g + Function j can be triggered ( That is, g + Able to target j ( To conduct the test, j ( ×g + =1, if it cannot be triggered then j ( ×g + =0.

[0094]

[0095] E tr+gger The larger the value, the more targeted the scenario is for testing the autonomous driving system's functions, and the more fully it can verify whether the autonomous driving system can accurately and safely complete driving operations under various complex conditions.

[0096] (3) Boundary condition coverage E border

[0097] The test scenarios need to cover the boundary conditions that the autonomous driving system may encounter. The test should examine whether the scenarios can effectively test the system's ability to cope under these boundary conditions. For example, if the autonomous driving system operates at speeds between 0 and 80 km / h, when testing speed control functions, it's necessary to test not only the normal range but also the boundary conditions at the highest and lowest limits. The test should examine whether the system's stability and safety control can be tested when the vehicle approaches these boundary speeds, and whether the system's response to acceleration and deceleration commands is correct under these boundary conditions. Therefore, based on the design operating range of the autonomous driving system, all boundary conditions must be clearly defined, including but not limited to road conditions (road damage, missing traffic signs), environmental factors (severe weather, low temperature, low visibility), and the status of traffic participants (high-speed target vehicles, non-motorized vehicles), considering the boundary condition set O = {o1, o2, ..., o...}. O} and scene element collection {element (mn}, determine the truth function σ(o) for the boundary conditions. + ),

[0098]

[0099] E border =σ(o1∧o2∧···∧o O );

[0100] When E borderWhen E = 1, the test scenario can cover all boundary conditions of the autonomous driving system. border When = 0, there are uncovered boundary conditions.

[0101] The dimension of hazard complexity includes the richness of scene elements, the degree of change of scene elements, the rate of change of traffic flow speed, and the complexity of traffic flow.

[0102] (1) Scene element richness E num

[0103] A rich array of traffic elements can better simulate complex real-world traffic environments; therefore, the completeness of traffic participants, infrastructure, and ad-hoc events in the test scenario must be considered. The more scene elements a scenario includes, the richer it is. Different scene elements, such as roads (straight roads, curves, slopes, etc.), environments (sunny days, rainy days, foggy days, and daytime, nighttime, etc.), and traffic participants (different types of motor vehicles and non-motor vehicles, pedestrians with different walking speeds and behavior patterns, and interactions between traffic participants), all influence the vehicle's perception and decision-making, and better test the autonomous driving system's ability to comply with traffic rules.

[0104] E num =|{element (mn} test |;

[0105] {element (mn} test E is a collection of scene elements. num The cardinality of the set is the largest value, which indicates a higher degree of richness.

[0106] (2) Scene element change degree E change

[0107] Autonomous driving systems need to operate stably in complex and ever-changing environments. A high degree of variability in scene element attributes allows for testing whether the autonomous driving system can correctly and quickly perceive, make decisions, and control the vehicle when faced with frequently changing scenarios (e.g., when a special vehicle suddenly appears on the road, the autonomous driving system needs to react quickly by braking or avoiding it). If the scene element attributes remain almost unchanged in a test scenario, then this test scenario may not be sufficient to test the performance of the autonomous driving system, and the test scenario needs to be optimized. The set of scene elements {element} (mn} test The time variation sequence is {S(t)} + ) (mn}, that is, a scene element (mn In t + The attribute value at time is S(t) + )(mn , i∈[0,T]. The number of attribute changes for each scene element is calculated using the indicator function 1(x).

[0108]

[0109] E change The larger the value, the higher the degree of change in scene elements, and the better it can be used to assess the reliability and safety response capabilities of the autonomous driving system.

[0110] (3) Traffic flow velocity change rate E T[VC

[0111] The magnitude and frequency of speed changes for each traffic participant in the traffic flow of the statistical test scenario are calculated, with the speed standard deviation being σ. veloc+ty The frequency of the velocity change is f veloc+ty ,

[0112] E T[VC =σ veloc+ty ×f veloc+ty ;

[0113] E T[VC The larger the value, the more dynamic the traffic flow, and the more difficult it is for the autonomous driving system to cope.

[0114] (4) Traffic flow complexity E T[C

[0115] Traffic flow density (TFD) is defined as the number of traffic participants in a unit road length within a test scenario, U T] The total number of traffic participants is given, and length is the road length.

[0116]

[0117] Traffic behavior diversity is considered in the test scenario, taking into account the types of driving tasks performed by traffic participants. For a test scenario, there are driving tasks {d1, d2, ..., d...} D}, where D is the number of driving tasks.

[0118]

[0119] E T[C The higher the value, the more numerous and diverse the traffic participant information the autonomous driving system needs to identify, process, and predict, representing a more complex scenario.

[0120] Furthermore, the comprehensive calculation layer assigns weights to each evaluation indicator and performs a weighted summation based on the values ​​of each evaluation indicator and their corresponding weights to obtain the comprehensive evaluation result of the test scenario. In this process, the values ​​of each evaluation indicator are standardized before being weighted and summed.

[0121] Furthermore, the process of assigning weights to each evaluation indicator includes:

[0122] The average expert score for each evaluation indicator is obtained, and the average expert score for all evaluation indicators is normalized to obtain the subjective weight of each evaluation indicator. Principal component analysis is used to obtain the objective weight of each evaluation indicator. Based on the subjective and objective weights, the combined weight of each evaluation indicator is obtained using the Lagrange function.

[0123] Furthermore, the process of obtaining the objective weight of each evaluation indicator includes:

[0124] The values ​​of various evaluation indicators for several scenarios are standardized; the covariance matrix between the evaluation indicators is calculated based on the standardized values; the covariance matrix is ​​decomposed into eigenvalues ​​and corresponding eigenvectors; the eigenvectors corresponding to the top k largest eigenvalues ​​are selected as principal components; the principal components are regressed to the original evaluation indicators to obtain the objective weights of each evaluation indicator.

[0125] Specifically, the subjective evaluation method uses expert scoring, with Z experts scoring 15 evaluation indicators, resulting in a Z×15 evaluation matrix. For the v-th evaluation indicator (v=1,2,...15), the z-th expert scores it as e. zv (z=1,2,...Z), then the average score of this indicator is

[0126]

[0127] The average scores of the 15 evaluation indicators were normalized, θ v It is the subjective weight of the v-th evaluation indicator.

[0128]

[0129] The objective evaluation method uses principal component analysis to calculate the values ​​of each evaluation index for each of the U scenarios. For the u-th scenario (u = 1, 2, ..., U), e uv This is the calculated value of the v-th evaluation index. It is the mean of the v-th evaluation indicator. The data is then standardized.

[0130]

[0131] Calculate the covariance matrix between the indicators based on the standardized data. This is a 15×15 matrix, where the element in the i-th row and j-th column represents the covariance between the i-th and j-th evaluation indicators. Since the data has been standardized, the mean of the i-th and j-th evaluation indicators is also considered. All are 0.

[0132]

[0133] For the covariance matrix [related +( Perform eigenvalue decomposition and calculate the eigenvalues ​​λ. v and corresponding feature vector u v According to λ v Sort by size and select the k largest λ values. v The corresponding u v As principal components, this results in a cumulative variance contribution rate exceeding 90%.

[0134]

[0135] Principal component regression is applied to the original evaluation indicators to obtain the weights of each indicator. The objective weight ρ of the v-th evaluation indicator is... v It can be represented as

[0136]

[0137] u v It is the eigenvector of the j-th principal component, [sta v [sta] is the standardized data matrix. uv The vth column vector of ].

[0138] Based on the subjective weight θ of the vth evaluation index v and objective weight ρ v The combined weight ω can be obtained. v The optimal solution ω is obtained by constructing the Lagrange function. v ,

[0139] ω v =αθ v +(1-α)ρ v ;

[0140] α is the subjective preference coefficient, and 1-α is the objective preference coefficient. If there is no preference, α can be set to 0.5. After determining the weight of each evaluation indicator in the test scenario, the combined weight ω is used... v and the results of standardized data index calculations v The comprehensive evaluation results are obtained.

[0141]

[0142] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An evaluation method for a test scenario of compliance with road traffic regulations for autonomous driving, characterized in that, Includes the following steps: Construct a seven-layer logical scene architecture with scene elements as the smallest unit; A scene set is obtained based on the seven-layer logical scene architecture; wherein, the seven-layer logical scene architecture includes a road layer, a traffic infrastructure layer, a road and traffic facility temporary event layer, a traffic participant layer, an environment layer, a communication state layer, and a vehicle state layer; each layer of the seven-layer logical scene architecture includes several types of attributes, and each type of attribute includes several types of scene elements; the scene set is the Cartesian product of different scene elements in different attributes; A multi-dimensional test scenario evaluation model is constructed, the scenario elements of each test scenario are extracted, and the data are input into the multi-dimensional test scenario evaluation model to obtain the comprehensive evaluation results of each test scenario. The multi-dimensional test scenario evaluation model includes an evaluation index value calculation layer and a comprehensive calculation layer; The evaluation index value calculation layer calculates the evaluation index value from four dimensions: authenticity and rationality, regulatory coverage, test adaptability, and hazard complexity. The dimensions of regulatory coverage include the degree of violation by traffic participants, coverage of regulatory provisions, matching degree of driving tasks, and frequency of regulatory testing; The test adaptability dimensions include the degree of achievement of test objectives, the relevance of functional tests, and the coverage of boundary conditions. The comprehensive calculation layer assigns weights to each evaluation indicator and performs a weighted summation based on each evaluation indicator value and its corresponding weight value to obtain the comprehensive evaluation result of the test scenario. The weighted summation calculation is performed after standardizing the data of each evaluation indicator value. The process of assigning weights to each evaluation indicator includes: The average expert score for each evaluation indicator is obtained, and the average expert score for all evaluation indicators is normalized to obtain the subjective weight of each evaluation indicator. Principal component analysis is used to obtain the objective weight of each evaluation indicator. Based on the subjective and objective weights, the combined weight of each evaluation indicator is obtained using the Lagrange function. The process of obtaining the objective weight of each evaluation indicator includes: The values ​​of various evaluation indicators for several scenarios are standardized; the covariance matrix between the evaluation indicators is calculated based on the standardized values; the covariance matrix is ​​decomposed into eigenvalues ​​and corresponding eigenvectors; the eigenvectors are selected as principal components based on the eigenvalues; the principal components are regressed to the original evaluation indicators to obtain the objective weights of each evaluation indicator.

2. The evaluation method for the compliance test scenario of autonomous driving road traffic regulations according to claim 1, characterized in that, The dimensions of realism and rationality include the degree of realism in environmental reproduction, the rationality of scene parameters, the accuracy of driving behavior simulation, and the accuracy of weather simulation. The dimension of hazard complexity includes the richness of scene elements, the degree of change of scene elements, the rate of change of traffic flow speed, and the complexity of traffic flow.

Images