Automatic driving algorithm evaluation method and driving scene fingerprint determination method and device

Abstract

The invention discloses an automatic driving algorithm evaluation method and a driving scene fingerprint determination method and device. The evaluation method comprises the following steps: acquiring a scene to be detected and a driving behavior to be detected; calculating a scene fingerprint of the scene to be detected, wherein the scene fingerprint is used for uniquely identifying one driving scene; based on the similarity of the scene fingerprints, a target scene group matched with the scene fingerprint of the scene to be detected is determined from multiple scene groups, the scene groups comprise scene fingerprints of scenes driven by multiple expert drivers, and the multiple scene fingerprints in each scene group are the same and / or similar; when the to-be-detected driving behavior is located in an expected driving behavior feasible region corresponding to the target scene group, it is determined that the to-be-detected automatic driving algorithm meets the driving expectation, and the expected driving behavior feasible region is a feasible region formed by a plurality of expert driving behaviors meeting the driving expectation; and when the to-be-detected driving behavior is located outside the expected driving behavior feasible region corresponding to the target scene group, determining that the to-be-detected automatic driving algorithm does not meet the driving expectation.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and more specifically, to an autonomous driving algorithm evaluation method, a driving scene fingerprint determination method and device. Background Art

[0002] Currently, there are two main methods for evaluating autonomous driving algorithms: (1) There is no clear reference truth value, and general evaluation indicators are set (such as safety indicators, passability indicators, and physical perception indicators), and the performance of the algorithm on general indicators is focused on; (2) There is a clear reference truth value, and the algorithm effect is evaluated by calculating the difference between the output of the autonomous driving algorithm and the reference truth value.

[0003] By comparing the two methods, we can see that the second method has a more significant effect on improving the autonomous driving algorithm. However, the collection and labeling of true value data is not easy. The current main method relies on expert drivers to collect data in a given scenario, but this method also has some problems: (1) The driving behaviors collected by experts vary in scenarios, and it is difficult to distinguish whether different data belong to the same scenario or similar scenarios; (2) For the same scenario currently defined, the behaviors of experts are also different. Not all expert data can be used as evaluation data, but evaluation processing is required; (3) For the same scenario, different experts have different driving behaviors, which we can all consider reasonable, but this makes it difficult to evaluate the driving behavior of the autonomous driving algorithm in the future. Summary of the invention

[0004] The present application provides an autonomous driving algorithm evaluation method, a driving scene fingerprint determination method and a device, which can automatically, accurately and efficiently evaluate the autonomous driving algorithm to be tested without the need for manual labeling of true values.

[0005] The specific technical solutions are as follows:

[0006] In a first aspect, an embodiment of the present application provides an autonomous driving algorithm evaluation method, the method comprising:

[0007] Acquire a scene to be tested and a driving behavior to be tested, wherein the scene to be tested is a driving scene for autonomous driving based on the autonomous driving algorithm to be tested, and the driving behavior to be tested is a driving behavior generated by the vehicle during the autonomous driving process of the scene to be tested based on the autonomous driving algorithm to be tested;

[0008] Calculating a scene fingerprint of the scene to be tested, wherein the scene fingerprint is used to uniquely identify a driving scene;

[0009] Based on the similarity of the scene fingerprints, determining a target scene group that matches the scene fingerprint of the scene to be tested from multiple scene groups, wherein the scene group includes scene fingerprints of scenes driven by multiple expert drivers, and multiple scene fingerprints in each of the scene groups are the same and / or similar;

[0010] When the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, determining that the automatic driving algorithm to be tested meets the driving expectation, wherein the feasible domain of the expected driving behavior is a feasible domain composed of multiple expert driving behaviors that meet the driving expectation;

[0011] When the driving behavior to be tested is outside the feasible domain of the expected driving behavior corresponding to the target scenario group, it is determined that the automatic driving algorithm to be tested does not meet the driving expectation.

[0012] It can be seen from the above scheme that after obtaining the scene to be tested and the driving behavior to be tested, the scene fingerprint of the scene to be tested can be calculated first, and then based on the similarity of the scene fingerprint, the target scene group that matches the scene fingerprint of the scene to be tested can be determined from multiple scene groups. Finally, when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scene group, it can be determined that the autonomous driving algorithm to be tested meets the driving expectations, otherwise it does not meet the driving expectations. Compared with the traditional method of evaluating autonomous driving algorithms based on reference truth values, the embodiment of the present application can first determine the feasible domain of expected driving behaviors of multiple scene groups based on the expert driving behaviors of multiple experts in multiple driving scenes, realize scene grouping and aggregation of expert driving behaviors, and thus realize automatic, accurate and efficient evaluation of the autonomous driving algorithm to be tested without the need for manual annotation of truth values.

[0013] In a first possible implementation manner of the first aspect, the calculating the scene fingerprint of the scene to be tested includes:

[0014] Obtain obstacles, navigation routes and map topology in the scene to be tested;

[0015] Performing feature coding on the obstacle to obtain a first coding result;

[0016] Performing feature coding on the first coding result and the navigation route to obtain a second coding result;

[0017] Performing feature encoding on the first encoding result and the map topology structure to obtain a third encoding result;

[0018] concatenating the first encoding result, the second encoding result, and the third encoding result to obtain a fourth encoding result;

[0019] The fourth encoding result is calculated based on a mapping function to obtain a scene fingerprint of the scene to be tested.

[0020] It can be seen from the above scheme that the embodiment of the present application can obtain a scene fingerprint that can uniquely identify the scene to be tested by orderly encoding the obstacles, navigation routes and map topology structures in the scene to be tested.

[0021] In a second possible implementation manner of the first aspect, the obstacle includes an obstacle feature, an obstacle type number, an obstacle sequence number, an obstacle position, and an obstacle position offset, and feature encoding the obstacle to obtain a first encoding result includes:

[0022] Encoding the obstacle feature based on a first encoding network to obtain a first obstacle encoding result;

[0023] Encoding the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result;

[0024] Based on a third coding network, respectively encode the obstacle position and the obstacle position offset to obtain a third obstacle coding result and a fourth obstacle coding result;

[0025] concatenating the second obstacle encoding result, the third obstacle encoding result, and the fourth obstacle encoding result to obtain a fifth obstacle encoding result;

[0026] The fifth obstacle coding result, the obstacle type number and the obstacle sequence number are encoded based on the second coding network to obtain the first coding result.

[0027] It can be seen from the above scheme that the embodiment of the present application can obtain a first coding result for generating a scene fingerprint by orderly encoding obstacle features, obstacle type numbers, obstacle serial numbers, obstacle positions and obstacle position offsets.

[0028] In a third possible implementation manner of the first aspect, the navigation route includes a navigation information feature, a navigation topology element number, and a navigation topology element sequence number, and feature encoding is performed on the first coding result and the navigation route to obtain a second coding result, including:

[0029] Encoding the navigation information feature based on the first coding network to obtain a navigation coding result;

[0030] The navigation topology element number, the navigation topology element sequence number, the navigation coding result, and the first coding result are encoded based on the second coding network to obtain the second coding result.

[0031] It can be seen from the above scheme that the embodiment of the present application can improve the uniqueness of the scene fingerprint by fusing the first encoding result to encode the navigation topology element number, the navigation topology element sequence number, and the navigation encoding result.

[0032] In a fourth possible implementation manner of the first aspect, the map topology structure includes a map topology feature, a map topology element number, and a map topology element sequence number, and feature encoding is performed on the first encoding result and the map topology structure to obtain a third encoding result, including:

[0033] Encoding the map topology feature based on the first encoding network to obtain a map topology encoding result;

[0034] The map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result are encoded based on the second encoding network to obtain the third encoding result.

[0035] It can be seen from the above scheme that the embodiment of the present application can improve the uniqueness of the scene fingerprint by fusing the first encoding result to encode the map topology element number, the map topology element sequence number, and the map topology encoding result.

[0036] In a fifth possible implementation manner of the first aspect, a method for determining a feasible domain of expected driving behaviors corresponding to each of the scenario groups includes:

[0037] For each of the scenario groups, when the expert driving behaviors corresponding to the driving scenarios in the scenario group are the same, the feasible domain of the expected driving behaviors corresponding to the scenario group includes the same expert driving behaviors and their proportions;

[0038] When the expert driving behaviors corresponding to the various driving scenarios in the scenario group are different, the expert driving behaviors with scores greater than or equal to a preset score threshold are screened out, and the feasible domain of the expected driving behavior corresponding to the scenario group is determined based on the screened multiple expert driving behaviors and the proportion of various expert driving behaviors.

[0039] It can be seen from the above scheme that the feasible domain of expected driving behavior determined by the embodiment of the present application can include not only excellent expert driving behavior, but also the proportion of various expert driving behaviors, so as to provide a basis for scoring the driving behavior to be tested.

[0040] In a sixth possible implementation manner of the first aspect, when the driving behavior to be tested is within a feasible domain of the expected driving behavior corresponding to the target scenario group, the method further includes:

[0041] Determine an expert driving behavior that is closest to the driving behavior to be tested within the feasible domain of the expected driving behavior corresponding to the target scenario group;

[0042] The score of the approximate expert driving behavior is used as the score of the driving behavior to be tested.

[0043] In a second aspect, an embodiment of the present application provides a driving scene fingerprint determination method, the method comprising:

[0044] Obtain obstacles, navigation routes and map topology in driving scenarios;

[0045] Performing feature coding on the obstacle to obtain a first coding result;

[0046] Performing feature coding on the first coding result and the navigation route to obtain a second coding result;

[0047] Performing feature encoding on the first encoding result and the map topology structure to obtain a third encoding result;

[0048] concatenating the first encoding result, the second encoding result, and the third encoding result to obtain a fourth encoding result;

[0049] The fourth encoding result is calculated based on a mapping function to obtain a scene fingerprint of the driving scene.

[0050] It can be seen from the above scheme that the embodiment of the present application can obtain a scene fingerprint that can uniquely identify the driving scene by orderly encoding obstacles, navigation routes and map topology structures in the driving scene.

[0051] In a first possible implementation manner of the second aspect, the obstacle includes an obstacle feature, an obstacle type number, an obstacle sequence number, an obstacle position, and an obstacle position offset, and feature encoding the obstacle to obtain a first encoding result includes:

[0052] Encoding the obstacle feature based on a first encoding network to obtain a first obstacle encoding result;

[0053] Encoding the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result;

[0054] Based on a third coding network, respectively encode the obstacle position and the obstacle position offset to obtain a third obstacle coding result and a fourth obstacle coding result;

[0055] concatenating the second obstacle encoding result, the third obstacle encoding result, and the fourth obstacle encoding result to obtain a fifth obstacle encoding result;

[0056] The fifth obstacle coding result, the obstacle type number and the obstacle sequence number are encoded based on the second coding network to obtain the first coding result.

[0057] In a second possible implementation manner of the second aspect, the navigation route includes a navigation information feature, a navigation topology element number, and a navigation topology element sequence number, and feature encoding is performed on the first coding result and the navigation route to obtain a second coding result, including:

[0058] Encoding the navigation information feature based on the first coding network to obtain a navigation coding result;

[0059] The navigation topology element number, the navigation topology element sequence number, the navigation coding result, and the first coding result are encoded based on the second coding network to obtain the second coding result.

[0060] In a third possible implementation manner of the second aspect, the map topology structure includes a map topology feature, a map topology element number, and a map topology element sequence number, and feature encoding is performed on the first encoding result and the map topology structure to obtain a third encoding result, including:

[0061] Encoding the map topology feature based on the first encoding network to obtain a map topology encoding result;

[0062] The map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result are encoded based on the second encoding network to obtain the third encoding result.

[0063] In a fourth possible implementation manner of the second aspect, the first encoding network includes a multilayer perceptron MLP; and / or,

[0064] The second encoding network includes any one of context gating CG, multi-scale context gating MCG, and self-attention mechanism; and / or,

[0065] The third encoding network includes a long short-term memory network LSTM.

[0066] In a third aspect, an embodiment of the present application provides an autonomous driving algorithm evaluation device, the device comprising:

[0067] an acquisition unit, configured to acquire a scene to be tested and a driving behavior to be tested, wherein the scene to be tested is a driving scene for autonomous driving based on the autonomous driving algorithm to be tested, and the driving behavior to be tested is a driving behavior generated by a vehicle during the autonomous driving process of the scene to be tested based on the autonomous driving algorithm to be tested;

[0068] A calculation unit, used to calculate a scene fingerprint of the scene to be tested, wherein the scene fingerprint is used to uniquely identify a driving scene;

[0069] A screening unit, configured to determine, from a plurality of scene groups, a target scene group that matches the scene fingerprint of the scene to be tested based on the similarity of the scene fingerprints, wherein the scene group includes scene fingerprints of scenes driven by a plurality of expert drivers, and a plurality of scene fingerprints in each of the scene groups are identical and / or similar;

[0070] A determination unit is used to determine that the autonomous driving algorithm to be tested meets the driving expectation when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, wherein the feasible domain of the expected driving behavior is a feasible domain composed of multiple expert driving behaviors that meet the driving expectation; and to determine that the autonomous driving algorithm to be tested does not meet the driving expectation when the driving behavior to be tested is outside the feasible domain of the expected driving behavior corresponding to the target scenario group.

[0071] In a first possible implementation manner of the third aspect, the computing unit includes:

[0072] An acquisition module, used to acquire obstacles, navigation routes and map topology in the scene to be tested;

[0073] A first encoding module, used to perform feature encoding on the obstacle to obtain a first encoding result;

[0074] A second encoding module, used for performing feature encoding on the first encoding result and the navigation route to obtain a second encoding result;

[0075] A third encoding module, used for performing feature encoding on the first encoding result and the map topology structure to obtain a third encoding result;

[0076] a concatenation module, configured to concatenate the first encoding result, the second encoding result and the third encoding result to obtain a fourth encoding result;

[0077] A calculation module is used to calculate the fourth encoding result based on a mapping function to obtain a scene fingerprint of the scene to be tested.

[0078] In a second possible implementation manner of the third aspect, the obstacle includes an obstacle feature, an obstacle type number, an obstacle sequence number, an obstacle position, and an obstacle position offset, and the first encoding module is used to:

[0079] Encoding the obstacle feature based on a first encoding network to obtain a first obstacle encoding result;

[0080] Encoding the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result;

[0081] Based on a third coding network, respectively encode the obstacle position and the obstacle position offset to obtain a third obstacle coding result and a fourth obstacle coding result;

[0082] concatenating the second obstacle encoding result, the third obstacle encoding result, and the fourth obstacle encoding result to obtain a fifth obstacle encoding result;

[0083] The fifth obstacle coding result, the obstacle type number and the obstacle sequence number are encoded based on the second coding network to obtain the first coding result.

[0084] In a third possible implementation manner of the third aspect, the navigation route includes a navigation information feature, a navigation topology element number, and a navigation topology element sequence number, and the second encoding module is used to:

[0085] Encoding the navigation information feature based on the first coding network to obtain a navigation coding result;

[0086] The navigation topology element number, the navigation topology element sequence number, the navigation coding result, and the first coding result are encoded based on the second coding network to obtain the second coding result.

[0087] In a fourth possible implementation manner of the third aspect, the map topology structure includes a map topology feature, a map topology element number, and a map topology element sequence number, and the third encoding module is used to:

[0088] Encoding the map topology feature based on the first encoding network to obtain a map topology encoding result;

[0089] The map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result are encoded based on the second encoding network to obtain the third encoding result.

[0090] In a fifth possible implementation manner of the third aspect, the determination unit is further used to, for each of the scenario groups, when the expert driving behaviors corresponding to the respective driving scenarios in the scenario group are the same, the feasible domain of the expected driving behaviors corresponding to the scenario group includes the same expert driving behaviors and their proportions; when the expert driving behaviors corresponding to the respective driving scenarios in the scenario group are different, screen out the expert driving behaviors with a score greater than or equal to a preset score threshold, and determine the feasible domain of the expected driving behavior corresponding to the scenario group based on the screened multiple expert driving behaviors and the proportions of the various expert driving behaviors.

[0091] In a sixth possible implementation manner of the third aspect, the determination unit is further used to determine, when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, an expert driving behavior within the feasible domain of the expected driving behavior corresponding to the target scenario group that is closest to the driving behavior to be tested; and use the score of the close expert driving behavior as the score of the driving behavior to be tested.

[0092] It can be seen from the above scheme that after obtaining the scene to be tested and the driving behavior to be tested, the scene fingerprint of the scene to be tested can be calculated first, and then based on the similarity of the scene fingerprint, the target scene group that matches the scene fingerprint of the scene to be tested can be determined from multiple scene groups. Finally, when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scene group, it can be determined that the autonomous driving algorithm to be tested meets the driving expectations, otherwise it does not meet the driving expectations. Compared with the traditional method of evaluating autonomous driving algorithms based on reference truth values, the embodiment of the present application can first determine the feasible domain of expected driving behaviors of multiple scene groups based on the expert driving behaviors of multiple experts in multiple driving scenes, realize scene grouping and aggregation of expert driving behaviors, and thus realize automatic, accurate and efficient evaluation of the autonomous driving algorithm to be tested without the need for manual annotation of truth values.

[0093] In a fourth aspect, an embodiment of the present application provides a driving scene fingerprint determination device, the device comprising:

[0094] An acquisition unit, used to acquire obstacles, navigation routes and map topology in the driving scene;

[0095] A first encoding unit, used to perform feature encoding on the obstacle to obtain a first encoding result;

[0096] A second encoding unit, used for performing feature encoding on the first encoding result and the navigation route to obtain a second encoding result;

[0097] A third encoding unit, used for performing feature encoding on the first encoding result and the map topology structure to obtain a third encoding result;

[0098] a concatenation unit, configured to concatenate the first coding result, the second coding result and the third coding result to obtain a fourth coding result;

[0099] A calculation unit is used to calculate the fourth encoding result based on a mapping function to obtain a scene fingerprint of the driving scene.

[0100] In a first possible implementation manner of the fourth aspect, the obstacle includes an obstacle feature, an obstacle type number, an obstacle sequence number, an obstacle position, and an obstacle position offset, and the first encoding unit includes:

[0101] A first encoding module, used to encode the obstacle feature based on a first encoding network to obtain a first obstacle encoding result;

[0102] a second encoding module, configured to encode the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result;

[0103] a third encoding module, configured to encode the obstacle position and the obstacle position offset respectively based on a third encoding network to obtain a third obstacle encoding result and a fourth obstacle encoding result;

[0104] a splicing module, configured to splice the second obstacle coding result, the third obstacle coding result and the fourth obstacle coding result to obtain a fifth obstacle coding result;

[0105] The fourth encoding module is used to encode the fifth obstacle encoding result, the obstacle type number and the obstacle sequence number based on the second encoding network to obtain the first encoding result.

[0106] In a second possible implementation manner of the fourth aspect, the navigation route includes a navigation information feature, a navigation topology element number, and a navigation topology element sequence number, and the second encoding unit includes:

[0107] a fifth encoding module, configured to encode the navigation information feature based on the first encoding network to obtain a navigation encoding result;

[0108] The sixth encoding module is used to encode the navigation topology element number, the navigation topology element sequence number, the navigation encoding result and the first encoding result based on the second encoding network to obtain the second encoding result.

[0109] In a third possible implementation manner of the fourth aspect, the map topology structure includes a map topology feature, a map topology element number, and a map topology element sequence number, and the third encoding unit includes:

[0110] a seventh encoding module, configured to encode the map topology feature based on the first encoding network to obtain a map topology encoding result;

[0111] An eighth encoding module is used to encode the map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result based on the second encoding network to obtain the third encoding result.

[0112] In a fourth possible implementation manner of the fourth aspect, the first encoding network includes a multilayer perceptron MLP; and / or,

[0113] The second encoding network includes any one of context gating CG, multi-scale context gating MCG, and self-attention mechanism; and / or,

[0114] The third encoding network includes a long short-term memory network LSTM.

[0115] It can be seen from the above scheme that the embodiment of the present application can obtain a scene fingerprint that can uniquely identify the driving scene by orderly encoding obstacles, navigation routes and map topology structures in the driving scene.

[0116] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any possible implementation of the first aspect or any possible implementation of the second aspect.

[0117] In a sixth aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0118] one or more processors;

[0119] The processor is coupled to a storage device, and the storage device is used to store one or more programs;

[0120] When one or more programs are executed by one or more processors, the electronic device implements the method described in any possible implementation manner of the first aspect or any possible implementation manner of the second aspect.

[0121] In a seventh aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer or a processor, the computer or the processor executes the method described in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0122] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0123] Figure 1 A flowchart of an autonomous driving algorithm evaluation method provided in an embodiment of the present application;

[0124] Figure 2 A flowchart of a scene fingerprint calculation method provided in an embodiment of the present application;

[0125] Figure 3 A flowchart of another method for calculating scene fingerprints provided in an embodiment of the present application;

[0126] Figure 4 A schematic diagram of a flow chart of a driving scene fingerprint determination method provided in an embodiment of the present application;

[0127] Figure 5 A block diagram of a device for evaluating an autonomous driving algorithm provided in an embodiment of the present application;

[0128] Figure 6 A block diagram of a driving scene fingerprint determination device provided in an embodiment of the present application;

[0129] Figure 7 A schematic diagram of the structure of an electronic device or computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0130] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0131] It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The terms "including" and "having" in the embodiments of the present application and the accompanying drawings and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0132] Figure 1 The flowchart of a method for evaluating an autonomous driving algorithm is shown below. The method can be applied to electronic devices or computer devices, and can be applied to terminals or servers. Specifically, the method can include the following steps:

[0133] S110: Acquire the scene to be tested and the driving behavior to be tested.

[0134] The scenarios to be tested are driving scenarios for autonomous driving based on the autonomous driving algorithm to be tested. The scenarios to be tested include static scenarios and dynamic scenarios. Static scenarios are driving scenarios at a certain moment, and dynamic scenarios are driving scenarios at multiple consecutive moments. Each driving scenario includes a map topology, navigation routes, and obstacles. Obstacles include static obstacles and dynamic obstacles. Static obstacles include static objects placed around the road, such as stones, etc., and dynamic obstacles include vehicles, pedestrians, animals, etc.

[0135] The driving behavior to be tested is the driving behavior generated by the vehicle in the process of autonomous driving based on the autonomous driving algorithm to be tested in the test scenario. Driving behavior includes driving decisions, vehicle trajectory, speed, acceleration, etc.

[0136] S120: Calculate the scene fingerprint of the scene to be tested.

[0137] Among them, the scene fingerprint is used to uniquely identify a driving scene.

[0138] There are many methods for calculating scene fingerprints, for example Figure 2 As shown, the obstacles, navigation routes and map topology of the scene to be tested can be directly obtained, and then a certain encoding algorithm is used to encode the obstacles, navigation routes and map topology respectively, and after splicing the encoding results of the three, the message digest of the spliced ​​encoding results is calculated as the scene fingerprint of the scene to be tested.

[0139] In order to improve the accuracy of scene fingerprints, an embodiment of the present application provides a method for calculating scene fingerprints, which includes steps A1-A6:

[0140] A1. Obtain obstacles, navigation routes and map topology in the scene to be tested.

[0141] A2. Perform feature encoding on the obstacle to obtain a first encoding result.

[0142] Among them, obstacles include obstacle features, obstacle type number, obstacle sequence number, obstacle position and obstacle position offset. Obstacle features include obstacle type, size, heading angle, perceived shape, speed, acceleration, trajectory, etc. The obstacle type number is used to uniquely identify the obstacle type. The obstacle sequence number is the sequence number of the obstacle in the sequence number written for each element in the map in a preset order. The obstacle position refers to the coordinates of the obstacle, and the obstacle position offset refers to the displacement information of the dynamic obstacle.

[0143] like Figure 3 As shown, the method for encoding features of obstacles in an embodiment of the present application includes: encoding obstacle features based on a first encoding network to obtain a first obstacle encoding result; encoding the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result; encoding the obstacle position and the obstacle position offset respectively based on a third encoding network to obtain a third obstacle encoding result and a fourth obstacle encoding result; concatenating the second obstacle encoding result, the third obstacle encoding result and the fourth obstacle encoding result to obtain a fifth obstacle encoding result; encoding the fifth obstacle encoding result, the obstacle type number and the obstacle sequence number based on the second encoding network to obtain the first encoding result.

[0144] The first coding network, the second coding network and the third coding network may be the same or different.

[0145] The first encoding network includes an MLP (Multilayer Perceptron); and / or,

[0146] The second encoding network includes any one of CG (Context Gating), MCG (Multi Context Gating), and self-attention mechanism; and / or,

[0147] The third encoding network includes LSTM (Long Short-Term Memory).

[0148] Of course, the method of encoding the features of obstacles is not limited to this, and other methods may also be used. For example, a coding network may be directly used to encode the obstacle features, obstacle type number, obstacle sequence number, obstacle position and obstacle position offset together.

[0149] A3. Perform feature encoding on the first encoding result and the navigation route to obtain a second encoding result.

[0150] The navigation route includes navigation information features, navigation topology element numbers and navigation topology element serial numbers. The navigation information features include navigation route tracks, etc. The difference between navigation topology element numbers and navigation topology element serial numbers is that navigation topology element serial numbers are ordered in the map, while navigation topology element numbers are not ordered. Both can uniquely identify a navigation topology element.

[0151] Methods for encoding the first encoding result and the navigation route include but are not limited to the following two methods: (1) Figure 3 As shown, based on the first coding network, the navigation information feature is encoded to obtain a navigation coding result; based on the second coding network, the navigation topology element number, the navigation topology element sequence number, the navigation coding result and the first coding result are encoded to obtain a second coding result; (2) directly based on the first coding network, the second coding network or any coding network in the third coding network, the first coding result, the navigation information feature, the navigation topology element number and the navigation topology element sequence number are encoded to obtain a second coding result.

[0152] A4. Perform feature encoding on the first encoding result and the map topology structure to obtain a third encoding result.

[0153] The map topology structure includes map topology features, map topology element numbers and map topology element serial numbers. The map topology features include the map topology element structure. The difference between the map topology element number and the map topology element serial number is that the map topology element serial number is sequential in the map, while the map topology element number is not sequential. Both can uniquely identify a map topology element.

[0154] Methods for encoding the first encoding result and the map topology structure include but are not limited to the following two methods: (1) Figure 3 As shown, based on the first coding network, the map topology feature is encoded to obtain a map topology encoding result; based on the second coding network, the map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result are encoded to obtain a third encoding result. (2) Directly based on any coding network of the first coding network, the second coding network or the third coding network, the first encoding result, the map topology feature, the map topology element number and the map topology element sequence number are encoded to obtain a third encoding result.

[0155] A5. Concatenate the first encoding result, the second encoding result, and the third encoding result to obtain a fourth encoding result.

[0156] There is no restriction on the order of splicing the first coding result, the second coding result and the third coding result, as long as the splicing order of calculating all driving scenes is unified. For example, they can be spliced ​​in the order of first coding result-second coding result-third coding result, or in the order of third coding result-second coding result-first coding result.

[0157] A6. Calculate the fourth encoding result based on the mapping function to obtain a scene fingerprint of the scene to be tested.

[0158] The mapping function may be a function that can transform an input of any length into an output of a fixed length. For example, the mapping function may be a hash function or other hash functions.

[0159] S130: Based on the similarity of the scene fingerprints, determine a target scene group that matches the scene fingerprint of the scene to be tested from the multiple scene groups.

[0160] The scenario group includes scenario fingerprints of driving scenarios of multiple expert drivers, and multiple scenario fingerprints in each scenario group are the same and / or similar.

[0161] Before evaluating the autonomous driving algorithm to be tested, multiple expert drivers may drive in multiple driving scenarios to obtain expert driving behaviors in each driving scenario, wherein each driving scenario may include one or more expert driving behaviors. Given multiple driving scenarios of each expert driver, the scene fingerprints of the multiple driving scenarios may be calculated respectively, and then the similarity of these scene fingerprints is calculated, and scene fingerprints with the same similarity or a similarity greater than or equal to a preset similarity threshold are divided into a scene group.

[0162] The method for determining the scene fingerprint similarity includes but is not limited to: (1) calculating the Hamming distance between two scene fingerprints. The smaller the Hamming distance, the greater the similarity; (2) calculating the cosine similarity between the two scene fingerprints. The greater the cosine similarity, the greater the similarity of the scene fingerprints.

[0163] In practical applications, two or more scene fingerprints with the same similarity can be merged into one scene fingerprint and saved in the scene group, or they can be not merged. Moreover, when they are not merged, the problem of reducing the accuracy of the feasible domain of the expected driving behavior due to the subsequent update of the scene fingerprint calculation method can be avoided.

[0164] After obtaining the scene fingerprint of the scene to be tested, the similarity between the scene fingerprint of the scene to be tested and each scene fingerprint in multiple scene groups can be calculated respectively, and the scene fingerprint with the highest similarity that is greater than or equal to the similarity threshold is selected, and the scene group where the selected scene fingerprint belongs is used as the target scene group that matches the scene fingerprint of the scene to be tested.

[0165] S140: When the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, it is determined that the automatic driving algorithm to be tested meets the driving expectations.

[0166] Among them, driving expectation can be understood as: expecting the autonomous driving effect based on the autonomous driving algorithm to achieve the driving effect of an expert driver. The feasible domain of expected driving behavior is a feasible domain composed of multiple expert driving behaviors that meet the driving expectation. The feasible domain of expected driving behavior includes each expert driving behavior and can also include the proportion of each expert driving behavior.

[0167] The method for determining the feasible domain of expected driving behaviors corresponding to each scenario group includes: for each scenario group, when the expert driving behaviors corresponding to the various driving scenarios in the scenario group are the same, the feasible domain of expected driving behaviors corresponding to the scenario group includes the same expert driving behaviors and their proportions; when the expert driving behaviors corresponding to the various driving scenarios in the scenario group are different, expert driving behaviors with scores greater than or equal to a preset score threshold are screened out, and the feasible domain of expected driving behaviors corresponding to the scenario group is determined based on the screened multiple expert driving behaviors and the proportions of various expert driving behaviors.

[0168] Methods for scoring expert driving behavior include but are not limited to the following: (1) evaluation and scoring based on cost function; (2) evaluation and scoring based on deep learning evaluation network; (3) manual evaluation and scoring.

[0169] The following takes driving behavior including driving decision as an example to explain the method of determining the feasible domain of expected driving behavior:

[0170] The first case: for a specific intersection and certain navigation information, in the absence of obstacle interference, the vehicle enters from the first lane on the right and selects an exit lane (it may exit from the first lane on the right, or it may exit from the second lane on the right...). If the lane selection aggregation results of multiple expert drivers are all the same, for example, they all exit from the first lane on the right (abbreviated as right first), then the feasible domain of the expected driving behavior is determined to be Ag: [right first: 1.0].

[0171] The second case: for a specific intersection and certain navigation information, in the absence of obstacle interference, the vehicle enters from the first lane on the right and chooses an exit lane (it may exit from the first lane on the right, or it may exit from the second lane on the right...), and the lane aggregation selected by multiple expert drivers is not unique. For example, 8 choose the first lane on the right and 2 choose the second lane on the right. Then the feasible domain of the expected driving behavior is determined to be Ag: [first lane on the right: 0.8, second lane on the right: 0.2].

[0172] After obtaining the target scenario group, the driving behavior to be tested can be matched with the feasible domain of the expected driving behavior corresponding to the target scenario group; if the feasible domain of the expected driving behavior corresponding to the target scenario group contains the driving behavior to be tested, it is determined that the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, and the autonomous driving algorithm to be tested meets the driving expectations; if the feasible domain of the expected driving behavior corresponding to the target scenario group does not contain the driving behavior to be tested, it is determined that the driving behavior to be tested is outside the feasible domain of the expected driving behavior corresponding to the target scenario group, and the autonomous driving algorithm to be tested does not meet the driving expectations. Among them, when the feasible domain of the expected driving behavior contains the driving trajectory, the driving trajectory to be tested in the driving behavior to be tested can be calculated for similarity with the expert driving trajectory in the feasible domain of the expected driving behavior. If the similarity is greater than or equal to the preset similarity threshold, it can be considered that the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group.

[0173] S150: When the driving behavior to be tested is outside the feasible domain of the expected driving behavior corresponding to the target scenario group, it is determined that the automatic driving algorithm to be tested does not meet the driving expectations.

[0174] The autonomous driving algorithm evaluation method provided in the embodiment of the present application can first calculate the scene fingerprint of the scene to be tested after obtaining the scene to be tested and the driving behavior to be tested, and then determine the target scene group that matches the scene fingerprint of the scene to be tested from multiple scene groups based on the similarity of the scene fingerprints, and finally, when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scene group, it can be determined that the autonomous driving algorithm to be tested meets the driving expectations, otherwise it does not meet the driving expectations. Compared with the traditional method of evaluating autonomous driving algorithms based on reference truth values, the embodiment of the present application can first determine the feasible domain of expected driving behaviors of multiple scene groups based on the expert driving behaviors of multiple experts in multiple driving scenes, realize scene grouping and aggregation of expert driving behaviors, and thus realize automatic, accurate and efficient evaluation of the autonomous driving algorithm to be tested without the need for manual annotation of truth values.

[0175] In a possible implementation, when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, the expert driving behavior closest to the driving behavior to be tested within the feasible domain of the expected driving behavior corresponding to the target scenario group can also be determined; and the score of the closest expert driving behavior is used as the score of the driving behavior to be tested.

[0176] Among them, taking the score of the closest expert driving behavior as the score of the driving behavior to be tested can mean: taking the proportion of the closest expert driving behavior as the score of the driving behavior to be tested, and the score range is [0,1]; it can also mean: taking the score of the closest expert driving behavior based on cost function, deep learning evaluation network, manual evaluation, etc. as the score of the driving behavior to be tested.

[0177] Figure 4 The following is a flow chart of a method for determining a driving scene fingerprint. The method can be applied to an electronic device or a computer device, and can be applied to a terminal or a server. Specifically, the method can include the following steps:

[0178] S210: Obtain obstacles, navigation routes, and map topology in the driving scene.

[0179] Driving scenarios include static scenarios and dynamic scenarios. Static scenarios are driving scenarios at a certain moment, and dynamic scenarios are driving scenarios at multiple moments in a row. Each driving scenario includes map topology, navigation routes, and obstacles. Obstacles include static obstacles and dynamic obstacles. Static obstacles include static objects placed around the road, such as stones, and dynamic obstacles include vehicles, pedestrians, animals, etc.

[0180] S220: Perform feature encoding on the obstacle to obtain a first encoding result.

[0181] The obstacles include obstacle characteristics, obstacle type number, obstacle sequence number, obstacle position and obstacle position offset.

[0182] The method of encoding the features of obstacles includes but is not limited to the following two: (1) Encoding the obstacle features based on the first encoding network to obtain the first obstacle encoding result; encoding the first obstacle encoding result, the obstacle type number and the obstacle serial number based on the second encoding network to obtain the second obstacle encoding result; encoding the obstacle position and the obstacle position offset based on the third encoding network to obtain the third obstacle encoding result and the fourth obstacle encoding result; splicing the second obstacle encoding result, the third obstacle encoding result and the fourth obstacle encoding result to obtain the fifth obstacle encoding result; encoding the fifth obstacle encoding result, the obstacle type number and the obstacle serial number based on the second encoding network to obtain the first encoding result. (2) Directly encoding the obstacle features, the obstacle type number, the obstacle serial number, the obstacle position and the obstacle position offset together based on any one of the first encoding network, the second encoding network or the third encoding network to obtain the first encoding result.

[0183] The first coding network, the second coding network and the third coding network may be the same or different.

[0184] The first encoding network includes a multi-layer perceptron MLP; and / or,

[0185] The second encoding network includes any one of context gating CG, multi-scale context gating MCG, and self-attention mechanism; and / or,

[0186] The third encoding network includes a long short-term memory network LSTM.

[0187] S230: Feature encoding is performed on the first encoding result and the navigation route to obtain a second encoding result.

[0188] The navigation route includes navigation information characteristics, navigation topology element numbers and navigation topology element sequences.

[0189] Methods for encoding the features of the first coding result and the navigation route include but are not limited to the following two: (1) encoding the navigation information features based on the first coding network to obtain the navigation coding result; encoding the navigation topology element number, the navigation topology element sequence number, the navigation coding result and the first coding result based on the second coding network to obtain the second coding result. (2) directly encoding the first coding result, the navigation information features, the navigation topology element number and the navigation topology element sequence number based on any coding network of the first coding network, the second coding network or the third coding network to obtain the second coding result.

[0190] S240: Perform feature encoding on the first encoding result and the map topology structure to obtain a third encoding result.

[0191] The map topology structure includes map topology features, map topology element numbers and map topology element sequences.

[0192] Methods for feature encoding the first coding result and the map topology structure include but are not limited to the following two: (1) Encoding the map topology feature based on the first coding network to obtain the map topology coding result; encoding the map topology element number, the map topology element sequence number, the map topology coding result and the first coding result based on the second coding network to obtain the third coding result. (2) Directly encoding the first coding result, the map topology feature, the map topology element number and the map topology element sequence number based on any coding network of the first coding network, the second coding network or the third coding network to obtain the third coding result.

[0193] S250: Concatenate the first encoding result, the second encoding result, and the third encoding result to obtain a fourth encoding result.

[0194] The embodiment of the present application may not limit the order of splicing the first coding result, the second coding result, and the third coding result, as long as the splicing order of calculating all driving scenes is unified. For example, they may be spliced ​​in the order of first coding result-second coding result-third coding result, or in the order of third coding result-second coding result-first coding result.

[0195] S260: Calculate the fourth encoding result based on the mapping function to obtain a scene fingerprint of the driving scene.

[0196] The mapping function may be a function that can transform an input of any length into an output of a fixed length. For example, the mapping function may be a hash function or other hash functions.

[0197] The driving scene fingerprint determination method provided in the embodiment of the present application can obtain a scene fingerprint that can uniquely identify the driving scene by orderly encoding obstacles, navigation routes and map topology structures in the driving scene.

[0198] Based on the above method embodiment, another embodiment of the present application provides an automatic driving algorithm evaluation device, such as Figure 5 As shown, the device comprises:

[0199] An acquisition unit 310 is used to acquire a scene to be tested and a driving behavior to be tested, wherein the scene to be tested is a driving scene for autonomous driving based on the autonomous driving algorithm to be tested, and the driving behavior to be tested is a driving behavior generated by the vehicle during the autonomous driving process of the scene to be tested based on the autonomous driving algorithm to be tested;

[0200] A calculation unit 320, configured to calculate a scene fingerprint of the scene to be tested, wherein the scene fingerprint is used to uniquely identify a driving scene;

[0201] A screening unit 330 is used to determine a target scene group that matches the scene fingerprint of the scene to be tested from multiple scene groups based on the similarity of the scene fingerprints, wherein the scene group includes scene fingerprints of scenes driven by multiple expert drivers, and multiple scene fingerprints in each of the scene groups are the same and / or similar;

[0202] The determination unit 340 is used to determine that the autonomous driving algorithm to be tested meets the driving expectations when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, wherein the feasible domain of the expected driving behavior is a feasible domain composed of multiple expert driving behaviors that meet the driving expectations; when the driving behavior to be tested is outside the feasible domain of the expected driving behavior corresponding to the target scenario group, determine that the autonomous driving algorithm to be tested does not meet the driving expectations.

[0203] In a possible implementation, the calculation unit 320 includes:

[0204] An acquisition module, used to acquire obstacles, navigation routes and map topology in the scene to be tested;

[0205] A first encoding module, used to perform feature encoding on the obstacle to obtain a first encoding result;

[0206] A second encoding module, used for performing feature encoding on the first encoding result and the navigation route to obtain a second encoding result;

[0207] A third encoding module, used for performing feature encoding on the first encoding result and the map topology structure to obtain a third encoding result;

[0208] a concatenation module, configured to concatenate the first encoding result, the second encoding result and the third encoding result to obtain a fourth encoding result;

[0209] A calculation module is used to calculate the fourth encoding result based on a mapping function to obtain a scene fingerprint of the scene to be tested.

[0210] In a possible implementation, the obstacle includes obstacle features, obstacle type number, obstacle sequence number, obstacle position, and obstacle position offset, and the first encoding module is used to:

[0211] Encoding the obstacle feature based on a first encoding network to obtain a first obstacle encoding result;

[0212] Encoding the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result;

[0213] Based on a third coding network, respectively encode the obstacle position and the obstacle position offset to obtain a third obstacle coding result and a fourth obstacle coding result;

[0214] concatenating the second obstacle encoding result, the third obstacle encoding result, and the fourth obstacle encoding result to obtain a fifth obstacle encoding result;

[0215] The fifth obstacle coding result, the obstacle type number and the obstacle sequence number are encoded based on the second coding network to obtain the first coding result.

[0216] In a possible implementation, the navigation route includes a navigation information feature, a navigation topology element number and a navigation topology element sequence number, and the second encoding module is used to:

[0217] Encoding the navigation information feature based on the first coding network to obtain a navigation coding result;

[0218] The navigation topology element number, the navigation topology element sequence number, the navigation coding result, and the first coding result are encoded based on the second coding network to obtain the second coding result.

[0219] In a possible implementation, the map topology structure includes a map topology feature, a map topology element number and a map topology element sequence number, and the third encoding module is used to:

[0220] Encoding the map topology feature based on the first encoding network to obtain a map topology encoding result;

[0221] The map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result are encoded based on the second encoding network to obtain the third encoding result.

[0222] In a possible implementation, the determination unit 340 is further used to, for each of the scenario groups, when the expert driving behaviors corresponding to the respective driving scenarios in the scenario group are the same, the feasible domain of the expected driving behaviors corresponding to the scenario group includes the same expert driving behaviors and their proportions; when the expert driving behaviors corresponding to the respective driving scenarios in the scenario group are different, screen out the expert driving behaviors having a score greater than or equal to a preset score threshold, and determine the feasible domain of the expected driving behavior corresponding to the scenario group based on the screened multiple expert driving behaviors and the proportions of the various expert driving behaviors.

[0223] In a possible implementation, the determination unit 340 is further used to determine, when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, an expert driving behavior within the feasible domain of the expected driving behavior corresponding to the target scenario group that is closest to the driving behavior to be tested; and use the score of the close expert driving behavior as the score of the driving behavior to be tested.

[0224] The autonomous driving algorithm evaluation device provided in the embodiment of the present application can first calculate the scene fingerprint of the scene to be tested after obtaining the scene to be tested and the driving behavior to be tested, and then determine the target scene group that matches the scene fingerprint of the scene to be tested from multiple scene groups based on the similarity of the scene fingerprints, and finally, when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scene group, it can be determined that the autonomous driving algorithm to be tested meets the driving expectations, otherwise it does not meet the driving expectations. Compared with the traditional method of evaluating autonomous driving algorithms based on reference truth values, the embodiment of the present application can first determine the feasible domain of expected driving behaviors of multiple scene groups based on the expert driving behaviors of multiple experts in multiple driving scenes, realize scene grouping and aggregation of expert driving behaviors, and thus realize automatic, accurate and efficient evaluation of the autonomous driving algorithm to be tested without the need for manual annotation of truth values.

[0225] Based on the above method embodiment, another embodiment of the present application provides a driving scene fingerprint determination device, such as Figure 6As shown, the device comprises:

[0226] An acquisition unit 410 is used to acquire obstacles, navigation routes and map topology in the driving scene;

[0227] A first encoding unit 420, configured to perform feature encoding on the obstacle to obtain a first encoding result;

[0228] A second encoding unit 430 is used to perform feature encoding on the first encoding result and the navigation route to obtain a second encoding result;

[0229] A third encoding unit 440 is used to perform feature encoding on the first encoding result and the map topology structure to obtain a third encoding result;

[0230] A concatenation unit 450, configured to concatenate the first coding result, the second coding result and the third coding result to obtain a fourth coding result;

[0231] The calculation unit 460 is used to calculate the fourth encoding result based on a mapping function to obtain a scene fingerprint of the driving scene.

[0232] In a possible implementation, the obstacle includes obstacle features, obstacle type number, obstacle sequence number, obstacle position, and obstacle position offset, and the first encoding unit 420 includes:

[0233] A first encoding module, used to encode the obstacle feature based on a first encoding network to obtain a first obstacle encoding result;

[0234] a second encoding module, configured to encode the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result;

[0235] a third encoding module, configured to encode the obstacle position and the obstacle position offset respectively based on a third encoding network to obtain a third obstacle encoding result and a fourth obstacle encoding result;

[0236] a splicing module, configured to splice the second obstacle coding result, the third obstacle coding result and the fourth obstacle coding result to obtain a fifth obstacle coding result;

[0237] The fourth encoding module is used to encode the fifth obstacle encoding result, the obstacle type number and the obstacle sequence number based on the second encoding network to obtain the first encoding result.

[0238] In a possible implementation, the navigation route includes navigation information features, navigation topology element numbers and navigation topology element sequences, and the second encoding unit 430 includes:

[0239] a fifth encoding module, configured to encode the navigation information feature based on the first encoding network to obtain a navigation encoding result;

[0240] The sixth encoding module is used to encode the navigation topology element number, the navigation topology element sequence number, the navigation encoding result and the first encoding result based on the second encoding network to obtain the second encoding result.

[0241] In a possible implementation, the map topology structure includes a map topology feature, a map topology element number and a map topology element sequence number, and the third encoding unit 440 includes:

[0242] a seventh encoding module, configured to encode the map topology feature based on the first encoding network to obtain a map topology encoding result;

[0243] An eighth encoding module is used to encode the map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result based on the second encoding network to obtain the third encoding result.

[0244] In a possible implementation, the first encoding network includes a multi-layer perceptron MLP; and / or,

[0245] The second encoding network includes any one of context gating CG, multi-scale context gating MCG, and self-attention mechanism; and / or,

[0246] The third encoding network includes a long short-term memory network LSTM.

[0247] The driving scene fingerprint determination device provided in the embodiment of the present application can obtain a scene fingerprint that can uniquely identify the driving scene by orderly encoding obstacles, navigation routes and map topology structures in the driving scene.

[0248] Based on the above method embodiments, another embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any of the above embodiments is implemented.

[0249] Based on the above method embodiment, another embodiment of the present application provides an electronic device or a computer device, such as Figure 7 As shown, including:

[0250] one or more processors 510;

[0251] The processor 510 is coupled to a storage device 520, and the storage device 520 is used to store one or more programs;

[0252] When the one or more programs are executed by the one or more processors 510, the electronic device or computer device implements the method described in any of the above embodiments.

[0253] Based on the above embodiments, another embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer or a processor, the computer or the processor executes the method described in any of the above embodiments.

[0254] The above device embodiment corresponds to the method embodiment and has the same technical effect as the method embodiment. For specific description, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For specific description, please refer to the method embodiment part, which will not be repeated here. It can be understood by those skilled in the art that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application.

[0255] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0256] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for evaluating an autonomous driving algorithm, characterized in that: The method comprises: Acquire a scene to be tested and a driving behavior to be tested, wherein the scene to be tested is a driving scene for autonomous driving based on the autonomous driving algorithm to be tested, and the driving behavior to be tested is a driving behavior generated by the vehicle during the autonomous driving process of the scene to be tested based on the autonomous driving algorithm to be tested; Calculating a scene fingerprint of the scene to be tested, wherein the scene fingerprint is used to uniquely identify a driving scene; Based on the similarity of the scene fingerprints, determining a target scene group that matches the scene fingerprint of the scene to be tested from multiple scene groups, wherein the scene group includes scene fingerprints of scenes driven by multiple expert drivers, and multiple scene fingerprints in each of the scene groups are the same and / or similar; When the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, determining that the automatic driving algorithm to be tested meets the driving expectation, wherein the feasible domain of the expected driving behavior is a feasible domain composed of multiple expert driving behaviors that meet the driving expectation; When the driving behavior to be tested is outside the feasible domain of the expected driving behavior corresponding to the target scenario group, it is determined that the automatic driving algorithm to be tested does not meet the driving expectation.

2. The method according to claim 1, characterized in that The calculating the scene fingerprint of the scene to be tested includes: Obtain obstacles, navigation routes and map topology in the scene to be tested; Performing feature coding on the obstacle to obtain a first coding result; Performing feature coding on the first coding result and the navigation route to obtain a second coding result; Performing feature encoding on the first encoding result and the map topology structure to obtain a third encoding result; concatenating the first encoding result, the second encoding result, and the third encoding result to obtain a fourth encoding result; The fourth encoding result is calculated based on a mapping function to obtain a scene fingerprint of the scene to be tested.

3. The method according to claim 2, characterized in that The obstacle includes obstacle features, obstacle type number, obstacle sequence number, obstacle position and obstacle position offset, and feature encoding is performed on the obstacle to obtain a first encoding result, including: Encoding the obstacle feature based on a first encoding network to obtain a first obstacle encoding result; Encoding the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result; Based on a third coding network, respectively encode the obstacle position and the obstacle position offset to obtain a third obstacle coding result and a fourth obstacle coding result; concatenating the second obstacle encoding result, the third obstacle encoding result, and the fourth obstacle encoding result to obtain a fifth obstacle encoding result; The fifth obstacle coding result, the obstacle type number and the obstacle sequence number are encoded based on the second coding network to obtain the first coding result.

4. The method according to claim 3, characterized in that The navigation route includes navigation information features, navigation topology element numbers and navigation topology element sequences, and feature encoding is performed on the first coding result and the navigation route to obtain a second coding result, including: Encoding the navigation information feature based on the first coding network to obtain a navigation coding result; The navigation topology element number, the navigation topology element sequence number, the navigation coding result, and the first coding result are encoded based on the second coding network to obtain the second coding result.

5. The method according to claim 4, characterized in that The map topology structure includes a map topology feature, a map topology element number, and a map topology element sequence number. Feature encoding is performed on the first encoding result and the map topology structure to obtain a third encoding result, including: Encoding the map topology feature based on the first encoding network to obtain a map topology encoding result; The map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result are encoded based on the second encoding network to obtain the third encoding result.

6. The method according to claim 1, characterized in that The method for determining the feasible domain of the expected driving behavior corresponding to each of the scenario groups includes: For each of the scenario groups, when the expert driving behaviors corresponding to the driving scenarios in the scenario group are the same, the feasible domain of the expected driving behaviors corresponding to the scenario group includes the same expert driving behaviors and their proportions; When the expert driving behaviors corresponding to the various driving scenarios in the scenario group are different, the expert driving behaviors with scores greater than or equal to a preset score threshold are screened out, and the feasible domain of the expected driving behavior corresponding to the scenario group is determined based on the screened multiple expert driving behaviors and the proportion of various expert driving behaviors.

7. The method according to claim 1, characterized in that When the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, the method further includes: Determine an expert driving behavior that is closest to the driving behavior to be tested within the feasible domain of the expected driving behavior corresponding to the target scenario group; The score of the approximate expert driving behavior is used as the score of the driving behavior to be tested.

8. A driving scene fingerprint determination method, characterized in that: The method comprises: Obtain obstacles, navigation routes and map topology in driving scenarios; Performing feature coding on the obstacle to obtain a first coding result; Performing feature coding on the first coding result and the navigation route to obtain a second coding result; Performing feature encoding on the first encoding result and the map topology structure to obtain a third encoding result; concatenating the first encoding result, the second encoding result, and the third encoding result to obtain a fourth encoding result; The fourth encoding result is calculated based on a mapping function to obtain a scene fingerprint of the driving scene.

9. The method according to claim 8, characterized in that The obstacle includes obstacle features, obstacle type number, obstacle sequence number, obstacle position and obstacle position offset, and feature encoding is performed on the obstacle to obtain a first encoding result, including: Encoding the obstacle feature based on a first encoding network to obtain a first obstacle encoding result; Encoding the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result; Based on a third coding network, respectively encode the obstacle position and the obstacle position offset to obtain a third obstacle coding result and a fourth obstacle coding result; concatenating the second obstacle encoding result, the third obstacle encoding result, and the fourth obstacle encoding result to obtain a fifth obstacle encoding result; The fifth obstacle coding result, the obstacle type number and the obstacle sequence number are encoded based on the second coding network to obtain the first coding result.

10. The method according to claim 9, characterized in that The navigation route includes navigation information features, navigation topology element numbers and navigation topology element sequences, and feature encoding is performed on the first coding result and the navigation route to obtain a second coding result, including: Encoding the navigation information feature based on the first coding network to obtain a navigation coding result; The navigation topology element number, the navigation topology element sequence number, the navigation coding result, and the first coding result are encoded based on the second coding network to obtain the second coding result.

11. The method according to claim 10, characterized in that The map topology structure includes a map topology feature, a map topology element number, and a map topology element sequence number. Feature encoding is performed on the first encoding result and the map topology structure to obtain a third encoding result, including: Encoding the map topology feature based on the first encoding network to obtain a map topology encoding result; The map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result are encoded based on the second encoding network to obtain the third encoding result.

12. The method according to any one of claims 9 to 11, characterized in that: The first encoding network comprises a multi-layer perceptron MLP; and / or, The second encoding network includes any one of context gating CG, multi-scale context gating MCG, and self-attention mechanism; and / or, The third encoding network includes a long short-term memory network LSTM.

13. An automatic driving algorithm evaluation device, characterized in that: The device comprises: an acquisition unit, configured to acquire a scene to be tested and a driving behavior to be tested, wherein the scene to be tested is a driving scene for autonomous driving based on the autonomous driving algorithm to be tested, and the driving behavior to be tested is a driving behavior generated by a vehicle during the autonomous driving process of the scene to be tested based on the autonomous driving algorithm to be tested; A calculation unit, used to calculate a scene fingerprint of the scene to be tested, wherein the scene fingerprint is used to uniquely identify a driving scene; A screening unit, configured to determine, from a plurality of scene groups, a target scene group that matches the scene fingerprint of the scene to be tested based on the similarity of the scene fingerprints, wherein the scene group includes scene fingerprints of scenes driven by a plurality of expert drivers, and a plurality of scene fingerprints in each of the scene groups are identical and / or similar; A determination unit is used to determine that the autonomous driving algorithm to be tested meets the driving expectation when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, wherein the feasible domain of the expected driving behavior is a feasible domain composed of multiple expert driving behaviors that meet the driving expectation; and to determine that the autonomous driving algorithm to be tested does not meet the driving expectation when the driving behavior to be tested is outside the feasible domain of the expected driving behavior corresponding to the target scenario group.

14. The device according to claim 13, characterized in that The determination unit is further configured to, for each of the scenario groups, when the expert driving behaviors corresponding to the respective driving scenarios in the scenario group are the same, the feasible domain of the expected driving behaviors corresponding to the scenario group includes the same expert driving behaviors and their proportions; when the expert driving behaviors corresponding to the respective driving scenarios in the scenario group are different, screen out the expert driving behaviors having a score greater than or equal to a preset score threshold, and determine the feasible domain of the expected driving behavior corresponding to the scenario group based on the screened multiple expert driving behaviors and the proportions of the various expert driving behaviors.

15. The device according to claim 13 or 14, characterized in that The determination unit is further configured to determine, when the driving behavior to be tested is within the feasible domain of the expected driving behavior corresponding to the target scenario group, an expert driving behavior that is closest to the driving behavior to be tested within the feasible domain of the expected driving behavior corresponding to the target scenario group; The score of the approximate expert driving behavior is used as the score of the driving behavior to be tested.

16. A driving scene fingerprint determination device, characterized in that: The device comprises: An acquisition unit, used to acquire obstacles, navigation routes and map topology in the driving scene; A first encoding unit, used to perform feature encoding on the obstacle to obtain a first encoding result; A second encoding unit, used for performing feature encoding on the first encoding result and the navigation route to obtain a second encoding result; A third encoding unit, used for performing feature encoding on the first encoding result and the map topology structure to obtain a third encoding result; a concatenation unit, configured to concatenate the first coding result, the second coding result and the third coding result to obtain a fourth coding result; A calculation unit is used to calculate the fourth encoding result based on a mapping function to obtain a scene fingerprint of the driving scene.

17. The device according to claim 16, characterized in that The obstacle includes obstacle features, obstacle type number, obstacle sequence number, obstacle position and obstacle position offset, and the first encoding unit includes: A first encoding module, used to encode the obstacle feature based on a first encoding network to obtain a first obstacle encoding result; a second encoding module, configured to encode the first obstacle encoding result, the obstacle type number and the obstacle sequence number based on a second encoding network to obtain a second obstacle encoding result; a third encoding module, configured to encode the obstacle position and the obstacle position offset respectively based on a third encoding network to obtain a third obstacle encoding result and a fourth obstacle encoding result; a splicing module, configured to splice the second obstacle coding result, the third obstacle coding result and the fourth obstacle coding result to obtain a fifth obstacle coding result; The fourth encoding module is used to encode the fifth obstacle encoding result, the obstacle type number and the obstacle sequence number based on the second encoding network to obtain the first encoding result.

18. The device according to claim 17, characterized in that The navigation route includes navigation information features, navigation topology element numbers and navigation topology element sequences, and the second encoding unit includes: a fifth encoding module, configured to encode the navigation information feature based on the first encoding network to obtain a navigation encoding result; The sixth encoding module is used to encode the navigation topology element number, the navigation topology element sequence number, the navigation encoding result and the first encoding result based on the second encoding network to obtain the second encoding result.

19. The device according to claim 18, characterized in that The map topology structure includes a map topology feature, a map topology element number and a map topology element sequence number, and the third encoding unit includes: a seventh encoding module, configured to encode the map topology feature based on the first encoding network to obtain a map topology encoding result; An eighth encoding module is used to encode the map topology element number, the map topology element sequence number, the map topology encoding result and the first encoding result based on the second encoding network to obtain the third encoding result.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements any one of claims 1 to 7, or implements the method according to any one of claims 8 to 12.

21. An electronic device, characterized in that: The electronic device comprises: one or more processors; The processor is coupled to a storage device, and the storage device is used to store one or more programs; When the one or more programs are executed by the one or more processors, the electronic device implements any one of claims 1 to 7, or implements the method according to any one of claims 8 to 12.

Images