Evaluation method of autonomous driving system, training method and device of related model

Abstract

The application discloses an evaluation method of an automatic driving system. The method comprises the following steps: obtaining an original driving data set; processing the original driving data set by using an automatic driving system evaluation model to obtain a multi-dimensional evaluation result of the automatic driving system corresponding to the original driving data set, wherein the automatic driving system evaluation model is used for multi-dimensional evaluation of the automatic driving system. The application also discloses a training method and equipment of a related model. The application improves the objectivity and universality of the evaluation of the automatic driving system, and realizes multi-dimensional evaluation of the automatic driving system.

Description

Technical Field

[0001] The disclosed embodiments of this application relate to the field of autonomous driving technology, and more specifically, to evaluation methods for autonomous driving systems, training methods for related models, and devices. Background Technology

[0002] An objective and universally applicable evaluation system for autonomous driving systems, combined with mainstream autonomous driving simulation systems, can efficiently improve the iteration efficiency of autonomous driving algorithms and save road testing resources. Currently, common evaluation systems for autonomous driving systems rely on human experience, statistical methods, or single-dimensional methods (such as drawing dragons, crossing lines, or tactile perception) to label each item and then use small-scale models to fit human judgments. Such methods suffer from problems such as single evaluation dimensions, low objectivity and accuracy, and poor timeliness and universality. Summary of the Invention

[0003] According to embodiments of this application, this application proposes an evaluation method for an autonomous driving system, a training method for related models, and an apparatus to solve the above-mentioned problems, improve the objectivity and universality of the evaluation of autonomous driving systems, and realize multi-dimensional evaluation of autonomous driving systems.

[0004] The first aspect of this application discloses a method for evaluating a driving system, comprising: acquiring an original driving dataset; processing the original driving dataset using an autonomous driving system evaluation model to obtain a multi-dimensional evaluation result of the autonomous driving system corresponding to the original driving dataset, wherein the autonomous driving system evaluation model is used to evaluate the autonomous driving system in multiple dimensions.

[0005] In some embodiments, inputting the original driving dataset into an autonomous driving evaluation model includes: extracting features from the original driving dataset to obtain input feature information corresponding to the original driving dataset; vectorizing the input feature information to obtain a vector representation of the input feature information, wherein the vector representation of the input feature information includes a vector representation of the scene description feature information, a first vector representing the corresponding description type, and a second vector sequence representing the corresponding question type; superimposing the vector representation of the scene description feature information with the first vector representing the corresponding description type to obtain a first vector sequence corresponding to the scene description feature information; and linking the first vector sequence and the second vector sequence representing the corresponding question type to input them into the autonomous driving system evaluation model to train the autonomous driving system evaluation model.

[0006] In some embodiments, the method further includes: using the evaluation results corresponding to the input problem description information to obtain driving performance information of the autonomous driving system.

[0007] The second aspect of this application discloses a training method for an evaluation model of an autonomous driving system, comprising: collecting an original driving dataset to obtain scene description information corresponding to the original driving dataset, wherein the scene description information includes at least one type of problem data; performing pre-processing on the original driving dataset to obtain input feature information corresponding to the original driving dataset, wherein the input feature information includes scene description feature information; and using the input feature information to train the evaluation model of the autonomous driving system, wherein the evaluation model of the autonomous driving system is used to implement the evaluation method of the autonomous driving system described in the first aspect.

[0008] In some embodiments, the step of performing preset processing on the original driving dataset to obtain input feature information corresponding to the original driving dataset includes: performing feature extraction on the original driving dataset to obtain input feature information corresponding to the original driving dataset; the step of using the input feature information for training the autonomous driving system evaluation model includes: performing vectorization processing on the input feature information to obtain a vector representation of the input feature information, wherein the vector representation of the input feature information includes a vector representation of the scene description feature information, a first vector for representing the corresponding description type, and a second vector sequence for representing the corresponding question type; superimposing the vector representation of the scene description feature information with the first vector for representing the corresponding description type to obtain a first vector sequence corresponding to the scene description feature information; linking the first vector sequence and the second vector sequence for representing the corresponding question type to input into the autonomous driving system evaluation model to achieve training of the autonomous driving system evaluation model.

[0009] In some embodiments, the step of extracting features from the original driving dataset to obtain input feature information corresponding to the original driving dataset includes: constructing vehicle feature information based on at least one frame of historical vehicle data; constructing object feature information based on object information in at least one frame of historical data in the perception environment; constructing lane line feature information based on lane line information and traffic light information in the perception map; and constructing path feature information based on the vehicle's path selection information.

[0010] In some embodiments, before performing feature extraction on the original driving dataset, the method includes: cleaning the original driving dataset to obtain a cleaned original driving dataset for performing the feature extraction.

[0011] In some embodiments, using the input feature information for training the autonomous driving system evaluation model further includes: splitting the original driving dataset according to a preset ratio, using it as a training dataset and a test dataset respectively, wherein the training dataset is used to optimize the autonomous driving system evaluation model, and the test dataset is used to perform multi-dimensional evaluation of the autonomous driving system.

[0012] The third aspect of this application discloses an electronic device including a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the evaluation method for the autonomous driving system described in the first aspect, or to implement the training method for the evaluation model of the autonomous driving system described in the second aspect.

[0013] The fourth aspect of this application discloses a non-volatile computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the evaluation method for the autonomous driving system described in the first aspect, or the method for implementing the evaluation model for the autonomous driving system described in the second aspect.

[0014] The beneficial effects of this application are: obtaining the original driving dataset, processing the original driving dataset using the autonomous driving system evaluation model to obtain the multi-dimensional evaluation results of the autonomous driving system corresponding to the original driving dataset, wherein the autonomous driving system evaluation model is used to evaluate the autonomous driving system in multiple dimensions, further improving the objectivity and universality of the evaluation of the autonomous driving system, and realizing the multi-dimensional evaluation of the autonomous driving system. Attached Figure Description

[0015] The present application will be further described below with reference to the accompanying drawings and embodiments. In the drawings:

[0016] Figure 1 This is a flowchart illustrating the evaluation method for a driving system according to an embodiment of this application;

[0017] Figure 2 This is a flowchart illustrating the training method of the evaluation model for an autonomous driving system according to an embodiment of this application.

[0018] Figure 3 This is a schematic diagram illustrating the training process of the autonomous driving system evaluation model according to an embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0020] Figure 5 This is a schematic diagram of the structure of a non-volatile computer-readable storage medium according to an embodiment of this application. Detailed Implementation

[0021] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, "many" in this application means two or more. Moreover, the term "at least one" in this application means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. Furthermore, the terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0023] To enable those skilled in the art to better understand the technical solution of this application, the technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Please see Figure 1 , Figure 1 This is a flowchart illustrating an evaluation method for an autonomous driving system according to an embodiment of this application. This method can be applied to electronic devices with computing or other functions. It should be noted that if substantially the same result is obtained, the method of this application does not necessarily require further clarification. Figure 1 The sequence of processes shown is limited.

[0025] In some possible implementations, this method can be implemented by the processor calling computer-readable instructions stored in memory, such as... Figure 1 As shown, the method may include the following steps:

[0026] S11: Obtain the original driving dataset.

[0027] Raw driving datasets include raw point cloud data, image data, radar data, GPS / IMU data, etc., acquired through sensor devices. These sensor devices include image sensors and radar sensors. The radar sensors can be radar devices used for autonomous driving and meeting accuracy requirements, providing point cloud perception. Image sensors, such as cameras, can be used to acquire image data. Radar sensors, such as millimeter-wave radar and lidar, can be used to acquire point cloud data. Image sensors and radar sensors can be mounted on a mobile device, such as an autonomous vehicle. LiDAR can include mechanical lidar, semi-solid-state lidar, or solid-state lidar.

[0028] In one application scenario, an autonomous vehicle is driving on a road. Image sensors installed on the autonomous vehicle acquire image data to describe the environmental space in which the onboard equipment is located, resulting in an initial dataset. Radar sensors are used to acquire point cloud data to describe the environmental space in which the onboard equipment is located, resulting in a raw driving dataset.

[0029] Each sensor perceives and captures a raw driving dataset to describe the environmental space in which the vehicle-mounted device is located. Each raw driving dataset corresponds to one sensor, and thus at least two sensors capture at least two raw driving datasets. The types of raw driving datasets include, but are not limited to, image data and point cloud data.

[0030] Obtain the raw driving dataset, which may include at least one type of problem data, such as problem data of different types of planning that meet the scene coverage distribution, as input problem description information, such as solid line crossing problem data, wrong road taking problem data, dragon drawing problem data (the vehicle cannot guarantee driving in the middle and frequently draws dragons to the left and right), haptic problem data, etc.

[0031] S12: The original driving dataset is processed using the autonomous driving system evaluation model to obtain the multi-dimensional evaluation results of the autonomous driving system corresponding to the original driving dataset. The autonomous driving system evaluation model is used to evaluate the autonomous driving system in multiple dimensions.

[0032] An autonomous driving system evaluation model is used to process the original driving dataset to obtain multi-dimensional evaluation results of the autonomous driving system corresponding to the original driving dataset. For example, this could be the probability of different problem types corresponding to the original driving dataset. The autonomous driving system evaluation model is used to perform multi-dimensional evaluation of the autonomous driving system, and the model can be trained based on a pre-defined method.

[0033] In this embodiment, the original driving dataset is obtained, and the autonomous driving system evaluation model is used to process the original driving dataset to obtain the multi-dimensional evaluation results of the autonomous driving system corresponding to the original driving dataset. The autonomous driving system evaluation model is used to evaluate the autonomous driving system from multiple dimensions, which further improves the objectivity and universality of the evaluation of the autonomous driving system and realizes the multi-dimensional evaluation of the autonomous driving system.

[0034] In some embodiments, the original driving dataset is processed using an autonomous driving system evaluation model, including: extracting features from the original driving dataset to obtain input feature information corresponding to the original driving dataset; vectorizing the input feature information to obtain a vector representation of the input feature information, wherein the vector representation of the input feature information includes a vector representation of scene description feature information and a first vector used to represent the corresponding description type; superimposing the vector representation of the scene description feature information with the first vector used to represent the corresponding description type to obtain a first vector sequence corresponding to the scene description feature information; and linking the first vector sequence with a second vector sequence used to represent the corresponding problem type to output a multi-dimensional evaluation result, thereby realizing a multi-dimensional evaluation of the autonomous driving system.

[0035] Feature extraction is performed on the original driving dataset to obtain the corresponding input feature information, which includes scene description feature information. The input feature information is then vectorized to obtain a vector representation of the input feature information. This vector representation includes a vector representation of the scene description feature information and a first vector representing the corresponding description type. For example, this can be achieved through Bezier curve mapping to control point coordinates or other preprocessing networks to transform the input feature information into a vectorized feature description, such as the vector representation of historical data features of autonomous vehicles ({Ego Car Token}), object information features ({Object token}), lane line features ({lane token}), and vehicle path features ({Routing Token}). An embedding layer is then used to obtain the first vector representing the corresponding description type, i.e., the vector representation of the description type of different scene description features, such as Type-1, Type-2, Type-3, etc., where Type-1 can be a vector representation of the feature type of historical data, Type-2 can be a vector representation of the feature type of object information, and so on.

[0036] At this point, the vector representation of the scene description feature information is superimposed with a first vector representing the corresponding description type to obtain a first vector sequence corresponding to the scene description feature information. Then, the first vector sequence is linked with a second vector sequence representing the corresponding problem type. This second vector sequence can represent different problem types, such as {issue1, issue2, ...}, where issue1 can be a vector representation of solid line problems, issue2 can be a vector representation of wrong-way problems, and so on. Further, a Transformer network structure is used for processing, outputting the multi-dimensional evaluation result to obtain the probabilities of different problem types corresponding to the input feature information, such as {Label1, Label2, Label3, ...}, where Label1 can be defined as {solid line problem, 0}, probability Label2 can be defined as {dragon drawing problem, 1}, ..., thereby achieving the evaluation of the autonomous driving system.

[0037] Furthermore, in some embodiments, the evaluation results corresponding to the input problem description information are used to obtain information on the driving performance of the autonomous driving system.

[0038] By utilizing the evaluation results corresponding to the input problem description information, we can obtain information on the driving effect of the autonomous driving system. In other words, we can use the model output results to evaluate the driving effect of the autonomous driving system / planning and control full-scenario regression test on the simulation platform.

[0039] Please see Figure 2 , Figure 2 This is a flowchart illustrating the training method for an evaluation model of an autonomous driving system according to an embodiment of this application. The subject executing this method can be an electronic device with computing capabilities, such as a microcomputer, a server, and mobile devices such as laptops and tablets.

[0040] It should be noted that if substantially the same result is obtained, the method of this application is not based on... Figure 2 The sequence of processes shown is limited.

[0041] In some possible implementations, this method can be implemented by the processor calling computer-readable instructions stored in memory, such as... Figure 1 As shown, the method may include the following steps:

[0042] S21: Collect the original driving dataset to obtain the scene description information corresponding to the original driving dataset, wherein the scene description information includes at least one type of problem data.

[0043] Furthermore, scene description information corresponding to the original driving dataset is obtained. This scene description information includes at least one type of problem data, such as road test problem data of different types that meet the scene coverage distribution, used as positive samples for model training. Examples include solid line crossing problem data, wrong route problem data, "dragging the line" problem data (the vehicle cannot guarantee driving in the middle and frequently darts left and right), and haptic problem data. In addition, the scene description information also includes excellent human driving data that meet the scene coverage distribution, i.e., driving data with a low probability of belonging to any type of problem, used as negative samples for model training.

[0044] S22: Perform pre-processing on the original driving dataset to obtain the input feature information corresponding to the original driving dataset. The input feature information includes scene description feature information.

[0045] The original driving dataset is pre-processed, such as by extracting sample input features, to obtain the input feature information corresponding to the original driving dataset, which is the data feature of the input data to be evaluated. The input feature information can include scene description feature information, such as the feature information of the historical data of the autonomous vehicle {Ego Car Traj}, the feature information of the object information {Object Traj}, the feature information of the lane line {lane}, and the feature information of the vehicle path {Routing}, etc.

[0046] S23: Use the input feature information to train the evaluation model for the autonomous driving system.

[0047] By performing pre-processing on the original driving dataset, the input feature information corresponding to the original driving dataset is obtained. The input feature information is then used to train the autonomous driving system evaluation model. For example, the transformer model is used to process the input feature information to train the autonomous driving system evaluation model, and then the multi-dimensional information of the autonomous driving system is output as the result, such as the probability of occurrence of different problem types corresponding to the input feature information.

[0048] The autonomous driving system evaluation model is used to implement the above-mentioned evaluation method for autonomous driving systems. That is, to obtain the original driving dataset, process the original driving dataset using the autonomous driving system evaluation model, and obtain the multi-dimensional evaluation results of the autonomous driving system corresponding to the original driving dataset. The specific details will not be elaborated here.

[0049] In some embodiments, the original driving dataset is subjected to pre-processing to obtain input feature information corresponding to the original driving dataset, including: extracting features from the original driving dataset to obtain input feature information corresponding to the original driving dataset; using the input feature information for training an autonomous driving system evaluation model, including: vectorizing the input feature information to obtain a vector representation of the input feature information, the vector representation of the input feature information including a vector representation of scene description feature information, a first vector representing the corresponding description type, and a second vector sequence representing the corresponding problem type; superimposing the vector representation of the scene description feature information with the first vector representing the corresponding description type to obtain a first vector sequence corresponding to the scene description feature information; linking the first vector sequence and the second vector sequence representing the corresponding problem type to output the multi-dimensional evaluation result, thereby realizing the training of the autonomous driving system evaluation model.

[0050] The original driving dataset is pre-processed, that is, feature extraction is performed on the original driving dataset to obtain the input feature information corresponding to the original driving dataset, wherein the input feature information includes scene description feature information. The input feature information is vectorized to obtain a vector representation of the input feature information. This vector representation includes a vector representation of scene description features, a first vector representing the corresponding description type, and a second vector sequence representing the corresponding problem type. For example, this can be achieved through Bezier curve mapping to control point coordinates or other preprocessing networks to transform the input feature information into a vectorized feature description, resulting in a vector representation of the scene description features. Examples include vector representations of historical data features of autonomous vehicles ({Ego Car Token}), object information features ({Object token}), lane line features ({lane token}), and vehicle path features ({Routing Token}). The embedding layer then obtains the first vector representing the corresponding description type, i.e., vector representations of different scene description feature types, such as Type-1, Type-2, Type-3, etc., where Type-1 can be a vector representation of historical data features, Type-2 can be a vector representation of object information features, etc. Finally, the embedding layer obtains the second vector sequence representing the corresponding problem type, i.e., vectors representing different problem types, such as {issue1, issue...}. 2, ...}, where issue1 can be a vector representation of the solid line problem data, issue2 can be a vector representation of the wrong path problem data, etc.

[0051] At this point, the vector representation of the scene description feature information is superimposed with a first vector representing the corresponding description type to obtain a first vector sequence corresponding to the scene description feature information. Then, the first vector sequence is linked with a second vector sequence representing the corresponding problem type. Further, a Transformer network structure is used for processing to output the multi-dimensional evaluation result, obtaining the probabilities of different problem types corresponding to the input feature information, such as {Label1, Label2, Label3, ...}, where probability Label1 can correspond to issue1, probability Label2 can correspond to issue2, and so on, thereby enabling the training of the autonomous driving system evaluation model.

[0052] In some embodiments, feature extraction is performed on the original driving dataset to obtain input feature information corresponding to the original driving dataset, including: constructing vehicle feature information based on at least one frame of historical vehicle data; constructing object feature information based on object information in at least one frame of historical data in the perception environment; constructing lane line feature information based on lane line information and traffic light information in the perception map; and constructing path feature information based on the vehicle's path selection information.

[0053] Feature extraction is performed on the original driving dataset to obtain the corresponding input feature information. This input feature information can include vehicle feature information, object feature information, lane line feature information, and path feature information. Specifically, vehicle feature information is constructed based on at least one frame of historical vehicle data, for example, based on the vehicle's current moment and m frames of historical data from the past. The m frames of data are used to construct vehicle features {egoCar Traj}; based on object information from at least one frame of historical data in the perceived environment, object feature information is constructed, for example, based on object information from the current moment and past m frames of historical data in the perceived environment. , , The m-frame data is constructed into {Object1 Traj, Object2 Traj……Objectn Traj}; based on the lane line information and traffic light information {lane1, lane2,……laneX} in the perception map, lane line feature information {lane1, lane2,……laneX} is constructed; based on the vehicle's route selection information {Routing}, route feature information {Routing} is constructed.

[0054] To facilitate understanding, the training process of an autonomous driving system evaluation model will be illustrated with an example, such as... Figure 3 As shown, Figure 3This is a schematic diagram illustrating the training process of the autonomous driving system evaluation model according to an embodiment of this application. The process involves collecting an original driving dataset to obtain scene description information corresponding to the dataset. This scene description information includes at least one type of problem data. Problem data from different types of road tests that satisfy the scene coverage distribution are used as positive samples for model training, while excellent human driving data that satisfies the scene coverage distribution are used as negative samples. Feature extraction is performed on the original driving dataset to obtain corresponding input feature information, such as feature information of historical data of autonomous vehicles {Ego Car Traj}, feature information of object information {Object1 Traj, Object2 Traj}, feature information of lane lines {lane1, lane2}, feature information of vehicle paths {Routing}, etc.

[0055] The input feature information is vectorized to obtain vector representations of the input feature information. These vector representations include vector representations of scene description features, such as those obtained by mapping Bezier curves to control point coordinates. Examples include vector representations of historical data features of autonomous vehicles ({Ego Car Token}), object information features ({Object1 token, Object2 token}), lane line features ({lane1 token, lane2 token}), and vehicle path features ({Routing Token}). An embedding layer is then used to obtain a first vector ({Type-1, Type-2, Type-3, Type-4}) representing the corresponding description type, and a second vector sequence ({issue 1, issue 2, issue 3}) representing the corresponding issue type.

[0056] Furthermore, the vector representations of the above scene description feature information are superimposed with the first vector {Type-1, Type-2, Type-3, Type-4} used to represent the corresponding description types to obtain the first vector sequence corresponding to the scene description feature information. Then, the first vector sequence is linked with the second vector sequence {issue 1, issue 2, issue 3} used to represent the corresponding problem types to be input into the autonomous driving system evaluation model, such as the Transformer model, to obtain the probability of each problem type in the input feature information, such as {Label1, Label2, Label3}, where probability Label1 can be defined as {solid line problem, 0}, probability Label2 can be defined as {dragon drawing problem, 1}, and probability Label3 can be defined as {point braking problem, 2}.

[0057] Specifically, the vector representation of the input feature information for any scene data is {Ego Car Traj}, {Object1Traj, Object2 Traj……Object n Traj}, {lane1, lane2……lanex}, {Routing}, {Label}, where {Ego Car Traj} is a 1×D dimensional vector; {Object1 Traj, Object2 Traj……Objectn Traj} is an n×D dimensional vector; {lane1, lane2……lanex} is an x×D dimensional vector; {Routing} is a 1×D dimensional vector; and {Label} is an i×D dimensional vector representing the i types of lane data in the scene. Therefore, the input feature matrix is... The model outputs an i×D matrix after passing through the transformer network structure. This matrix is ​​then processed through the classification layer and the softmax layer to obtain the probabilities of the i labels for the given scene. During model training, the model gradually converges through iteration of the objective function L.

[0058] In some embodiments, before feature extraction from the original driving dataset, the method includes: cleaning the original driving dataset to obtain a cleaned original driving dataset for feature extraction.

[0059] Before feature extraction from the original driving dataset, i.e. after obtaining the scene description information corresponding to the original driving dataset, the original driving dataset is cleaned. This involves cleaning the data according to preset conditions to obtain a balanced target dataset, which serves as the cleaned original driving dataset for feature extraction.

[0060] In some embodiments, using input feature information for training an autonomous driving system evaluation model further includes: splitting the original driving dataset according to a preset ratio, using it as a training dataset and a test dataset respectively, wherein the training dataset is used to optimize the autonomous driving system evaluation model, and the test dataset is used to evaluate the autonomous driving system in multiple dimensions.

[0061] The original driving dataset is split according to a preset ratio, such as 8:2 or 7:3, with no specific limitation. For example, if the original driving dataset is split into an 8:2 ratio to be used as a training dataset and a test dataset respectively, then 2 / 10 of the original driving dataset can be used to optimize the evaluation model of the autonomous driving system, and 8 / 10 of the original driving dataset can be used to evaluate the autonomous driving system in multiple dimensions.

[0062] Specifically, model iteration and optimization are achieved by updating the dataset, using the following objective function L:

[0063]

[0064] in, Preset weighting coefficients; Probabilities for different problem types; Describe the type of problem, for example, there are N types of problems, such as drawing a dragon, crossing the line, speeding, etc. Indicates input feature information, For problematic data, Data representing excellent human driving performance.

[0065] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0066] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. The electronic device 40 includes a memory 41 and a processor 42 coupled to each other. The processor 42 is used to execute program instructions stored in the memory 41 to implement the steps of the above-described training method embodiment for the autonomous driving system evaluation model, or to implement the steps of the above-described evaluation method embodiment for the driving system. In a specific implementation scenario, the electronic device 40 may include, but is not limited to, a microcomputer or a server.

[0067] Specifically, processor 42 controls itself and memory 41 to implement the steps of the training method embodiment of the above-described autonomous driving system evaluation model, or to implement the steps of the above-described driving system evaluation method embodiment. Processor 42 can also be called a CPU (Central Processing Unit), and may be an integrated circuit chip with signal processing capabilities. Processor 42 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 42 can be implemented using integrated circuit chips.

[0068] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a non-volatile computer-readable storage medium according to an embodiment of this application. The non-volatile computer-readable storage medium 50 is used to store a computer program 501. When the computer program 501 is executed by a processor, for example by the aforementioned... Figure 4 When the processor 42 in the embodiment is executed, it is used to implement the steps of the above-described training method embodiment for the evaluation model of the autonomous driving system, or to implement the steps of the above-described evaluation method embodiment for the driving system.

[0069] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0070] In the several embodiments provided in this application, it should be understood that the disclosed methods and related devices can be implemented in other ways. For example, the related device implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication disconnection shown or discussed may be indirect coupling or communication disconnection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0071] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0072] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0073] Those skilled in the art will readily recognize that numerous modifications and variations can be made to the apparatus and method while maintaining the teachings of this application. Therefore, the above disclosure should be considered limited only by the scope of the appended claims.

Claims

1. An evaluation method for an autonomous driving system, characterized in that, include: Obtain the raw driving dataset; The original driving dataset is processed using an autonomous driving system evaluation model to obtain a multi-dimensional evaluation result of the autonomous driving system corresponding to the original driving dataset, wherein the autonomous driving system evaluation model is used to evaluate the autonomous driving system in multiple dimensions. The step of processing the original driving dataset using an autonomous driving system evaluation model includes: Feature extraction is performed on the original driving dataset to obtain the input feature information corresponding to the original driving dataset; The input feature information is vectorized to obtain a vector representation of the input feature information, which includes a vector representation of scene description feature information and a first vector for characterizing the corresponding description type. The vector representation of the scene description feature information is superimposed with the first vector used to characterize the corresponding description type to obtain the first vector sequence corresponding to the scene description feature information; The first vector sequence is linked with the second vector sequence used to represent the corresponding problem type, and then the multi-dimensional evaluation result is output to achieve a multi-dimensional evaluation of the autonomous driving system.

2. The method according to claim 1, characterized in that, Further includes: By using the evaluation results corresponding to the input problem description information, the driving effect information of the autonomous driving system can be obtained.

3. A training method for an evaluation model of an autonomous driving system, characterized in that, include: Collect raw driving datasets to obtain scene description information corresponding to the raw driving datasets, wherein the scene description information includes at least one type of problem data; The original driving dataset is subjected to a preset processing to obtain the input feature information corresponding to the original driving dataset, the input feature information including scene description feature information; The input feature information is used to train the autonomous driving system evaluation model, wherein the autonomous driving system evaluation model is used to implement the evaluation method of the autonomous driving system according to any one of claims 1 to 2.

4. The method according to claim 3, characterized in that, The step of performing pre-processing on the original driving dataset to obtain the input feature information corresponding to the original driving dataset includes: Feature extraction is performed on the original driving dataset to obtain the input feature information corresponding to the original driving dataset; The step of using the input feature information for training the autonomous driving system evaluation model includes: The input feature information is vectorized to obtain a vector representation of the input feature information. The vector representation of the input feature information includes a vector representation of the scene description feature information, a first vector for representing the corresponding description type, and a second vector sequence for representing the corresponding question type. The vector representation of the scene description feature information is superimposed with the first vector used to characterize the corresponding description type to obtain the first vector sequence corresponding to the scene description feature information; The first vector sequence and the second vector sequence used to represent the corresponding problem type are linked together to output the multi-dimensional evaluation result, thereby enabling the training of the autonomous driving system evaluation model.

5. The method according to claim 4, characterized in that, The step of extracting features from the original driving dataset to obtain the input feature information corresponding to the original driving dataset includes: Construct vehicle feature information based on at least one frame of historical vehicle data; Construct object feature information based on object information from at least one frame of historical data in the perceived environment; Based on lane line information and traffic light information in the perception map, lane line feature information is constructed; Based on the vehicle's route selection information, route feature information is constructed.

6. The method according to claim 4, characterized in that, Before performing feature extraction on the original driving dataset, the following steps are included: The original driving dataset is cleaned to obtain the cleaned original driving dataset, which is then used for feature extraction.

7. The method according to claim 4, characterized in that, Using the input feature information for training the autonomous driving system evaluation model further includes: The original driving dataset is split into a training dataset and a test dataset according to a preset ratio. The training dataset is used to optimize the evaluation model of the autonomous driving system, and the test dataset is used to evaluate the autonomous driving system in multiple dimensions.

8. An electronic device, characterized in that, The system includes a memory and a processor that are coupled to each other, the processor being used to execute program instructions stored in the memory to implement the evaluation method of the autonomous driving system according to any one of claims 1 to 2, or to implement the training method of the evaluation model of the autonomous driving system according to any one of claims 3 to 7.

9. A non-volatile computer-readable storage medium storing program instructions thereon, characterized in that, When the program instructions are executed by the processor, they implement the evaluation method for the autonomous driving system according to any one of claims 1 to 2, or the training method for the evaluation model of the autonomous driving system according to any one of claims 3 to 7.

Images