Automated evaluation method and device for vehicle data, electronic equipment and vehicle

Abstract

This invention discloses an automated evaluation method, apparatus, electronic device, and vehicle for vehicle data. The method includes: responding to a data upload command applied to an interactive interface, reading multiple vehicle datasets collected from real vehicles from a source location carried in the data upload command, and uploading the vehicle datasets to a target location corresponding to the data upload command; responding to an evaluation task request applied to the interactive interface, processing the vehicle data in the vehicle datasets based on the target device information carried in the evaluation task request to obtain target vehicle data; determining at least one data point to be tested from the target vehicle data based on at least one target data type carried in the evaluation task request; determining a target evaluation algorithm from multiple evaluation algorithms that corresponds one-to-one with the at least one target data type, and performing parallel evaluation on the at least one data point to be tested based on the at least one target evaluation algorithm to obtain a target evaluation result; and displaying the target evaluation result on the interactive interface.

Description

Technical Field

[0001] This invention relates to the fields of intelligent vehicles and autonomous driving, and more specifically, to an automated evaluation method, apparatus, electronic device, and vehicle for vehicle data. Background Technology

[0002] In the development of autonomous driving systems, it is often necessary to evaluate the collected vehicle data, especially comparing differences between different hardware devices and software versions. However, due to the different manufacturers of the hardware, the collected data formats often differ, and different data types often require different evaluation algorithms. When using related technologies to evaluate vehicle data, users need to switch between different tools to ensure that vehicle data from different hardware devices and software versions can be evaluated. This makes the entire evaluation process complex and time-consuming, resulting in low efficiency in vehicle data evaluation.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides an automated evaluation method, apparatus, electronic device, and vehicle for vehicle data, to at least address the technical problem of low evaluation efficiency of vehicle data.

[0005] According to one aspect of the present invention, an automated evaluation method for vehicle data is provided, comprising: responding to a data upload command applied to an interactive interface, reading vehicle datasets collected from multiple real vehicles from a source location carried by the data upload command, and uploading the vehicle datasets to a target location corresponding to the data upload command, wherein the vehicle data in the vehicle datasets includes data of multiple data types and device information of acquisition devices corresponding to different data types, the device information of acquisition devices on different real vehicles being different, and the device information including manufacturer information and version information of the acquisition devices; responding to an evaluation task request applied to the interactive interface, processing the vehicle data in the vehicle datasets based on the target device information carried by the evaluation task request to obtain target vehicle data; determining at least one test data from the target vehicle data based on at least one target data type carried by the evaluation task request, wherein at least one test data corresponds one-to-one with at least one target data type; determining a target evaluation algorithm corresponding one-to-one with at least one target data type from multiple evaluation algorithms, and performing parallel evaluation on at least one test data based on at least one target evaluation algorithm to obtain a target evaluation result, wherein the multiple evaluation algorithms are used to characterize pre-encapsulated evaluation algorithms corresponding one-to-one with multiple data types; and displaying the target evaluation result on the interactive interface.

[0006] Optionally, uploading the vehicle dataset to the target location corresponding to the data upload command includes: obtaining the target project name set and target data label set carried in the data upload command, wherein the target project name set corresponds one-to-one with the vehicle dataset, and different target project names are used to represent the project to which the corresponding vehicle data belongs, and vehicle data of different projects are stored in different locations; the target data label set corresponds one-to-one with the vehicle dataset, and different target data labels are used to group the corresponding vehicle data; determining the target location corresponding to each vehicle data based on the target project name and target data label corresponding to each vehicle data in the vehicle dataset; and storing each vehicle data in the target location corresponding to each vehicle data.

[0007] Optionally, the method further includes one of the following: displaying multiple pre-created project names on the interactive interface; in response to a selection instruction to select multiple project names, determining the project name corresponding to the selection instruction as a target project name; in response to a creation instruction applied to the interactive interface, creating a target project based on the creation instruction, and using the project name of the target project as a target project name.

[0008] Optionally, based on the target device information carried in the evaluation task request, the vehicle data in the vehicle dataset is processed to obtain the target vehicle data, including: determining the initial vehicle data that matches the target device information from the vehicle dataset based on the target device information; determining whether it is necessary to parse the initial vehicle data; and in response to the need to parse the initial vehicle data, parsing the initial vehicle data to obtain the target vehicle data.

[0009] Optionally, determining whether initial vehicle data needs to be parsed includes: determining the file type of the file storing the initial vehicle data; determining that initial vehicle data needs to be parsed in response to the file type being a preset file type; and determining that initial vehicle data does not need to be parsed in response to the file type not being a preset file type.

[0010] Optionally, the method further includes: in response to the file type not being a preset file type, performing template verification on the initial vehicle data to obtain a template verification result; in response to the template verification result indicating that the initial vehicle data has passed verification, determining that the initial vehicle data is the target vehicle data.

[0011] Optionally, the test data is stored in an object storage service in a serialized manner. The filename of the test data stored in the object storage service is the same as the filename of the target vehicle data, but the file extension of the test data stored in the object storage service is different from that of the target vehicle data. Parallel evaluation of at least one test data point is performed based on at least one target evaluation algorithm to obtain a target evaluation result. This includes: reading at least one test data point from the object storage service and obtaining the corresponding ground truth data for at least one test data point. The ground truth data consists of data collected from multiple real vehicles under normal conditions. Each test data point is evaluated based on the corresponding target evaluation algorithm and the corresponding ground truth data to obtain an evaluation result for each test data point. The evaluation results corresponding to at least one test data point are then summarized to obtain the target evaluation result.

[0012] Optionally, the method further includes: in response to the failure to parse the initial vehicle data, recording the abnormal information that occurs during the parsing process and determining the task status of the evaluation task request as a failure state; in response to the failure to evaluate any test data, recording the abnormal information that occurs during the evaluation process and determining the task status of the evaluation task request as a failure state; in response to the template verification result indicating that the target vehicle data verification fails, recording the abnormal information that occurs during the verification process and determining the task status of the evaluation task request as a failure state; and in response to the task status of the evaluation task request being a failure state, displaying the recorded abnormal information on the interactive interface.

[0013] According to another aspect of the present invention, an automated vehicle data evaluation device is also provided, comprising: an upload module, configured to respond to a data upload command applied to an interactive interface, read vehicle datasets collected from multiple real vehicles from a source location carried in the data upload command, and upload the vehicle datasets to a target location corresponding to the data upload command; wherein the vehicle data in the vehicle datasets includes data of multiple data types, and device information of the acquisition devices corresponding to different data types, and the device information of the acquisition devices on different real vehicles is different, and the device information includes manufacturer information and version information of the acquisition devices; and a processing module, configured to respond to an evaluation task request applied to the interactive interface, and based on the target location carried in the evaluation task request... The system comprises the following modules: a target vehicle information module, a vehicle dataset module, and a display module. The target vehicle data is processed to obtain target vehicle data. A determination module is used to determine at least one test data point from the target vehicle data based on at least one target data type carried in the evaluation task request, wherein each test data point corresponds one-to-one with at least one target data type. An evaluation module is used to determine the target evaluation algorithm corresponding one-to-one with at least one target data type from multiple evaluation algorithms, and to perform parallel evaluation on at least one test data point based on the at least one target evaluation algorithm to obtain the target evaluation result. The multiple evaluation algorithms are used to characterize pre-encapsulated evaluation algorithms that correspond one-to-one with multiple data types. The target evaluation result is displayed on an interactive interface.

[0014] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a storage device, wherein the storage device is used to store one or more programs, which, when executed by one or more processors, cause the one or more processors to perform an automated evaluation method for vehicle data as described above.

[0015] According to another aspect of the present invention, a vehicle is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform an automated evaluation method for vehicle data as described above.

[0016] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0017] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0018] According to another aspect of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0019] According to another aspect of the present invention, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of the present invention.

[0020] In this embodiment of the invention, in response to a data upload command applied to the interactive interface, multiple vehicle datasets collected from real vehicles are read from the source location carried in the data upload command, and the vehicle datasets are uploaded to the target location corresponding to the data upload command; in response to an evaluation task request applied to the interactive interface, the vehicle data in the vehicle dataset is processed based on the target device information carried in the evaluation task request to obtain target vehicle data; based on at least one target data type carried in the evaluation task request, at least one data to be tested is determined from the target vehicle data; a target evaluation algorithm corresponding one-to-one with at least one target data type is determined from multiple evaluation algorithms, and the at least one data to be tested is evaluated in parallel based on the at least one target evaluation algorithm to obtain a target evaluation result; the target evaluation result is displayed on the interactive interface. It is noteworthy that this application provides an automated evaluation method. The entire evaluation process only requires the user to upload vehicle data and input an evaluation task request, eliminating the need for the user to switch between different tools, thereby enabling automated testing of vehicle data with different hardware devices and different version information. Moreover, throughout the evaluation process, vehicle data with different hardware devices and different versions can be processed based on the target device information. Furthermore, pre-encapsulated evaluation algorithms for the target data type can be invoked to perform parallel evaluation on at least one piece of data to be tested. This achieves the purpose of processing and parallelizing vehicle data with different hardware devices and different versions, greatly reducing the evaluation workload and shortening the evaluation cycle, further improving evaluation efficiency, and thus solving the technical problem of low evaluation efficiency for vehicle data. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0022] Figure 1 This is a flowchart of an automated evaluation method for vehicle data according to an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of an automated evaluation system architecture according to an embodiment of the present invention;

[0024] Figure 3 This is a flowchart of an automated evaluation method according to an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of an automated vehicle data evaluation device according to an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] According to an embodiment of the present invention, an automated evaluation method for vehicle data is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] Figure 1 This is a flowchart of an automated evaluation method for vehicle data according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0030] Step S102: In response to the data upload command applied to the interactive interface, read vehicle datasets collected from multiple real vehicles from the source location carried by the data upload command, and upload the vehicle datasets to the target location corresponding to the data upload command. The vehicle data in the vehicle dataset contains data of multiple data types, as well as device information of the acquisition devices corresponding to different data types. The device information of the acquisition devices on different real vehicles is different. The device information includes the manufacturer information and version information of the acquisition devices.

[0031] The aforementioned interactive interface can be a human-computer interaction interface used for information transmission and operation between the user and the client device. Optionally, by inputting data upload commands on the interactive interface, the vehicle dataset can be uploaded to the server, and the vehicle dataset can be evaluated.

[0032] The interactive interface can include the following aspects:

[0033] Input interface: Used to facilitate users to input information or instructions. Devices for inputting information or instructions may include keyboards, mice, touch screens, voice recognition, etc.

[0034] Output interface: Used to output information or results to the user. Devices that output information or results may include displays, audio players, printers, etc.

[0035] Control interface: Used to facilitate user control of the computer. Common control interfaces include buttons, sliders, menus, etc.

[0036] Feedback interface: Used to provide users with operation feedback and prompts so that users can understand the status and results of the operation. Common feedback interfaces include dialog boxes, progress bars, and prompts.

[0037] The aforementioned multiple real vehicles can be vehicles equipped with different hardware devices or different software versions. Each real vehicle can drive on real roads, thus collecting one vehicle data point. The vehicle data collected by multiple real vehicles constitutes a vehicle dataset. The vehicle data in the aforementioned vehicle dataset can refer to data collected by different acquisition devices installed on multiple real vehicles. For example, it can include, but is not limited to, vehicle surrounding environment data and vehicle status data. Vehicle surrounding environment data can include at least one of the following: radar point cloud data, ultrasonic data, and image data. Vehicle status data can include at least one of the following: vehicle positioning information, attitude, direction of movement, speed, acceleration, turning radius, etc. Since the device information of the acquisition devices on different real vehicles differs, in order to distinguish vehicle data with different device information, the aforementioned vehicle data can also include device information. This device information includes, but is not limited to, information indicating the hardware of the acquisition device, i.e., manufacturer information, and information indicating the software version of the acquisition device, i.e., version information.

[0038] The aforementioned data upload command can be generated by user interaction on the interface, indicating that the vehicle dataset needs to be uploaded to the server or automated evaluation system. For example, the data upload command could be generated by clicking the "Data Upload" button on the interface, by pressing a specific button on the keyboard, or by voice control. To achieve the purpose of uploading the vehicle dataset to the server or automated evaluation system, the data upload command can carry the source location information. This source location indicates the current storage location of the vehicle dataset, allowing the client device to read the vehicle dataset from the source location and complete the data upload.

[0039] The aforementioned target location can be the storage location of the vehicle dataset on the server or in the automated evaluation system. Since the vehicle data contains data of multiple data types and the corresponding equipment information of the acquisition devices, and considering that vehicle data with the same equipment information or the same data type can be processed in the same way, the vehicle dataset can be stored based on the equipment information and data type to determine the target location corresponding to the vehicle dataset, that is, to determine the storage location of each vehicle data in the vehicle dataset.

[0040] In one optional embodiment, this application provides an automated evaluation method. Users can operate on an interactive interface to select vehicle datasets stored at a source location and generate data upload commands. The server or automated evaluation system can then read the vehicle datasets from the source location and upload them to the target location. For example, a user can store multiple vehicle datasets collected from real vehicles on a hard drive. Optionally, the hard drive can be connected to a client device, and the user can issue a data upload command on the interactive interface, instructing the client device to read the vehicle data stored on the hard drive and upload it to the server for automated evaluation.

[0041] Step S104: In response to the evaluation task request applied to the interactive interface, based on the target device information carried in the evaluation task request, process the vehicle data in the vehicle dataset to obtain the target vehicle data.

[0042] The aforementioned evaluation task request can be an instruction to the server or automated evaluation system to perform automated evaluation of vehicle data. Optionally, the evaluation task request can be generated by the user clicking virtual buttons on the interactive interface, pressing specific keys on the keyboard, or through voice interaction. Since the automated evaluation of vehicle data requires comparing the differences between different device hardware and different software versions, the evaluation task request can include the hardware and software version of the device to be evaluated, i.e., the target device information mentioned above.

[0043] In one optional embodiment, since vehicle data collected by acquisition devices with different hardware and software versions often have different formats—for example, the format could be BAG (a file format used to record and play back data streams) or DATA (a file format used to store and transmit vehicle data), or readable formats such as CSV (Comma Separated Values), or video formats such as MP4 (MPEG-4 Part 14) or AVI (Audio Video Interleaving)—the vehicle data in the vehicle dataset can be processed using the target device information. This includes parsing the vehicle data or converting its format, thereby identifying the target vehicle data that matches the target device information. This target vehicle data can then be automatically evaluated by a server or an automated evaluation system.

[0044] Step S106: Based on at least one target data type carried in the evaluation task request, determine at least one test data from the target vehicle data, wherein at least one test data corresponds one-to-one with at least one target data type.

[0045] At least one of the target data types mentioned above can be the data type of vehicle data that the user wishes to evaluate, such as image data, radar point cloud data, etc. Considering that users often need to evaluate vehicle data of different data types from the same device hardware or the same software version, or vehicle data of the same data type from different device hardware or different software versions, the number of target data types here can be one or more, which can be determined according to actual needs.

[0046] At least one of the aforementioned test data can be used to represent the data that needs to be evaluated. Optionally, since the vehicle data contains multiple data types, it is necessary to determine the test data to be evaluated from the target vehicle data during the actual evaluation process. For example, if the vehicle speed needs to be evaluated, the vehicle speed can be read only from the target vehicle data during the actual evaluation. Since the number of target data types can be one or more, the number of test data is also one or more, and the quantity relationship between the two is one-to-one.

[0047] In one optional embodiment, since the vehicle data contains multiple data types, the field value of the signal field corresponding to each target data type can be read from the target vehicle data to obtain at least one test data, which can then be automatically evaluated by a server or automated evaluation system.

[0048] Step S108: Determine the target evaluation algorithm corresponding to at least one target data type from multiple evaluation algorithms, and perform parallel evaluation on at least one test data based on at least one target evaluation algorithm to obtain the target evaluation result. The multiple evaluation algorithms are used to characterize the pre-encapsulated evaluation algorithms corresponding to multiple data types.

[0049] Considering that different evaluation algorithms are often required for different data types to be evaluated, and that the target data type desired by users often differs in different evaluation processes, to achieve automated evaluation of vehicle data, a suitable evaluation algorithm can be pre-encapsulated for each data type contained in the vehicle data, and multiple evaluation algorithms can be stored in a database.

[0050] In one optional embodiment, after obtaining at least one data to be tested, for each data to be tested, a target evaluation algorithm can be determined from multiple evaluation algorithms stored in the database according to the corresponding target data type. Then, the target evaluation algorithm can be used to evaluate the data to be tested to obtain the evaluation result of the data to be tested. Then, by summarizing the evaluation results of at least one data to be tested, the target evaluation result is obtained.

[0051] It should be noted that when there are multiple sets of data to be tested, in order to further improve the evaluation efficiency, the multiple sets of data can be evaluated in parallel. For example, after determining multiple sets of data A, B and C to be tested, the target evaluation algorithm a corresponding to data A can be used to evaluate data A, the target evaluation algorithm b corresponding to data B can be used to evaluate data B in parallel, and the target evaluation algorithm c corresponding to data C can be used to evaluate data C in parallel.

[0052] Step S110: Display the target evaluation results on the interactive interface.

[0053] In one optional embodiment, after obtaining the evaluation results, the target evaluation results can be displayed on the interactive interface in a visual form. For example, a corresponding text-based evaluation report can be generated based on the target evaluation results and displayed on the interactive interface, or the evaluation results can be displayed on the interactive interface by uploading corresponding graphs, bar charts, or other images.

[0054] Figure 2 This is a schematic diagram of an automated evaluation system architecture according to an embodiment of the present invention, such as... Figure 2 As shown, the system includes a data import and management layer, a data processing layer, and a display and interaction layer.

[0055] The main function of the data import and management layer is to upload vehicle data collected from users' actual vehicles to the local server and create an index for each file in the database, recording relevant information such as device manufacturer, software version, file name, and storage path. The file format storing vehicle data can be BAT or DATA format, or processed readable formats such as CSV, or video file formats such as MP4 or AVI. Afterwards, it allows searching by project (such as evaluation requirements, evaluation tasks, etc.) and other tags, and batch creation of tasks at the file granularity. The data import and management layer supports functions such as dedicated projects, data upload, and data management.

[0056] The dedicated project feature is used to tag data and isolate data from different projects. When creating a project, specific characters can be added to the project name. By selecting a project name containing these characters when uploading data, the system can associate the unique tags required by the automated evaluation system, such as manufacturer and version. A project name can be created once and used multiple times for data uploads; alternatively, a project can be created each time an evaluation task is initiated. The purpose of creating projects is threefold: first, to isolate them from other projects; second, to personalize and record unique tags; and third, to quickly filter the desired data files based on the project name when creating an evaluation task.

[0057] The data upload function, also known as activating the upload service, is used to upload vehicle data to the target location of the server or automated evaluation system. The data upload function can determine the source and target locations, as well as the names of any existing projects. By enabling the upload service on the local server, all files in the source location directory can be uploaded to the target location. The data here is mainly divided into two categories: the data being tested and the ground truth data generated by the ground truth system.

[0058] The data management function allows users to create different evaluation tasks, delete redundant vehicle data, and index vehicle data, meaning they can search for data based on different items and data tags. The data management function displays a progress bar during data upload and monitors the success or failure of file uploads, while also supporting the resumption of failed uploads. After file uploads are complete, the data management function supports basic operations such as adding, deleting, modifying, and querying, as well as batching or individually selecting files to create datasets for evaluation tasks.

[0059] The main function of the data processing layer is to parse BAG or DATA format files, identify the data to be tested, and analyze and calculate evaluation metrics. The data processing layer supports parsing, evaluation, and other functions.

[0060] Optionally, the parsing module receives the evaluation task, parses the file to be parsed (bag or data format file) using a general parsing program, reads the signal field corresponding to the target data type (topic) required for the evaluation, that is, obtains at least one test data, and serializes and saves it to OSS by using the original filename and changing the file extension. In the above process, the parsing module can return which data types were parsed and the status indicating whether the test data has been parsed completely. That is, the parsing function can determine the file type and, based on the determination result, choose to process the file containing vehicle data by direct parsing or data verification, thereby obtaining at least one test data. Furthermore, a task can be created based on at least one test data, and at least one test data can be uploaded to the Object Storage Service (OSS).

[0061] The evaluation module supports evaluation requirements for multiple versions of data. Specifically, it can read at least one set of test data stored in OSS for signal extraction, extract signals for index calculation, and compare the extracted signals with signals under normal vehicle conditions. In other words, the evaluation module supports result comparison, further index calculation, and saves the results. The evaluation program, as a separate process, is called by the backend service upon successful parsing and parameter passing. The evaluation module receives the parameters, obtains the filename and target data type to be evaluated, reads the data stored in OSS based on the filename, adapts different algorithms to different target data types to calculate different evaluation indices, and obtains the evaluation results after calculation. These results are then stored in the Tidb database (a distributed relational database supporting online transaction processing and online analytical processing).

[0062] Other modules can record erroneous data when errors occur and update the parsing or evaluation status in real time. Optionally, if an exception occurs at any stage from task creation to the end of the evaluation module, the error information will be recorded, and the task status will be updated to failure. The template download function allows users to download and view the column name requirements for vehicle data or structured data stored in CSV format.

[0063] The main function of the display and interaction layer is to allow users to view, export, and tag videos of the evaluation results. This layer supports displaying evaluation results through indicator curves, map trajectories, and video playback, and also supports the export of evaluation results and test data. Optionally, the backend retrieves evaluation results from the Tidb database and sends them to the frontend. The frontend can then visualize the results; for example, it can display indicator curves; it can also display vehicle trajectories on a map based on the deflected latitude and longitude; if videos are available, the frontend can play them synchronously, pause playback, capture the current frame, and tag the data with selected or custom tags; the frontend can also visualize lane-level high-precision maps; similarly, it can export test data stored on OSS; it can also export evaluation results stored on Tidb; and users can predefine report templates on the frontend, allowing the frontend to populate evaluation results and generate reports in the populated areas. Users can input the evaluation criteria they expect to achieve, and the front end can then display on the criterion curve which time period meets the criteria and which time period does not, and on the map trajectory which route meets the criteria and which route does not; during video playback, the cursor on the criterion curve moves, the vehicle moves along the trajectory on the map, and the high-precision map is dynamically loaded, creating a coordinated effect.

[0064] In this embodiment of the invention, in response to a data upload command applied to the interactive interface, multiple vehicle datasets collected from real vehicles are read from the source location carried in the data upload command, and the vehicle datasets are uploaded to the target location corresponding to the data upload command; in response to an evaluation task request applied to the interactive interface, the vehicle data in the vehicle dataset is processed based on the target device information carried in the evaluation task request to obtain target vehicle data; based on at least one target data type carried in the evaluation task request, at least one data to be tested is determined from the target vehicle data; a target evaluation algorithm corresponding one-to-one with at least one target data type is determined from multiple evaluation algorithms, and the at least one data to be tested is evaluated in parallel based on the at least one target evaluation algorithm to obtain a target evaluation result; the target evaluation result is displayed on the interactive interface. It is noteworthy that this application provides an automated evaluation method. The entire evaluation process only requires the user to upload vehicle data and input an evaluation task request, eliminating the need for the user to switch between different tools, thereby enabling automated testing of vehicle data with different hardware devices and different version information. Moreover, throughout the evaluation process, vehicle data with different hardware devices and different versions can be processed according to the target device information, and evaluation algorithms pre-encapsulated for the target data type can be called to perform parallel evaluation on at least one data to be tested, thereby greatly reducing the evaluation workload and shortening the evaluation cycle, further improving the evaluation efficiency, and thus solving the technical problem of low evaluation efficiency of vehicle data.

[0065] Optionally, uploading the vehicle dataset to the target location corresponding to the data upload command includes: obtaining the target project name set and target data label set carried in the data upload command, wherein the target project name set corresponds one-to-one with the vehicle dataset, and different target project names are used to represent the project to which the corresponding vehicle data belongs, and vehicle data of different projects are stored in different locations; the target data label set corresponds one-to-one with the vehicle dataset, and different target data labels are used to group the corresponding vehicle data; determining the target location corresponding to each vehicle data based on the target project name and target data label corresponding to each vehicle data in the vehicle dataset; and storing each vehicle data in the target location corresponding to each vehicle data.

[0066] In one optional embodiment, when uploading a vehicle dataset, the user can select a target project name from a list of pre-created project names displayed on the interface, or create a target project name according to their needs. Furthermore, the user can select a target data label from a list of different data labels displayed on the interface, or define a target data label themselves. Optionally, after obtaining the target project name and target data label, the target location can be determined based on these labels, ensuring that vehicle data for the same project can be stored in the same location, and vehicle data for different projects can be stored in different locations. Vehicle data with different data labels can be grouped into different groups, and vehicle data with the same data label can be grouped into the same group. After determining the target location, the vehicle dataset is stored at the corresponding target location.

[0067] Optionally, the method further includes one of the following: displaying multiple pre-created project names on the interactive interface; in response to a selection instruction to select multiple project names, determining the project name corresponding to the selection instruction as a target project name; in response to a creation instruction applied to the interactive interface, creating a target project based on the creation instruction, and using the project name of the target project as a target project name.

[0068] In one optional embodiment, the user can pre-create multiple project names. When uploading a vehicle dataset, the user can issue a selection command via any command sending method to choose the target project name from the multiple project names. Furthermore, if none of the multiple project names meet the user's needs, the user can issue a creation command via any command sending method through the interactive interface, thereby creating the target project based on the creation command and setting the target project name as the target project name.

[0069] In another alternative embodiment, the user can directly issue a creation command on the interactive interface using any command sending method, thereby creating the target project based on the creation command and determining the name of the target project as the target project name.

[0070] Optionally, based on the target device information carried in the evaluation task request, the vehicle data in the vehicle dataset is processed to obtain the target vehicle data, including: determining the initial vehicle data that matches the target device information from the vehicle dataset based on the target device information; determining whether it is necessary to parse the initial vehicle data; and in response to the need to parse the initial vehicle data, parsing the initial vehicle data to obtain the target vehicle data.

[0071] In one optional embodiment, when a user issues an evaluation task request in any command sending format, the vehicle data containing the target device information carried in the evaluation task request can be determined from the vehicle dataset to obtain initial vehicle data. Considering that some vehicle data formats (e.g., bag format) cannot directly read data of different data types, i.e., the test data cannot be directly read, the storage format of the initial vehicle data can be used to determine whether parsing of the initial vehicle data is necessary. For example, if the initial vehicle data is in a pre-defined format, parsing is required; if the initial vehicle data is not in a pre-defined format, parsing is not required. For example, if the initial vehicle data is stored in bag format, parsing is required; if the initial vehicle data is not stored in bag format, parsing is not required. If parsing of the initial vehicle data is determined to be necessary, it can be performed to obtain the target vehicle data.

[0072] Optionally, determining whether initial vehicle data needs to be parsed includes: determining the file type of the file storing the initial vehicle data; determining that initial vehicle data needs to be parsed in response to the file type being a preset file type; and determining that initial vehicle data does not need to be parsed in response to the file type not being a preset file type.

[0073] The file types mentioned above may include, but are not limited to: bag format, data format, csv format, mp4 format, or avi format.

[0074] The preset file types mentioned above can be either bag format or data format.

[0075] In one alternative embodiment, the storage format of the target vehicle data can be determined based on the file type of the file containing the target vehicle data. Therefore, the target vehicle data only needs to be parsed when the file type is .bag or .data. In this case, it can be determined whether the target vehicle data needs to be parsed based on whether the file type is .bag or .data.

[0076] Optionally, the method further includes: in response to the file type not being a preset file type, performing template verification on the initial vehicle data to obtain a template verification result; in response to the template verification result indicating that the initial vehicle data has passed verification, determining that the initial vehicle data is the target vehicle data.

[0077] In an optional embodiment, assuming the file type is other than bag or data format, such as csv, mp4, or avi format, then it is not necessary to parse the initial vehicle data. The verification template can be used directly for verification, and after the verification is passed, the initial vehicle data can be used as the target vehicle data. The aforementioned verification template can be a verification template stored in the evaluation system in advance.

[0078] Optionally, the test data is stored in an object storage service in a serialized manner. The filename of the test data stored in the object storage service is the same as the filename of the target vehicle data, but the file extension of the test data stored in the object storage service is different from that of the target vehicle data. Parallel evaluation of at least one test data point is performed based on at least one target evaluation algorithm to obtain a target evaluation result. This includes: reading at least one test data point from the object storage service and obtaining the corresponding ground truth data for at least one test data point. The ground truth data consists of data collected from multiple real vehicles under normal conditions. Each test data point is evaluated based on the corresponding target evaluation algorithm and the corresponding ground truth data to obtain an evaluation result for each test data point. The evaluation results corresponding to at least one test data point are then summarized to obtain the target evaluation result.

[0079] In one optional embodiment, to ensure that the server or automated evaluation system can perform parallel evaluation of the test data, the filename of the test data can remain unchanged while the suffix is ​​changed, thereby changing the format of the test data. Optionally, the formatted test data can be stored sequentially in an object storage service in the cloud. Further, to achieve the purpose of evaluating the test data, while reading the test data from the target vehicle data, truth data generated by the truth system is also read from the target vehicle data. This truth data can be the data collected from the vehicle under normal conditions. The truth data is then compared with the test data to evaluate the vehicle data. Then, a target evaluation algorithm can be used to compare the test data with the truth data to achieve the purpose of evaluating the test data, thereby obtaining the evaluation result corresponding to each test data. Optionally, the evaluation result can be a text result or a graphical result. Finally, the evaluation results corresponding to at least one data point to be tested can be summarized to obtain the target evaluation result. The summarization here can be done by concatenating the evaluation results corresponding to at least one data point to be tested in sequence, or by superimposing the evaluation results corresponding to at least one data point to be tested, but it is not limited to these. The specific implementation method of summarization can be determined according to actual needs.

[0080] Optionally, displaying the target evaluation results on the interactive interface includes: displaying the target evaluation results through a target image and displaying the target evaluation results on the interactive interface, wherein the target image includes at least one of the following: an indicator curve, a map trajectory, and a target video.

[0081] In one optional embodiment, after obtaining the target evaluation results, the target evaluation results can be displayed on the interactive interface in at least one form, such as indicator curves, map trajectories, or target videos. For example, different performance indicators of the vehicle can be displayed through indicator curves, the driving path of the vehicle can be displayed through map trajectories, or the driving process of the vehicle can be displayed through target videos, thereby realizing the visualization of the target evaluation results.

[0082] Optionally, the method further includes: in response to the failure to parse the initial vehicle data, recording the abnormal information that occurs during the parsing process and determining the task status of the evaluation task request as a failure state; in response to the failure to evaluate any test data, recording the abnormal information that occurs during the evaluation process and determining the task status of the evaluation task request as a failure state; in response to the template verification result indicating that the target vehicle data verification fails, recording the abnormal information that occurs during the verification process and determining the task status of the evaluation task request as a failure state; and in response to the task status of the evaluation task request being a failure state, displaying the recorded abnormal information on the interactive interface.

[0083] In one optional embodiment, during the parsing of the initial vehicle data, the parsing process can be monitored in real time. If an abnormal situation occurs during the parsing process, the abnormal situation can be recorded and displayed on the interactive interface in any form such as text, alarm, or flashing light.

[0084] In another optional embodiment, during the evaluation of any test data, the evaluation process can be monitored in real time. If an abnormal situation occurs during the evaluation process, the abnormal situation can be recorded and displayed on the interactive interface in any form such as text, alarm, or flashing light.

[0085] In another alternative embodiment, during the template verification of the initial vehicle data, the evaluation process can be monitored in real time. If an abnormal situation occurs during the evaluation process, the abnormal situation can be recorded and displayed on the interactive interface in any form such as text, alarm, or flashing light.

[0086] Figure 3 This is a flowchart of an automated evaluation method according to an embodiment of the present invention, such as... Figure 3As shown, after starting the evaluation, the specific project can be identified first, that is, the target project name can be determined. Next, data can be uploaded, that is, the vehicle dataset can be uploaded to the local server, and an evaluation task can be created. Then, based on the target device information carried in the evaluation task, the initial vehicle data containing the target device information is determined, and the file type of the file storing the initial vehicle data is determined. Only vehicle data of the target type, such as bag or data format, needs to be parsed. Therefore, based on whether the file type is bag or data format, it can be determined which files in the vehicle dataset need to be parsed. It should be noted that since the vehicle data that needs to be parsed in this task may have been parsed in other evaluation tasks, and the parsing / upload status will be "updated" after each parsing (the default status is unparsed), when initiating the evaluation task, the request body will include an indicator of whether the file has been parsed. If the file type is the target type, that is, bag or data format, then the initial vehicle data needs to be parsed. After parsing the target vehicle data, the evaluation module needs to adapt different indicator calculation algorithms based on different target data types. Therefore, the parsing module needs to return to the backend which data types it parsed. Then, based on at least one target data type carried in the evaluation task, it reads at least one test data from the target vehicle data and uploads it to the object storage service. The "determine whether to upload to OSS" and "determine whether to parse" functions share the same identifier. Since the evaluation module uniformly reads test data from OSS, readable data or structured data such as CSV files that do not require parsing also need to be uploaded to OSS. If the file type is not the target type, the evaluation module needs to specify the column names for each column to calculate different evaluation indicators. Therefore, it needs to verify whether the column names of the uploaded readable data or structured data (such as CSV files) are consistent with the predefined column names. This requires template validation of the initial vehicle data. If the validation fails, the error needs to be recorded. If the validation passes, based on at least one target data type carried in the evaluation task, it reads at least one test data from the target vehicle data and uploads it to the object storage service. At this point, the file parsing and successful saving / uploading status can be updated so that the latest parsing / uploading status can be obtained when initiating a subsequent evaluation task. Optionally, during the parsing process, the parsing process needs to be recorded and the status updated. Furthermore, after parsing is completed, at least one target evaluation algorithm needs to be used to perform parallel evaluation on at least one piece of test data, and the target evaluation results should be displayed. The target evaluation results are stored in the Tidb database. If any abnormality occurs at any stage, the error information should be recorded, and the task status should be determined as evaluation failure. This should be displayed on the page to inform the user of the error.

[0087] The above solution unifies the evaluation process and encapsulates the evaluation algorithm implementation, eliminating the existing workflow where data collected by the vehicle's industrial control computer needs to be manually uploaded and migrated to a local server. This allows autonomous driving developers to focus more on results analysis, avoiding complex multi-stage connection issues and improving the efficiency of development iterations. Furthermore, since raw data is typically stored in bag format, with a single bag often several gigabytes in size, the automated evaluation system saves the information used in development during data processing and storage, effectively supporting secondary use without needing to focus on the original data files. This significantly reduces data storage costs and increases data utilization.

[0088] Therefore, the above-mentioned technical solution of this application has the following advantages: Data uploaded under a dedicated project can be customized with unique tags according to requirements; projects are isolated and do not affect each other; during parsing, data of the signal level that needs to be stored for a relatively long time is extracted according to the data type, and serialized and archived for storage, reducing the occupation of cloud resources and reducing the impact of irrelevant data when the evaluation program reads OSS; since the parsed data has already been saved, and the video has also saved tagged continuous frame images, the original large-sized package files and videos are no longer needed, greatly saving local server storage space; the parsed data is stored in OSS, and given the characteristics of OSS itself, large amounts of data can be stored in the form of objects, reducing the cost of storing in a database and reducing the pressure of database read and write operations; the evaluation program can directly read the data stored on OSS without mounting it, improving the speed of evaluation calculation and saving evaluation time; parsing and evaluation are decoupled, and the task status is updated and error information is recorded simultaneously, allowing users to know the task status and error reasons at different stages, providing a user-friendly experience; the result display interface accesses the cloud database and OSS through the backend interface, providing a diversified display interface and interaction, making business support convenient and user-friendly.

[0089] According to another aspect of the present invention, an automated evaluation device for vehicle data is also provided. Figure 4 This is a schematic diagram of an automated vehicle data evaluation device according to an embodiment of the present invention, such as... Figure 4 As shown, the device includes:

[0090] The upload module 402 is used to respond to the data upload command applied to the interactive interface, read vehicle datasets collected from multiple real vehicles from the source location carried by the data upload command, and upload the vehicle datasets to the target location corresponding to the data upload command. The vehicle data in the vehicle dataset contains data of multiple data types, as well as the device information of the acquisition device corresponding to different data types. The device information of the acquisition device on different real vehicles is different. The device information includes the manufacturer information and version information of the acquisition device.

[0091] The processing module 404 is used to respond to the evaluation task request applied to the interactive interface, and to process the vehicle data in the vehicle dataset based on the target device information carried in the evaluation task request to obtain the target vehicle data.

[0092] The determination module 406 is used to determine at least one test data from the target vehicle data based on at least one target data type carried in the evaluation task request, wherein the at least one test data corresponds one-to-one with at least one target data type.

[0093] Evaluation module 408 is used to determine the target evaluation algorithm corresponding to at least one target data type from multiple evaluation algorithms, and to perform parallel evaluation on at least one test data based on at least one target evaluation algorithm to obtain the target evaluation result. The multiple evaluation algorithms are used to characterize the pre-encapsulated evaluation algorithms corresponding to multiple data types.

[0094] Display module 410 is used to display the target evaluation results on the interactive interface.

[0095] Optionally, the upload module 402 includes: a first acquisition unit, used to acquire the target project name set and target data tag set carried by the data upload instruction, wherein the target project name set corresponds one-to-one with the vehicle dataset, and different target project names are used to characterize the project to which the corresponding vehicle data belongs, and vehicle data of different projects are stored in different locations, the target data tag set corresponds one-to-one with the vehicle dataset, and different target data tags are used to group the corresponding vehicle data; a first determination unit, used to determine the target location corresponding to each vehicle data based on the target project name and target data tag corresponding to each vehicle data in the vehicle dataset; and a storage unit, used to store each vehicle data in the target location corresponding to each vehicle data.

[0096] Optionally, the device may further include one of the following: a selection module for displaying multiple pre-created project names on an interactive interface, and in response to a selection instruction to select multiple project names, determining the project name corresponding to the selection instruction as a target project name; and a creation module for responding to a creation instruction applied to the interactive interface, creating a target project based on the creation instruction, and using the project name of the target project as a target project name.

[0097] Optionally, the processing module 404 includes: a second determining unit, used to determine initial vehicle data matching the target device information from the vehicle dataset based on the target device information; a third determining unit, used to determine whether it is necessary to parse the initial vehicle data; and a parsing unit, used to parse the initial vehicle data in response to the need to parse the initial vehicle data to obtain the target vehicle data.

[0098] Optionally, the third determining unit is further configured to determine the file type of the file storing the initial vehicle data; in response to the file type being a preset file type, determine that the initial vehicle data needs to be parsed; in response to the file type not being a preset file type, determine that the initial vehicle data does not need to be parsed.

[0099] Optionally, the device further includes: a verification module, used to perform template verification on the initial vehicle data in response to the file type not being a preset file type, and obtain a template verification result; the determination module 406 is also used to determine the initial vehicle data as target vehicle data in response to the template verification result indicating that the initial vehicle data has passed verification.

[0100] Optionally, the test data is stored in an object storage service in a serialized manner. The filename of the test data stored in the object storage service is the same as the filename of the target vehicle data, but the file extension of the test data stored in the object storage service is different from that of the target vehicle data. The evaluation module 408 includes: a second acquisition unit, used to read at least one test data from the object storage service and acquire the corresponding ground truth data for at least one test data, where the ground truth data is data collected from multiple real vehicles under normal conditions; an evaluation unit, used to evaluate each test data based on the corresponding target evaluation algorithm and the corresponding ground truth data, obtaining the evaluation result corresponding to each test data; and a summarization unit, used to summarize the evaluation results corresponding to at least one test data to obtain the target evaluation result.

[0101] Optionally, the device further includes: a first recording module, used to record abnormal information occurring during the parsing process in response to the failure of parsing the initial vehicle data, and to determine that the task status of the evaluation task request is a failure state; a second recording module, used to record abnormal information occurring during the evaluation process in response to the failure of evaluating any test data, and to determine that the task status of the evaluation task request is a failure state; a third recording module, used to record abnormal information occurring during the verification process in response to the template verification result indicating that the target vehicle data verification fails, and to determine that the task status of the evaluation task request is a failure state; the display module 410 is also used to display the recorded abnormal information on the interactive interface in response to the task status of the evaluation task request being a failure state.

[0102] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a storage device, wherein the storage device is used to store one or more programs, which, when executed by one or more processors, cause the one or more processors to perform an automated evaluation method for vehicle data as described above.

[0103] According to another aspect of the present invention, a vehicle is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform an automated evaluation method for vehicle data according to any of the above embodiments.

[0104] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0105] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0106] According to another aspect of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0107] According to another aspect of the present invention, a computer program is also provided, which, when executed by a processor, implements the methods described in the various embodiments of the present invention.

[0108] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0110] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0112] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0113] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An automated evaluation method for vehicle data, characterized in that, include: In response to a data upload command applied to the interactive interface, the system reads vehicle datasets collected from multiple real vehicles from the source location carried by the data upload command, and uploads the vehicle datasets to the target location corresponding to the data upload command. The vehicle data in the vehicle datasets contains data of multiple data types, as well as device information of the acquisition devices corresponding to different data types. The device information of the acquisition devices on different real vehicles is different. The device information includes the manufacturer information and version information of the acquisition devices. In response to the evaluation task request applied to the interactive interface, the vehicle data in the vehicle dataset is processed based on the target device information carried in the evaluation task request to obtain the target vehicle data. Based on at least one target data type carried in the evaluation task request, at least one data to be tested is determined from the target vehicle data, wherein the at least one data to be tested corresponds one-to-one with the at least one target data type; From multiple evaluation algorithms, a target evaluation algorithm corresponding to the at least one target data type is determined, and the at least one test data is evaluated in parallel based on the at least one target evaluation algorithm to obtain the target evaluation result. The multiple evaluation algorithms are used to characterize the pre-encapsulated evaluation algorithms corresponding to the multiple data types. The target evaluation results are displayed on the interactive interface.

2. The method according to claim 1, characterized in that, Uploading the vehicle dataset to the target location corresponding to the data upload command includes: Obtain the target project name set and target data tag set carried by the data upload instruction, wherein the target project name set corresponds one-to-one with the vehicle dataset, and different target project names are used to represent the project to which the corresponding vehicle data belongs, and vehicle data of different projects are stored in different locations; the target data tag set corresponds one-to-one with the vehicle dataset, and different target data tags are used to group the corresponding vehicle data. Based on the target project name and target data label corresponding to each vehicle data in the vehicle dataset, the target location corresponding to each vehicle data is determined. Each vehicle data is stored in the target location corresponding to each vehicle data.

3. The method according to claim 2, characterized in that, The method also includes one of the following: Multiple pre-created project names are displayed on the interactive interface. In response to a selection instruction to select from the multiple project names, the project name corresponding to the selection instruction is determined as a target project name. In response to the creation command applied to the interactive interface, a target project is created based on the creation command, and the project name of the target project is used as a target project name.

4. The method according to claim 1, characterized in that, Based on the target device information carried in the evaluation task request, the vehicle data in the vehicle dataset is processed to obtain the target vehicle data, including: Based on the target device information, initial vehicle data matching the target device information is determined from the vehicle dataset; Determine whether the initial vehicle data needs to be parsed; In response to the need to parse the initial vehicle data, the initial vehicle data is parsed to obtain the target vehicle data.

5. The method according to claim 4, characterized in that, Determine whether the initial vehicle data needs to be parsed, including: Determine the file type of the file containing the initial vehicle data; In response to the fact that the file type is a preset file type, it is determined that the initial vehicle data needs to be parsed; In response to the fact that the file type is not the preset file type, it is determined that the initial vehicle data does not need to be parsed.

6. The method according to claim 5, characterized in that, The method further includes: In response to the fact that the file type is not the preset file type, the initial vehicle data is subjected to template verification to obtain the template verification result; In response to the template verification result indicating that the initial vehicle data has passed verification, the initial vehicle data is determined to be the target vehicle data.

7. The method according to claim 1, characterized in that, The test data is stored in an object storage service in a serialized manner. The filename of the test data stored in the object storage service is the same as the filename of the target vehicle data, and the suffix of the test data stored in the object storage service is different from the suffix of the target vehicle data. Parallel evaluation of the at least one test data is performed based on at least one target evaluation algorithm to obtain target evaluation results, including: Read the at least one test data from the object storage service and obtain the truth data corresponding to the at least one test data, wherein the truth data is data collected by the multiple real vehicles under normal conditions; Each data point to be tested is evaluated based on the corresponding target evaluation algorithm and the corresponding ground truth data, and the evaluation result corresponding to each data point to be tested is obtained. The evaluation results corresponding to the at least one test data are summarized to obtain the target evaluation result.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: In response to the failure to parse the initial vehicle data, the abnormal information that occurred during the parsing process is recorded, and the task status of the evaluation task request is determined to be a failed state. In response to the failure of any test data evaluation, the abnormal information that occurs during the evaluation process is recorded, and the task status of the evaluation task request is determined to be a failed state. If the template verification result indicates that the target vehicle data verification fails, the abnormal information that occurred during the verification process is recorded, and the task status of the evaluation task request is determined to be a failed state. If the task status of the evaluation task request is in a failed state, the recorded exception information is displayed on the interactive interface.

9. An automated evaluation device for vehicle data, characterized in that, include: The upload module is used to respond to the data upload command applied to the interactive interface, read multiple vehicle datasets collected from real vehicles from the source location carried by the data upload command, and upload the vehicle datasets to the target location corresponding to the data upload command. The vehicle data in the vehicle dataset contains data of multiple data types, as well as the device information of the acquisition device corresponding to different data types. The device information of the acquisition device is different on different real vehicles. The device information includes the manufacturer information and version information of the acquisition device. The processing module is used to respond to the evaluation task request applied to the interactive interface, and to process the vehicle data in the vehicle dataset based on the target device information carried in the evaluation task request to obtain the target vehicle data. The determination module is used to determine at least one data to be tested from the target vehicle data based on at least one target data type carried in the evaluation task request, wherein the at least one data to be tested corresponds one-to-one with the at least one target data type; The evaluation module is used to determine the target evaluation algorithm corresponding to the at least one target data type from multiple evaluation algorithms, and to perform parallel evaluation on the at least one test data based on the at least one target evaluation algorithm to obtain the target evaluation result. The multiple evaluation algorithms are used to characterize the pre-encapsulated evaluation algorithms corresponding to the multiple data types. The display module is used to display the target evaluation results on the interactive interface.

10. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the automated evaluation method for vehicle data as described in any one of claims 1-8.

Images