Intelligent driving data evaluation method and device, electronic equipment and storage medium
By utilizing container instances to execute evaluation operator code in intelligent driving data evaluation and performing differentiated behaviors based on scenario matching relationships, the problem of insufficient flexibility and efficiency in existing evaluation methods is solved, achieving high efficiency, configurability, and maintainability in intelligent driving data evaluation.
Patent Information
- Application Number
- CN202512009937.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-07
AI Technical Summary
Existing methods for evaluating intelligent driving data have systemic flaws in terms of flexibility, efficiency, and reproducibility, which severely restrict the efficient iteration and collaborative innovation of intelligent driving technology.
By acquiring intelligent driving data shards and evaluation task configuration files, and based on the scenario matching relationship between evaluation configuration parameters and driving data shards for different driving scenarios, the evaluation operator code is executed using container instances. This enables differentiated behavior of the same evaluation operator code in different scenarios, supporting high reusability and flexible orchestration of operators.
It significantly improves the configurability and maintainability of assessment tasks, and enables efficient execution of complex assessment processes and consistency of results.
Smart Images
Figure CN121807719A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent driving technology, and more specifically, to an intelligent driving data evaluation method, device, electronic device, and storage medium. Background Technology
[0002] In the research and development of intelligent driving, the automated evaluation of the performance of core algorithms such as perception, planning, and control on real data is a key step in verifying the performance of these core algorithms.
[0003] Existing assessment methods are divided into fixed-indicator cloud-based assessment and scripted local assessment. Fixed-indicator cloud-based assessment uses a set of fixed assessment indicators built into the cloud platform. After users upload driving data using a core algorithm, the platform executes preset calculation logic to generate a report. Scripted local assessment downloads driving data using the core algorithm from the platform to the local machine, uses scripting tools to write assessment logic on a local server, completes the analysis, and then manually organizes and uploads the results.
[0004] The above-mentioned scheme has systemic defects in terms of evaluation flexibility, efficiency, and reproducibility, which seriously restricts the efficient iteration and collaborative innovation of intelligent driving technology. Summary of the Invention
[0005] The purpose of this application is to address the shortcomings of the prior art by providing a method, apparatus, electronic device, and storage medium for evaluating intelligent driving data, so as to improve the flexibility, efficiency, and reproducibility of the evaluation.
[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a method for evaluating intelligent driving data, the method comprising: Obtain multiple driving data slices and assessment task configuration files to be evaluated. The assessment task configuration files include: identification information of at least one assessment operator code and assessment configuration parameters for each assessment operator code for at least one driving scenario. Based on the identification information of the at least one evaluation operator code, obtain the image file of the at least one evaluation operator code from the preset image repository. The image file is constructed based on the evaluation operator code and the evaluation dependency environment and is pre-stored in the preset image repository. The number of container instances is determined based on the number of the multiple driving data shards; Based on the number of container instances, create multiple container instances on multiple compute nodes; Based on the scenario matching relationship between the multiple driving data shards and the evaluation configuration parameters of the at least one driving scenario, the image file of the at least one evaluation operator code, the multiple driving data shards, and the evaluation configuration parameters of the corresponding driving scenario are allocated to the multiple container instances. The operator execution engine in the multiple container instances loads the at least one evaluation operator code, and executes the at least one evaluation operator code according to the allocated driving data shards and evaluation configuration parameters to generate multiple driving evaluation results.
[0007] Optionally, before allocating the image file of the at least one evaluation operator code, the multiple driving data shards, and the evaluation configuration parameters of the corresponding driving scenarios to the multiple container instances based on the scenario matching relationship between the multiple driving data shards and the evaluation configuration parameters of the at least one driving scenario, the method further includes: Acquire data features from multiple driving data segments, wherein the data features include at least one or more of the following: road type, weather information, and road congestion information; Based on the scenario matching conditions defined in at least one evaluation configuration parameter and the data characteristics of the multiple driving data segments, the matching relationship between the multiple driving data segments and the at least one evaluation configuration parameter is determined.
[0008] Optionally, before obtaining the image file of the at least one evaluation operator code from a preset image repository based on the address of the at least one evaluation operator code, the method further includes: Define an operator base class, which is used to generate core computation methods, and design the evaluation operator code of the core computation methods based on the operator base class; The target return value type and standard data access interface of the core calculation method are defined in the operator base class, such that the return value type of the evaluation operator code is the target return value type, and the data access interface of the evaluation operator code is the standard data access interface.
[0009] Optionally, the method further includes: The operator result output specification is defined in the operator base class. The operator result output specification includes: multiple indicator fields and the required type of each indicator field; Define the type conversion function for the indicator field.
[0010] Optionally, determining the number of container instances based on the number of the plurality of driving data shards includes: The number of container instances is determined based on the number of the multiple driving data shards and the maximum parallelism of the preset computing cluster, wherein the preset computing cluster includes the multiple computing nodes.
[0011] Optionally, after allocating the image file of the at least one evaluation operator code, the multiple driving data shards, and the evaluation configuration parameters of the corresponding driving scenarios to the multiple container instances based on the scenario matching relationship between the multiple driving data shards and the evaluation configuration parameters of the at least one driving scenario, the method further includes: Real-time monitoring of the evaluation speed of the multiple driving data segments and the working status of the multiple computing nodes; The number of container instances is adjusted based on the evaluation speed of the multiple driving data shards and the working status of the multiple computing nodes.
[0012] Optionally, the process by which the operator execution engine in the container instance executes the evaluation operator code includes: The operator execution engine is started and the evaluation operator code is loaded; Convert the evaluation operator code into an instantiated operator object; Inject a task context, which includes at least: the task execution identifier and node resource information of the evaluation operator code; Based on the allocated driving data segments and evaluation configuration parameters, the evaluation operator code is executed to generate the evaluation results; The evaluation results are converted into a structured dictionary and stored.
[0013] Secondly, embodiments of this application provide an intelligent driving data evaluation device, the device comprising: The information acquisition module is used to acquire multiple driving data slices to be evaluated and evaluation task configuration files. The evaluation task configuration files include: identification information of at least one evaluation operator code and evaluation configuration parameters for each evaluation operator code for at least one driving scenario. The image file acquisition module is used to acquire the image file of the at least one evaluation operator code from a preset image repository based on the identifier of the at least one evaluation operator code. The image file is constructed based on the evaluation operator code and the evaluation dependency environment and is pre-stored in the preset image repository. The instance quantity determination module is used to determine the number of container instances based on the number of the multiple driving data shards; The container instance creation module is used to create multiple container instances on multiple compute nodes according to the number of container instances. The task allocation module is used to allocate the image file of the at least one evaluation operator code, the multiple driving data slices, and the evaluation configuration parameters of the corresponding driving scenarios to the multiple container instances based on the scenario matching relationship between the multiple driving data slices and the evaluation configuration parameters of the at least one driving scenario. The at least one evaluation operator code is loaded by the operator execution engine in the multiple container instances, and the at least one evaluation operator code is executed according to the allocated driving data slices and evaluation configuration parameters to generate multiple driving evaluation results.
[0014] Optionally, the device further includes: The parameter matching module is used to acquire data features of multiple driving data segments, the data features including at least one or more of road type, weather information, and road congestion information; and to determine the matching relationship between the multiple driving data segments and the at least one evaluation configuration parameter based on the scenario matching conditions defined in the at least one evaluation configuration parameter and the data features of the multiple driving data segments.
[0015] Optionally, the device further includes: A standard definition module is used to define an operator base class, which is used to generate a core computation method, and to design the evaluation operator code of the core computation method based on the operator base class. The target return value type and standard data access interface of the core computation method are defined in the operator base class, such that the return value type of the evaluation operator code is the target return value type, and the data access interface of the evaluation operator code is the standard data access interface.
[0016] Optionally, the standard definition module is also used to define the operator result output specification in the operator base class, the operator result output specification including: multiple indicator fields and the required type of each indicator field; and to define the type conversion function of the indicator fields.
[0017] Optionally, the instance quantity determination module is specifically used to determine the number of container instances based on the number of the multiple driving data shards and the maximum parallelism of the preset computing cluster, wherein the preset computing cluster includes the multiple computing nodes.
[0018] Optionally, the instance quantity determination module is further configured to monitor the evaluation speed of the multiple driving data shards and the working status of the multiple computing nodes in real time; and adjust the number of container instances based on the evaluation speed of the multiple driving data shards and the working status of the multiple computing nodes.
[0019] Optionally, the process by which the operator execution engine in the container instance executes the evaluation operator code includes: The operator execution engine is started and the evaluation operator code is loaded; the evaluation operator code is converted into an instantiated operator object; a task context is injected, which includes at least: the task execution identifier and node resource information of the evaluation operator code; the evaluation operator code is executed according to the allocated driving data shards and evaluation configuration parameters to generate the evaluation result; the evaluation result is converted into a structured dictionary and stored.
[0020] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the intelligent driving data evaluation method as described in any of the first aspects.
[0021] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the intelligent driving data evaluation method as described in any of the first aspects.
[0022] The beneficial effects of this application are: The intelligent driving data evaluation method, device, electronic device, and storage medium provided in this application acquire intelligent driving data slices and evaluation task configuration files. Based on the scenario matching relationship between evaluation configuration parameters for different driving scenarios and driving data slices, they achieve differentiated behavior of the same evaluation operator code. This supports testing of driving data slices under different scenarios using the same evaluation operator code, realizing high reusability and flexible arrangement of operators. Complex evaluation processes can be implemented by combining multiple independent operators, significantly improving the configurability and maintainability of evaluation tasks. Attached Figure Description
[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0024] Figure 1 This is an architecture diagram of the intelligent driving data evaluation system provided in the embodiments of this application; Figure 2 A flowchart illustrating the intelligent driving data evaluation method provided in this application embodiment. Figure 1 ; Figure 3 A flowchart illustrating the intelligent driving data evaluation method provided in this application embodiment. Figure 2 ; Figure 4 A flowchart illustrating the intelligent driving data evaluation method provided in this application embodiment. Figure 3 ; Figure 5 A flowchart illustrating the intelligent driving data evaluation method provided in this application embodiment. Figure 4 ; Figure 6 This is a schematic diagram of the structure of the intelligent driving data evaluation device provided in the embodiments of this application; Figure 7 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.
[0026] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0027] In the description of this application, it should be noted that if the terms "upper", "lower", etc. appear to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of this application is usually placed in, it is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.
[0028] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0029] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.
[0030] To better understand the intelligent driving data evaluation method provided in this application, the intelligent driving data evaluation system used in this application will be introduced below.
[0031] Figure 1 The architecture diagram of the intelligent driving data evaluation system provided in the embodiments of this application is as follows: Figure 1 As shown, the intelligent driving data evaluation system includes: algorithm development, platform core layer, infrastructure layer, and infrastructure layer. The algorithm development includes multiple evaluation operator development teams, which develop evaluation operator code for intelligent driving data. The evaluation operator code evaluates the performance of intelligent driving algorithms for intelligent vehicle applications based on intelligent driving data.
[0032] The core layer of the platform runs an operator code repository management module, an image builder, an image repository, and a task scheduling module. The operator code repository management module interacts with the Git repository to manage the operator code versions submitted by the evaluation operator development team. The evaluation operator development team can send the modifications made to the evaluation operator code to the Git repository. The Git repository generates a new version of the evaluation operator code based on the original version and the modifications. The operator code repository management module monitors the Git repository to obtain the new version of the evaluation operator code when the Git repository generates a new version of the evaluation operator code.
[0033] The image builder is used to build the source code files of the evaluation operator code and its running dependencies into an executable Docker image file, and store the Docker image file in an image repository.
[0034] The task scheduling module, acting as the command center of the cloud platform, is responsible for receiving, parsing, and scheduling evaluation tasks submitted by users through the web interface.
[0035] The infrastructure layer includes the Kubernetes cluster, which serves as a container orchestration platform and the underlying engine for task execution. The Kubernetes cluster includes multiple compute nodes, which are elastic computing resource pools. Each compute node can dynamically create and run one or more container instances (Pods) to execute specific evaluation computing tasks.
[0036] A container instance is the basic unit of task execution. Each container instance runs an operator execution engine as a main process. This engine dynamically loads, initializes, and executes the evaluation operator code, and is responsible for saving the operator results after the task execution is completed.
[0037] The data layer or infrastructure layer adopts a distributed storage method to centrally store massive amounts of raw CAN data and the final generated evaluation results.
[0038] The following describes the specific implementation of the intelligent driving data evaluation method provided in this application with reference to the embodiments.
[0039] Figure 2 A flowchart illustrating the intelligent driving data evaluation method provided in this application embodiment. Figure 1 ,like Figure 2 As shown, the method may include: S101. Obtain multiple driving data slices to be evaluated and evaluation task configuration files. The evaluation task configuration files include: identification information of at least one evaluation operator code, and evaluation configuration parameters for each evaluation operator code for at least one driving scenario.
[0040] In this embodiment, driving data segments are independent data units collected and divided by onboard sensors during the operation of an intelligent driving vehicle using various intelligent driving algorithms. Each driving data segment includes a continuous or discrete segment of sensor data and its corresponding vehicle status, environmental perception, and decision logs. Specifically, driving data from a continuous day's driving can be divided according to preset time intervals to obtain multiple driving data segments; alternatively, driving data can be divided based on the intelligent driving algorithm and driving scenario used by the intelligent driving vehicle to obtain multiple driving data segments.
[0041] Each driving data slice may include a LiDAR point cloud sequence, camera image data, millimeter-wave radar data, high-precision positioning trajectory, vehicle control signals, and timestamp-aligned semantic annotation information. The vehicle control signals may include steering wheel angle and accelerator or brake pedal opening.
[0042] In some embodiments, multiple driving data slices can be pre-segmented by a data preprocessing platform and stored in a distributed file system. Each data slice is named with a unique identifier and is accompanied by metadata tags. The metadata tags can include at least some or all of the following: geographical location range, collection time period, weather type, road type, and traffic density. Weather type can be, for example, sunny, rainy, or foggy weather. Road type can be, for example, highway, urban, or rural. Traffic density can be, for example, congested or smooth traffic.
[0043] The assessment task configuration file is a structured JSON configuration file submitted by the user through the web interface provided by the cloud platform. It describes the set of test logic to be executed in this assessment task and its running parameters. The configuration file is uploaded to the task management system of the cloud platform and downloaded and parsed by the task scheduling module during the task startup phase.
[0044] In one example, the test task configuration file includes: The identification information of the evaluation operator code can be the address of the evaluation operator code in the preset mirror repository, and can also include the version information of the evaluation operator code.
[0045] The evaluation configuration parameters include the driving scenarios and corresponding parameters that the parameters are adapted to. For example, for a target monitoring performance evaluation operator, its configuration parameters may include: the applicable scenario is "night scene", the IOU threshold is 0.5, and the confidence level filtering range is [0.3, 1.0].
[0046] In some embodiments, for a segment of driving data, one evaluation operator code or multiple evaluation operator codes may be executed. If multiple evaluation operator codes are executed, the test task configuration file may also include the execution order of the multiple evaluation operator codes.
[0047] Furthermore, for one or more evaluation operator codes, evaluations can be performed in various driving scenarios. Therefore, evaluation configuration parameters for various driving scenarios can be included, enabling flexible reuse of "one operator for multiple scenarios".
[0048] S102. Based on the identification information of at least one evaluation operator code, obtain the image file of at least one evaluation operator code from the preset image repository. The image file is constructed based on the evaluation operator code and the evaluation dependent environment and is stored in the preset image repository in advance.
[0049] In this embodiment, a preset image repository stores multiple pre-built evaluation operator image files. Each evaluation operator image file includes: the evaluation operator code itself, the evaluation dependent environment, and a standardized execution script. The evaluation dependent environment includes an operating system, runtime environment, and third-party dependency libraries that are compatible with the evaluation operator code. The standardized execution script is used to automatically initialize the execution environment, load evaluation configuration parameters, and trigger the operator execution engine to call the evaluation logic of the evaluation operator code when the container starts.
[0050] S103. Determine the number of container instances based on the number of multiple driving data shards.
[0051] In this embodiment, multiple driving data slices can be used to independently execute the same evaluation operator for batch evaluation.
[0052] To improve the overall testing throughput of the system, the number of container instances to be created is dynamically determined based on the number of multiple driving data shards.
[0053] In some embodiments, the number of container instances can be set to be equal to the number of driving data shards, according to the parallel processing mode of "one shard, one container".
[0054] S104. Based on the number of container instances, create multiple container instances on multiple compute nodes.
[0055] In this embodiment, after determining the required number of container instances, the system calls the management interface of the underlying container orchestration platform to create the corresponding number of container instances in batches in the K8S cluster composed of multiple compute nodes. Each container instance is used to independently load and execute the evaluation operator code.
[0056] S105. Based on the scenario matching relationship between multiple driving data shards and evaluation configuration parameters of at least one driving scenario, the image file of at least one evaluation operator code, multiple driving data shards, and evaluation configuration parameters of the corresponding driving scenario are allocated to multiple container instances. The operator execution engine in the multiple container instances loads at least one evaluation operator code, and executes at least one evaluation operator code according to the allocated driving data shards and evaluation configuration parameters to generate multiple driving evaluation results.
[0057] In this embodiment, multiple driving data segments and evaluation configuration parameters for at least one driving scenario are matched to determine the evaluation configuration parameters corresponding to each driving data segment.
[0058] After matching the driving data shards with the evaluation configuration parameters, the image file of at least one evaluation operator code, each driving data shard, and the corresponding evaluation configuration parameters are packaged into a task unit. The task scheduling module sends each task unit to the corresponding container instance one by one according to the running status of each container instance.
[0059] After each container instance receives the task instruction, its built-in operator execution engine starts working. Based on the image file of at least one evaluation operator code, it determines the running environment of the operator execution engine, loads at least one evaluation operator code into the running space of the operator execution engine, reads each driving data shard and evaluation configuration parameters, runs the core computing logic of at least one evaluation operator code, and generates driving evaluation results for each driving data shard and for at least one evaluation operator code.
[0060] In some embodiments, if there is one evaluation operator code, a single evaluation operator code is executed to generate a driving evaluation result for each driving data segment; if there are multiple evaluation operator codes, multiple evaluation operator codes are executed in execution order to generate multiple driving evaluation results for each driving data segment.
[0061] It should be noted that in each evaluation task, all driving data slices execute the same evaluation operator code, but the evaluation configuration parameters may be the same or different. This allows the same evaluation operator code to be reused to complete different driving data evaluations for different driving scenarios. Specifically, if the evaluation configuration parameters of the driving data slice and the evaluation operator code do not match for at least one driving scenario, the default configuration parameters in the evaluation operator code will be used for that driving data.
[0062] Driving test results can evaluate the effectiveness of intelligent driving algorithms that generate driving data slices, and thus optimize the intelligent driving algorithms based on the driving test results.
[0063] The intelligent driving data evaluation method provided in the above embodiments obtains intelligent driving data segments and evaluation task configuration files. Based on the scenario matching relationship between evaluation configuration parameters for different driving scenarios and driving data segments, it realizes differentiated behavior of the same evaluation operator code. It supports the testing of the same evaluation operator code for driving data segments in different scenarios, realizes high reusability and flexible arrangement of operators, and enables complex evaluation processes to be implemented by combining multiple independent operators, which greatly improves the configurability and maintainability of evaluation tasks.
[0064] In one possible implementation, Figure 3 A flowchart illustrating the intelligent driving data evaluation method provided in this application embodiment. Figure 2 ,like Figure 3 As shown, before S105 allocates the image file of at least one evaluation operator code, multiple driving data shards, and evaluation configuration parameters of the corresponding driving scenario to multiple container instances based on the scenario matching relationship between multiple driving data shards and evaluation configuration parameters of at least one driving scenario, the method may further include: S201. Obtain data features from multiple driving data segments. The data features shall include at least one or more of the following: road type, weather information, and road congestion information.
[0065] In this embodiment, each driving data segment is an autonomous driving operation segment collected from the test vehicle. The data characteristics of each driving data segment can be determined based on the metadata tags of each driving data segment, or the data characteristics can be determined by extracting features from the driving data segments.
[0066] Road types can include highways, urban arterial roads, ramps, rural roads, parking lots, etc. The category of road segments in driving data slices can be identified through high-precision map matching modules or visual semantic segmentation models.
[0067] Weather information can include sunny, rainy, snowy, and foggy weather. It can be determined by combining the results of visual analysis from vehicle sensors such as rain sensors and light intensity sensors, external meteorological API interfaces, image defogging algorithms, and visibility estimation algorithms.
[0068] Road congestion information can be categorized as smooth traffic, slow traffic, congestion, or stagnation. It can be determined using a rule engine or a lightweight classification model based on vehicle speed change curves, distance to the vehicle in front, traffic light status, and surrounding vehicle density.
[0069] S202. Based on the scenario matching conditions defined in at least one evaluation configuration parameter and the data characteristics of multiple driving data segments, determine the matching relationship between multiple driving data segments and at least one evaluation configuration parameter.
[0070] In this embodiment, at least one evaluation configuration parameter bound to each evaluation operator code is parsed, and defined scene matching conditions are extracted from it. For example, the scene matching condition indicates that the evaluation configuration parameter is suitable for data sharding under the combined scenario of "high speed + rain and fog + congestion + night".
[0071] Subsequently, the data features of the driving data segments are matched with the scene matching conditions. The matching method can be exact matching or confidence matching. Exact matching must ensure that the data features meet all scene matching conditions, while confidence matching can determine the matching of data with a matching degree greater than a preset threshold based on the matching degree of the data features and scene matching conditions.
[0072] The intelligent driving data evaluation method provided in the above embodiments divides the evaluation configuration parameters and driving data slices into slices, realizing a refined evaluation strategy based on driving scenarios, which significantly improves the intelligence level of driving data evaluation and the utilization rate of evaluation operator code.
[0073] In one possible implementation, before step S102, which retrieves the image file of at least one evaluation operator code from a preset image repository based on the identification information of at least one evaluation operator code, the method may further include: Define an operator base class, which is used to generate core computation methods. The evaluation operator code for the core computation methods is designed based on the operator base class. The target return value type and standard data access interface of the core computation methods are defined in the operator base class, so that the return value type of the evaluation operator code is the target return value type and the data access interface of the evaluation operator code is the standard data access interface.
[0074] In this embodiment, in order to achieve a modular, reusable and highly cohesive architecture for the intelligent driving evaluation system, a mandatory operator base class (BaseOperator) is defined as a unified interface contract that all operator development teams must inherit and implement when developing evaluation operator code.
[0075] The base class `BaseOperator` inherits from the cloud platform's default public interface protocol `Operator` and is encapsulated using an abstract base class mechanism. This ensures that all subclasses must override its core computation method `compute()`, otherwise instantiation will fail. This method serves as the sole entry point for the evaluation operator, carrying the specific evaluation logic. When developing new evaluation operators, the operator development team must derive from this base class and implement the `compute()` method. After the operator development team submits the evaluation operator code, the cloud platform first verifies whether it inherits from the operator base class. If it does not inherit from the operator base class, the code is considered invalid.
[0076] To ensure consistency of evaluation results and automation of subsequent processing, the operator base class explicitly defines the return type of core calculation methods through type annotations as either a single structured result dictionary (Dict) or a list of multiple results (List[Dict]). During code submission, the return format can be checked to determine if it conforms to the specifications; if not, the code can be intercepted, improving system robustness.
[0077] To further reduce the complexity of operator code development for the operator development team, a set of standardized data access APIs are built into the operator base class, which hides the implementation details such as underlying data storage format, file path, and network location. All subclasses can directly call these methods to obtain the required information without having to parse the raw logs or build complex I / O processes themselves.
[0078] For example, a standard data access interface may include: has_data(channel_name): Checks if data exists in the specified channel; read_data(channel_name, field_list): Reads standardized time series data; get_label(): Retrieves scene label metadata; get_env(), get_config(): Retrieves runtime configuration parameters.
[0079] The intelligent driving data evaluation method provided in the above embodiments introduces a mandatory operator base class and standardized contract. Developers do not need to understand the underlying mechanisms such as distributed scheduling, containerized deployment and data hanging path. They only need to write the core calculation method according to the interface contract to achieve complete decoupling between user logic and platform environment. All operator outputs are structured dictionaries, which instruct the platform to automatically parse, extract indicators and visualize them, avoiding the manual sorting cost caused by messy format and improving the consistency and aggregability of evaluation results.
[0080] In one possible implementation, the method may further include: Define the operator result output specification in the operator base class. The operator result output specification includes: multiple indicator fields and the required type for each indicator field; define the type conversion function for the indicator fields.
[0081] In this embodiment, in order to solve the problems of difficulty in uniform processing, failure to enter data into the database, or abnormal visualization caused by the output format defined by each algorithm team during the intelligent driving evaluation process, the system further defines a domain-specific operator result output specification in the operator base class. The operator result output specification clearly stipulates the structured data format that must be returned after the evaluation operator is executed, and supports two modes according to different application scenarios.
[0082] Specifically, for overall evaluation of driving data segments, i.e., evaluation using a single evaluation operator code, a single structured result dictionary can be requested. For fine-grained analysis of driving data segments, i.e., evaluation using multiple evaluation operator codes, a list of multiple structured result dictionaries can be requested.
[0083] The structured results dictionary specifies various indicator fields and whether each indicator is required. For required indicator fields, the operator development team must include the required indicator fields in the output results when developing the evaluation operator code. For non-required indicator fields, the operator development team can determine whether the output results need to include them as needed. In addition, the operator development team can also set custom fields in the evaluation operator code.
[0084] To further enhance the development experience and prevent risks caused by implicit type conversion, a set of type conversion helper functions can be provided for all evaluation operator code to call.
[0085] For example, "status": to_str("pass") means ensuring that the result of "status" is converted to a string and stored.
[0086] The intelligent driving data evaluation method provided in the above embodiments ensures that the output results of multiple evaluation operator codes developed by multiple operator development teams can be automatically identified, extracted and stored by the cloud platform, achieving full-domain parseability of the evaluation results; and ensures the consistency of data type when entering the database through mandatory type declaration and safe conversion functions.
[0087] In one possible implementation, the process of determining the number of container instances based on the number of multiple driving data shards in S103 above may include: The number of container instances is determined based on the number of multiple driving data shards and the maximum parallelism of the preset computing cluster, which includes multiple computing nodes.
[0088] In this embodiment, considering that the actual computing cluster has a resource limit, the maximum parallelism of the computing cluster is introduced as a hard limit. The maximum parallelism can be pre-configured by the operation and maintenance personnel, reflecting the maximum number of concurrent tasks that the underlying hardware resources of the computing cluster can support.
[0089] Specifically, based on the maximum parallelism C_max and the number of multiple driving data shards N, the smaller value between the maximum parallelism C_max and the number of multiple driving data shards N is determined to be the number of container instances.
[0090] Specifically, if the maximum parallelism C_max is less than the number of multiple driving data shards N, the number of container instances is determined to be the maximum parallelism C_max; if the maximum parallelism C_max is greater than the number of multiple driving data shards N, the number of container instances is determined to be the number of multiple driving data shards N.
[0091] Furthermore, Figure 4 A flowchart illustrating the intelligent driving data evaluation method provided in this application embodiment. Figure 3 ,like Figure 4 As shown, the method may further include: S301: Real-time monitoring of the evaluation speed of multiple driving data segments and the working status of multiple computing nodes.
[0092] S302. Adjust the number of container instances based on the evaluation speed of multiple driving data shards and the working status of multiple computing nodes.
[0093] In this embodiment, after the evaluation task begins execution, the task scheduling module monitors two key indicators in real time: the processing progress of the evaluation task and the cluster resource status. The processing progress is the number of data shards that have been evaluated within a unit of time, and the cluster resource status is the average utilization rate of the processors and memory of all computing nodes. When the evaluation speed is slow and the cluster resources are sufficient (e.g., the average utilization rate is below 70%), the number of container instances is automatically increased to improve the processing speed. When the cluster resources are strained (e.g., the average utilization rate is above 85%), the number of container instances is appropriately reduced to ensure the stable operation of the system.
[0094] The intelligent driving data evaluation method provided in the above embodiments adjusts the number of container instances according to the evaluation speed of multiple driving data shards and the working status of multiple computing nodes, which can ensure a balance between processing speed and system stability.
[0095] In one possible implementation, Figure 5 A flowchart illustrating the intelligent driving data evaluation method provided in this application embodiment. Figure 4 ,like Figure 5 As shown, the process of the operator execution engine in the container instance executing the evaluation operator code includes: S401. Start the operator execution engine and load the evaluation operator code.
[0096] S402. Convert the evaluation operator code into an instantiated operator object.
[0097] S403. Inject the task context. The task context shall include at least the task execution identifier of the evaluation operator code and node resource information.
[0098] S404. Based on the allocated driving data segments and evaluation configuration parameters, execute the evaluation operator code to generate the evaluation results.
[0099] S405. Convert the evaluation results into a structured dictionary and store them.
[0100] In this embodiment, in order to achieve seamless integration, high-performance execution, and transparent access to resources between the evaluation operator code and the platform runtime environment, a rotating operator execution engine is deployed inside each container instance. As the core component connecting the evaluation logic and the underlying infrastructure, this engine is responsible for the full lifecycle management from code loading and context injection to result collection, providing near-native code execution efficiency while ensuring security isolation.
[0101] Once the container instance completes initialization, the operator execution engine automatically starts as the main process, loads the evaluation operator code into the process space of the operator execution engine, calls the constructor to convert the evaluation operator code into instantiated operator objects, and after the operator object is instantiated, the execution engine automatically injects a runtime task context into it. The task context may include: task execution identifier, node resource information, data accessor, and configuration parameter handle. The task execution identifier is a globally unique task number used for log tracking and result attribution. The node resource information includes the number of available CPU cores, memory capacity, GPU model, etc. of the current computing node. The data accessor encapsulates the ability to read driving data shards from shared storage, and the configuration parameter handle points to the evaluation configuration parameters of the current task shard.
[0102] The `prepare()` method is called to perform pre-processing, such as model loading and cache initialization. Then, the core computation method `compute()` in the evaluation operator code is called to execute the core evaluation logic, passing in the data sharding and configuration parameters. After execution, the return result is obtained, converted into a structured dictionary, and serialized for storage. The `cleanup()` method is then called to release resources, close connections, and clean up temporary files. During the conversion of the result into a structured dictionary, the return value is forcibly validated to ensure it conforms to the structured dictionary format, and field types are safely converted using type conversion functions.
[0103] The intelligent driving data evaluation method provided in the above embodiments does not require developers to write any infrastructure adaptation code. They only need to implement business logic, which can be automatically scheduled and executed in a distributed cluster. Based on the dynamic loading mechanism, operator versions can be flexibly switched without rebuilding the container. Only the hot update of operators and hot switching of versions are possible. All tasks go through the same execution path, which facilitates unified monitoring, log collection and fault diagnosis.
[0104] In one possible implementation, the method may further include: The software dependencies of the testing environment are packaged into standardized deployment units, which include at least: an operating system consistent with the testing environment, the software development kit of the testing environment, and third-party dependency libraries.
[0105] To address the common issue in existing intelligent driving evaluation systems where systems run locally but encounter errors in the cloud, the software dependencies of the evaluation environment, such as operating systems, software development kits, and third-party libraries, can be packaged into standardized deployment units. This allows operator development teams to run the same environment locally as the evaluation environment, enabling them to develop and debug test operator code locally. They can set breakpoints, view variable values, and step through the compute() method in the PyCharm interface, running entirely within the same dependency environment as the cloud platform, truly achieving a "what you see is what you get" development experience.
[0106] Based on the above method embodiments, this application provides an intelligent driving data evaluation device. Figure 6 This is a schematic diagram of the structure of the intelligent driving data evaluation device provided in the embodiments of this application, as shown below. Figure 6 As shown, the device may include: The information acquisition module 501 is used to acquire multiple driving data slices to be evaluated and the evaluation task configuration file. The evaluation task configuration file includes: identification information of at least one evaluation operator code and evaluation configuration parameters for each evaluation operator code for at least one driving scenario. The image file acquisition module 502 is used to acquire the image file of at least one evaluation operator code from the preset image repository based on the identifier of at least one evaluation operator code. The image file is constructed based on the evaluation operator code and the evaluation dependency environment and is pre-stored in the preset image repository. The instance number determination module 503 is used to determine the number of container instances based on the number of multiple driving data shards; The container instance creation module 504 is used to create multiple container instances on multiple compute nodes based on the number of container instances. The task allocation module 505 is used to allocate the image file of at least one evaluation operator code, multiple driving data slices and evaluation configuration parameters of corresponding driving scenarios to multiple container instances based on the scenario matching relationship between multiple driving data slices and evaluation configuration parameters of at least one driving scenario. The operator execution engine in the multiple container instances loads at least one evaluation operator code and executes at least one evaluation operator code according to the allocated driving data slices and evaluation configuration parameters to generate multiple driving evaluation results.
[0107] Optionally, the device may further include: The parameter matching module is used to acquire data features of multiple driving data segments, including at least one or more of the following: road type, weather information, and road congestion information; and to determine the matching relationship between multiple driving data segments and at least one evaluation configuration parameter based on the scenario matching conditions defined in at least one evaluation configuration parameter and the data features of multiple driving data segments.
[0108] Optionally, the device may further include: The standard definition module is used to define operator base classes, which are used to generate core computation methods. Evaluation operator code for core computation methods is designed based on the operator base classes. The target return value type and standard data access interface of the core computation methods are defined in the operator base classes, so that the return value type of the evaluation operator code is the target return value type and the data access interface of the evaluation operator code is the standard data access interface.
[0109] Optionally, the standard definition module is also used to define the operator result output specification in the operator base class. The operator result output specification includes: multiple indicator fields and the required type for each indicator field; and defining the type conversion function for the indicator fields.
[0110] Optionally, the instance number determination module 503 is specifically used to determine the number of container instances based on the number of multiple driving data shards and the maximum parallelism of the preset computing cluster, wherein the preset computing cluster includes multiple computing nodes.
[0111] Optionally, the instance number determination module 503 is also used to monitor the evaluation speed of multiple driving data shards and the working status of multiple computing nodes in real time; and adjust the number of container instances based on the evaluation speed of multiple driving data shards and the working status of multiple computing nodes.
[0112] Optionally, the process by which the operator execution engine in the container instance executes the evaluation operator code includes: Start the operator execution engine and load the evaluation operator code; convert the evaluation operator code into instantiated operator objects; inject the task context, which includes at least: the task execution identifier of the evaluation operator code and node resource information; execute the evaluation operator code according to the allocated driving data slices and evaluation configuration parameters, and generate evaluation results; convert the evaluation results into a structured dictionary and store them.
[0113] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.
[0114] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).
[0115] Figure 7 A schematic diagram of the electronic device provided in the embodiments of this application, such as... Figure 7 As shown, the electronic device 600 may include a processor 601, a storage medium 602, and a bus. The storage medium 602 stores program instructions executable by the processor 601. When the electronic device 600 is running, the processor 601 communicates with the storage medium 602 via the bus, and the processor 601 executes the program instructions to perform the above-described method embodiment. Specific implementation methods and technical effects are similar and will not be repeated here.
[0116] Optionally, this application also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the above-described method embodiments.
[0117] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0119] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0120] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0121] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for evaluating intelligent driving data, characterized in that, The method includes: Obtain multiple driving data slices and assessment task configuration files to be evaluated. The assessment task configuration files include: identification information of at least one assessment operator code and assessment configuration parameters for each assessment operator code for at least one driving scenario. Based on the identification information of the at least one evaluation operator code, obtain the image file of the at least one evaluation operator code from the preset image repository. The image file is constructed based on the evaluation operator code and the evaluation dependency environment and is pre-stored in the preset image repository. The number of container instances is determined based on the number of the multiple driving data shards; Based on the number of container instances, create multiple container instances on multiple compute nodes; Based on the scenario matching relationship between the multiple driving data shards and the evaluation configuration parameters of the at least one driving scenario, the image file of the at least one evaluation operator code, the multiple driving data shards, and the evaluation configuration parameters of the corresponding driving scenario are allocated to the multiple container instances. The operator execution engine in the multiple container instances loads the at least one evaluation operator code, and executes the at least one evaluation operator code according to the allocated driving data shards and evaluation configuration parameters to generate multiple driving evaluation results.
2. The method as described in claim 1, characterized in that, Before allocating the image file of the at least one evaluation operator code, the multiple driving data shards, and the evaluation configuration parameters of the corresponding driving scenarios to the multiple container instances based on the scenario matching relationship between the multiple driving data shards and the evaluation configuration parameters of the at least one driving scenario, the method further includes: Acquire data features from multiple driving data segments, wherein the data features include at least one or more of the following: road type, weather information, and road congestion information; Based on the scenario matching conditions defined in at least one evaluation configuration parameter and the data characteristics of the multiple driving data segments, the matching relationship between the multiple driving data segments and the at least one evaluation configuration parameter is determined.
3. The method as described in claim 1, characterized in that, Before obtaining the image file of the at least one evaluation operator code from a preset image repository based on the address of the at least one evaluation operator code, the method further includes: Define an operator base class, which is used to generate core computation methods, and design the evaluation operator code of the core computation methods based on the operator base class; The target return value type and standard data access interface of the core calculation method are defined in the operator base class, such that the return value type of the evaluation operator code is the target return value type, and the data access interface of the evaluation operator code is the standard data access interface.
4. The method as described in claim 3, characterized in that, The method further includes: The operator result output specification is defined in the operator base class. The operator result output specification includes: multiple indicator fields and the required type of each indicator field; Define the type conversion function for the indicator field.
5. The method as described in claim 1, characterized in that, Determining the number of container instances based on the number of the multiple driving data shards includes: The number of container instances is determined based on the number of the multiple driving data shards and the maximum parallelism of the preset computing cluster, wherein the preset computing cluster includes the multiple computing nodes.
6. The method as described in claim 1, characterized in that, After allocating the image file of the at least one evaluation operator code, the multiple driving data shards, and the evaluation configuration parameters of the corresponding driving scenarios to the multiple container instances based on the scenario matching relationship between the multiple driving data shards and the evaluation configuration parameters of the at least one driving scenario, the method further includes: Real-time monitoring of the evaluation speed of the multiple driving data segments and the working status of the multiple computing nodes; The number of container instances is adjusted based on the evaluation speed of the multiple driving data shards and the working status of the multiple computing nodes.
7. The method as described in claim 1, characterized in that, The process by which the operator execution engine in the container instance executes the evaluation operator code includes: The operator execution engine is started and the evaluation operator code is loaded; Convert the evaluation operator code into an instantiated operator object; Inject a task context, which includes at least: the task execution identifier and node resource information of the evaluation operator code; Based on the allocated driving data segments and evaluation configuration parameters, the evaluation operator code is executed to generate the evaluation results; The evaluation results are converted into a structured dictionary and stored.
8. A smart driving data evaluation device, characterized in that, The device includes: The information acquisition module is used to acquire multiple driving data slices to be evaluated and evaluation task configuration files. The evaluation task configuration files include: identification information of at least one evaluation operator code and evaluation configuration parameters for each evaluation operator code for at least one driving scenario. The image file acquisition module is used to acquire the image file of the at least one evaluation operator code from a preset image repository based on the identifier of the at least one evaluation operator code. The image file is constructed based on the evaluation operator code and the evaluation dependency environment and is pre-stored in the preset image repository. The instance quantity determination module is used to determine the number of container instances based on the number of the multiple driving data shards; The container instance creation module is used to create multiple container instances on multiple compute nodes according to the number of container instances. The task allocation module is used to allocate the image file of the at least one evaluation operator code, the multiple driving data slices, and the evaluation configuration parameters of the corresponding driving scenarios to the multiple container instances based on the scenario matching relationship between the multiple driving data slices and the evaluation configuration parameters of the at least one driving scenario. The at least one evaluation operator code is loaded by the operator execution engine in the multiple container instances, and the at least one evaluation operator code is executed according to the allocated driving data slices and evaluation configuration parameters to generate multiple driving evaluation results.
9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the intelligent driving data evaluation method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, performs the steps of the intelligent driving data evaluation method as described in any one of claims 1 to 7.