Automatic driving simulation task scheduling method, electronic device, vehicle and storage medium

Abstract

This application provides an autonomous driving simulation task scheduling method, electronic device, vehicle, and storage medium. The method includes: acquiring relationship data between scenarios and simulation algorithms, resource pool status information, task execution status information, and user feedback and behavioral information on simulation task scheduling; inputting the above information into a simulation task reinforcement learning model to obtain vehicle simulation task scheduling strategy information, and executing scheduling according to the strategy information. Thus, this method integrates multi-dimensional information such as scenario association, algorithm association, system resources, and user feedback, enabling the scheduling strategy to consider both business logic and user needs. While improving scheduling efficiency and shortening waiting time, it enhances the relevance of scheduling results to business scenarios and user needs, comprehensively improving the intelligence level and user experience of simulation task scheduling.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to an autonomous driving simulation task scheduling method, electronic device, vehicle, and storage medium. Background Technology

[0002] In autonomous driving simulation scenarios, task scheduling involves the rational allocation and orderly management of various tasks generated during the simulation process, ultimately achieving efficient utilization of simulation resources and ensuring that simulation tasks proceed stably as expected.

[0003] In related technologies, priority is set for simulation tasks, and tasks with higher priority are scheduled first. However, this method lacks consideration of scenario relevance, and the scheduling is detached from the core business. It has problems such as low scheduling efficiency, poor adaptability, and ignoring user feedback and behavior, resulting in a poor user experience, which urgently needs to be solved. Summary of the Invention

[0004] This application provides an autonomous driving simulation task scheduling method, electronic device, vehicle, and storage medium, aiming to improve the technical problem of low scheduling efficiency in related technologies, so as to enhance business relevance and improve user experience.

[0005] The first aspect of this application provides a method for scheduling autonomous driving simulation tasks, comprising the following steps: acquiring relationship data between scenarios and relationship data of simulation algorithms, wherein the relationship data between scenarios is used to characterize the real user needs of autonomous driving simulation tasks, and the relationship data of simulation algorithms is used to characterize the relationship between algorithm versions of autonomous driving simulation tasks; acquiring resource pool status information and task execution status information, wherein the resource pool status information is used to characterize the utilization rate of the resource pool, and the task execution status information is used to characterize the progress of task execution; acquiring user feedback information on simulation task scheduling and user behavior information; inputting the relationship data, the relationship data of simulation algorithms, the resource pool status information, the task execution status information, the feedback information, and the behavior information into a simulation task reinforcement learning model to obtain vehicle simulation task scheduling strategy information; and executing scheduling according to the simulation task scheduling strategy information.

[0006] The autonomous driving simulation task scheduling method according to the embodiments of this application obtains historical scene correlation data of vehicles, parses historical simulation task data to generate the algorithm under test, and combines user feedback and behavioral information input model to obtain scheduling strategy for execution, thereby improving scheduling efficiency, enhancing business relevance, and improving user experience.

[0007] In addition, the autonomous driving simulation task scheduling method according to the above embodiments of this application may also have the following additional technical features: According to one embodiment of this application, obtaining the correlation data between scenes specifically includes: parsing historical scene files to extract scene tag information and scene metadata; based on the execution records of historical simulation tasks, statistically analyzing the co-occurrence frequency and co-occurrence order of scenes in the same batch of tasks; associating and mapping the scene tag information, co-occurrence frequency, and co-occurrence order to generate explicit scene correlation data; the explicit scene correlation data includes associations with the same tag, associations with similar tags, and associations with subordinate tags; and implicit scene correlation data is obtained by adaptively mining during the scheduling and training process of the simulation task reinforcement learning model.

[0008] According to one embodiment of this application, obtaining relational data of simulation algorithms specifically includes: obtaining historical submission records of the algorithm under test; parsing the historical data of the simulation task to obtain the classification features of a single code submission and the correlation features between multiple code submissions; and generating relational data of the simulation algorithm based on the classification features of a single code submission and the correlation features between multiple code submissions.

[0009] According to one embodiment of this application, the association data, the relationship data of the simulation algorithm, the resource pool status information, the task execution status information, the feedback information, and the behavior information are input into the simulation task reinforcement learning model to obtain the vehicle's simulation task scheduling strategy information. Specifically, this includes: splicing and fusing the association data, the relationship data of the simulation algorithm, the resource pool status information, the task execution status information, the feedback information, and the behavior information to obtain the initial task scheduling information; and inputting the initial task scheduling information into the simulation task reinforcement learning model to obtain the simulation task scheduling strategy information.

[0010] According to one embodiment of this application, after executing the scheduling based on the simulation task scheduling strategy information, the method further includes: collecting environmental feedback data after the scheduling execution, including changes in resource pool utilization, task waiting time, user satisfaction rating, time difference between task completion and user viewing, and scenario / algorithm similarity of continuously executing tasks; performing weighted calculation on the environmental feedback data based on a preset multi-objective reward function to obtain a cumulative reward value; and using the cumulative reward value and the corresponding state-action interaction data to update the parameters of the simulation task reinforcement learning model online.

[0011] According to one embodiment of this application, the simulation task reinforcement learning model is generated in the following manner: During the model initialization phase, the policy network parameters are set to zero, causing the scheduling policy to degenerate into a first-in-first-out queue policy, completing the cold start; during the actual operation of the platform, the interaction data of the four-tuple of state, action, reward, and next state are continuously collected; the interaction data are optimized using a preset reinforcement learning algorithm to optimize the policy gradient or value function, and the model parameters are iteratively updated so that the scheduling policy gradually approaches the global optimum. According to one embodiment of this application, the simulation task scheduling policy information includes a sorted queue of tasks to be executed, and the scheduling is performed according to the simulation task scheduling policy information, including: parsing the sorted queue to obtain a list of tasks to be executed; and calling the corresponding resources from the resource pool according to the list of tasks to be executed for simulation testing.

[0012] A second aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned autonomous driving simulation task scheduling method.

[0013] The electronic device according to the embodiments of this application implements the above-described autonomous driving simulation task scheduling method. Therefore, by acquiring historical scene correlation data of the vehicle, parsing historical simulation task data to generate a simulation algorithm, and combining user feedback and behavioral information input models to obtain a scheduling strategy for execution, scheduling efficiency is improved, business relevance is enhanced, and user experience is improved.

[0014] A third aspect of this application provides a vehicle that includes the aforementioned electronic equipment.

[0015] The vehicle according to the embodiments of this application includes electronic devices that implement the aforementioned autonomous driving simulation task scheduling method. Thus, by acquiring historical scene correlation data of the vehicle, parsing historical simulation task data to generate a simulation algorithm, and combining user feedback and behavioral information input models to obtain a scheduling strategy for execution, scheduling efficiency is improved, business relevance is enhanced, and user experience is improved.

[0016] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described autonomous driving simulation task scheduling method.

[0017] The computer-readable storage medium according to the embodiments of this application implements the above-described autonomous driving simulation task scheduling method when the computer program stored thereon is executed by a processor. Based on the above-described autonomous driving simulation task scheduling method, scheduling efficiency is improved, business relevance is enhanced, and user experience is improved. Attached Figure Description

[0018] Figure 1This is a flowchart of an autonomous driving simulation task scheduling method provided in some embodiments of this application; Figure 2 This is a flowchart of an autonomous driving simulation task scheduling method provided in some specific embodiments of this application; Figure 3 This is a block diagram of an electronic device provided in some embodiments of this application; Figure 4 This is a block diagram of a vehicle provided in some embodiments of this application. Detailed Implementation

[0019] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] The autonomous driving simulation task scheduling method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0021] Please refer to Figure 1 The autonomous driving simulation task scheduling method provided in this application includes the following steps: S1. Obtain the correlation data between scenarios and the correlation data of simulation algorithms. The correlation data between scenarios is used to characterize the real user needs of autonomous driving simulation tasks, and the correlation data of simulation algorithms is used to characterize the correlation between algorithm versions of autonomous driving simulation tasks.

[0022] The correlation data may refer to the similarity or subordinate relationship between the tags carried by the scenes, as well as the co-occurrence frequency of different scenes being selected in the same batch in historical simulation tasks; the tags include scene type, vehicle behavior, and weather conditions. In some embodiments, the correlation data also includes implicit relationships, which are automatically mined and reflected during the reinforcement learning training process of the scheduler. That is, the model learns the potential relationships between scenes that cannot be directly defined by static label rules by analyzing user viewing behavior and feedback data.

[0023] Relationship data for simulation algorithms can refer to the relationships between different algorithm versions. For example, this relationship data can be the similarity between different algorithm versions based on Git (Global Information Tracker, a distributed version control system) metadata. Git metadata includes the branch, committer, commit time, and involved code modules. In some embodiments, the relationship data for simulation algorithms also includes implicit relationships, which are dynamically learned and reflected during the scheduler's reinforcement learning training process. That is, the model discovers potential, unlabeled, correlations between algorithm versions based on the user's viewing order and comparison behavior of simulation results for different algorithm versions.

[0024] Furthermore, the correlation data between scenarios is used to characterize the real user needs of autonomous driving simulation tasks, and the correlation data of simulation algorithms is used to characterize the correlation between algorithm versions of autonomous driving simulation tasks.

[0025] Specifically, the correlation data between scenarios and the relationship data of simulation algorithms can be obtained from the vehicle's historical scenario files and simulation task history records. The vehicle's historical scenario files are collections of various scenario data recorded by autonomous vehicles during actual road driving, using sensors such as cameras, LiDAR, and millimeter-wave radar, combined with onboard systems. This data typically contains rich information, such as the vehicle's own state, including speed, acceleration, steering angle, and position; surrounding environmental information, including the driving trajectories of other vehicles, pedestrian dynamics, traffic light status, road signs, and weather conditions; and timestamp information used to mark the sequence of events. Obtaining the vehicle's historical scenario files means collecting and storing records of the vehicle's past real-world driving scenarios through sensors and onboard systems. The simulation task history records originate from task data accumulated by the autonomous driving simulation platform during its historical operation, including the algorithm version information tested in each simulation task and its corresponding Git commit records.

[0026] After acquiring a large number of historical scenario files and simulation task history records, it is necessary to analyze the inherent connections between these scenarios and simulation task history records to generate correlation data and simulation algorithm relationship data. Specifically, the generation of correlation data is achieved by extracting and analyzing features from historical scenario files to find the correlations between different scenarios. For example, temporal correlation: after a scenario of a vehicle driving at low speed in a congested section, there may be frequent scenarios of accelerating into a smooth section, showing a sequential relationship between the two. Causal correlation: for example, a scenario of a vehicle braking suddenly ahead may lead to a scenario of this vehicle braking suddenly, showing a causal triggering relationship. Similar feature correlation: for example, two different scenarios may both contain the features of rainy weather, highways, and high traffic volume, showing a correlation due to similar environmental or traffic conditions. The generation of simulation algorithm relationship data is achieved by parsing Git metadata in the simulation task history records, specifically including extracting the branch name, committer identity, commit timestamp, and modified code module path corresponding to each algorithm version, and calculating the similarity between different algorithm versions based on the above metadata. For example, algorithm versions originating from the same branch or completed by the same committer are considered to have a high correlation.

[0027] The generated correlation data and the correlation data of the simulation algorithm are essentially a summary of the rules of how and why scenarios are connected. The purpose is to provide a basis for subsequent simulation task scheduling. For example, during simulation testing, related scenarios can be automatically linked according to the correlation to more realistically simulate continuous actual driving processes and improve the effectiveness and efficiency of simulation.

[0028] Compared to simulation task scheduling schemes in related technologies that lack consideration of scenario correlation, the embodiments of this application, based on the correlation of historical scenarios, can automatically connect scenario chains that conform to actual driving logic, allowing the simulation process to restore the scenario flow rules of real roads, avoiding logical disconnect between simulation and actual driving, and thus more accurately testing the autonomous driving system's ability to cope with continuous complex scenarios.

[0029] S2, obtain resource pool status information and task execution status information. The resource pool status information is used to characterize the utilization rate of the resource pool, and the task execution status information is used to characterize the progress of task execution.

[0030] The resource pool status information refers to a set of parameters that reflect the current usage of computing resources on the simulation platform. For example, the resource pool status information includes currently available computing resources, the number of running tasks, and their resource usage.

[0031] Task execution status information can refer to a set of parameters that reflect the current execution progress of a running task. For example, task execution status information includes the time the running task has been running and the estimated remaining time.

[0032] Furthermore, resource pool status information is used to characterize the utilization rate of the resource pool, and task execution status information is used to characterize the progress of task execution.

[0033] By simultaneously acquiring resource pool status information, task execution status information, and historical simulation task data, a multi-dimensional input foundation covering system resource dimensions and task execution dimensions is provided for the subsequent construction of the state space of the reinforcement learning model. This enhances the scheduler's real-time awareness of computing resources to avoid resource idleness or overload, improves the controllability of task execution progress to reduce task waiting time, and supports algorithm relationship mining through the analysis of historical simulation task data to achieve batch scheduling based on algorithm similarity. Ultimately, it achieves multi-dimensional state fusion of resource status, task progress, and algorithm relationship information, overcoming the limitations of traditional scheduling methods that only focus on a single dimension, and improving the comprehensiveness and accuracy of scheduling decisions.

[0034] S3 obtains user feedback information on simulation task scheduling and user behavior information.

[0035] Specifically, user feedback is subjective input, which is information actively transmitted by users based on past simulation test experiences, current test goals, or expected results. For example, evaluations of previous simulation task scheduling results, such as low test efficiency in a certain type of scenario or insufficient test coverage of the algorithm in complex road conditions, or clear test requirement instructions, such as prioritizing the verification of the algorithm's emergency braking performance, focusing on testing urban road congestion scenarios, or suggestions on simulation task priorities and resource allocation.

[0036] User feedback can be collected through proactive interactive entry points. For example, a dedicated feedback interface or module can be set up to allow users to submit feedback through text input, option selection, voice messages, etc. After the simulation task is completed, a pop-up questionnaire can be triggered to ask users for their opinions on task priority, latency, scene aggregation, algorithm aggregation, and the rationality of resource allocation. In addition, users can submit personalized requests through customer service interfaces, instant messaging tools, etc., and this information can be stored in the feedback database in real time through interface integration.

[0037] User behavior information is the collection of objective behavioral data, which consists of various recordable operation trajectories generated by users during their interaction with the simulation system. Examples include the types of simulation tasks created by users in the past, the types of test scenarios frequently selected, the testing preferences for different algorithm versions, and the configuration of task parameters such as test duration, scenario difficulty, and the time patterns of task submission and adjustment.

[0038] User behavior information can be obtained through passive recording and log analysis of user interactions. For example, when a user creates a task, selects a test scenario, adjusts algorithm parameters, sets task priorities, or pauses or terminates a task on a simulation platform, the timestamps, content, and frequency of these operations are automatically recorded. Simultaneously, by analyzing user login times, task submission patterns, and adjustments to historical scheduling results, behavioral characteristics can be extracted. This behavioral data is written to the log system in real time, and after data cleaning and structuring, it forms usable behavioral feature data.

[0039] By receiving user feedback and acquiring user behavior information, and by comprehensively outlining the user's core needs, testing preferences, and operating habits, the scheduler ensures that the generated scheduling strategy accurately matches the user's actual usage requirements, making simulation task scheduling more targeted and practical. The scheduler can directly optimize based on user behavior and further uncover implicit relationships between scenarios, algorithms, and other elements based on specific user behaviors.

[0040] Compared to related technologies that ignore user feedback and behavior, the embodiments of this application are based on a user feedback mechanism that combines subjective and objective factors, which can fully outline user needs, thereby making the scheduling strategy more in line with the actual usage needs of users, achieving personalized demand matching, and improving user experience.

[0041] S4. The correlation data, the relationship data of the simulation algorithm, the resource pool status information, the task execution status information, the feedback information, and the behavior information are input into the simulation task reinforcement learning model to obtain the vehicle simulation task scheduling strategy information.

[0042] Specifically, relational data provides a scenario-level logical foundation for scheduling, ensuring that simulation tasks can be rationally ordered based on dependencies or progression between scenarios. The relational data of the simulation algorithm, as a characteristic representation of the test objects, determines the type of scenario to be matched and the test intensity. Resource pool status information and task execution status information provide real-time constraints on scheduling at the system resource level. User feedback and behavioral information constrain the scheduling direction from the perspective of user needs, ensuring that the strategy aligns with actual usage goals.

[0043] By inputting relational data, simulation algorithm relational data, resource pool status information, task execution status information, feedback information, and behavioral information into the simulation task reinforcement learning model, the model can understand the inherent relationships between these input data through continuous learning and iterative optimization. For example, the basic framework of the test sequence is determined based on the scene relational relationships; test resources for related algorithms are prioritized based on the algorithm version similarity reflected by the algorithm relational data; task priorities are adjusted based on user feedback; and parameter configurations are optimized based on user behavior habits. Simultaneously, scheduling decisions are dynamically adjusted by referring to the resource pool status and task execution progress.

[0044] The simulation task scheduling strategy information output by the reinforcement learning model includes specific task execution order, matching scheme between scenario and algorithm, resource allocation ratio, priority ranking, etc. This scheduling strategy takes into account the objective laws of scenario logic and algorithm characteristics, and responds to changes in system resource status in real time, fully responding to the subjective needs of users, thereby achieving efficient and accurate autonomous driving simulation task scheduling.

[0045] Compared to related technologies that prioritize all simulation tasks and schedule high-priority tasks first, the implementation of this application uses a scheduling strategy output by a reinforcement learning model that includes specific details such as scenario-algorithm matching schemes and resource allocation ratios. It also incorporates dynamic information such as resource pool status and task execution progress. Based on the inherent correlation of input data, it can tilt resources toward core testing needs, reduce resource waste and task inefficiency, and improve the efficiency of autonomous driving simulation task scheduling.

[0046] S5 executes scheduling based on the simulation task scheduling strategy information.

[0047] Specifically, when executing simulation task scheduling, the simulation task process is started automatically or semi-automatically based on these strategy information. First, the corresponding simulation tasks are activated in priority order and matched with preset test scenarios. Second, according to the resource allocation scheme in the strategy, the required computing nodes, simulation engine resources, etc. are allocated to the tasks to avoid resource conflicts and improve utilization efficiency. At the same time, during the task execution process, the implementation of the scheduling strategy is monitored in real time, such as whether the scenario switching is completed as planned, whether the algorithm runs as expected, and whether the resource consumption is within a reasonable range.

[0048] If anomalies occur, such as insufficient resources or scene matching deviations, dynamic corrections can be made based on the preset adjustment rules in the strategy to ensure that the simulation task proceeds efficiently and in an orderly manner, ultimately achieving comprehensive and accurate testing of the algorithm under test and verifying its performance in various scenarios.

[0049] Therefore, this embodiment improves scheduling efficiency, enhances business relevance, and improves user experience by acquiring relationship data between scenarios, parsing simulation algorithm relationship data generated from historical simulation task data, and combining user feedback and behavioral information input models to obtain scheduling strategies for execution.

[0050] To clearly illustrate the above embodiment, in some embodiments of this application, obtaining the relationship data between scenes specifically includes: parsing historical scene files to extract scene tag information and scene metadata; based on the execution records of historical simulation tasks, statistically analyzing the co-occurrence frequency and co-occurrence order of scenes in the same batch of tasks; mapping the scene tag information, co-occurrence frequency, and co-occurrence order to generate explicit scene relationship data; the explicit scene relationship data includes same-tag association, similar-tag association, and subordinate-tag association; and implicit scene relationship data is obtained through adaptive mining during the scheduling and training process of the simulation task reinforcement learning model.

[0051] Scene label information can refer to classification labels used to identify scene characteristics, such as scene type labels (e.g., urban roads, highways, rural roads), vehicle behavior labels (e.g., going straight, changing lanes, making a U-turn, emergency braking), environmental condition labels (e.g., rainy days, snowy days, foggy days, nighttime), and traffic status labels (e.g., congestion, smooth traffic, accidents).

[0052] Scene metadata can refer to data that describes the attributes of a scene file itself, such as the scene file's creation time, modification time, file size, data format, version number, scene library category, author information, and the scene's unique identifier.

[0053] The co-occurrence frequency and co-occurrence order of scenarios in the same batch of tasks can refer to quantitative indicators obtained after statistical analysis of historical simulation task records; among them, the co-occurrence frequency represents the number of times any two or more scenarios are selected simultaneously in the same batch of simulation tasks, and the co-occurrence order represents the order in which multiple scenarios appear in the same batch of tasks.

[0054] Explicit scene association data refers to the quantifiable degree of association between scenes, which can be directly calculated through preset rules or algorithms based on scene label information, scene metadata, and historical co-occurrence statistics. This degree of association can be explicitly expressed and stored, and can be obtained without model training.

[0055] Furthermore, explicit scene association data includes associations with the same label, associations with similar labels, and associations with subordinate labels.

[0056] Implicit scene association data refers to potential relationship patterns between scenes that cannot be directly defined by static label rules or simple co-occurrence statistics, but need to be automatically learned and mined by reinforcement learning models during the scheduling and training process by analyzing dynamic information such as user viewing behavior, feedback data, and task execution results.

[0057] Implicit scene relationship data can be adaptively mined by the simulation task reinforcement learning model during the scheduling and training process.

[0058] Specifically, historical scene files are parsed to extract scene information, such as scene tag information, like tags for city roads, traffic congestion, and rainy days. Scene metadata is also included, which refers to data describing the scene file's own attributes. Examples include the scene file's creation time, modification time, file size, data format, version number, its category in the scene library, author information, and the scene's unique identifier.

[0059] Historical scene files are parsed to extract scene tag information and scene metadata. Based on historical simulation task execution records, the co-occurrence frequency and co-occurrence order of scenes in the same batch of tasks are statistically analyzed. The above information is associated and mapped to generate explicit scene association data. Implicit scene association data is obtained by adaptively mining during the scheduling and training process of the simulation task reinforcement learning model.

[0060] Therefore, this embodiment, through a combination of explicit and implicit scene association acquisition methods, can not only quickly establish a basic scene association framework using computable static rules, but also dynamically mine deep association patterns in a real scheduling environment through reinforcement learning models. This enables subsequent task scheduling strategies to simultaneously consider both the explicit logical associations and implicit usage habit associations of the scenes, thereby improving the rationality of scene coverage, the smoothness of task chaining, and the accuracy of matching user needs in simulated task scheduling.

[0061] In some embodiments of this application, obtaining the relationship data of the simulation algorithm specifically includes: obtaining the historical submission records of the algorithm under test; parsing the historical data of the simulation task to obtain the classification features of a single code submission and the correlation features between multiple code submissions; and generating the relationship data of the simulation algorithm based on the classification features of a single code submission and the correlation features between multiple code submissions.

[0062] The algorithm under test can refer to the version of the algorithm that needs to be tested and verified in the autonomous driving simulation task, including core functional modules of autonomous driving such as perception algorithm, decision planning algorithm, and control algorithm.

[0063] The historical commit records of the algorithm under test refer to the commit logs generated by developers after modifying the algorithm code through a version control system (such as Git). Each commit record includes a unique commit identifier, branch name, committer information, commit timestamp, and the path of the modified code module.

[0064] Furthermore, the historical data of the simulation task is analyzed to obtain the classification features of individual code submissions and the correlation features between multiple code submissions.

[0065] The algorithm under test in a simulation task originates from user code submissions; it is a collection of code versions. Each code submission generates a version of the algorithm that can be released for testing. Analyzing historical data from the simulation task reveals the classification characteristics of individual code submissions, such as classification by code branch, submitter, or business type. It also reveals the correlation characteristics between multiple code submissions; for example, some code submissions for tested modules may be completely identical, while others may be similar. Similar submissions mean they are on the same branch in the control system or the time interval between submissions is relatively short.

[0066] Based on the classification characteristics of individual code submissions and the correlation characteristics between multiple code submissions, relationship data for simulation algorithms is generated. Specifically, when two algorithm versions' code submissions belong to the same development branch, or are submitted at similar times, or are completed by the same submitter, or the modified code modules have dependencies, the two algorithm versions are considered to have a high correlation; otherwise, the correlation is low. The relationship data of simulation algorithms is a quantitative representation of the correlation between different algorithm versions, used to guide subsequent scheduling strategies to centrally schedule tasks of algorithm versions with high correlation.

[0067] Therefore, this embodiment obtains the historical commit records of the algorithm under test and parses them to obtain the classification features of individual code commits and the correlation features between multiple code commits, thereby generating relationship data of the simulation algorithms. This scheme can achieve a quantitative expression of the correlation between different algorithm versions. Based on this relationship data, the scheduling system can centrally schedule tasks of algorithm versions with high correlation (such as tasks belonging to the same development branch, with similar commit times, or completed by the same committer), thereby reducing the overhead of repeated loading and initialization caused by algorithm environment switching and improving the utilization efficiency of computing resources. At the same time, users can compare and view the simulation results of related algorithm versions within a concentrated time period, shortening the waiting time and improving the convenience and efficiency of test comparison. In addition, by mining the algorithm correlation relationships, the scheduling strategy can better fit the actual algorithm development and testing logic, making the execution order of simulation tasks match the natural evolution process of algorithm versions, thereby enhancing the professionalism and business relevance of the scheduling results.

[0068] In some embodiments of this application, the correlation data, the relationship data of the simulation algorithm, the resource pool status information, the task execution status information, the feedback information, and the behavior information are input into the simulation task reinforcement learning model to obtain the vehicle's simulation task scheduling strategy information. Specifically, this includes: splicing and fusing the correlation data, the relationship data of the simulation algorithm, the resource pool status information, the task execution status information, the feedback information, and the behavior information to obtain the initial task scheduling information; and inputting the initial task scheduling information into the simulation task reinforcement learning model to obtain the simulation task scheduling strategy information.

[0069] Among them, the initial information for task scheduling can refer to the comprehensive feature vector formed by splicing and fusing multi-source heterogeneous data in a unified data format. This feature vector covers the dimensions of scene association, algorithm association, system resources, task execution, and user subjective and objective aspects, and serves as the direct data source for the input layer of the reinforcement learning model.

[0070] Furthermore, the initial task scheduling information is input into the simulation task reinforcement learning model to obtain simulation task scheduling strategy information. Specifically, the simulation task reinforcement learning model adopts a deep reinforcement learning network structure. Its input layer receives the initial task scheduling information (i.e., the comprehensive feature vector), performs feature extraction and nonlinear transformation through several hidden layers, and the output layer outputs the probability distribution of each candidate scheduling action or directly outputs the optimal scheduling action. This scheduling action includes, but is not limited to, the reordering result of the queue of tasks to be executed, the selection decision of the next task to be executed, and the allocation scheme of tasks and computing resources. During the training process, the model continuously updates the network parameters based on the feedback of the reward function, gradually converging to the optimal scheduling strategy. During actual scheduling, the model calculates and outputs the corresponding simulation task scheduling strategy information in real time based on the initial task scheduling information input at the current moment. This strategy information is used to guide the simulation platform to execute tasks in a specified order.

[0071] Therefore, this embodiment, by splicing and fusing multi-dimensional data and inputting it into the reinforcement learning model, enables the model to simultaneously perceive scene association, algorithm association, resource status, task progress, and user needs, avoiding the one-sidedness of single-dimensional decision-making. The model uses deep networks to mine the deep interaction relationships between various dimensions, outputting better scheduling decisions, thereby improving resource utilization, shortening task waiting time, and better meeting users' personalized testing needs. In some embodiments of this application, after executing scheduling according to the simulation task scheduling strategy information, the method further includes: collecting environmental feedback data after scheduling execution, including changes in resource pool utilization, task waiting time, user active satisfaction rating, time difference between task completion and user viewing, and scene / algorithm similarity of continuously executed tasks; performing weighted calculation on the environmental feedback data based on a preset multi-objective reward function to obtain a cumulative reward value; and using the cumulative reward value and corresponding state-action interaction data to update the parameters of the simulation task reinforcement learning model online.

[0072] Among them, environmental feedback data refers to observable quantitative indicators generated by changes in the environmental state after the execution of scheduling actions, which are used to evaluate the quality of the current scheduling decision.

[0073] Furthermore, the environmental feedback data includes changes in resource pool utilization, task waiting time, user-initiated satisfaction ratings, the time difference between task completion and user viewing, and the scenario / algorithm similarity of continuously executing tasks. Specifically, changes in resource pool utilization refer to the difference in the proportion of computational resources used before and after scheduling, reflecting an improvement or decrease in resource utilization efficiency, and are collected in real-time by the resource monitoring module. Task waiting time refers to the time elapsed from when a task enters the queue to when it begins execution, and is recorded in the task scheduling log. User-initiated satisfaction ratings refer to scheduling satisfaction scores (e.g., 1-5 points) actively submitted by users through the feedback interface, subdivided into latency satisfaction, scenario aggregation satisfaction, and algorithm aggregation satisfaction, and are collected from the active feedback collection interface. The time difference between task completion and user viewing refers to the time interval between the task completion time and the time when the user first views the result, and is obtained from the correlation matching between task status logs and user operation logs. The scenario / algorithm similarity of continuously executing tasks refers to the similarity between adjacent executing tasks in scenario tag features or algorithm Git metadata, and is obtained from real-time calculation of scenario association data and simulation algorithm relationship data.

[0074] A multi-objective reward function refers to a function expression that assigns different weights to multiple environmental feedback indicators and then sums them up. It is used to quantify multi-dimensional scheduling objectives (such as high efficiency, low latency, and high satisfaction) into a single reward value, which serves as the optimization objective for model updates. It is understood that the multi-objective reward function can be set by relevant technical personnel according to the actual situation, and there are no specific restrictions.

[0075] As a specific embodiment of this example, the multi-objective reward function can be expressed by the following formula: R=α·R1-β·R2-γ·R3+δ·R4+ε·R5; R1 is a resource efficiency reward; for example, a positive reward of +0.1 is given when the CPU utilization in the resource pool exceeds 70%, and a negative reward of -0.05 is given when it is below 30%. R2 is a latency penalty, which can be 0.01 times the current task's waiting time in the queue (in seconds); the longer the waiting time, the greater the penalty. R3 is a user waiting penalty; for example, a fixed negative reward of -0.5 is given when a user is waiting for the result of a specific task in the foreground and the task has not yet been completed. R4 is a user satisfaction reward; a positive reward of +0.2 is given when the user's actively submitted scheduling satisfaction rating (1-5 points) reaches 4 points or higher. R5 is a batch processing reward; for example, a positive reward of +0.1 is given when two consecutively executed tasks have a scene label similarity or algorithm Git branch similarity exceeding 80%.

[0076] α, β, γ, δ, and ε are the weight coefficients corresponding to each item, which can be exemplarily set to 0.3, 0.25, 0.25, 0.1, and 0.1 respectively. The specific values ​​can be adjusted by those skilled in the art according to the actual scheduling objectives. Through the above multi-objective reward function, the model can comprehensively balance resource efficiency, waiting latency, user experience, and batch processing benefits during training, gradually converging to the optimal scheduling strategy.

[0077] The cumulative reward value refers to the sum of the immediate reward calculated according to the multi-objective reward function from the current moment, combined with the future expected reward after discounting and accumulation, and is used to evaluate the long-term benefits of the current scheduling action.

[0078] Furthermore, the parameters of the reinforcement learning model for the simulation task are updated online using the accumulated reward value and the corresponding state-action interaction data. Specifically, after each scheduling execution, the system stores experience samples in the experience pool; when the experience pool accumulates to a certain number, a reinforcement learning algorithm is used to randomly sample a small batch of samples from the experience pool, calculate the loss function, and update the model parameters along the gradient descent direction; the updated model is immediately deployed online for scheduling decisions in the next time step, realizing online continuous learning and iterative optimization.

[0079] In reinforcement learning models, the three core elements of the interaction problem between the agent (i.e., the scheduler model) and the environment (i.e., the entire autonomous driving simulation platform, including task queues, resource pools, and user behavior) include the state space, action space, and reward function.

[0080] In the embodiments of this application, the state space of the reinforcement learning model is a combination of queue content, task features, and system state. The queue content is the current queue of tasks to be scheduled. Task features include the order of the tasks themselves, as well as relational data and the correlation features of the algorithm under test. The system state includes the resource pool state, the task execution result state (e.g., execution time), and user feedback and behavioral information (e.g., viewing task order, viewing specific task times).

[0081] The action space of a reinforcement learning model is the scheduling decision that can be made at each time step, namely, generating a new queue containing each task to be scheduled, or an empty element (i.e., waiting). Specifically, the queue can be fine-tuned, completely rescheduled, or even, when there is higher privilege, the currently executing task can be superseded.

[0082] In resource scheduling optimization for autonomous driving simulation, the reward function of reinforcement learning models mainly includes common reward functions such as resource pool utilization and task waiting latency, as well as user waiting time, such as the waiting time when a user views the results on the simulation platform and the results do not appear in time.

[0083] Based on relational data, the algorithm under test, feedback information, and behavioral information, an initial ranking of simulation tasks with priority is generated. This initial ranking is used as the initial input to the simulation task reinforcement learning model, enabling the model to be trained based on real-time scheduling data while executing the scheduling, thus avoiding the cold start problem.

[0084] The task scheduler is a wrapper around the reinforcement learning model. The training and scheduling processes of the simulation task reinforcement learning model are carried out synchronously in the scheduler, thereby obtaining the vehicle simulation task scheduling strategy information.

[0085] Therefore, this embodiment integrates multi-dimensional information such as scene association, algorithm association, resource status, and user feedback into the state space, enabling the model to fully perceive scheduling decision elements; it achieves fine-grained scheduling adjustments through a flexible action space; it clarifies the optimization direction through a multi-objective reward function; and it avoids the cold start problem through an integrated training and scheduling mechanism. Ultimately, it achieves adaptive evolution of scheduling strategies, improves resource utilization, shortens waiting time, and enhances the relevance of scheduling results to business needs.

[0086] In some embodiments of this application, the simulation task reinforcement learning model is generated as follows: During the model initialization phase, the policy network parameters are set to zero, causing the scheduling policy to degenerate into a first-in-first-out (FIFO) queue policy, thus completing a cold start; during the actual operation of the platform, the interaction data of the four-tuples of state, action, reward, and next state are continuously collected; the interaction data are optimized using a preset reinforcement learning algorithm to optimize the policy gradient or value function, iteratively updating the model parameters so that the scheduling policy gradually approaches the global optimum. Specifically, during the model initialization phase, since all policy network parameters are set to zero, the model does not yet have any intelligent scheduling capabilities and automatically reverts to the simple and reliable default scheduling rule of FIFO, that is, executing tasks in the order they arrive. This design ensures that the system can run stably as soon as it goes online, avoiding the uncertainty and inefficiency caused by the randomness of the initial policy in traditional reinforcement learning, thus completing a cold start.

[0087] During actual platform operation, the system continuously collects the state, action, reward, and next state generated by each scheduling decision, forming a four-tuple of interactive data. The state includes the current task queue content, task characteristics, resource pool status, task execution status, and user feedback and behavior information; the action is the task queue sorting result output by the model; the reward is the immediate feedback value calculated based on a multi-objective reward function; and the next state is the new state the environment updates to after the action is executed.

[0088] The interaction data is optimized using a pre-defined reinforcement learning algorithm to perform policy gradient or value function optimization, and the model parameters are iteratively updated. For example, the pre-defined reinforcement learning algorithm can be a proximal policy optimization algorithm, and there are no specific limitations. Mini-batch four-tuple samples are randomly sampled from the experience pool, the loss function is calculated, and the policy network parameters are updated along the gradient descent direction. After each round of parameter updates, the new model is immediately deployed online for subsequent scheduling decisions.

[0089] Thus, through multiple iterations, the model parameters are gradually adjusted from zero towards the optimal level. The scheduling strategy also evolves from a simple first-in-first-out approach to an intelligent scheduling strategy that comprehensively considers scene correlation, algorithm correlation, resource status, and user needs, ultimately approaching the global optimum. This generation method achieves seamless integration of model training and actual scheduling, enabling the model to continuously evolve in the real environment.

[0090] In some embodiments of this application, the simulation task scheduling strategy information includes a sorted queue of tasks to be executed. Executing scheduling according to the simulation task scheduling strategy information includes: parsing the sorted queue to obtain a list of tasks to be executed; and calling corresponding resources from the resource pool to perform simulation testing based on the list of tasks to be executed.

[0091] Specifically, the sorted queue is first parsed, and the task identifiers to be executed are extracted in the order from front to back of the queue, generating an ordered list of tasks to be executed. Then, the scheduler allocates the necessary computing resources, including CPU cores, GPU cores, memory, and disk I / O bandwidth, to each task from the resource pool according to the task list, and starts the execution of the simulation task. When a task is completed, the scheduler automatically removes it from the task list and continues to schedule the next task, until all tasks in the task list are completed.

[0092] Therefore, this embodiment accurately converts the sorted queue output by the model into executable resource call instructions, achieving a seamless mapping from scheduling strategy to physical execution. This ensures that simulation tasks are executed in an ordered manner according to the optimized sequence, improving the accuracy of scheduling instruction execution and the standardization of resource allocation. As a specific embodiment of this application, such as... Figure 2 As shown, the autonomous driving simulation task scheduling method may include the following steps: S201, acquire the vehicle's historical scene files, simulation task historical data, and the historical submission records of the algorithm under test.

[0093] S202: Obtain the correlation data between scenarios based on the historical scenario files, and obtain the relationship data of the simulation algorithm based on the historical data of the simulation task and the historical submission records of the algorithm under test.

[0094] S203: Obtain resource pool status information, task execution status information, user feedback on simulation task scheduling, and user behavior information.

[0095] S204, the association data, simulation algorithm relationship data, resource pool status information, task execution status information, user feedback information and user behavior information are spliced ​​and merged to generate initial task scheduling information.

[0096] S205 uses the initial information of task scheduling as the input to the simulation task reinforcement learning model, so that the model can be trained online based on the real-time generated scheduling data while executing the scheduling, and obtain the simulation task scheduling strategy information.

[0097] S206, parse the sorted queue in the scheduling policy information to obtain the list of tasks to be executed, and call the corresponding resources from the resource pool according to the list of tasks to be executed for simulation testing.

[0098] S207 collects environmental feedback data after scheduling execution, calculates the cumulative reward value based on the multi-objective reward function, and updates the reinforcement learning model parameters online.

[0099] Therefore, the autonomous driving simulation task scheduling method of this application can achieve adaptive and continuous optimization of scheduling strategies, improve resource utilization efficiency, shorten task waiting time, and enhance the correlation between scheduling results and business needs and user preferences, thereby comprehensively improving the intelligence level and execution efficiency of autonomous driving simulation task scheduling.

[0100] Figure 3 This is a block diagram of an electronic device provided in some embodiments of this application.

[0101] like Figure 3 As shown, this application embodiment also provides an electronic device 300 including: a memory 310, a processor 320, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned autonomous driving simulation task scheduling method.

[0102] The electronic device according to the embodiments of this application can implement the above-mentioned autonomous driving simulation task scheduling method when the processor executes a computer program. Based on the above-mentioned autonomous driving simulation task scheduling method, scheduling efficiency is improved, business relevance is enhanced, and user experience is improved.

[0103] This application also provides a vehicle, such as... Figure 4 As shown, vehicle 400 includes the aforementioned electronic equipment 300.

[0104] The vehicle according to the embodiments of this application includes electronic devices. When the processor executes a computer program, the above-mentioned autonomous driving simulation task scheduling method can be implemented. Based on the above-mentioned autonomous driving simulation task scheduling method, scheduling efficiency is improved, business relevance is enhanced, and user experience is improved.

[0105] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described autonomous driving simulation task scheduling method.

[0106] The computer-readable storage medium according to the embodiments of this application implements the above-described autonomous driving simulation task scheduling method when the computer program stored thereon is executed by a processor. Based on the above-described autonomous driving simulation task scheduling method, scheduling efficiency is improved, business relevance is enhanced, and user experience is improved.

[0107] In this application, "multiple" refers to two or more.

[0108] In this application, unless otherwise expressly defined, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0109] The terms “first,” “second,” “third,” “fourth,” etc., in this application (if present) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0110] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0111] Unless otherwise specified, all steps in this application may be performed sequentially or randomly. For example, if the method includes steps A and B, it means that the method may include steps A and B performed sequentially, or it may include steps B and A performed sequentially. For example, if the method may also include step C, it means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or it may include steps A, C, and B, or it may include steps C, A, and B, etc.

[0112] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for scheduling autonomous driving simulation tasks, characterized in that, include: Acquire the correlation data between scenarios and the correlation data of simulation algorithms, wherein the correlation data between scenarios is used to characterize the real user needs of autonomous driving simulation tasks, and the correlation data of simulation algorithms is used to characterize the correlation between algorithm versions of autonomous driving simulation tasks; Obtain resource pool status information and task execution status information, wherein the resource pool status information is used to characterize the utilization rate of the resource pool, and the task execution status information is used to characterize the progress of task execution; Obtain user feedback on simulation task scheduling and the user's behavioral information; The correlation data, the relationship data of the simulation algorithm, the resource pool status information, the task execution status information, the feedback information, and the behavior information are input into the simulation task reinforcement learning model to obtain the vehicle simulation task scheduling strategy information. The scheduling is performed according to the simulation task scheduling strategy information.

2. The autonomous driving simulation task scheduling method according to claim 1, characterized in that, The acquisition of the relationship data between scenarios specifically includes: Parse historical scene files to extract scene tag information and scene metadata; Based on the execution records of historical simulation tasks, the co-occurrence frequency and co-occurrence order of scenarios in the same batch of tasks are statistically analyzed; The scene label information, co-occurrence frequency, and co-occurrence order are associated and mapped to generate explicit scene association data; the explicit scene association data includes associations with the same label, associations with similar labels, and associations with subordinate labels; And implicit scene correlation data are obtained by adaptively mining during the scheduling and training process through the reinforcement learning model of the simulation task.

3. The autonomous driving simulation task scheduling method according to claim 2, characterized in that, The acquisition of relational data for the simulation algorithm specifically includes: Obtain the historical submission records of the algorithm under test; The historical data of the simulation task is parsed to obtain the classification features of a single code submission and the correlation features between multiple code submissions; Based on the classification features of the individual code submissions and the correlation features among the multiple code submissions, the relationship data of the simulation algorithm is generated.

4. The autonomous driving simulation task scheduling method according to claim 1, characterized in that, The step of inputting the association data, the relationship data of the simulation algorithm, the resource pool status information, the task execution status information, the feedback information, and the behavior information into the simulation task reinforcement learning model to obtain the vehicle's simulation task scheduling strategy information specifically includes: The correlation data, the relationship data of the simulation algorithm, the resource pool status information, the task execution status information, the feedback information, and the behavior information are spliced ​​and fused to obtain the initial information for task scheduling. The initial task scheduling information is input into the simulation task reinforcement learning model to obtain the simulation task scheduling strategy information.

5. The autonomous driving simulation task scheduling method according to claim 1 or 4, characterized in that, After executing the scheduling according to the simulation task scheduling policy information, the process further includes: Collect environmental feedback data after scheduling execution. The environmental feedback data includes changes in resource pool utilization, task waiting time, user-initiated satisfaction rating, time difference between task completion and user viewing, and scenario / algorithm similarity of continuous task execution. The environmental feedback data is weighted and calculated based on a preset multi-objective reward function to obtain a cumulative reward value. The parameters of the reinforcement learning model for the simulation task are updated online using the accumulated reward value and the corresponding state-action interaction data.

6. The autonomous driving simulation task scheduling method according to claim 1, characterized in that, The simulation task reinforcement learning model is generated in the following way: During the model initialization phase, the policy network parameters are set to zero, causing the scheduling policy to degenerate into a first-in-first-out queue policy, thus completing the cold start. During the actual operation of the platform, the interaction data of the four-tuple of status, action, reward and next status is continuously collected; The interaction data is optimized using a preset reinforcement learning algorithm to perform policy gradient or value function optimization, and the model parameters are iteratively updated so that the scheduling strategy gradually approaches the global optimum.

7. The autonomous driving simulation task scheduling method according to claim 1, characterized in that, The simulation task scheduling strategy information includes a sorted queue of tasks to be executed, and the step of executing scheduling according to the simulation task scheduling strategy information includes: The sorted queue is parsed to obtain a list of tasks to be executed; Based on the task list, the corresponding resources are retrieved from the resource pool for simulation testing.

8. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the autonomous driving simulation task scheduling method as described in any one of claims 1-7.

9. A vehicle, characterized in that, It includes the electronic device as described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the autonomous driving simulation task scheduling method as described in any one of claims 1-7.

Images