Vehicle-to-Everything (V2X) system scheduling methods and V2X resource scheduling methods

Abstract

This invention relates to the field of autonomous driving, and discloses a vehicle-to-everything (V2X) system scheduling method and a V2X resource scheduling method. The method acquires autonomous driving service requests sent by simulated vehicles and sends an inquiry action request to a resource scheduler based on the autonomous driving service requests. It receives action decisions sent by the resource scheduler based on the inquiry action requests and allocates tasks based on the action decisions. According to the task allocation results, it calls a preset interface to create a sub-task container, and processes the autonomous driving sub-tasks in the autonomous driving service requests through the sub-task container to obtain processing result data. When the autonomous driving service request processing is completed, the computing resources occupied by the autonomous driving service request are released. Because resource planning to generate action decisions is achieved through the resource scheduler, and the system scheduler allocates autonomous driving sub-tasks based on these action decisions, the real-time performance and responsiveness of autonomous driving are improved, enhancing vehicle intelligence.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a vehicle network system scheduling method and a vehicle network resource scheduling method. Background Technology

[0002] With the rapid development of Internet of Vehicles (IoV) and autonomous driving technologies, numerous autonomous driving services have emerged. Therefore, vehicles are no longer just transportation tools, but also computing devices that support these services. However, as virtualization technology matures, the demand for computing resources is becoming increasingly high.

[0003] Existing vehicle-to-everything (V2X) system resource calculation methods mostly consider the allocation of communication resources or homogeneous computing resources. This allocation method is difficult to adapt to the allocation of computing resources under heterogeneous computing resources.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a vehicle network system scheduling method and a vehicle network resource scheduling method, aiming to solve the technical problem that existing vehicle network system resource calculation methods are difficult to adapt to the allocation of computing resources under easily accessible computing resources.

[0006] To achieve the above objectives, the present invention provides a vehicle-to-everything (V2X) system scheduling method. The method is applied to a system scheduler in a resource scheduling system, which further includes a resource scheduler and a simulated vehicle. The method includes:

[0007] The system acquires the autonomous driving service request sent by the simulated vehicle and sends an inquiry action request to the resource scheduler based on the autonomous driving service request; the autonomous driving service request includes multiple autonomous driving sub-tasks.

[0008] The system receives action decisions sent by the resource scheduler based on the query action request and performs task allocation based on the action decisions; the task allocation includes allocating computing resources and edge server nodes to carry the tasks for each autonomous driving subtask in the autonomous driving service request.

[0009] Based on the task allocation result, a preset interface is called to create a subtask container. The subtask container is used to process the autonomous driving subtask in the autonomous driving service request and obtain the processing result data.

[0010] When the autonomous driving service request is completed, the computing resources occupied by the autonomous driving service request are released, and the processing result data is sent to the simulated vehicle.

[0011] Optionally, the method further includes:

[0012] Upon receiving the action decision, a time window is generated;

[0013] When the time window ends, obtain the current state of the resource scheduling system;

[0014] The resource scheduling reward and the next state of the resource scheduling system are determined based on the current state and the action decision.

[0015] The system scheduler's state group information is determined based on the current state, the action decision, the resource scheduling reward, and the next state.

[0016] The status group information is sent to the resource scheduler to update the decision policy; the decision policy is used to match action decisions for query action requests.

[0017] Optionally, the step of calling a preset interface to create a subtask container based on the task allocation result, processing the autonomous driving subtasks in the autonomous driving service request through the subtask container, and obtaining processing result data includes:

[0018] Based on the task allocation result, a preset interface is invoked to create a subtask container corresponding to the autonomous driving subtask in the edge server node;

[0019] The autonomous driving domain data in the autonomous driving subtask is input into the subtask container in units of frames for processing to obtain processing result data; the processing result data includes the I / O latency, computation latency, average computation latency and overall task completion latency of the autonomous driving subtask.

[0020] Optionally, the step of releasing the computing resources occupied by the autonomous driving service request when the autonomous driving service request processing is completed includes:

[0021] When the autonomous driving service request processing is completed, obtain the subtask identifier of each autonomous driving subtask in the autonomous driving service request;

[0022] The edge server node corresponding to each of the autonomous driving sub-tasks is determined based on the sub-task identifier;

[0023] Call the resource interface to release the computing resources occupied by the autonomous driving subtasks in each edge server node.

[0024] Optionally, after the step of releasing the computing resources occupied by the autonomous driving service request when the autonomous driving service request processing is completed, the method further includes:

[0025] When the computing resources are released, a resource update completion request is sent to the resource scheduler to update the system state recorded by the resource scheduler.

[0026] This invention acquires autonomous driving service requests sent by a simulated vehicle and sends inquiry action requests to a resource scheduler based on these requests. The autonomous driving service request includes multiple autonomous driving sub-tasks. The system receives action decisions sent by the resource scheduler based on the inquiry action requests and allocates tasks based on these decisions. Task allocation includes allocating computing resources and edge server nodes to each autonomous driving sub-task in the autonomous driving service request. Based on the task allocation results, a preset interface is called to create a sub-task container. The sub-task container processes the autonomous driving sub-tasks in the autonomous driving service request, obtaining processing result data. Upon completion of the autonomous driving service request processing, the computing resources occupied by the request are released, and the processing result data is sent back to the simulated vehicle. Because resource planning is implemented through the resource scheduler to generate action decisions, and the system scheduler allocates autonomous driving sub-tasks based on these decisions, and establishes corresponding sub-task containers on each edge server node based on the task allocation results, and processes the autonomous driving sub-tasks through these containers, the real-time performance and responsiveness of autonomous driving are improved, enhancing the intelligence and autonomy of the vehicle.

[0027] Furthermore, to achieve the above objectives, the present invention also proposes a vehicle-to-everything (V2X) resource scheduling method, which is applied to a resource scheduler in a resource scheduling system. The resource scheduling system further includes a system scheduler, and the method comprises:

[0028] Upon receiving an inquiry action request sent by the system scheduler, the system determines the action decision corresponding to the inquiry action request based on the decision strategy.

[0029] The action decision is sent to the system scheduler;

[0030] Receive state group information generated by the system scheduler based on the action decision, and store the state group information as training experience;

[0031] When the stored training experience meets the training condition threshold, the agent is trained based on the training experience to update the decision-making strategy.

[0032] Optionally, the step of determining the action decision corresponding to the inquiry action request based on the decision strategy when receiving the inquiry action request sent by the system scheduler includes:

[0033] Upon receiving an inquiry action request from the system scheduler, the current state of the resource scheduling system is determined.

[0034] Obtain a decision strategy, and determine the local action decision corresponding to the query action request based on the current state and the decision strategy;

[0035] Explore the action space and obtain exploration action decisions;

[0036] The local action decision and the exploration action decision are compared to obtain the target action decision, which is the action decision corresponding to the inquiry action request.

[0037] Optionally, the step of exploring the action space to obtain exploration action decisions includes:

[0038] Generate random numbers from a preset uniform distribution;

[0039] When the random number is less than a preset threshold, a random action is selected from the selectable actions in the action space;

[0040] When the random number is not less than the preset threshold, a Gaussian action is selected based on the learned knowledge, and the Gaussian action satisfies a Gaussian distribution.

[0041] The exploration action decision is obtained based on the Gaussian action or the random action.

[0042] Optionally, the step of training the agent based on the training experience to update the decision-making policy includes:

[0043] The priority of the training experience is calculated using temporal difference error;

[0044] The stored training experience is sampled according to the priority to obtain the sampled experience.

[0045] The agent is trained based on the sampling experience to update the decision-making strategy.

[0046] Optionally, the method further includes:

[0047] Upon receiving a resource update completion request from the system scheduler, the recorded system status is updated.

[0048] When this invention receives an inquiry action request from the system scheduler, it determines the corresponding action decision based on a decision policy; sends the action decision to the system scheduler; receives state group information generated by the system scheduler based on the action decision, and stores the state group information as training experience; when the stored training experience meets the training condition threshold, it trains the agent based on the training experience to update the decision policy. Because the agent is trained based on the state group information sent by the system scheduler, thereby updating the decision policy, the resource scheduler can be gradually optimized in a continuous iteration process, improving the robustness of the resource scheduling system.

[0049] Furthermore, to achieve the above objectives, the present invention also proposes a resource scheduling system, which includes a resource caller, a system caller, and a simulated vehicle;

[0050] The resource caller, system caller, and simulated vehicle are used to implement the functionality of any of the methods described above. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the execution of an autonomous driving subtask in one embodiment of the present invention.

[0052] Figure 2 This is a flowchart illustrating the first embodiment of the vehicle-to-everything (V2X) system scheduling method of the present invention.

[0053] Figure 3 This is a flowchart illustrating the second embodiment of the vehicle network system scheduling method of the present invention;

[0054] Figure 4 This is a flowchart illustrating the overall process of one embodiment of the vehicle network system scheduling method of the present invention.

[0055] Figure 5 This is a flowchart illustrating the first embodiment of the vehicle network resource scheduling method of the present invention;

[0056] Figure 6 This is a schematic diagram of the overall process of one embodiment of the vehicle network resource scheduling method of the present invention;

[0057] Figure 7 This is a flowchart illustrating the first embodiment of the resource scheduling system of the present invention.

[0058] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0059] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0060] It should be noted that with the rapid development of Internet of Vehicles (IoV) and autonomous driving technologies, existing onboard computing capabilities are no longer sufficient to meet the ever-increasing computing demands. In particular, some perception-related subtasks are computationally very time-consuming, and the limitations of onboard computing devices in terms of computing power, energy, and heat make it impossible to process these tasks entirely within the vehicle. Autonomous vehicles require a large amount of computation to successfully process the real-time tasks generated by each vehicle. However, due to the high mobility of vehicles, it is difficult to meet the latency-sensitive requirements of these computational tasks. Therefore, in one aspect of this invention, the powerful computing capabilities provided by a mobile edge computing (MEC) cloud platform are utilized to address these challenges.

[0061] In existing vehicle-to-everything (V2X) systems, solutions using MEC (Multi-access Edge Computing) to provide computing power neglect the impact of traffic conditions on the system model. During peak traffic periods, more computing requests need to be offloaded, while edge computing devices face relatively high pressure. Therefore, a reasonable resource allocation strategy is required to ensure the completion of computing tasks. Furthermore, most existing computing resource management methods in MEC environments treat autonomous driving as a black box model, ignoring the different computing requirements of each subtask and the different requirements for device types and resources.

[0062] Furthermore, these methods typically assume a linear relationship between resources allocated to autonomous driving tasks and latency, using the formula: processing latency = task's CPU cycles / server's CPU frequency, to establish the relationship between processing latency, tasks, and computing resources. Analysis reveals that this relationship is actually highly non-linear and depends on the computing device itself. In this context, it is crucial to identify the key factors to consider when allocating resources, quickly determine the required amount of virtual heterogeneous resources to minimize overall computational latency, and establish a suitable service model for the overall system.

[0063] However, due to the variability of dynamic workloads, the complex interactions between autonomous driving subtasks can trigger cascading effects when providing resources for interrelated subtasks. If we want to model the system directly, we need either a well-designed autonomous driving system model that requires complex prior knowledge of the system, or a data-driven model that requires a large amount of training data from real autonomous driving systems (i.e., resource allocation feedback) that may consume a lot of resources.

[0064] The existing solutions mentioned above are rule-based and heuristic methods. These methods rely on a complete understanding of the autonomous driving system and a carefully designed model, but they cannot handle dynamic system states. Furthermore, heuristic methods are based on fixed rules and policies and may not be able to proactively explore new solutions.

[0065] In one implementation, computational resource scheduling can be based on reinforcement learning. Choosing reinforcement learning for computational resource scheduling allows for adaptive adjustments based on environmental changes and feedback. Then, resource scheduling decisions are optimized based on real-time system status and feedback, thereby adapting to dynamic system states and changing workloads. Furthermore, the resource scheduling strategy can be continuously learned and improved through interaction with the environment.

[0066] Specifically, this invention proposes a resource scheduling system based on a container orchestration platform and a reinforcement learning framework, which can be applied to the Internet of Vehicles (IoV). Building upon this system, this invention further proposes an IoV system scheduling method and an IoV resource scheduling method. These methods allow for the separate control of the system scheduler and resource scheduler within the resource scheduling system, thereby ensuring the validation of different types of heterogeneous computing resource scheduling algorithms.

[0067] It should be noted that a container orchestration platform is a tool or system that can be used to automate the deployment, orchestration, and management of containerized applications. The platform used in this invention can be Kubernetes, Docker, Google Cloud, etc., and this embodiment of the invention is not limited thereto.

[0068] In one embodiment, in order to improve the efficiency of the resource scheduling system, this embodiment of the invention uses the Kubernetes platform as the container orchestration platform to design the resource scheduling system.

[0069] It should be noted that, in the first aspect of the present invention, namely the vehicle network system scheduling method of the present invention, a method for verifying resource scheduling algorithms is designed, which can verify and optimize resource scheduling algorithms; in the second aspect of the present invention, namely the vehicle network resource scheduling method of the present invention, a resource scheduler suitable for distributed systems and a resource scheduling algorithm that can be used for the resource scheduler are designed, which can realize resource planning of resource scheduling system.

[0070] It should be explained that a resource scheduling system can include a resource scheduler, a system scheduler, and simulated vehicles.

[0071] It should be noted that the simulated vehicle can be used to simulate the autonomous driving request of a vehicle, and a Poisson process is used to simulate the time interval between the autonomous driving request generated by the vehicle and the arrival of the resource scheduler. The autonomous driving request may include at least one autonomous driving subtask, which may be the same or different; this invention does not impose any limitations on this.

[0072] In one implementation, the execution order of autonomous driving subtasks can be as follows: Figure 1As shown, Figure 1 The execution flowchart for autonomous driving video frame processing tasks.

[0073] It should be noted that, based on autonomous driving video frames, vehicle localization can be performed to obtain the vehicle's location results; target detection and target tracking can also be performed to obtain target information; based on the vehicle's localization results and target information, information fusion can be performed, and task processing results and subsequent planned actions can be output.

[0074] Understandably, simulating the request patterns and traffic load of autonomous driving requests can help understand the system load of the resource scheduling system, facilitating system design and performance evaluation.

[0075] It should be noted that in practical applications, a computing resource limit can be set for edge server nodes. Based on the computing resource limit, the arrival rate and time window size in the Poisson process can be adjusted to assess the workload of the resource scheduler.

[0076] It should be explained that a resource scheduler is a module that can be used to plan the computing resources in a resource scheduling system. Based on the vehicle-to-everything (V2X) resource scheduling method, the resource scheduler can determine the computing resources and edge server nodes that need to be allocated to each autonomous driving subtask in an autonomous driving service request.

[0077] It should be noted that the system scheduler is a module that can be used to perform system scheduling of resource scheduling systems. Based on the vehicle-to-everything (V2X) system scheduling method, the system scheduler can allocate autonomous driving subtasks in autonomous driving service requests to edge server nodes and allocate computing resources to these autonomous driving subtasks.

[0078] It should be understood that the resource scheduling system can realize the planning of computing resources for the Internet of Vehicles through the resource scheduler; and the resource scheduling system can realize the system scheduling of the Internet of Vehicles through the system scheduler, thereby realizing the allocation and execution of autonomous driving sub-tasks.

[0079] It is understandable that by simulating different autonomous driving service requests through a vehicle, the resource scheduling system of this invention can be continuously iterated, thereby obtaining a resource scheduling system that can provide users with fast and efficient computing services.

[0080] Specifically, refer to Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the vehicle network system scheduling method of the present invention.

[0081] In this embodiment, the vehicle-to-everything (V2X) system scheduling method includes the following steps:

[0082] Step S10: Obtain the autonomous driving service request sent by the simulated vehicle, and send an inquiry action request to the resource scheduler based on the autonomous driving service request; the autonomous driving service request includes multiple autonomous driving sub-tasks.

[0083] It should be noted that the execution entity of the method in this embodiment is the aforementioned system scheduler. This system scheduler is a virtual module in a resource scheduling system, which can be used to implement system scheduling of the resource scheduling system.

[0084] It should be understood that the resource scheduling system can be applied to servers, computers or other terminal devices with data processing and program execution functions, and can also be applied to electronic devices with the same or similar functions. This embodiment of the invention does not limit this.

[0085] It should be noted that when the system scheduler receives an autonomous driving service request sent by the simulated vehicle, it can generate an inquiry action request corresponding to the autonomous driving service request and send it to the resource scheduler, so that the resource scheduler can perform resource allocation planning for each autonomous driving subtask in the autonomous driving service request based on the inquiry action request.

[0086] It should be understood that autonomous driving subtasks refer to the process of breaking down the overall task into multiple independent and relatively independent subtasks during the realization of autonomous driving, and solving the specific problems of each subtask. Specifically, autonomous driving subtasks may include perception tasks (such as object detection, target tracking, road condition recognition, etc.), localization tasks (acquiring location information, attitude information, etc.), scene understanding tasks, human-computer interaction tasks, control tasks, etc., and the embodiments of the present invention do not impose any limitations on them.

[0087] It should be understood that an inquiry action request is a request that enables the resource scheduling system to determine driving task request information. The inquiry action request may include information such as the request type of the autonomous driving service request, the number of subtasks, the subtask type, the subtask identifier, and the subtask content. Based on this inquiry action request, the resource scheduling system can allocate edge server nodes and the computing resources occupied by each autonomous driving subtask on the edge server nodes.

[0088] In one implementation, the resource scheduling system can be based on reinforcement learning for computational resource scheduling. Choosing reinforcement learning for computational resource scheduling allows for adaptive adjustments based on environmental changes and feedback. Then, resource scheduling decisions are optimized based on real-time system status and feedback, thus adapting to dynamic system states and changing workloads. Furthermore, it can continuously learn and improve resource scheduling strategies through interaction with the environment. However, most reinforcement learning-based vehicular network (V2N) computational resource scheduling schemes are still in the experimental simulation stage, neglecting the potential problems caused by heterogeneous computing resource demands in real-world operating environments. Moreover, experimental simulations rarely consider the potential problems of V2N service systems built on physical platforms, and different computing resources and their demands vary significantly in V2N environments. This invention, by using a system scheduler and a resource scheduler respectively, considers the resource allocation problem of communication systems under heterogeneous computing resources, making it more suitable for computational resource allocation scenarios in real-world environments.

[0089] In its implementation, the system scheduler receives autonomous driving requests arriving in each time window. Upon receiving a request, it sends an inquiry request to the resource scheduler based on that request. This collaborative work between the system scheduler and the resource scheduler improves the computational efficiency of the resource scheduling system.

[0090] Step S20: Receive the action decision sent by the resource scheduler based on the query action request, and perform task allocation based on the action decision; the task allocation includes allocating computing resources and edge server nodes to carry the task for each autonomous driving subtask in the autonomous driving service request.

[0091] It should be understood that action decision-making refers to the action decision selected by the resource scheduler based on the currently perceived environmental information and task objectives. Action decision-making is a core part of the autonomous driving system. By continuously acquiring autonomous driving sub-tasks and their corresponding information, it can provide specific driving decisions to control the vehicle's behavior. The system scheduler can allocate autonomous driving sub-tasks based on action decisions, that is, determine the edge server nodes and required computing resources to execute each autonomous driving sub-task, and upload each autonomous driving sub-task to the corresponding edge server nodes for execution.

[0092] In its implementation, the system scheduler receives action decisions from the resource scheduler based on query action requests, and determines the amount of computing resources allocated to each autonomous driving subtask and which edge server node each subtask needs to upload to based on these action decisions. Since resource planning to generate action decisions is achieved through the resource scheduler, and system scheduling is performed by the system scheduler based on these action decisions, resource scheduling and system scheduling within the resource scheduling system can be conducted separately, improving the efficiency of the resource scheduling system.

[0093] Step S30: Based on the result of the task allocation, call the preset interface to create a subtask container, and process the autonomous driving subtask in the autonomous driving service request through the subtask container to obtain the processing result data.

[0094] It should be noted that the aforementioned preset interfaces are also the application interfaces provided by the container orchestration platform. Through the various application interfaces provided by the container orchestration platform, control over edge server nodes can be achieved, such as container deployment and management, resource management, configuration management, etc. This embodiment of the invention does not impose any limitations on this.

[0095] It is understandable that the above task allocation results are the computing resources and edge server nodes allocated to the autonomous driving subtasks. Based on the computing resources allocated to the autonomous driving subtasks, the resource consumption of the subtask containers to be created can be determined; based on the edge server nodes allocated to the autonomous driving subtasks, the creation address of the subtask containers to be created can be determined.

[0096] It should be understood that the system scheduler creates subtask containers corresponding to each autonomous driving subtask in the edge computing nodes according to the task allocation results, and uploads the corresponding autonomous driving subtasks when the subtask containers are created, thereby completing the processing of each autonomous driving subtask.

[0097] It is understood that the above processing result data can be the processing results corresponding to each autonomous driving sub-task and the corresponding delay information. The delay information can include: I / O latency, computation latency, average computation latency, overall task completion latency, etc. for each autonomous driving sub-task. Different autonomous driving sub-tasks may have different processing results, and this embodiment of the invention does not limit this.

[0098] In its implementation, the system scheduler can call the application interface provided by the container orchestration platform based on the task allocation results, and create subtask containers in the edge server nodes. These subtask containers then process the autonomous driving subtasks and obtain the processing results. Because subtask containers corresponding to the autonomous driving subtasks are created in each edge server node based on the task allocation results, and the autonomous driving subtasks are processed through these containers, the real-time performance and responsiveness of autonomous driving are improved, enhancing the vehicle's intelligence and autonomy.

[0099] Step S40: When the autonomous driving service request is completed, the computing resources occupied by the autonomous driving service request are released, and the processing result data is sent to the simulated vehicle.

[0100] It should be understood that, in order to save computing costs and improve computing efficiency, the system scheduler can release the computing resources occupied by each autonomous driving subtask when the autonomous driving service request processing is completed, that is, when each autonomous driving subtask is completed. Specifically, the step of releasing the computing resources occupied by the autonomous driving service request when the autonomous driving service request processing is completed includes:

[0101] Step S41: When the autonomous driving service request processing is completed, obtain the subtask identifier of each autonomous driving subtask in the autonomous driving service request;

[0102] Step S42: Determine the edge server node corresponding to each of the autonomous driving sub-tasks based on the sub-task identifier;

[0103] Step S43: Call the resource interface to release the computing resources occupied by the autonomous driving subtasks in each edge server node.

[0104] Understandably, in resource scheduling scenarios, in order to identify tasks, task identifiers are usually set in each task. These task identifiers can be used to identify tasks. Through the subtask identifiers of autonomous driving subtasks, the edge server node carrying the autonomous driving subtask and the subtask container in the edge server node used to execute the autonomous driving subtask can be identified.

[0105] It should be understood that when the edge server node and subtask container corresponding to the autonomous driving subtask are determined, the application service interface provided by the container orchestration platform can be called to release computing resources, thereby improving the processing efficiency of the resource scheduling system.

[0106] Furthermore, when the computing resources are released, the system scheduler can also send a resource update completion request to the resource scheduler, so that the resource scheduler updates the recorded system state.

[0107] Understandably, when an autonomous driving service request is completed, the processing result data is sent to the simulated vehicle to mark the completion of an autonomous driving request.

[0108] It should be noted that, in this embodiment of the invention, a complete Poisson process is defined as one complete iteration, that is, the process from the generation of an autonomous driving request to its completion. At the end of each iteration, the system waits for all requests to complete, indicating that the environment has been reset, and then proceeds to the next iteration. By setting multiple iterations and a loop, the performance of the resource scheduling algorithm is optimized.

[0109] This invention, in its embodiments, acquires autonomous driving service requests sent by a simulated vehicle and sends inquiry action requests to a resource scheduler based on these requests. The autonomous driving service request includes multiple autonomous driving sub-tasks. The system receives action decisions sent by the resource scheduler based on the inquiry action requests and allocates tasks based on these decisions. Task allocation includes allocating computing resources and edge server nodes to each autonomous driving sub-task in the autonomous driving service request. Based on the task allocation results, a preset interface is invoked to create a sub-task container. The sub-task container processes the autonomous driving sub-tasks in the autonomous driving service request, obtaining processing result data. Upon completion of the autonomous driving service request processing, the computing resources occupied by the request are released, and the processing result data is sent back to the simulated vehicle. Because resource planning is implemented through the resource scheduler to generate action decisions, and the system scheduler allocates autonomous driving sub-tasks based on these decisions, and establishes sub-task containers corresponding to the autonomous driving sub-tasks on each edge server node based on the task allocation results, and processes the autonomous driving sub-tasks through these containers, the real-time performance and responsiveness of autonomous driving are improved, enhancing the intelligence and autonomy of the vehicle.

[0110] Based on the first embodiment of the vehicle network system scheduling method of the present invention, a second embodiment of the vehicle network system scheduling method of the present invention is proposed.

[0111] like Figure 3 As shown, Figure 3 This is a flowchart illustrating the second embodiment of the vehicle network system scheduling method of the present invention.

[0112] Reference Figure 3In this embodiment, when the system scheduler receives an action decision from the resource scheduler, it can initiate a series of processing steps. During this process, the system scheduler can set a system time window and send the corresponding state transition quadruple (s, a, r, s') to the resource scheduler at the end of the system time window for policy updates by the agents in the resource scheduler. Here, the quadruple represents the current state (s) observed by the system scheduler when making the action decision, the action decision (a), the reward (r), and the next state after the state transition (s'). That is, the method also includes:

[0113] Step S21: Upon receiving the action decision, generate a time window;

[0114] Step S22: When the time window ends, obtain the current state of the resource scheduling system;

[0115] Step S23: Determine the resource scheduling reward and the next state of the resource scheduling system based on the current state and the action decision;

[0116] Step S24: Determine the state group information of the system scheduler based on the current state, the action decision, the resource scheduling reward, and the next state;

[0117] Step S25: Send the status group information to the resource scheduler to update the decision strategy; the decision strategy is used to match action decisions for query action requests.

[0118] It should be noted that rewards refer to the positive feedback received by the system scheduler in autonomous vehicles during action decision-making and behavior execution. Through resource scheduling rewards, the resource scheduler can be used to evaluate action strategies and make corresponding optimizations to enhance the scheduling capabilities of the resource scheduling system.

[0119] In reinforcement learning, rewards are typically represented as a scalar value, which can be a real number or a discrete category. After the system scheduler determines action decisions, allocates and executes autonomous driving subtasks, it can obtain a reward signal (i.e., resource scheduling reward) based on the processing results and driving performance. This reward signal can be positive, negative, or zero, used to express the quality of the action. A positive number indicates that the action contributes significantly to the task objective, a negative number indicates that the action contributes less, and zero indicates that the action has no significant impact on the task objective. Through continuous interaction, learning and optimization are performed based on the obtained reward signals to maximize the long-term accumulated reward.

[0120] It's important to note that state (s) represents the current environmental state, which is the system's observation or feature vector at a given moment. In autonomous driving systems, state can include information such as the vehicle's position, speed, acceleration, and the positions and speeds of surrounding vehicles and obstacles. State describes the situation and environmental conditions faced by the agent at the current moment. a (action): Action represents the behavior chosen by the agent in the resource scheduler at a given moment. In autonomous driving systems, action can be control commands such as steering angle, braking force, and acceleration. Action describes the strategy adopted by the agent to achieve a certain goal. s' (next state): The next state represents the new environmental state the system transitions to after executing action a. In autonomous driving systems, the next state is usually new environmental information perceived by the agent, such as updated vehicle position, speed, and the positions of obstacles and other vehicles. r (reward): Reward represents the immediate feedback the system receives after executing action a and transitioning to the next state s'. Reward can be a real number or a discrete category used to evaluate the quality of the action. In autonomous driving systems, rewards can be set based on driving performance metrics, such as obstacle avoidance, maintaining a safe following distance, and smooth acceleration and deceleration.

[0121] It is understandable that the aforementioned state group information is also known as the quadruple (s, a, r, s'). By sending the state group information to the resource scheduler, the agents in the resource scheduler can be updated, thereby completing the decision-making strategy update and achieving further optimization of the resource scheduling system.

[0122] It should be explained that the specific duration of the aforementioned time window can be set and optimized according to the specific implementation requirements, and this embodiment of the invention does not impose any restrictions on it.

[0123] To further improve the processing efficiency of autonomous driving sub-tasks, the step of creating a sub-task container by calling a preset interface based on the task allocation result, and processing the autonomous driving sub-tasks in the autonomous driving service request through the sub-task container to obtain processing result data includes:

[0124] Step S31: Based on the task allocation result, call the preset interface to create a subtask container corresponding to the autonomous driving subtask in the edge server node;

[0125] Step S32: Input the autonomous driving domain data in the autonomous driving subtask into the subtask container in units of frames for processing to obtain processing result data; the processing result data includes the I / O latency, computation latency, average computation latency and overall task completion latency of the autonomous driving subtask.

[0126] It should be noted that once the subtask container is created, the system scheduler can process autonomous driving domain data within a segment of the autonomous driving subtask on a frame-by-frame basis to simulate the real-time processing of each frame during autonomous driving. Specifically, the system scheduler adjusts the overall workload of the resource scheduling system by setting an appropriate number of input frames for the subtask container to adapt to different clusters. Each frame of data is input into the subtask container for processing, and the system scheduler can record the corresponding subtask I / O latency and computation latency during the processing of each frame, thus providing relevant reference indicators for subsequent optimization of the resource scheduling system.

[0127] like Figure 4 As shown, Figure 4 This is a flowchart illustrating the overall process of one embodiment of the vehicle network system scheduling method of the present invention.

[0128] Reference Figure 4 The system scheduler can receive autonomous driving requests sent by simulated vehicles and generate inquiry action requests to the resource scheduler based on the autonomous driving requests, so that the resource scheduler can determine the action decision based on the inquiry action requests.

[0129] Furthermore, the system scheduler can accept action decisions sent by the resource scheduler, allocate on-the-fly subtasks according to the action decisions, set and process time windows, and send the obtained quadruple to the resource scheduler.

[0130] Furthermore, upon completion of the driving subtask allocation, a subtask container is created based on the allocation results, and the autonomous driving subtask is processed through the subtask container. Resources are released after all autonomous driving subtasks have been processed. Upon completion of resource release, a resource update completion request is sent to the resource scheduler to update the system state. Simultaneously, the processing result of the autonomous driving request is returned to the simulated vehicle to mark the end of one iteration.

[0131] This solution provides a vehicle-to-everything (V2X) system scheduling method that can optimize resource scheduling algorithms. It takes into account the V2X scenario and autonomous driving technology, and designs a resource scheduling system based on a container orchestration platform and using virtualization technology, which is also applicable in environments with extremely scarce computing resources.

[0132] This invention, through the design of the system scheduler, records the subtask I / O latency and computation latency of each frame of data when processing autonomous driving requests. It provides a way to comprehensively analyze different subtasks, which can help to more accurately obtain the system's operating status and optimize system performance. It also provides a reference for the subsequent design of corresponding resource scheduling algorithms.

[0133] This invention also tested the stability of the system under different load conditions. Under high load conditions, the system was able to capture some abnormal signals and process them appropriately, thereby ensuring the smooth operation of the entire system and the collection of data, laying the foundation for subsequent work.

[0134] Specifically, this invention implements a distributed system resource scheduling system based on a container orchestration platform, providing efficient autonomous driving services by leveraging the powerful capabilities of the container orchestration platform. The system can process tasks based on a cluster of multiple edge servers, each equipped with different computing resources, such as CPUs and GPUs. By decomposing autonomous driving service requests into tasks, each task can be containerized, and sufficient computing resources within the cluster can be utilized to ensure service latency and quality. Simultaneously, this resource scheduling system exhibits good scalability and flexibility, capable of adapting to autonomous driving services of varying scales and requirements.

[0135] This embodiment generates a time window upon receiving an action decision; at the end of the time window, it obtains the current state of the resource scheduling system; determines the resource scheduling reward and the next state of the resource scheduling system based on the current state and the action decision; determines the state group information of the system scheduler based on the current state, action decision, resource scheduling reward, and next state; and sends the state group information to the resource scheduler to update the decision strategy. The decision strategy is used to match action decisions with query action requests. By setting a time window and determining the state group information of the system scheduler at the end of the time window and sending it to the resource scheduler, the decision strategy is updated, the iterative optimization of the resource scheduling system is completed, and the resource scheduling efficiency of the resource scheduling system is improved.

[0136] Furthermore, the present invention also proposes a vehicle network resource scheduling method, which is applied to the resource scheduler of a resource scheduling system.

[0137] In one implementation, reinforcement learning-based resource scheduling algorithms are typically implemented using PyTorch or TensorFlow libraries. This invention constructs a resource scheduler that can be used in Python to handle concurrent requests. It is applicable to resource scheduling in distributed environments, especially in scenarios where corresponding actions need to be allocated at the granularity of autonomous driving subtasks, and the concurrency will be significantly improved.

[0138] Specifically, this invention constructs a resource scheduler based on a Python Web framework. The specific framework used can be the Flask framework or other frameworks, and this embodiment of the invention does not limit this.

[0139] In one implementation, this invention fully leverages the advantages of the Flask framework, utilizing the built-in WSGI server to handle concurrent requests, making it suitable for distributed environments in resource scheduling systems. The resource scheduler constructed in this invention can also be applied to deploying resource scheduling algorithms based on non-reinforcement learning. Furthermore, this invention's vehicle-to-everything (V2X) resource scheduling method incorporates additional improvements for real-world scenarios, contributing to the stable operation of the system.

[0140] Reference Figure 5 , Figure 5 This is a flowchart illustrating the first embodiment of the vehicle network resource scheduling method of the present invention.

[0141] like Figure 5 As shown, the method includes the following steps:

[0142] Step S100: Upon receiving an inquiry action request sent by the system scheduler, determine the action decision corresponding to the inquiry action request based on the decision strategy;

[0143] Step S200: Send the action decision to the system scheduler.

[0144] It should be noted that the execution entity of the method in this embodiment is the aforementioned resource scheduler. This resource scheduler can be a virtual module in a resource scheduling system, used to implement resource scheduling within the system.

[0145] It should be understood that the resource scheduling system can be applied to servers, computers or other terminal devices with data processing and program execution functions, and can also be applied to electronic devices with the same or similar functions. This embodiment of the invention does not limit this.

[0146] It should be noted that when the resource scheduler receives an action request from the system scheduler, it can observe and select the corresponding action, such as allocating edge server nodes and computing resources for the autonomous driving subtask. After the resource scheduler determines the action corresponding to the action request, it can generate the corresponding action decision and send it to the system scheduler.

[0147] It should be explained that the above observations refer to obtaining information about the environment through the vehicle's perception system, such as the state and characteristics of vehicles, roads, and other obstacles.

[0148] In one implementation, the observation step can be performed by the system scheduler, which then sends the obtained environmental information to the resource scheduler. Alternatively, the observation can be performed by the resource scheduler itself. This embodiment of the invention does not limit this step.

[0149] It should be noted that the decision strategy is the core of the resource scheduler. In the resource scheduler, the agent determines the action decision corresponding to the query action request through the decision strategy, and updates the decision strategy according to the four-tuple returned by the system scheduler, thereby realizing the continuous iterative optimization of the agent.

[0150] It should be understood that an intelligent agent is an entity capable of perceiving its environment, making decisions, and executing actions. An intelligent agent is a virtual entity in a resource scheduling system that can continuously interact with the environment to optimize and update its decision-making strategies through learning, planning, and decision-making, based on information and objectives within the environment.

[0151] In its implementation, when the resource scheduler receives an action request from the system scheduler, it determines the corresponding action decision based on a decision strategy and sends this decision to the system scheduler. The system scheduler then allocates and executes tasks according to this action decision. Because resource planning is achieved through the resource scheduler to generate action decisions, and the system scheduler performs system scheduling based on these decisions, resource scheduling and system scheduling can be performed separately, thus improving the efficiency of the resource scheduling system.

[0152] Step S300: Receive the state group information generated by the system scheduler based on the action decision, and store the state group information as training experience.

[0153] It is understandable that the state group information can be the four-tuple mentioned above. When the resource scheduler receives the four-tuple generated by the system scheduler based on action decisions, it can store the four-tuple as training experience in the experience replay pool. When the training experience stored in the experience replay pool reaches a certain amount, the agent can be trained based on the training experience to update the agent's decision-making strategy. That is, step S400: when the stored training experience meets the training condition threshold, the agent is trained based on the training experience to update the decision-making strategy, thereby optimizing the resource scheduler and improving the working efficiency of the resource scheduling system.

[0154] Specifically, the resource scheduler can accept a quadruple (s, a, r, s') from the system scheduler. When the conditions for starting training are met (e.g., the amount of training experience in the experience replay pool reaches a certain quantity), each subsequent new training experience storage request will trigger a decision policy update. The experience stored in the experience replay pool can be prioritized based on the importance of each sample. Specifically, this can be achieved by using a temporal error difference algorithm as the priority metric, and sampling the training experience in the experience replay pool according to the priority to obtain sampled experience. This sampled experience is then used to train the agent and update the decision policy.

[0155] At the end of an iteration, the resource scheduler can calculate the overall reward for that iteration to measure the effectiveness of the current decision-making strategy.

[0156] It is understood that the resource scheduler can also receive resource update completion requests sent by the system scheduler, and update the recorded system status upon receiving such a request. This system status may include: the number of queue requests, the set upper limit of computing resources, and the remaining amount of computing resources. In addition, there are some variables used to manage synchronization between concurrent environments, such as condition variables, mainly used to manage the concurrent execution of different operations such as resource updates, queue updates, and action queries. This embodiment of the invention does not impose limitations on these variables.

[0157] In this embodiment of the invention, upon receiving an inquiry action request from the system scheduler, the system determines the corresponding action decision based on a decision policy; sends the action decision to the system scheduler; receives state group information generated by the system scheduler based on the action decision, and stores the state group information as training experience; when the stored training experience meets the training condition threshold, the system trains the agent based on the training experience to update the decision policy. Because the agent is trained based on the state group information sent by the system scheduler, thereby updating the decision policy, the resource scheduler can be gradually optimized during continuous iteration, improving the robustness of the resource scheduling system.

[0158] Based on the embodiments described above, in order to find the possibility of global optimization, the step of determining the action decision corresponding to the query action request based on the decision strategy when receiving the query action request sent by the system scheduler includes:

[0159] Step S110: Upon receiving an inquiry action request sent by the system scheduler, determine the current state of the resource scheduling system.

[0160] It should be understood that the current state of the system is also the current system state. The information included in the system state can be found above and will not be repeated here.

[0161] It should be noted that when determining the current state of the system, the action decision corresponding to the query action request can be determined based on the decision strategy and the query action request and the current state of the system.

[0162] It should be explained that since the action decision is determined based on the agent's decision strategy, and the decision strategy is optimized based on multiple historical iterations, the obtained action decision is not necessarily the globally optimal strategy, but rather a locally optimal action decision. That is, step S120: Obtain the decision strategy, and determine the local action decision corresponding to the query action request based on the current state and the decision strategy.

[0163] It should be understood that the local action decision obtained based on the agent's decision-making strategy is a locally optimal strategy, which may or may not be a globally optimal strategy. To enable the agent to break through the scope of its learned knowledge, attempt to find potentially better actions, and allow the model to escape potentially trapped in locally optimal solutions, and to possess the possibility of finding the global optimum, this invention also designs a function for action space exploration, namely an exploration function, to ensure that the agent can smoothly provide reasonable actions in the initial stage, ensuring the stable operation of the system. Based on the exploration function, step S130 can be implemented: exploring the action space to obtain exploration action decisions.

[0164] It should be noted that before iteration, this invention sets a minimum computational resource requirement to prevent "0" resource allocation, thereby ensuring that the system caller can correctly create and run the subtask container. Specifically, this invention targets the resource configuration of the "Best Effort" category in Kubernetes, which does not set any restrictions on the CPU or memory of the subtask container in Kubernetes' Quality of Service (QoS) level. Under this configuration, the subtask container can use unused CPU resources on the edge server node, even if it exceeds the amount of resources it requests. In this case, processing the task may use all remaining CPU or GPU resources, exceeding the resource limit set for testing the goodness of resource scheduling, so such problems need to be avoided.

[0165] Furthermore, this invention sets a maximum allocated amount of computing resources. The purpose of this setting is primarily to ensure that, initially, excessive computing resources are not allocated to a single request due to algorithm weight initialization, resulting in insufficient resources for processing other requests. This maximum allocated amount of computing resources increases with the number of iterations, and its maximum should be equal to the remaining resources of the node.

[0166] It should be understood that an action space refers to the set of actions an agent can choose in reinforcement learning. It represents the set of all possible actions the agent can perform at each time step. The action space can be discrete or continuous, depending on the nature and requirements of the problem. In a discrete action space, the actions an agent can choose are finite and discrete, such as driving left or right. In a continuous action space, the actions an agent can choose are continuous and unrestricted, such as choosing a steering angle or driving speed within a range.

[0167] It should be noted that the exploration function of this invention employs an exploration-exploitation strategy, specifically an epsilon-greedy strategy. In one implementation, in the exploration part, random numbers can be generated from a preset uniform distribution and compared with a preset threshold epsilon. When the random number is less than epsilon, a random action can be selected from the selectable actions in the action space; when the random number is not less than epsilon, this constitutes the exploitation part. In the exploitation part, an action can be selected based on medical knowledge, and this action follows a Gaussian distribution. This Gaussian action is then used as the action corresponding to the mean sampling to obtain the exploration action, which is further used to generate an exploration action decision.

[0168] Understandably, by setting the size of epsilon, the relationship between exploration and exploitation can be balanced, and the size of epsilon and the variance of the Gaussian distribution can be changed with the number of iterations.

[0169] Understandably, when exploring the action space using the exploration function, the obtained action decision may result in a situation where the remaining computing resources are less than the set minimum computing resource requirement. In this case, the processing of the autonomous driving request will be blocked, and the request will continue to wait in the queue for processing. Only after another autonomous driving request task completes and releases resources will the request be reawakened to continue attempting to obtain the corresponding action decision.

[0170] In one implementation, the local action decision can be compared with the exploratory action strategy to obtain a better target action decision. That is, step S140: compare the local action decision and the exploratory action decision to obtain the target action decision, which is the action decision corresponding to the query action request.

[0171] It should be noted that this invention does not limit the comparison method for action decision-making, and the method can be selected according to the actual application.

[0172] It is understandable that when an action decision is obtained that satisfies the minimum computing resource requirement and the maximum allocated computing resource, the action decision can be sent to the system scheduler, so that the system scheduler can allocate and process subtasks according to the action decision.

[0173] In one implementation, this invention constructs a novel multi-threaded resource scheduler that, with the aid of the Flask framework, implements request sending and receiving and agent training in a distributed environment. To meet the requirements of a distributed environment, this invention chooses to use the Flask framework to construct the multi-threaded agent server, ensuring normal operation in a distributed environment.

[0174] Specifically, this multi-threaded resource scheduler has the following key functions:

[0175] Resource scheduling decision distribution in response to system requests: By receiving requests from the system scheduler, the resource scheduler can intelligently distribute resource scheduling decisions (i.e. generate action decisions) based on the current state and task requirements, ensuring that tasks can be completed on time and optimizing overall resource utilization.

[0176] Agent policy updates: The scheduler can update the agent's policy using a reinforcement learning framework based on different feedback and learning algorithms. Over time, the agent will gradually improve its decision-making ability through continuous learning and adjustment, thereby enhancing the effectiveness and accuracy of resource scheduling.

[0177] Reasoning and Decision-Making: When the system needs to schedule resources, the resource scheduler can reason based on the current agent's policy (i.e., decision-making policy) and make corresponding decisions. This process is automated; through optimization algorithms and learning models, the resource scheduler can effectively respond to different resource demands and provide high-quality action decisions.

[0178] It should be understood that the resource scheduler facilitates the deployment and verification of reinforcement learning resource scheduling algorithms. Furthermore, in situations of resource scarcity, this resource scheduler design allows requests to wait for processing, increasing the system's fault tolerance and robustness. This design ensures stable system operation during sudden peak loads while making better use of system resources.

[0179] Furthermore, this design facilitates the direct deployment of corresponding algorithms, takes into account both reinforcement learning and non-reinforcement learning application scenarios, proposes a unified scheduling architecture, considers the process of the system transitioning from a static state to stable operation, and demonstrates strong adaptability.

[0180] Furthermore, this design fully leverages the advantages of the Flask framework in handling concurrent requests and, based on this, designs a scheduler suitable for distributed environments, which is highly adaptable to modern high-concurrency, high-load computing environments.

[0181] Furthermore, this design addresses the potential for over-allocation of resources during the algorithm's initialization phase. A dynamic resource allocation limit restricts excessive resource allocation initially, and gradually increases the allocated computational resources as the number of iterations increases. This ensures smooth and fair processing of all requests within the system and improves overall system utilization efficiency in the long run. A minimum resource limit is also considered to avoid issues such as allocating "0" resources and container creation failures that occur in vehicle-to-everything (V2X) scheduling methods.

[0182] like Figure 6 As shown, Figure 6 This is a schematic diagram of the overall process of one embodiment of the vehicle network resource scheduling method of the present invention.

[0183] Reference Figure 6 The resource scheduler can receive query action requests sent by the system scheduler, and select actions based on the current system state and the query action request, thereby determining the action decision and sending the action decision to the system scheduler.

[0184] Furthermore, the resource scheduler can also receive four-tuple data sent by the system scheduler and add this four-tuple data as training experience to the experience replay pool. When the training experience in the experience replay pool meets the training conditions, the training experience is extracted for agent training to update the agent's decision-making strategy.

[0185] Furthermore, when the resource scheduler receives a quadruple, it can also determine whether the current iteration is complete. If it is complete, it calculates the total reward based on the reward in each quadruple and measures the merits (good or bad) of the decision-making strategy based on the reward.

[0186] Furthermore, the resource scheduler can also receive resource update completion requests sent by the system scheduler, and update the system status recorded in the resource scheduler when it receives such requests.

[0187] In this embodiment of the invention, upon receiving an inquiry action request from the system scheduler, the current state of the resource scheduling system is determined; a decision strategy is obtained; and a local action decision corresponding to the inquiry action request is determined based on the current state and the decision strategy; an exploration is performed in the action space to obtain an exploration action decision; and the local action decision and the exploration action decision are compared to obtain a target action decision, which is the action decision corresponding to the inquiry action request. Because exploring whether a better action decision exists in the action space upon obtaining a local action decision helps the agent break through its current knowledge limitations, enabling the system to escape potentially trapped in locally optimal solutions and possessing the possibility of finding the global optimum.

[0188] Based on the embodiments of the vehicle network system scheduling method and vehicle network resource scheduling method of the present invention described above, a first embodiment of the resource scheduling system of the present invention is proposed.

[0189] like Figure 7 As shown, Figure 7 This is a flowchart illustrating the first embodiment of the resource scheduling system of the present invention.

[0190] The resource scheduling system includes a system scheduler and a resource scheduler.

[0191] It should be noted that, in order to iteratively update the resource scheduling system, the system also includes simulated vehicles.

[0192] It should be noted that the specific implementation of the resource scheduling system in this embodiment and the specific implementation of other embodiments can be referred to the above text, and will not be repeated here.

[0193] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0194] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0195] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0196] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A scheduling method for a vehicle-to-everything (V2X) system, characterized in that, The method is applied to a system scheduler in a resource scheduling system, the resource scheduling system further including a resource scheduler and a simulated vehicle, and the method includes: The system obtains the autonomous driving service request sent by the simulated vehicle and sends an inquiry action request to the resource scheduler based on the autonomous driving server request; the autonomous driving server request includes multiple autonomous driving sub-tasks. The system receives action decisions sent by the resource scheduler based on the query action request and performs task allocation based on the action decisions; the task allocation includes allocating computing resources and edge server nodes to carry the tasks for each autonomous driving subtask in the autonomous driving service request. Based on the task allocation result, a preset interface is called to create a subtask container. The subtask container is used to process the autonomous driving subtask in the autonomous driving service request and obtain the processing result data. When the autonomous driving service request is completed, the computing resources occupied by the autonomous driving service request are released, and the processing result data is sent to the simulated vehicle. 2.The IoT system scheduling method of claim 1, wherein, The method further includes: Upon receiving the action decision, a time window is generated; When the time window ends, obtain the current state of the resource scheduling system; The resource scheduling reward and the next state of the resource scheduling system are determined based on the current state and the action decision. The system scheduler's state group information is determined based on the current state, the action decision, the resource scheduling reward, and the next state. The status group information is sent to the resource scheduler to update the decision policy; the decision policy is used to match action decisions for query action requests. 3.The method of Claim 1, wherein The step of calling a preset interface to create a subtask container based on the task allocation result, processing the autonomous driving subtasks in the autonomous driving service request through the subtask container, and obtaining processing result data includes: Based on the task allocation result, a preset interface is invoked to create a subtask container corresponding to the autonomous driving subtask in the edge server node; The autonomous driving domain data in the autonomous driving subtask is input into the subtask container in units of frames for processing to obtain processing result data; the processing result data includes the I / O latency, computation latency, average computation latency and overall task completion latency of the autonomous driving subtask. 4.The method of Claim 1, wherein The step of releasing the computing resources occupied by the autonomous driving service request when the autonomous driving service request processing is completed includes: When the autonomous driving service request processing is completed, obtain the subtask identifier of each autonomous driving subtask in the autonomous driving service request; The edge server node corresponding to each of the autonomous driving sub-tasks is determined based on the sub-task identifier; Call the resource interface to release the computing resources occupied by the autonomous driving subtasks in each edge server node. 5.The method of Claim 1-4, wherein, After the step of releasing the computing resources occupied by the autonomous driving service request when the autonomous driving service request processing is completed, the method further includes: When the computing resources are released, a resource update completion request is sent to the resource scheduler to update the system state recorded by the resource scheduler. 6.The method of scheduling for the Internet of Vehicles system according to claim 2, wherein, The method further includes: When the resource scheduler receives an inquiry action request sent by the system scheduler, it determines the action decision corresponding to the inquiry action request based on the decision strategy. The resource scheduler sends the action decision to the system scheduler; The resource scheduler receives state group information generated by the system scheduler based on the action decision, and stores the state group information as training experience. When the stored training experience meets the training condition threshold, the resource scheduler trains the agent based on the training experience to update the decision-making strategy.

7. The vehicle-to-everything (V2X) system scheduling method as described in claim 6, characterized in that, When the resource scheduler receives an inquiry action request sent by the system scheduler, the steps for determining the action decision corresponding to the inquiry action request based on the decision strategy include: When the resource scheduler receives an inquiry action request sent by the system scheduler, it determines the current state of the resource scheduling system. The resource scheduler acquires a decision strategy and determines the local action decision corresponding to the query action request based on the current state and the decision strategy. The resource scheduler explores the action space and obtains exploration action decisions; The resource scheduler compares the local action decision and the exploration action decision to obtain the target action decision, which is the action decision corresponding to the query action request.

8. The vehicle-to-everything (V2X) system scheduling method as described in claim 7, characterized in that, The resource scheduler explores the action space and obtains the steps for exploration action decisions, including: The resource scheduler generates random numbers from a preset uniform distribution; When the random number is less than a preset threshold, the resource scheduler selects a random action from the selectable actions in the action space. When the random number is not less than the preset threshold, the resource scheduler selects a Gaussian action based on learned knowledge, and the Gaussian action satisfies a Gaussian distribution. The resource scheduler obtains exploration action decisions based on the Gaussian action or the random action.

9. The vehicle-to-everything (V2X) system scheduling method as described in claim 6, characterized in that, The step of the resource scheduler training the agent based on the training experience to update the decision-making strategy includes: The resource scheduler calculates the priority of the training experience based on time difference errors; The resource scheduler samples the stored training experience according to the priority to obtain sampled experience. The resource scheduler trains the agent based on the sampling experience to update the decision-making strategy.

10. The vehicle-to-everything (V2X) system scheduling method as described in any one of claims 6-9, characterized in that, The method further includes: When the resource scheduler receives a resource update completion request from the system scheduler, it updates the recorded system status.

Images