Real-time cooperative task scheduling method and device for vehicle-mounted operating system

Abstract

The invention discloses a real-time cooperative task scheduling method and device for a vehicle-mounted operating system, and belongs to the field of intelligent vehicle-mounted systems and edge computing. The method comprises the following steps: collecting vehicle tracks and terminal and server state data in a vehicle-edge-cloud cooperative system in real time; using a PatchTST model to predict the future position of the vehicle based on the historical trajectory; constructing a collaborative scheduling model taking the vehicle and the edge server as intelligent agents; based on local observation, the vehicle agent generates an unloading decision, and the edge server agent generates a task migration decision; a multi-agent near-end strategy optimization algorithm for partial reward decoupling is adopted, and a strategy is optimized under a centralized training and distributed execution framework; task unloading, migration and dynamic result return are executed according to the decision; and calculating rewards based on the task completion condition and the system state, and continuously updating model parameters. According to the method, the task completion rate of vehicle-mounted task scheduling in a dynamic complex environment is effectively improved, and the average time delay is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to a real-time cooperative task scheduling method and device for a vehicle-mounted operating system, and belongs to the field of intelligent vehicle-mounted systems and edge computing, in particular to a real-time cooperative task scheduling method for a vehicle-mounted operating system. BACKGROUND

[0002] With the continuous improvement of the intelligentization and networking of automobiles, vehicle-mounted operating systems are becoming the core software platform of intelligent automobiles, carrying multiple key functions such as automatic driving, vehicle information entertainment, remote monitoring, and vehicle-road cooperation. The tasks generated by these functions have characteristics such as high real-time performance, high computational complexity, and strong space-time correlation, which pose strict requirements on vehicle-mounted computing resources. However, vehicle embedded hardware has inherent limitations in computing power, storage capacity, and energy supply, making it difficult to independently meet the real-time processing needs of all tasks. Vehicle edge computing provides an important computing extension for vehicle-mounted systems by offloading part of the computing tasks to roadside units or the cloud, and has become a key technical path to achieve high-reliability and low-latency vehicle services.

[0003] From the technical status quo, existing vehicle task scheduling methods can be mainly divided into three categories: first, scheduling methods based on static strategies, such as priority scheduling and round-robin scheduling, which are simple to implement but lack adaptability to dynamic environments; second, scheduling methods based on optimization models, such as linear programming and dynamic programming, which can obtain optimal solutions under certain assumptions, but require high modeling accuracy and have exponential growth in complexity with problem size; third, scheduling methods based on machine learning, especially deep reinforcement learning methods, which can learn and optimize strategies through interaction with the environment, but still face challenges such as insufficient modeling of vehicle mobility and imperfect multi-agent collaboration mechanisms in the vehicle context.

[0004] Specifically, the existing technology mainly has the following deficiencies: first, most methods do not fully consider the impact of vehicle mobility on task scheduling, and the task scheduling decisions lack coordination with vehicle motion trajectories, which may result in tasks being scheduled to service nodes that the vehicle will soon leave, causing task interruption or backhaul delay; second, the existing scheduling architecture lacks flexibility, and most do not support dynamic migration of tasks between edge nodes, making it difficult to effectively address load imbalance issues; third, existing multi-agent scheduling methods have high coupling or complete independence in reward allocation and strategy coordination, limiting the overall performance of the system; finally, existing methods generally lack sufficient consideration of the characteristics of vehicle-mounted operating systems, such as real-time performance guarantee mechanisms and resource isolation requirements.

[0005] Therefore, there is an urgent need for a real-time task scheduling method and device specially oriented to a vehicle-mounted operating system environment, capable of deeply integrating vehicle mobility prediction, supporting dynamic task migration, and having an efficient multi-agent collaboration mechanism, to meet the urgent needs of intelligent networked vehicles for high-reliability, low-latency computing services. SUMMARY

[0006] Therefore, the present application provides a real-time collaborative task scheduling method and device for a vehicle-mounted operating system, aiming to realize forward-looking task scheduling and efficient agent collaboration by introducing a vehicle trajectory prediction module and a multi-agent reinforcement learning mechanism with partial reward decoupling, thereby improving task completion rate and reducing average latency in a dynamic and complex vehicular edge computing (VEC) environment.

[0007] To achieve the above-mentioned purpose, the present application discloses a real-time collaborative task scheduling device for a vehicle-mounted operating system, which combines Figure 1 implemented by a vehicle-edge-cloud collaborative system, the system includes: a plurality of vehicle terminals (VTs), a plurality of edge servers (RSUs) deployed on road side units, and a cloud server; the vehicle terminals are installed on vehicles and connected to the RSUs and the cloud server through a wireless network; the RSUs and the cloud server are interconnected through wired or high-speed wireless networks; the vehicle terminals, edge servers, and cloud server are all computer devices with computing and communication capabilities.

[0008] For a city topology scenario including a plurality of vehicle terminals , a plurality of edge servers of road side units , and a cloud server ; wherein, is a time slot; at time slot , there are vehicle terminals within the coverage range of any th edge server . is the set of vehicle terminals in the coverage blind area of the edge server.

[0009] Any th vehicle terminal has a certain computing power , a storage capacity of , and a historical observation trajectory at the current time slot is , wherein , is the backtracking window length.

[0010] Any Edge server CPU with a processing speed of deployed at intersections with a coverage radius of ; a storage capacity of , and a device transmission channel number of .

[0011] Cloud server CPU with a processing speed of , a device transmission channel number of , and nearly unlimited storage and computing resources.

[0012] The vehicle terminal generates a task in a time slot , and the task is recorded as , wherein , represents the total number of tasks generated by all vehicle terminals, represents the task data size, represents the number of CPU cycles required for the task, represents the task result data size, represents the task deadline, and if the result is not returned to the vehicle before the time , the task is considered to have failed.

[0013] The present application also discloses a real-time collaborative task scheduling method for a vehicle-mounted operating system, which combines Figure 2 and includes the following steps:

[0014] S1: Real-time collection of vehicle historical trajectory data, vehicle terminal state data, and edge server state data in the vehicle-edge-cloud collaborative system;

[0015] S2: Based on the historical trajectory sequence, a PatchTST-based trajectory prediction method is used to predict the vehicle position in the next time slots;

[0016] S3: A vehicle-edge-cloud collaborative task scheduling model is constructed, the vehicle and the edge server are modeled as multiple agents, and their strategy network and value network are initialized;

[0017] S4: In each decision-making time slot, the vehicle agent generates an unloading decision based on local observation;

[0018] S5: The edge server agent generates a task migration decision based on local observation;

[0019] S6: A partial reward decoupled multi-agent proximal policy optimization algorithm (PRD-MAPPO) is used, global information is used to optimize the agent strategy in the centralized training stage, and distributed decision-making is performed based on local observation in the execution stage.

[0020] S7: Based on the decision results, perform task unloading and migration, and after the task calculation is completed, dynamically select the optimal path based on the real-time location of the vehicle and send the results back to the vehicle;

[0021] S8: Calculate the rewards for each agent based on the task completion status and the status of the vehicle-edge-cloud collaborative system. Adopt an experience-based training mechanism to regularly update the trajectory prediction model and multi-agent policy network parameters to achieve continuous policy optimization.

[0022] Furthermore, the trajectory prediction method based on PatchTST in step S2 is specifically as follows:

[0023] S201: For any... Vehicle terminal, given subsequence length and sliding step size Will Split into overlapping subsequences ,in, The number of subsequences is , In order to collect evidence from lower levels, Adjacent subsequences exist Frame overlap;

[0024] S202: Will First, it is mapped to the input of the Transformer through linear projection, while position encoding is introduced to preserve timing information. ,in The projection matrix is ​​learnable. For the potential dimensions of Transformer, For the first Position encoding vectors of each subsequence;

[0025] S203: The projected and encoded temporal information is fed into a shared-weight Transformer encoder, which extracts temporal features through a multi-layer self-attention mechanism and a feedforward network, ultimately outputting the corresponding representation vectors for each channel. ;

[0026] S204: Characterization After being flattened into a one-dimensional vector, it is input into the linear prediction layer to generate the future. Vehicle position prediction for each time slot ;

[0027] S205: Supervised learning is used to train the Transformer, with the goal of minimizing the difference between the model's predicted trajectory and the true trajectory. The loss function is: ,in denote all learnable parameters, is the training set.

[0028] Further, the agent described in step S3 adopts the Actor-Critic paradigm, including an Actor network, i.e. a policy network , and two independent Critic networks, i.e. a Q network and a value network , which are represented by a tuple of five elements , , , , , , where is the reward, is the state transition probability, is the discount factor, is the state space, is the action space; in the execution phase, the Actor outputs a deterministic action according to the local observation feature ; in the training phase, the Q network and the value network will take the global state and joint action of all agents in the input time slot as input, the Q network is used to predict the expected future individual reward of the agent performing action in state , and the value network is used to predict the sum of future rewards of the agents in the relevant set of agents , where , , are the parameters to be trained.

[0029] Further, the step S4 is specifically:

[0030] S401: In each time slot , the vehicle agent obtains its local observation feature in the environment; wherein denotes the information of the edge server where the vehicle terminal is located at time slot predicting the vehicle's future 6 time steps, denotes the computing queue congestion degree of the edge server where the vehicle terminal is located; is the vehicle terminal to the edge server ; representing the vehicle terminal to the edge server , channel gain, representing the transmission distance, is the noise power accumulated in the signal bandwidth;

[0031] S402: the vehicle agent inputs its local observation feature into its policy network to generate its offloading decision;

[0032] The policy network outputs a discrete action probability distribution, and the vehicle agent samples according to the distribution to generate a discrete offloading decision . The action space of the offloading decision includes: (1) local execution, if the local resource meets the delay requirement, directly processed in the vehicle agent ; (2) edge offloading, offloading the task to the edge server through the V2I link; (3) cloud offloading, transmitting the task to the cloud server for processing through the backhaul link; wherein, , ;

[0033] S403: scheduling according to the generated offloading decision - if the decision is local execution, storing the task in the local computing queue to wait for CPU time slice round-robin scheduling; if the decision is offloading, storing the task in the wireless transmission queue, and sending data packets based on the channel state; and obtaining the reward and state of the next moment.

[0034] Further, the step S5 is specifically:

[0035] S501: in time slot , the edge server agent obtains its local observation feature in the environment; wherein, is the modeling of the task priority, is the time of task generation; is the load condition of all edge servers in time slot ;

[0036] S502: the edge server agent inputs the local observation feature into its policy network to generate its offloading decision;

[0037] The policy network outputs a discrete action probability distribution, and the edge server agent... Sampling is performed based on this distribution. Generate task scheduling decisions The action space for offloading decisions is: local processing, horizontal migration to a neighboring edge server with a lower load. Vertical migration and uploading to cloud servers Processing; among which, , ;

[0038] S503: Schedules tasks based on generated unloading decisions—classifying and managing local computing tasks, tasks awaiting migration, and tasks uploaded to the cloud, and dynamically adjusting the task flow of each queue based on migration decisions to achieve efficient task scheduling and forwarding; and receiving rewards for the next time step. and state .

[0039] Furthermore, step S6 in the training phase specifically includes:

[0040] S601: Each intelligent agent The value network constructs a global state through splicing. Joint actions of all intelligent agents Using the corresponding Q-network to predict the agent In state Next action Expected future individual returns It also outputs the weights of the attention mechanism in the Q-network. ;wherein, any intelligent agent The weight vector output by the value network middle, The remaining weights are in the range middle;

[0041] S602: Each agent Value networks utilize global state Joint actions of all intelligent agents Predictive agents The sum of future rewards of agents within the relevant set ;

[0042] S603: Filter out the current agent Set of "key neighbors" with significant influence It aggregates the reward information of these key neighbors to calculate the single-step advantage value after decoupling. ;in, For time slots intelligent agent the reward of the vehicle;

[0043] S604: calculating the decoupling advantage value ; wherein, is a discount factor, is a GAE (Generalized Advantage Estimation) parameter;

[0044] S605: constructing the loss function of the agent using the clipped objective function of the proximal policy optimization PPO algorithm combined with the entropy regularization term , and iteratively updating the parameters of the policy network Actor ; wherein, is a PPO clipping range hyperparameter; is a clipped objective function; is a mean value calculation; is the reward of the agent under the Actor network parameters ;

[0045] S606: respectively calculating the mean square error loss of the Q network and the value network of the agent , and updating the network parameters and by Adam.

[0046] Further, the "key neighbor" set in step S6 is , and the iterative update of the parameters of the policy network Actor is , wherein, is a set threshold, is the number of iterations, is a learning rate.

[0047] Further, the step S6 in the execution phase, based on the local observation, carries out the distributed decision-making described in steps S4 and S5, while collecting feedback in real time to train the network.

[0048] Further, the result return described in step S7 is specifically: after the task calculation is completed, the system queries the real-time location of the vehicle that generates the task. If is still within the coverage range of the RSU at the calculation completion point, it is directly returned through the V2I link. If has left but is located under another RSU coverage, the result is forwarded to through the inter-RSU optical fiber network, and then sent to the vehicle. If When in a coverage blind area, the result is cached through a cloud server or a neighboring RSU, and pushed after the vehicle enters the coverage area.

[0049] The application has the beneficial effects that: a real-time cooperative task scheduling method and device for a vehicle-mounted operating system are provided, a vehicle-edge-cloud three-level cooperative computing architecture is constructed, and dynamic intelligent scheduling of vehicle-mounted tasks between the local, edge and cloud is realized; a vehicle trajectory prediction module based on PatchTST is introduced, forward-looking perception of the vehicle movement trend is realized, task offloading decisions and vehicle movement trajectories are cooperatively optimized, and task interruption caused by the vehicle driving out of the coverage range is effectively avoided; a multi-agent reinforcement learning algorithm with partial reward decoupling is used, global information is used to optimize the cooperative strategy in the centralized training stage, and distributed real-time decisions are made based on local observations in the execution stage, which significantly improves the cooperation efficiency between multi-agents and the system scalability; through supporting dynamic task migration and flexible result return mechanism, load balancing between edge nodes is realized, and the real-time requirement of high-priority tasks is ensured. The experimental results show that, compared with the traditional method, the task completion rate and the average task time efficiency of the application are significantly improved, the task scheduling efficiency and service quality of the vehicle-mounted operating system in a dynamic complex environment are significantly improved, and the application has good engineering application value. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to make the purpose, technical scheme of the application clearer, the application provides the following drawings for illustration:

[0051] Figure 1 It is a schematic diagram of the vehicle-edge-cloud cooperative system in embodiment 1 of the application;

[0052] Figure 2 It is a flowchart of the real-time cooperative task scheduling method for a vehicle-mounted operating system in the application; wherein, the thin solid line arrow is the task scheduling direction, and the thick arrow is the network training feedback direction;

[0053] Figure 3 It is a schematic diagram of the task queue migration model in embodiment 1 of the application;

[0054] Figure 4 It is a network architecture diagram of the multi-agent proximal policy optimization algorithm with partial reward decoupling in embodiment 1 of the application;

[0055] Figure 5 It is a comparison effect diagram of the task completion rate under different task intensities in embodiment 1 of the application;

[0056] Figure 6 It is a comparison effect diagram of the task completion time under different task intensities in embodiment 1 of the application;

[0057] Figure 7This is a comparison chart of task completion rates under different task data scales in Embodiment 1 of the present invention;

[0058] Figure 8 This is a comparison chart of task completion times under different task data scales in Embodiment 1 of the present invention;

[0059] Figure 9 This is a comparison chart of task completion rates under different task time constraints in Embodiment 1 of the present invention;

[0060] Figure 10 This is a comparison diagram of task completion time under different task time constraints in Embodiment 1 of the present invention. Detailed Implementation

[0061] To make the objectives and technical solutions of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0062] Example 1: Taking a typical urban grid road as an example, in order to better realize the task scheduling efficiency and service quality of the vehicle operating system in a dynamic and complex environment, this invention provides a "real-time collaborative task scheduling device for vehicle operating systems".

[0063] Combination Figure 1 The device is implemented by a vehicle-edge-cloud collaborative system, which includes: Taiwan vehicle terminal , Edge servers deployed in roadside units A cloud server; the vehicle terminal is installed on the vehicle and equipped with an on-board computing unit, and is connected to the RSU and the cloud server via a wireless network; the RSUs are interconnected with each other and with the cloud server via wired or high-speed wireless networks, and are evenly distributed at the intersections of the main urban roads; the vehicle terminal, edge server and cloud server are all computer devices with computing and communication capabilities.

[0064] Any number The vehicle terminal has a certain computing power. Storage capacity is In the current time slot The historical observation trajectory is ,in , This is the length of the backtracking window.

[0065] Any number Taiwan edge server The processing speed is configured to be The CPU is deployed at the intersection, with a coverage radius of [missing information]. Storage capacity is , the device transmission channel number is .

[0066] The cloud server is configured with a CPU with a processing speed of , the device transmission channel number is , and has almost unlimited storage and computing resources.

[0067] The system covers a urban central area, and the roads are distributed in a grid shape. Each RSU is located at a crossroads. The system time is discretized into time slots , each time slot is seconds, which corresponds to the minimum decision period of vehicle state and task scheduling.

[0068] The first vehicle terminal generates a task at time slot , and the task is recorded as , wherein , represents the total number of tasks generated by all vehicle terminals, represents the task data size, represents the number of CPU cycles required by the task, represents the task result data size, represents the task deadline, and if the result is not returned to the vehicle before , the task is considered to be failed.

[0069] To verify the effectiveness of the method described in this embodiment, a corresponding simulation experiment environment is constructed.

[0070] In this embodiment, the key parameters of the system are set as follows: the transmission speed of the vehicle terminal to the RSU is 60Mbps, the transmission rate between RSUs is 100Mbps, the transmission speed of the vehicle terminal to the cloud server is 40Mbps, the transmission speed of the RSU to the cloud server is 80Mbps. The computing power of the vehicle terminal is 7~10G CPU cycles / s, the processing speed of the edge server is 48~60G CPU cycles / s, and the processing speed of the cloud server is 200G CPU cycles / s. The coverage radius of the edge server is 200m. The task data size ranges from 4~8 MB, the task result data size ranges from 0.4~0.8 MB, and the number of CPU cycles required by the task 22~26G CPU cycles, task deadline 7~12 seconds. Total number of vehicles 12, total number of edge servers 9, length of backtracking window 16. Hardware platform: a server with Intel Xeon Silver 4214R @ 2.40GHz processor, NVIDIA GeForce RTX 1080Ti (12GB video memory) and 128GB memory. Software environment: Python3.10.16 and PyTorch 1.12.1, all simulation codes run in this environment.

[0071] In addition, the present application provides a "real-time cooperative task scheduling method for vehicle-mounted operating system". In combination with Figure 2 , the following steps are included:

[0072] S1: Real-time collection of vehicle historical trajectory data, vehicle terminal state data and edge server state data in the vehicle-edge-cloud cooperative system.

[0073] Specifically, at the beginning of each time slot , the system performs the following data collection:

[0074] (1) Vehicle historical trajectory data: each vehicle uploads its historical observation trajectory composed of latitude and longitude coordinate sequences of consecutive time slots .

[0075] (2) Vehicle terminal state data: including current CPU utilization, remaining available storage, currently connected RSU, and local pending task queue length.

[0076] (3) Edge server state data: any RSU reports its CPU utilization, current computing task queue, available transmission channel number, and link status with adjacent RSUs.

[0077] All collected data are aggregated to a central coordinator (in this embodiment, the cloud server) through V2I link to form a global observation database.

[0078] S2: Based on its historical trajectory sequence, a PatchTST-based trajectory prediction method is used to predict the vehicle position in the next time slots. Specifically,

[0079] S201: Given subsequence length and sliding step the historical observed trajectory is split into overlapping sub-sequences (patches): wherein the number of sub-sequences is , for downscaling, adjacent sub-sequences exist frame overlap;

[0080] S202: the is first mapped to the input of the Transformer through linear projection, while introducing position encoding to preserve the temporal information wherein is a learnable projection matrix, is the latent dimension of the Transformer, is the position encoding vector of the th sub-sequence;

[0081] S203: the projected and encoded temporal information is fed into the Transformer encoder with shared weights (number of layers = 4, number of attention heads = 8), which extracts temporal features through multi-layer self-attention mechanism and feedforward network, and finally outputs the corresponding representation vector of the channel ;

[0082] S204: the representation is converted into a one-dimensional vector after the flatten operation, and then input into the linear prediction layer to generate the vehicle position prediction for the next time slots ;

[0083] S205: the Transformer is trained in a supervised learning manner, and the goal is to minimize the difference between the predicted trajectory and the real trajectory, and the loss function is: wherein represents all the learnable parameters, is the training set.

[0084] For the PatchTST multi-step vehicle trajectory module, the historical sequence , the predicted sequence and the length of the patch are set to 16, 6 and 5 respectively. The dimension of the model DModel is set to 128. The number of attention heads is 8, the MSE loss function is selected and the AdamW optimizer is used for offline training, and incremental update is performed every 24 hours after deployment, and the learning rate is 0.001.

[0085] S3: combine Figure 3, a vehicle-edge-cloud collaborative task scheduling model is constructed, and the vehicle and the edge server are modeled as multiple agents, and the strategy network and the value network thereof are initialized.

[0086] Further, the first agent adopts an Actor-Critic paradigm, including an Actor network, i.e., a strategy network , and two independent Critic networks, i.e., a Q network and a value network , which are represented by a tuple of five elements , , , , , wherein is the reward, is the state transition probability, is the discount factor, is the state space, is the action space; in the execution phase, the Actor outputs a deterministic action only according to the local observation feature ; in the training phase, the Q network and the value network will be trained on the global state and the joint action of all agents in the input time slot , the Q network is used to predict the expected future individual reward of the agent performing the action in the state , and the value network is used to predict the sum of the future rewards of the agents in the relevant set of agents , wherein , , are the parameters to be trained.

[0087] All strategy networks are three-layer MLPs, i.e., input layer (dimension depends on observation), hidden layer (64 dimensions, ReLU), output layer (action space dimension, Softmax). Two independent Critic networks are four-layer MLPs, and the final output is a scalar value. The Actor and Critic networks of all agents are initially independent, and the network weights are initialized by He.

[0088] For the strategy network Actor, the Q network and the value network in PRD-MAPPO, the Adam optimizer is used for training, the initial learning rate is 0.005 and decays over time, the discount factor of cumulative reward is set to 0.99, and the GAE hyperparameter The coefficient of the entropy term is 0.1. Each agent samples 300 experience data per round, and is divided into 5 mini-batch multiple updates, a total of 10 rounds of training.

[0089] S4: At each decision time slot, the vehicle agent generates an offloading decision based on the local observation. Specifically,

[0090] S401: At each time slot , the vehicle agent obtains the local observation features of itself in the environment ; wherein represents the information of the edge server where the vehicle terminal is located at time slot predicts the edge server where the vehicle will be in the next 6 time steps, represents the computing queue congestion degree of the edge server where the vehicle terminal is located; is the transmission rate of the vehicle terminal to the edge server ; represents the channel bandwidth of the vehicle terminal to the edge server , channel gain, represents the transmission distance, is the noise power accumulated in the signal bandwidth;

[0091] S402: The vehicle agent inputs its own local observation features into its own policy network to generate its own offloading decision;

[0092] The policy network outputs a discrete action probability distribution, and the vehicle agent samples according to the distribution to generate a discrete offloading decision , the action space of the offloading decision includes: (1) local execution , if the local resources meet the delay requirement, it is directly processed in the vehicle agent ; (2) edge offloading , offload tasks to the edge server through the V2I link; (3) cloud offloading , transmit tasks to the cloud server for processing through the backhaul link; wherein , S403: Schedule tasks based on the generated offload decision—if the decision is local execution, store the task in the local computing queue to await CPU time slice round-robin scheduling; if the decision is offload, store the task in the wireless transmission queue, transmit data packets based on channel state, and receive the reward for the next time step. and state .

[0093] S5: The edge server agent generates task migration decisions based on local observations. Specifically,

[0094] S501: In time slot Edge server intelligent agent Obtain its local observational features in the environment. ;in, It involves modeling task priorities. It is the time when the task was generated; It is a time slot Load status of all edge servers;

[0095] S502: Edge Server Intelligent Agent It inputs local observation features into its own policy network to generate its own unloading decision;

[0096] The policy network outputs a discrete action probability distribution, and the edge server agent... Sampling is performed based on this distribution. Generate task scheduling decisions The action space for unloading decisions is: Local processing at the same time Horizontal migration to a neighboring edge server with lower load , Vertical migration upload to cloud server Processing; among which, , , ;

[0097] S503: Schedules tasks based on generated unloading decisions—classifying and managing local computing tasks, tasks awaiting forwarding and migration, and tasks uploaded to the cloud, and dynamically adjusting the task flow of each queue based on migration decisions to achieve efficient task scheduling and forwarding; and receiving rewards for the next time step. and state .

[0098] S6: Combination Figure 4 A multi-agent proximal policy optimization algorithm with partial reward decoupling is adopted. In the centralized training phase, global information is used to optimize the agent policy, and in the execution phase, distributed decision-making is based on local observations.

[0099] 1. Training Phase (Centralized):

[0100] S601: Each intelligent agent The value network constructs a global state through splicing. Joint actions of all intelligent agents Using the corresponding Q-network to predict the agent In state Next action Expected future individual returns It also outputs the weights of the attention mechanism in the Q-network. ;wherein, any intelligent agent The weight vector output by the value network middle, The remaining weights are in the range middle;

[0101] S602: Value Network Utilizes Global State Joint actions of all intelligent agents Predictive agents The sum of future rewards of agents within the relevant set ;

[0102] S603: Filter out the current agent Set of "key neighbors" with significant influence It aggregates the reward information of these key neighbors to calculate the single-step advantage value after decoupling. ;in, For time slots intelligent agent The reward;

[0103] The set of "key neighbors" ,in, ;

[0104] S604: Calculate the decoupling advantage value ;in, As a discount factor, For GAE parameters;

[0105] S605: Constructing an agent by optimizing the pruning objective function of the PPO algorithm using a proximal strategy and incorporating an entropy regularization term. loss function And iteratively update the parameters of the policy network Actor. ;in, This refers to the hyperparameters for PPO clipping range. The objective function for pruning; Calculated as the mean; For intelligent agents In Actor network parameters The following rewards;

[0106] The parameters of the policy network Actor are iteratively updated as follows: in, For the number of iterations, The learning rate;

[0107] S606: Calculate the agents separately The mean squared error loss of the Q-network and value network is updated using Adam to adjust network parameters. and .

[0108] Calculate the mean squared error loss for the Q-network and the value network respectively. and ;

[0109] in, , , , These are the target values ​​for the Q network and the value network, respectively.

[0110] 2. Execution Phase (Distributed):

[0111] Each agent relies solely on its local observations, runs a local policy network to make real-time decisions, and does not need to access global information or communicate with other agents, thus achieving low-latency distributed scheduling.

[0112] S7: Based on the decision results, execute task unloading and migration, and after the task calculation is completed, dynamically select the optimal path based on the vehicle's real-time location and send the results back to the vehicle. Specifically,

[0113] (1) Execute tasks locally: in The processes are executed sequentially on the CPU, and the completion time is calculated.

[0114] (2) Edge execution of tasks: in or migration target It executes on the CPU and calculates the completion time.

[0115] (3) Execute tasks in the cloud: on the cloud server Execute the above and calculate the completion time.

[0116] After the task calculation is completed, the system queries the vehicle that generated the task. The real-time location. If If the data is still within the RSU coverage area of ​​the calculation completion point, it will be directly transmitted back via the V2I link. He has left, but is located in another RSU. Under coverage, the results are forwarded to vehicles through the inter-RSU fiber network , and then retransmitted to vehicles. If in the coverage blind area, the results are cached through the cloud server or neighboring RSU, and then pushed to vehicles when they enter the coverage area.

[0117] S8: According to the task completion and the state of the vehicle-edge-cloud collaborative system, the rewards of each agent are calculated, and the training mechanism based on experience replay is adopted to update the trajectory prediction model and the multi-agent policy network parameters regularly, so as to realize the continuous optimization of the policy.

[0118] After the action is executed, the system feeds back the corresponding reward of each agent according to the execution of the task. The reward design follows the optimization objective of this research.

[0119] The reward of the vehicle mainly includes the task local calculation and transmission delay:

[0120] , wherein is the waiting time for local calculation of the vehicle, is the time required for local calculation of the task, is the waiting time for transmission to the cloud or edge server, is the time spent on task transmission.

[0121] The reward of the RSU mainly includes the task completion delay and the load balancing reward: .

[0122] , wherein is the coefficient weight used to balance the importance of delay and load balancing, is the total time required for task completion; the reward for load balancing is , wherein , represent the load of the edge server before and after the task migration decision, respectively. Through the standard deviation and the mean , the influence on the imbalance degree before and after the migration decision is quantified, and the corresponding punishment is formulated to encourage migration to the RSU with lighter load.

[0123] Experience replay and regular update, specifically:

[0124] (1) The experience tuples of all agents are stored in a shared centralized experience replay pool with a capacity of .

[0125] (2) The system performs a training batch every time slots: randomly sample Update the Actor and Critic network parameters of all agents in step S6.

[0126] In order to better show the beneficial effects of the method of the present application, the present embodiment will compare the task completion rate and completion time of the method of the present application and the prior art. Among them, the prior art MADDPG [1] constructs an independent Actor and Critic network for each agent, and its Critic uses the joint action of all agents to evaluate the value of the global state.

[0127] [1] LOWE R, WU Y I, TAMAR A, et al. Multi-agent actor-critic for mixed cooperative-competitive environments [J]. Advances in neural information processing systems, 2017, 30.

[0128] The experimental results are shown in Table 1. Figures 5~10 As shown in Table 1, compared with the suboptimal MAPPO, the average task completion rate of the method of the present application is 4.0% higher, and the average task completion time is 6.02% faster, showing the efficiency advantage of the forward migration guidance in processing large communication overhead tasks.

[0129] The method of the present application shows consistent and significant superiority in both task completion rate and average task completion time in all test scenarios, fully proving its excellent performance, robustness and strong adaptability to dynamic environment.

[0130] Finally, it should be pointed out that the above preferred embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present application.

Claims

1. A real-time collaborative task scheduling method for in-vehicle operating systems, characterized in that, Includes the following steps: S1: Real-time collection of vehicle historical trajectory data, vehicle terminal status data, and edge server status data in the vehicle-edge-cloud collaborative system; S2: Based on its historical trajectory sequence, the PatchTST-based trajectory prediction method is used to predict the vehicle's position in several future time slots; S3: Construct a vehicle-edge-cloud collaborative task scheduling model, model vehicles and edge servers as multiple agents, and initialize their policy network and value network; S4: In each decision-making time slot, the vehicle agent generates an unloading decision based on local observations; S5: Edge server agents generate task migration decisions based on local observations; S6: Employs a partially reward-decoupled multi-agent proximal policy optimization algorithm (PRD-MAPPO), which optimizes agent policies using global information during the centralized training phase and makes distributed decisions based on local observations during the execution phase; S7: Based on the decision results, perform task unloading and migration, and after the task calculation is completed, dynamically select the optimal path based on the real-time location of the vehicle and send the results back to the vehicle; S8: Calculate the rewards for each agent based on the task completion status and the status of the vehicle-edge-cloud collaborative system. Adopt an experience-based training mechanism to regularly update the trajectory prediction model and multi-agent policy network parameters to achieve continuous policy optimization.

2. The real-time collaborative task scheduling method for in-vehicle operating systems according to claim 1, characterized in that, Step S2 specifically includes: S201: For any vehicle terminal, given the subsequence length and sliding step size, split its historical trajectory sequence into overlapping subsequences; S202: Map each subsequence to the input space of the Transformer through linear projection and introduce positional encoding; S203: The projected and encoded sequence is fed into a shared-weight Transformer encoder to extract temporal features and output a representation vector; S204: After converting the representation vector into a one-dimensional vector, input it into the linear prediction layer to generate vehicle position predictions for several future time slots; S205: The Transformer model is trained using supervised learning, and the loss function is the mean square error between the predicted trajectory and the true trajectory.

3. The real-time collaborative task scheduling method for in-vehicle operating systems according to claim 1, characterized in that, In step S3, each agent adopts the Actor-Critic paradigm, which includes a policy network as an Actor and two independent Critic networks, namely the Q network and the value network; during the execution phase, the Actor outputs actions based only on local observation features. During the training phase, the Q-network is used to predict the expected future individual reward of the agent when performing actions in the global state, while the value network is used to predict the sum future reward of the agents within the relevant set.

4. The real-time collaborative task scheduling method for in-vehicle operating systems according to claim 1, characterized in that, In step S4, the local observation features of the vehicle agent include: information on the edge server where the vehicle is predicted to be in several future time slots, the congestion level of the computing queue currently connected to the edge server, task information, edge server processing speed, vehicle terminal computing power, and transmission rate from the vehicle terminal to the edge server; the vehicle's own policy network outputs a discrete action probability distribution, and the vehicle agent samples according to this distribution to generate discrete unloading decisions; the action space of the unloading decision includes: local execution, edge unloading, and cloud unloading.

5. The real-time collaborative task scheduling method for in-vehicle operating systems according to claim 1, characterized in that, In step S5, the local observation features of the edge server agent include: edge server information, task information, edge server processing speed, task priority, and load status of all edge servers; the edge server's own policy network outputs a discrete action probability distribution, and the edge server agent samples according to this distribution to generate discrete offloading decisions; the action space of the migration decision includes: local processing, horizontal migration to neighboring edge servers, and vertical migration uploading to cloud servers.

6. The real-time collaborative task scheduling method for in-vehicle operating systems according to claim 1, characterized in that, The partially reward-decoupled multi-agent proximal policy optimization algorithm in step S6 includes the following during the training phase: S601: Each agent's value network receives the global state and the joint actions of all agents, outputs attention weights, and uses the Q-network to predict its expected future individual returns; S602: The value network of each agent predicts the sum of future returns of agents within the relevant set of agents; S603: Select the set that has a significant impact on the current agent based on the attention weight, and aggregate their rewards to calculate the decoupled single-step advantage value; S604: Calculate the decoupling advantage value based on the single-step advantage value; S605: Construct a loss function by combining the pruning objective function of the near-end policy optimization with the entropy regularization term, and update the policy network parameters; S606: Calculate the loss of the Q network and the value network respectively, and update their network parameters; During the execution phase, distributed decision-making as described in steps S4 and S5 is performed based on local observations, while feedback is collected in real time to train the network.

7. The real-time collaborative task scheduling method for in-vehicle operating systems according to claim 1, characterized in that, The dynamic result feedback mechanism in step S7 is as follows: after the task calculation is completed, the real-time location of the vehicle that generated the task is queried. If the vehicle is still within the coverage area of ​​the edge server at the point of completion of the calculation, the result is directly fed back. If the vehicle has left but is under the coverage of another edge server, the result is forwarded through the network between edge servers. If the vehicle is in a coverage blind spot, the result is cached through the cloud server or a nearby edge server and pushed after the vehicle enters the coverage area.

8. A real-time collaborative task scheduling apparatus for an in-vehicle operating system, applied to implementing the real-time collaborative task scheduling method for an in-vehicle operating system as described in any one of claims 1 to 7, characterized in that, The device is implemented by a vehicle-edge-cloud collaborative system, comprising: several vehicle terminals, several edge servers deployed on roadside units, and one cloud server; the vehicle terminals, edge servers, and cloud servers are all computer devices with computing and communication capabilities; the vehicle terminals are installed on vehicles and connected to the edge servers and cloud servers via wireless networks; the edge servers are interconnected with each other and with the cloud server via wired or high-speed wireless networks.

Images