Task offloading method for agent-assisted drl oriented to vehicle-side collaboration reasoning

Abstract

The application provides an agent-assisted DRL task offloading method for vehicle-edge collaborative reasoning, through a double-time-scale collaborative scheduling mechanism, active reasoning of the agent and numerical optimization of deep reinforcement learning are decoupled at the cache window level, deep semantic understanding of complex traffic scenes is realized while ensuring real-time scheduling performance. In each cache period, the agent generates a traffic semantic vector and a dynamic reward weight, which are used to enhance the state representation of reinforcement learning and drive the adaptive adjustment of the reward function, respectively. On this basis, a memory continuous updating mechanism assisted by deep reinforcement learning is designed, by aggregating and feeding back real interaction experience to the agent memory module, the online evolution of perception ability is realized without model parameter fine-tuning. Simulation results show that the proposed method is significantly better than the mainstream model optimization and DRL baseline in terms of task success rate, average reasoning delay and non-stationary scene adaptability.

Description

Technical Field

[0001] This invention belongs to the field of vehicle networking technology, specifically a task offloading (scheduling) method for Agent-Assisted DRL oriented vehicle-side collaborative reasoning. Background Technology

[0002] Retrieval-augmented Generation (RAG) is a task offloading method for Deep Reinforcement Learning (DRL) that enhances DRL. It accesses static and dynamic traffic knowledge through an MCP gateway and transforms heterogeneous domain knowledge into environmental perception factors, integrating them into the DRL training and inference process. However, this task offloading method still has limitations when dealing with non-stationary and complex traffic environments, mainly: 1) The efficiency weights and loss weights in the reward function are fixed constants preset by humans, which cannot be dynamically adjusted according to the risk level of the traffic scenario, resulting in insufficient risk avoidance ability of the agent in high-risk scenarios; 2) The environmental perception factors rely entirely on prior knowledge for generation, lacking perception and utilization of real-world DRL interaction experience. When there is a deviation between the actual traffic situation and the knowledge base description, factor errors will continuously affect decision quality and cannot self-correct.

[0003] The RAG method, essentially a large language model (LLM), is positioned as a static knowledge retrieval and transformation tool, its cognitive capabilities limited by the coverage and update frequency of the knowledge base. However, traffic scenarios in the integrated air-space-ground vehicle-to-everything (SAGVN) exhibit high spatiotemporal non-stationarity: road topology, traffic flow patterns, communication conditions, and unexpected events are interdependent and dynamically evolving. Pre-built knowledge bases struggle to fully cover the diverse states under complex traffic scenarios. To adapt to this continuously changing environment, the knowledge base must be frequently updated. LLM, on the other hand, requires periodic parameter fine-tuning to maintain generation quality. However, both models come with high maintenance costs. In practical vehicle-to-everything deployments, both face high maintenance costs and response latency.

[0004] Unlike LLMs, which are passive knowledge retrieval tools, LLM Agents (Large Language Model Intelligent Agents) introduce cognitive mechanisms such as perception, memory, planning, and action, enabling LLMs to proactively perceive the environment, accumulate experience, and autonomously plan and make decisions. [1] [2] Its advantage lies in the fact that it does not require explicit updates to the knowledge base or fine-tuning of model parameters. The agent relies on the collaborative operation of the memory and reasoning modules to achieve continuous cognition of the dynamic environment and adaptive policy evolution, thereby effectively coping with diverse traffic scenarios, including occasional anomalies. Combining the proactive reasoning of the LLM agent with the instantaneous decision-making capabilities of the DRL helps optimize the quality of state perception and the accuracy of reward modeling during task offloading, thereby improving the system's robustness in non-stationary dynamic scenarios.

[0005] Although LLM agents can provide rich state observations and policy considerations, building a SAGVN task offloading framework that enhances LLM agents still faces the following challenges, considering the time-scale heterogeneity between LLM and DRL, and the continuous deviation between prior knowledge and the real environment:

[0006] 1) Accuracy and reliability of semantic vector generation by LLM agents. LLM agents rely on multi-source knowledge such as terrain features and node load to generate semantic vectors to enable vehicle-to-everything (V2X) task offloading. During semantic extraction, important feature compression, and numerical mapping, agents are prone to information shifts or representational distortions. [3] Sun et al. [4] A multi-satellite MEC (Mobile Edge Computing) task offloading and resource allocation method based on LLM context learning capability is proposed. Network state information is encoded into natural language and input into the LLM as prompt words to generate offloading decisions. (Zhu et al.) [5] A method for UAV-assisted edge computing enhanced with LLM is proposed, which utilizes the semantic encoding capability of LLM to improve the agent's correlation modeling of the global state, thereby increasing the task completion rate and convergence speed. However, in the above method, the semantic signals generated by LLM and the feedback from the real environment do not form a closed-loop correction. How to improve the long-term decision reliability of SAGVN task offloading remains to be explored.

[0007] 2) Asynchronous Collaboration between LLM Agent Semantic Reasoning and DRL Numerical Optimization. The high-level semantic cognition of the LLM agent and the numerical iterative optimization of the DRL differ significantly in time scale, representation space, and update rhythm. The reasoning process of the LLM agent involves complex context understanding and multi-step chained reasoning, with computational overhead far exceeding that of the single-step policy update in the DRL. Calling the agent in every decision slot would introduce unacceptable reasoning delays. [6] CARD frame [7] The reward function code for reinforcement learning is generated and optimized through LLM iterations, and unnecessary RL training rounds are skipped through a trajectory preference evaluation mechanism to reduce interaction overhead. However, the high inference latency of LLM cannot meet the millisecond-level scheduling requirements of vehicle-to-everything (V2X) communication. Han et al. [8] A six-DOF flight control method based on LLM-guided DRL is proposed. LLM is used as a prior knowledge source to provide initial policy suggestions for DRL, thereby accelerating training convergence. However, since LLM guidance is only injected once in the early stages of training, semantic reasoning and policy learning cannot continuously coordinate during runtime. Summary of the Invention

[0008] To address the aforementioned challenges, this invention proposes a robust DRL framework enhanced with an LLM Agent, aiming to maximize task offloading completion rate in non-stationary vehicular network environments. Within this framework, the proactive planning of the LLM Agent and the experience-driven optimization capabilities of the DRL are integrated, achieving deep synergy between knowledge-driven and data-driven approaches.

[0009] Specifically, this invention is a task offloading method for agent-assisted DRL oriented vehicle-edge collaborative reasoning:

[0010] In the SAGVN scenario, which comprises vehicle terminals, ground base stations, drone base stations, LEO (Low Earth Orbit) satellite base stations, and MEC (Multi-access Edge Computing) controllers, the following components are integrated into a space-air-ground vehicle-to-everything (SAGVN) network:

[0011] Ground base stations, drone base stations, and low Earth orbit (LEO) satellite base stations serve as edge nodes; Let i be the set of vehicles i. , and These represent the sets of ground base stations, drone base stations, and LEO base stations, respectively.

[0012] The MEC controller is deployed in the network core layer of SAGVN and is responsible for global state aggregation, task queue management and offload policy scheduling.

[0013] The MEC controller includes an LLM Agent and a DRL scheduler;

[0014] LLM Agent: The Large Language Model (LLM) is constructed as an intelligent agent with active reasoning and feedback perception capabilities;

[0015] DRL Scheduler: A scheduling system based on Deep Reinforcement Learning (DRL);

[0016] The LLM Agent first receives external environment information and stores and maintains the historical scheduling experience of the MEC controller. Using the external environment information and historical scheduling experience as joint inputs, it comprehensively assesses the risk status of the current traffic scenario through the semantic understanding and chain reasoning capabilities of LLM. Finally, the reasoning result is transformed into a semantic vector that is directly used by the DRL scheduler.

[0017] During the task unloading and scheduling process: based on the vehicle Based on the location status and channel conditions, the task reasoning adopts two modes: local reasoning and cooperative reasoning. The former relies on the on-board computing unit to complete the task independently, while the latter migrates the computing load to ground base stations, UAV base stations or LEO base stations for execution.

[0018] A dual-timescale mechanism is employed to coordinate the real-time response of the agent's semantic reasoning and task unloading. Specifically, the timeline is divided into several equal-length cache periods. ; The internal structure is further subdivided into multiple task scheduling slots. ;

[0019] During cache cycle Initially, the agent integrates prior knowledge and real-world interaction experience to generate semantic vectors for enhanced collaborative reasoning and task scheduling; these semantic vectors are written into the agent's memory cache for reuse in various scheduling slots within the cache window; during the cache period... At the end, a statistical summary of the accumulated interaction experience. It is used as experience feedback to the Agent for cognitive correction and dynamic updating of semantic vectors;

[0020] In scheduling time slots Inside, the inference tasks generated by the vehicle terminal are uploaded to the base station;

[0021] Combining network conditions, vehicle locations, and semantic vectors and reward signals output by the Agent, the DRL scheduler makes task offloading decisions, and finally the MEC controller executes batch task offloading decisions.

[0022] This invention has the following three main contributions:

[0023] First, a dual-timescale LLM Agent-assisted task scheduling framework. The LLM Agent is integrated into the MEC controller to understand the traffic environment and task scheduling semantics. This framework drives the LLM Agent to perform semantic reasoning and update the cached results in a cache window period. Each scheduling slot within the cache window directly reuses the cached output, ensuring real-time scheduling performance while also taking into account the deep semantic understanding of complex scenarios.

[0024] Second, semantically enhanced DRL offline training. The agent assists DRL training from two dimensions: expanding the state space and enriching reward signals. During the cache period, the agent performs semantic vector generation and dynamic reward adjustment, continuously providing high-quality enhanced state representations and adaptive reward signals for DRL. Guided by the enhanced state and reward signals, the decision-making quality and risk adaptation capability of DRL in complex and dynamic traffic scenarios are effectively improved.

[0025] Third, the agent's memory is continuously updated. At the end of each cache cycle, the offloading strategies generated by multiple rounds of DRL are aggregated into a structured experience summary and fed back to the agent. By continuously integrating historical and latest experiences, the agent's memory module enables the agent to continuously evolve its perception capabilities without the need for parameter fine-tuning. Attached Figure Description

[0026] Figure 1 This represents a SAGVN scenario based on LLM Agent and DRL;

[0027] Figure 2 This represents a dual-timescale task scheduling framework based on an LLM Agent.

[0028] Figure 3 This represents the Agent-DRL intelligent agent framework;

[0029] Figures 4(a) to 4(c) show the impact of training rounds on different unloading performances, where...

[0030] Figure 4(a) shows the effect of training rounds on rewards.

[0031] Figure 4(b) shows the impact of training rounds on the task failure rate.

[0032] Figure 4(c) shows the probability density distribution of the reward value;

[0033] Figures 5(a) and 5(b) respectively show the impact of cache window size on different unloading performance.

[0034] Figure 5(a) illustrates the impact of cache window size on task failure rate.

[0035] Figure 5(b) illustrates the impact of cache window size on inference latency;

[0036] Figure 6 This indicates the impact of scenario risks on task failure rate;

[0037] Figure 7 This indicates the impact of the amount of spectrum resources on the success rate of the mission. Detailed Implementation

[0038] 1 Overview

[0039] To overcome the limitations of inaccurate reward signals and accumulated perception biases in non-stationary traffic environments, this invention proposes a task scheduling (unloading) method for Agent-enhanced DRL oriented towards vehicle-side collaborative reasoning.

[0040] This method employs a dual-timescale collaborative scheduling mechanism to decouple the agent's proactive reasoning from the numerical optimization of deep reinforcement learning at the cache window level, achieving deep semantic understanding of complex traffic scenarios while ensuring real-time scheduling performance. Within each cache period, the agent generates traffic semantic vectors and dynamic reward weights, which are used to enhance the state representation of reinforcement learning and drive the adaptive adjustment of the reward function, respectively.

[0041] Based on this, the present invention designs a deep reinforcement learning-assisted memory continuous update mechanism, which aggregates real interaction experience and feeds it back to the agent's memory module to realize the online evolution of perception ability without the need for fine-tuning of model parameters.

[0042] Simulation results show that the proposed method significantly outperforms mainstream model optimization and DRL baselines in terms of task success rate, average inference latency, and adaptability to non-stationary scenarios.

[0043] 2 System Model

[0044] like Figure 1 As shown, this scenario considers a space-air-ground integrated vehicle-to-everything (SAGVN) network composed of vehicle terminals, ground base stations, drones, LEOs (Legion-Oriented Arrays), and agent-based edge controllers. The set of vehicles is represented as... The set of edge nodes is represented as ,in , and These correspond to the sets of ground base stations, drone nodes, and low-Earth orbit satellites, respectively. The edge controller is deployed in the network core layer and is responsible for global state aggregation, task queue management, and offloading strategy scheduling. LLMAgent is integrated into the controller, semantically understanding traffic conditions and assessing scenario risks, providing knowledge-level support for the formulation of offloading strategies.

[0045] 2.1 Traffic Perception Based on LLM Agent

[0046] To support perception, understanding, and adaptive scheduling decisions in complex and dynamic traffic environments, the core architecture of the LLM Agent consists of four modules: perception, memory, reasoning, and generation.

[0047] 1) Road Network Environment Perception: Responsible for receiving external environment information, it serves as the input layer for the Agent's cognition. This module accesses three types of traffic knowledge: terrain communication descriptions in the road network knowledge base. Description of peak-hour traffic congestion communication And information on emergencies sensed in real time by IoT sensors and RSUs. The above three types of knowledge together constitute the agent's complete perceptual input of the current traffic environment.

[0048] 2) Scheduling Experience Memory: This module stores and maintains the historical scheduling experience of the edge controller. It continuously updates the actual interaction memory of the edge controller with each collaborating node during historical scheduling, using cache cycles as the unit. At the end of each cache cycle, the task scheduling results are statistically aggregated by the memory module to form an experience feedback summary that can be read by the inference module. .

[0049] 3) Scene Semantic Reasoning: Using traffic knowledge input from the perception module and historical experience maintained by the memory module as joint inputs, this module comprehensively assesses the risk status of the current traffic scene through LLM's semantic understanding and chain-like reasoning capabilities. The reasoning in this module follows a chain-like paradigm of "environmental understanding, risk quantification, and instruction generation," outputting structured scene analysis results to provide decision-making support for the action module.

[0050] 4) Semantic Vector Generation: The reasoning results of the reasoning module are converted into semantic vectors that can be directly used by the DRL, driving the controller to make more accurate and risk-adaptive task offloading decisions in complex dynamic scenarios.

[0051] 2.2 Agent-Assisted Edge Computing

[0052] like Figure 2 As shown, this section designs an LLM agent-assisted edge computing framework for task offloading and scheduling. In this framework, the LLM is constructed as an agent with proactive reasoning and feedback perception capabilities. Based on the vehicle... Based on the location status and channel conditions, task inference can choose between two modes: local inference and collaborative inference. The former relies on the on-board computing unit to complete the task independently, while the latter migrates the computing load to ground base stations, UAV nodes, or satellite platforms for execution.

[0053] The framework introduces a dual-timescale mechanism to coordinate the real-time response of the agent's semantic reasoning and task offloading. The timeline is divided into several equal-length cache windows (also known as "cache cycles"). The internal structure is further subdivided into multiple task scheduling slots. In the cache window Initially, the agent integrates prior knowledge and real-world interaction experience to generate semantic vectors that enhance collaborative reasoning and task scheduling. These semantic vectors are written to the agent's memory cache for reuse across scheduling slots within the cache window, effectively avoiding the inference latency introduced by repeatedly calling LLM in each scheduling slot. Within the cache window... At the end, a statistical summary of the accumulated interaction experience. This experience is fed back to the agent for cognitive correction and dynamic updating of semantic vectors. (During scheduling slots...) Inside, the inference tasks generated by the vehicle are uploaded to the base station. Combining network conditions, vehicle location, and semantic vectors and reward signals output by the agent, the edge controller performs batch task offloading decisions.

[0054] 2.3 Communication model

[0055] This section constructs a time-varying channel communication model to quantify the impact of vehicle movement characteristics on channel gain. Let Indicates vehicle With base station The time-varying decay factor. Assume at time... ,vehicle With base station The Euclidean distance between them is expressed as Channel gain between the two [9] Represented as

[0056] (1)

[0057] in, This is a reference distance, used as a benchmark for calibrating channel gain; For reference distance The corresponding channel gain; This is the path loss exponent, determined based on the base station type. Let... and For vehicles and base stations The transmit power. Spectrum resources in the base station are allocated to each vehicle in units of mutually orthogonal sub-channels.

[10]

[11] Assume the bandwidth of each sub-channel is... .make Indicates assignment to autonomous driving tasks The number of sub-channels (e.g., a street view image). Then, the number of vehicles... to base station Submit Task The uplink transmission rate at that time is calculated as follows:

[0058] (2)

[0059] in, This represents the average background noise. Let... Indicates assignment to task The number of sub-channels in the calculation result (e.g., image classification or detection result). Then, the base station... To the vehicle Return mission The downlink transmission rate at that time was predicted to be

[0060] (3).

[0061] 2.4 End-to-end inference latency modeling

[0062] The MEC controller needs to comprehensively consider factors such as vehicle location, base station load, and task latency when scheduling tasks. This section quantifies the service latency of tasks based on queuing theory. Autonomous driving tasks. Includes a set of characterization parameters ,in Represents the size of the task data (bits). Indicates task The time delay constraint (seconds).

[0063] A. Task unloading delay

[0064] Task offloading latency represents the time required for the receiving base station to transmit the autonomous driving task to the controller offloading queue and then to the cooperating base station processing queue. (Base station) The set of autonomous driving tasks collected within the coverage area is represented as The total number of elements in the set is .make Indicates vehicle With base station A connection was established, otherwise Within the MEC controller's coverage area, the average time for a task to be uploaded from the vehicle to the base station is calculated as follows:

[0065] (4).

[0066] The arrival of a task between a single vehicle and a base station is modeled as a Poisson process. Let the vehicle... The arrival rate of autonomous driving tasks is The arrival rate of tasks in the MEC controller's unloading queue is...

[0067] (5)

[0068] The offload queue processes only one autonomous driving task at a time. Task arrivals in the offload queue are modeled as an M / M / 1 queue model. To reflect the queue's busy level, the service intensity of the MEC controller's offload queue is defined as...

[0069] (6)

[0070] The enqueue rate for unloading tasks is determined by the task arrival rate, while the dequeue rate is determined by the transmission rate. When the enqueue rate exceeds the dequeue rate, tasks will accumulate, eventually leading to queue overflow. Therefore, to ensure queue stability, the service intensity must meet certain requirements.

[0071] (7)

[0072] In the unloading queue, the task is in the queue. The previous set of tasks is represented as Assuming the vehicle The generated autonomous driving task The latency from the base station to the controller and from the controller to the cooperating base station is Autonomous driving task The unloading delay is calculated as follows

[0073] (8).

[0074] B. Task reasoning delay

[0075] Task inference latency refers to the time it takes for an autonomous driving task to be processed from the time it enters the processing queue of the cooperating base station until it is completed. For locally processed tasks, task inference latency is the time it takes for the task to be completed in the local processing queue on the vehicle. Let a binary variable... ( The task is unloaded to the collaborator. (Processed locally on the vehicle; otherwise, the value is 0.) autonomous driving task Offload to cooperative base station The time spent on reasoning is expressed as .vehicle Local reasoning task The time spent is expressed as Therefore, cooperative base stations The average processing time for all autonomous driving tasks in the queue is calculated as follows:

[0076] (9)

[0077] The MEC controller assigns tasks from the offload queue to different collaborators for processing. Each collaborator processes the arrival of tasks in the queue according to a Poisson process. Let a binary variable... The controller will represent the vehicle Generated tasks Assigned to cooperative base stations Process, otherwise 0. The controller assigns the value to the base station. The percentage of tasks is expressed as

[0078] (10)

[0079] Cooperative base stations The arrival rate of tasks in the processing queue The arrival of tasks in the processing queue is modeled as an M / M / 1 queue model. Based on equations (5), (9), and (10), the cooperative base station... The service strength of the processing queue is defined as

[0080] (11)

[0081] To ensure queue stability, the service strength for processing the queue must meet the following requirements.

[0082] (12)

[0083] Tasks in the cooperative base station processing queue The previous task index set was Assuming the vehicle The generated autonomous driving task The latency required for processing by the cooperating base station or vehicle is Then the task Task inference latency calculation is

[0084] (13).

[0085] C. Result return delay

[0086] The result return latency represents the time required to transmit the result data from the cooperating base station back to the vehicle after the task inference is completed. Local processing tasks perform task inference on the vehicle side, so no task return is required. According to equation (3), the cooperating base station... Mid-autonomous driving tasks Send back to the vehicle The latency for the result return in this process is calculated as follows:

[0087] (14)

[0088] Task The total latency includes task unloading latency, task processing latency, and task return latency, which is the sum of equations (8), (13), and (14), expressed as:

[0089] (15).

[0090] 2.5 Problem Modeling

[0091] Define cache window Internal task completion indicator variables:

[0092] (16)

[0093] If task In delay constraints If the internal process is successfully completed, then ,otherwise Cache window The duration is expressed as Cache window Within, the set of task scheduling strategies is represented as The problem of maximizing the long-run average completion rate of autonomous driving tasks in the system is modeled as follows: .

[0094] :

[0095] (17a)

[0096] (17b)

[0097] (17c)

[0098] (17d)

[0099] (17e)

[0100] (17f)

[0101] Constraint (17a) ensures that each vehicle can only connect to a single base station for transmission and cannot establish connections with multiple base stations simultaneously. Constraint (17b) ensures that each task can only be assigned to one cooperating base station for processing. Under constraint (17c), each task can only select one task processing mode. Constraint (17d) ensures that the selected cooperating base station can continuously cover the vehicle throughout the entire task processing and result transmission process, especially in high-speed moving scenarios, to avoid task failure due to the vehicle moving out of the base station's coverage area. Constraints (17e) and (17f) ensure the stability of the controller offload queue and the processing queues of each base station.

[0102] 3. Method Design

[0103] like Figure 3 As shown, the Agent and DRL in the Agent-DRL intelligent agent framework cooperate deeply across two time scales. During the cache cycle... Initially, the LLM Agent integrates road network knowledge, traffic situation data, and historical experience summaries from the memory module to generate two types of decision support information: traffic semantic vectors to enhance the state representation of the DRL, and dynamic reward weights to drive the reward function to adaptively adjust according to scenario risk. During each scheduling slot within the cache period, the DRL reads this information from the cache, combines it with environmental observations to construct enhanced states, and executes policy training and offloading decisions. Within the cache period... At the end, the interactive experiences accumulated by DRL are aggregated into a structured experience summary. The feedback is sent to the Agent memory module to correct the semantic vector generation for the next cycle, covering knowledge guidance, DRL execution, experience feedback, and memory update, forming a closed-loop optimization link.

[0104] 3.1 Semantic Vector Generation and Dynamic Reward Adjustment

[0105] The agent deeply perceives and assesses the risks of the current traffic scenario, transforming heterogeneous traffic knowledge into structured decision-making basis that can be directly used by the DRL. Based on this, this section designs an agent-driven semantic vector and dynamic reward generation mechanism, which uniformly generates two types of decision support information at the beginning of each cache window: traffic semantic vectors and dynamic reward weights, providing reliable knowledge support for task scheduling decisions.

[0106] (1) Traffic semantic vector generation

[0107] During cache cycle Initially, the agent's perception module acquires three types of traffic knowledge. These correspond to terrain communication descriptions, peak-hour congestion descriptions, and real-time emergency information, respectively. This is based on the aforementioned knowledge and the experience feedback summary stored in the memory module. For joint input, edge nodes The traffic semantic vector is normalized to

[0108] (18)

[0109] pass With the introduction of DRL, the agent no longer relies solely on prior knowledge for reasoning, but incorporates the interaction experience accumulated by DRL in the real environment into the semantic vector generation process. It can continuously track the actual communication status of each collaborative node, making up for the lack of timeliness of pure prior reasoning in non-stationary scenarios.

[0110] (2) Dynamic reward weight generation

[0111] To address the issue of fixed reward weights failing to adapt to dynamic environments, the Agent quantifies and evaluates the current cache window. The overall risk level of the scenario in which the vehicle is located is represented by the generated scenario risk index.

[0112] (19)

[0113] The prompt template guides the LLM to assess the risks of the current scenario from three dimensions: 1) the impact of terrain occlusion on communication stability; 2) the pressure level of traffic density on base station queues during the current time period; and 3) the urgency of unforeseen events on task latency constraints. The assessment results from the three dimensions are aggregated into a single risk index through internal reasoning within the agent reasoning module. , The closer it is to 1, the higher the risk level of the current scenario.

[0114] based on Agent generation and edge nodes Dynamic performance weights that match scenario risks and loss weight The two weight generation functions are defined as follows:

[0115] (20)

[0116] and

[0117] (twenty one)

[0118] in and These are the preset baseline performance and baseline loss weights, respectively. The above mapping relationship has the following characteristics: when... (In high-risk scenarios) Performance rewards are narrowed, and agents receive less incentive for successful tasks, making them more inclined to choose collaborators with low risk and high reliability; at the same time The penalties for failure are increased, and tolerance for decision-making errors is reduced. When (In low-risk scenarios) Performance rewards are restored to normal levels to encourage agents to actively pursue efficiency; The punishment is moderate, allowing for appropriate decision-making exploration. This two-way adaptive mechanism enables differentiated guidance of agent behavior under different risk levels, ensuring that reward and punishment signals accurately reflect the decision-making cost structure of the current scenario.

[0119] 3.2 Offline Training of Semantic Augmentation DRL

[0120] Based on the traffic semantic vector and reward weights output by the agent, the task offloading problem is modeled as an extended Markov decision process. This scheme enhances the DRL training process from two dimensions: state representation and reward signal, forming an offline training mechanism that promotes mutual reinforcement between knowledge-driven and experience-driven approaches.

[0121] State space S: rounds Corresponding state This includes real-time information on vehicles, tasks, and base station load, as well as traffic semantic vectors generated by the Agent. In the round The state is represented as

[0122] (twenty two)

[0123] Action Space A: The edge controller filters the set of candidate collaborators that continuously cover the vehicle during task execution. In round... The task unloading action is represented as

[0124] (twenty three)

[0125] in Representative round The set of task unloading decisions within.

[0126] Cumulative Reward R: The Agent generates a scenario risk index at the beginning of each cache period. Based on this, dynamic performance weights are calculated. With loss weight (See equations (20) and (21)). In round In the context of the task, the reward for successfully completing it within the time delay constraint is...

[0127] (twenty four)

[0128] The penalty is as follows if the task fails to be completed within the time constraint:

[0129] (25)

[0130] In state Next action The total reward received is:

[0131] (26)

[0132] This reward design has dual adaptive characteristics. On the one hand, dynamic weights... and Risk index based on scenario Real-time adjustments are made to match the intensity of rewards and penalties with the level of danger in the current traffic situation; on the other hand, perception factors... After correction based on DRL experience feedback, the signal more closely reflects the actual communication status of each collaborating node, further ensuring the accuracy of the reward signal. The cumulative expected reward for DDQN with discounts is calculated as follows:

[0133] (27)

[0134] in As a discount factor, The total number of steps (rounds) for training. For all possible unloading sets, the training objective is to maximize the cumulative expected reward of the discount. DDQN maintains the following parameters: and The policy network and target network employ a dual-network mechanism to separate action selection and value evaluation. Within each decision slot, the interaction quadruple is stored in the experience replay pool. During training Random sampling batch This breaks the temporal correlation between samples. The optimal action for selecting the next state by the policy network is represented as follows:

[0135] (28)

[0136] The Q-value of this action is evaluated by the target network and calculated as follows:

[0137] (29)

[0138] The mean square error between the target Q value and the predicted Q value is

[0139] (30)

[0140] The above formula is used to construct the loss function of the policy network. By minimizing... Gradient updates are performed on the policy network parameters, and the target network parameters are updated. Every The algorithm synchronizes with the policy network once per step to maintain training stability. The offline training process for semantic augmentation is summarized as Algorithm 1.

[0141]

[0142] 3.3 DRL-assisted agent memory is continuously updated

[0143] For vehicle unloading decisions, the agent generates a traffic semantic vector and a state space and reward signal for a Dynamically Weighted Reward Stream (DRL). To improve the LLM's ability to perceive and utilize real-world interaction experience from the DRL, this section designs a DRL-assisted agent memory update mechanism. This mechanism executes on a rolling basis in cache cycles, forming a closed-loop chain of "Agent semantically enhancing DRL, DRL feedback improving Agent," enabling the agent to continuously evolve its perception capabilities without fine-tuning model parameters.

[0144] (1) Construction of experience summary

[0145] The original DRL interaction quadruple cannot be directly used as natural language input for LLM for two reasons: First, dimensionality redundancy. The state vector contains a large number of numerical features such as vehicle position, speed, channel gain, and queue load, which are high in dimensionality and contain a lot of redundant information, exceeding the context range that LLM can effectively process. Second, noise interference. Single-step interaction data fluctuates greatly due to environmental randomness, and directly inputting it into LLM can easily lead to unstable inference results.

[0146] Therefore, cache cycle The interaction data within the nodes was statistically aggregated into three types of interpretable statistics. Collaborating nodes The summary of experience feedback is represented as follows:

[0147] (31)

[0148] in, For period Inner selection edge nodes The average reward obtained when the node is used as an unloading target reflects the overall service quality actually provided by the node. The corresponding task failure rate measures its reliability in actual interaction; This represents the difference between the average reward of the current period and the previous period, characterizing the dynamic trend of the node's performance. The physical meanings of the three types of statistics are complementary: Reflects the absolute performance level of the node. Its reliability can be independently measured from the perspective of task completion. This captures the evolution of node performance over time, and the three together constitute a multi-dimensional characterization of the real interactive performance of each collaborating node.

[0149] (2) Memory-weighted update

[0150] Considering the non-stationarity of traffic conditions, a single-period experience summary may deviate from the true long-term performance of nodes due to short-term unforeseen events, while completely discarding historical experience would result in the loss of accumulated valuable knowledge. Therefore, historical experience and the latest experience are merged and updated by the memory module using an Exponential Moving Average (EMA) strategy.

[0151] (32)

[0152] in The forgetting factor controls the degree to which historical experience is retained. The experience stored in the memory module can gradually fade outdated information in an exponential manner, while retaining historical information that is valuable for reference in the current environment, thus achieving a balance between stability and adaptability.

[0153] Through this mechanism, the agent's perception capabilities can be continuously corrected as real-world interaction experience accumulates, without the need for fine-tuning model parameters. Compared to pure prior knowledge reasoning, this mechanism effectively compensates for issues such as insufficient timeliness of the knowledge base and biases in prior reasoning. Based on this, the agent provides a more accurate representation of the environmental state for DRL decisions in the next cache cycle, improving the robustness of decisions in complex dynamic scenarios.

[0154] 4 Experimental Design

[0155] The hardware environment, software stack, and road network data used in the simulation experiment are consistent with those in Chapter 3; detailed configurations are provided in Section 3.4.1. Based on this, and considering the characteristics of the LLM Agent-enhanced DRL method proposed in this invention, the simulation platform is further configured as follows.

[0156] The LLM Agent uses Qwen3-8B as its base model and is encapsulated as an independent microservice within the FastAPI framework, deployed on a local inference server. It receives perceptual input through a unified interface and outputs traffic semantic vectors and scenario risk indices. To reduce the additional latency introduced by LLM inference, the Agent employs a dual-timescale mechanism, performing semantic inference periodically in cache cycles. Within each cache cycle, the cached results are directly reused across scheduling slots, thus decoupling the LLM inference overhead from task scheduling decisions. Regarding the memory update module, at the end of each cache cycle, the interaction quadruples accumulated by DRL are statistically aggregated and converted into structured natural language descriptions. These descriptions are then weighted and fused using EMA and written into the Agent's memory module for use in generating semantic vectors for the next cycle. (Forgetting factor) The value was set to 0.6 to strike a balance between historical stability and recent response speed. The complete configuration of the experimental parameters is shown in Table 1. This section designs baseline comparison experiments and ablation experiments. The ablation experiment verifies the independent contribution of each component by progressively removing key modules; the specific configuration is shown in Table 2. The baseline comparison experiment selects three representative methods as performance benchmarks, including:

[0157] Baseline-1: Task scheduling is performed using the maximum signal-to-noise ratio (Max-SINR) criterion.

[12] During each scheduling decision, the edge controller measures the channel quality of each cooperating node in real time and assigns the task to the base station with the best current signal-to-noise ratio for execution, without the need for model training.

[0158] Baseline-2: Uses the DQN algorithm to make unloading decisions.

[13] This method constructs an observation space based on the current channel gain and the load status of each base station, and uses a deep Q-network to perform end-to-end optimization of the offloading strategy.

[0159] Baseline-3: Employs the DDQN algorithm as the offloading decision framework.

[14] This method, based on Baseline-2, further incorporates the vehicle's real-time position, speed, acceleration, and direction information into the state representation, and uses a dual-network mechanism to separate action selection and value evaluation, thus alleviating the problem of Q-value overestimation.

[0160] Baseline-4: This method integrates static road network knowledge and dynamic traffic situation based on the MCP framework. It incorporates the environmental perception factors generated by LLM into the training and inference process of DDQN in two ways: state enhancement and action space constraint. This serves as a direct comparison benchmark with the method of this invention.

[0161] Table 1 Default parameter settings

[0162]

[0163] The ablation settings of the proposed method are shown in Table 2.

[0164]

[0165] 5. Simulation Experiments and Result Analysis

[0166] 5.1 Impact of Training Rounds on Performance

[0167] Figures 4(a) to 4(c) illustrate the impact of training epochs on task unloading performance and the probability density distribution of the final reward value under different methods. The DRL algorithm learning rate was set to 0.005, and approximately 4300 data points were used for training. Figure 4(a) shows the impact of training epochs on cumulative reward. Proposed-8 consistently achieved the highest cumulative reward, stabilizing at approximately 3042 after convergence. This is attributed to the synergistic effect of the dynamic reward mechanism and the continuous memory update mechanism. The cumulative reward of Proposed-9 converged to approximately 2902, about 4.6% lower than that of Proposed-8. As training progressed, Proposed-9 continuously fed back experience summaries to the LLM Agent through the memory update mechanism, and the perceptual factor was continuously corrected, gradually widening the gap with Proposed-11.

[0168] Figure 4(b) illustrates the impact of training rounds on the task failure rate. The failure rate of Proposed-8 rapidly decreased from approximately 43% to approximately 18% in the first 10 rounds, eventually converging to approximately 6.8%. This indicates that the dynamic reward mechanism can deliver accurate risk feedback to the agent in the early stages of training, accelerating policy convergence. Proposed-9 eventually converged to approximately 8.6%, about 1.8 percentage points higher than Proposed-8, mainly due to insufficient penalty from the fixed reward weights in high-risk scenarios. Proposed-11 converged to approximately 10.4%, with the lack of experience summary feedback preventing the correction of perceptual biases.

[0169] Observing the probability density distribution of reward values ​​in Figure 4(c), the peak of the reward distribution of Proposed-8 is furthest to the right, and it exhibits the narrowest distribution width and the highest concentration, indicating that the agent driven by dynamic rewards and memory updates improves the stability of the unloading decision quality. The distribution of Proposed-11 is relatively wide and the peak is relatively low, and the cumulative bias of the perceptual factor leads to an increase in reward dispersion. The peak of Baseline-3 is furthest to the left and the distribution is the most diffuse, which reflects that the DRL method without knowledge guidance has insufficient decision stability in complex dynamic scenarios and is easily affected by environmental fluctuations.

[0170] 5.2 The impact of cache windows on performance

[0171] Figures 5(a) and 5(b) illustrate the impact of cache window size on task failure rate and average inference latency. The experiment gradually increased the cache window from 3 seconds to 13 seconds to examine its dual effect on system performance. From the trend of task failure rate in Figure 5(a), all methods exhibited a U-shaped curve characteristic of first decreasing and then increasing, with the optimal window appearing around 7 seconds. Proposed-8 achieved the lowest failure rate of 6.8% with a window of 7 seconds, a decrease of 9 percentage points compared to 15.8% with a window of 3 seconds. This is because an excessively small cache window leads to excessively high agent inference frequency, with LLM inference overhead consuming a large amount of scheduling time and compressing the available time window for task transmission and processing; while an excessively large cache window reduces the timeliness of semantic vectors and dynamic reward weights, failing to reflect changes in traffic conditions in a timely manner. When the window size increased to 13 seconds, the failure rate of Proposed-8 rebounded to 18.5%, while the failure rate of Proposed-10 rose to 24.7% due to the lack of a memory update mechanism. This indicates that the memory update mechanism is more valuable under larger cache windows, as it can continuously correct semantic vector biases through empirical feedback. Baseline-4 has a failure rate of 11.6% with a 7-second window, approximately 4.8 percentage points higher than Proposed-8, validating the superiority of the proposed method over RAG knowledge injection. Baseline-3, not involving LLM inference, has a fixed failure rate of 15.5%, outperforming methods with LLM when the window is small. However, when the window reaches 7 seconds, the failure rate of Proposed-8 is significantly lower than this baseline.

[0172] As shown in Figure 5(b) regarding the change in average inference latency, the inference latency of all methods monotonically decreases as the cache window increases. This is because a larger cache window means a lower frequency of Agent inference calls, and the LLM inference overhead is distributed across more scheduling slots. The latency of Proposed-8 decreased from 318ms with a 3-second window to 122ms with a 13-second window, a reduction of 61.6%. Baseline-3, which does not call LLM, has a constant latency of 98ms, representing the lower bound of semantic inference overhead. Proposed-8 has a latency of 198ms with a 7-second window, an increase of about 100ms compared to Baseline-3, while the failure rate decreased by 8.7 percentage points, indicating that a moderate inference latency overhead is traded for a significant improvement in decision quality.

[0173] 5.3 Impact of Reward Mechanism on Performance

[0174] To verify the adaptive capability of the dynamic reward mechanism, this section designs three typical scenarios under different risk levels, as shown in Table 3.

[0175] Table 3 Risk Classification for Typical Scenarios

[0176]

[0177] Figure 6 The changes in task failure rates under different risk levels in various scenarios are illustrated. In low-risk scenarios, the failure rates of Proposed-8 and Proposed-9 are 4.2% and 4.8%, respectively, with a difference of only 0.6 percentage points, indicating that the gain of the dynamic reward mechanism is limited when communication conditions are good. In medium-risk scenarios, mountainous tunnel terrain significantly obstructs ground base stations, with the failure rate of Proposed-8 at 7.8% and Proposed-9 rising to 11.3%, a difference of 3.5 percentage points. The agent guides the controller to prioritize cooperative nodes unaffected by terrain. In high-risk scenarios, traffic accidents ahead cause a surge in base station load, with the failure rate of Proposed-8 at 10.5% and Proposed-9 rising to 17.2%, a difference of 6.7 percentage points. After identifying high risks, the agent significantly compresses performance weights and increases loss weights, making the agent more inclined to choose conservative and reliable offloading strategies. The failure rates of Baseline-3 and Baseline-1 are 22.8% and 38.5%, respectively, indicating that methods lacking knowledge enhancement have insufficient decision-making ability in emergency scenarios.

[0178] 5.4 Impact of Resource Quantity on Performance

[0179] Figure 7 The changes in task completion rates for each method under different amounts of spectrum resources are shown. As the number of sub-channels increases from 10 to 22, the task completion rates of all methods show an upward trend, but the rate of increase differs significantly from the final performance. Proposed-8 consistently maintains the highest task completion rate across the entire resource range, stabilizing at approximately 92.1% after convergence. Baseline-1's completion rate ultimately reaches only 57.0%, reflecting the limitations of purely heuristic strategies in non-stationary traffic scenarios. A comparison is made between Proposed-8 and Proposed-10, which also employ a dynamic reward mechanism. With limited resources, the difference between the two is relatively limited; however, as the number of sub-channels increases, the difference gradually widens, reaching approximately 7.1% when the number of sub-channels is 22. This trend indicates that when communication transmission is no longer the primary bottleneck, the difference in decision quality becomes fully apparent. The memory update mechanism continuously drives the agent's memory update by leveraging the interaction experience accumulated by DRL in real-world environments, effectively correcting the discrepancy between prior knowledge and actual traffic conditions, thus bringing continuous performance gains.

[0180] 6. General Section

[0181] To address the issues of inaccurate reward signals and accumulated perception biases in task offloading decisions within an integrated air-space-ground vehicle-to-everything (V2X) environment, this paper proposes an Agent-enhanced DRL (Depth Regression-Regression) task offloading method. At the methodological level, this invention upgrades the LLM Agent from a static knowledge-injecting role to a dynamic collaborator with proactive reasoning and experience feedback capabilities. Through a dual-timescale collaborative scheduling mechanism, the Agent and DDQN achieve efficient decoupling at the cache window level, ensuring real-time scheduling performance while also considering deep semantic understanding in complex scenarios. The DRL-assisted continuous memory update mechanism enables the agent to achieve online evolution of perception capabilities without parameter fine-tuning, overcoming the limitations of traditional methods that rely on knowledge base updates or model fine-tuning. At the experimental level, ablation studies show that both the dynamic reward mechanism and the memory update mechanism make significant and independent contributions to overall performance, with optimal results when they work together. Comparative experiments further verify the comprehensive advantages of the proposed method compared to pure heuristic strategies and traditional DRL methods, especially demonstrating a significant improvement in decision robustness under high-risk, sudden event scenarios.

[0182] References

[0183] [1]Wang L, Ma C, Feng X, et al. A Survey on Large Language ModelBased Autonomous Agents[J]. Frontiers of Computer Science, 2024, 18(6):186345.

[0184] [2]Yao S, Zhao J, Yu D, et al. ReAct: Synergizing Reasoning andActing in Language Models[C] / / International Conference on LearningRepresentations (ICLR), 2023.

[0185] [3]Huang L, Yu W, Ma W, et al. A Survey on Hallucination in LargeLanguage Models: Principles, Taxonomy, Challenges, and Open Questions[J]. ACMTransactions on Information Systems, 2025, 43(2): 1-55.

[0186] [4]Sun M, Hou J, Qiu K, et al. LLM-Based Task Offloading and ResourceAllocation in Satellite Edge Computing Networks[J]. IEEE Transactions onVehicular Technology, 2025.

[0187] [5]Zhu F, Huang F, Yu Y, et al. Task Offloading with LLM-EnhancedMulti-Agent Reinforcement Learning in UAV-Assisted Edge Computing[J].Sensors, 2024, 25(1): 175.

[0188] [6]Zhao W X, Zhou K, Li J, et al. A Survey of Large Language Models[J]. arXiv preprint arXiv:2303.18223, 2023.

[0189] [7]Sun S, Liu R, Lyu J, et al. A Large Language Model-Driven RewardDesign Framework via Dynamic Feedback for Reinforcement Learning[J].Knowledge-Based Systems, 2025, 326: 114065.

[0190] [8]Han Y, Yang M, Ren Y, et al. Large Language Model GuidedReinforcement Learning Based Six-Degree-of-Freedom Flight Control[J]. IEEEAccess, 2024, 12: 89479-89492.

[0191] [9]Erceg V, Greenstein L J, Tjandra S Y, et al. An Empirically BasedPath Loss Model for Wireless Channels in Suburban Environments[J]. IEEEJournal on Selected Areas in Communications, 1999, 17(7): 1205-1211.

[0192]

[10] Zhang S, Luo H, Li J, et al. Hierarchical Soft Slicing to MeetMulti-Dimensional QoS Demand in Cache-Enabled Vehicular Networks[J]. IEEETransactions on Wireless Communications, 2020, 19(3): 2150-2162.

[0193]

[11] Chen Y, Wang Y, Liu M, et al. Network Slicing Enabled ResourceManagement for Service-Oriented Ultra-Reliable and Low-Latency VehicularNetworks[J]. IEEE Transactions on Vehicular Technology, 2020, 69(7): 7847-7862.

[0194]

[12] Ye H, Li G Y, Juang B H F. Deep Reinforcement Learning BasedResource Allocation for V2V Communications[J]. IEEE Transactions on VehicularTechnology, 2019, 68(4): 3163-3173.

[0195]

[13] Dai F, Liu G, Mo Q, et al. Task Offloading for Vehicular EdgeComputing with Edge-Cloud Cooperation[J]. World Wide Web, 2022, 25(5): 1999-2017.

[0196]

[14] Shen H, Tian Y, Wang T, et al. Slicing-Based Task Offloading inSpace-Air-Ground Integrated Vehicular Networks[J]. IEEE Transactions onMobile Computing, 2023, 23(5): 4009-4024.

Claims

1. A task offloading method for Agent-Assisted DRL oriented to vehicle-side collaborative reasoning, characterized by: exist Vehicle terminals, ground base stations, drone base stations, LEO (Low Earth Orbit) satellite base stations, and MEC (Multi-access Edge Computing) controllers together constitute the integrated air-space-ground vehicle-to-everything (SAGVN) scenario: Ground base stations, drone base stations, and low Earth orbit (LEO) satellite base stations serve as edge nodes; Let i be the set of vehicles i. , and These represent the sets of ground base stations, drone base stations, and LEO base stations, respectively. The MEC controller is deployed in the network core layer of SAGVN and is responsible for global state aggregation, task queue management and offload policy scheduling. The MEC controller includes an LLM Agent and a DRL scheduler; LLM Agent: The Large Language Model (LLM) is constructed as an intelligent agent with active reasoning and feedback perception capabilities; DRL Scheduler: A scheduling system based on Deep Reinforcement Learning (DRL); The LLM Agent first receives external environment information and stores and maintains the historical scheduling experience of the MEC controller. Using the external environment information and historical scheduling experience as joint inputs, it comprehensively assesses the risk status of the current traffic scenario through the semantic understanding and chain reasoning capabilities of LLM. Finally, the reasoning results are transformed into semantic vectors that DRL can directly use. During the task unloading and scheduling process: based on the vehicle Based on the location status and channel conditions, the task reasoning adopts two modes: local reasoning and cooperative reasoning. The former relies on the on-board computing unit to complete the task independently, while the latter migrates the computing load to ground base stations, UAV base stations or LEO base stations for execution. A dual-timescale mechanism is employed to coordinate the real-time response of the agent's semantic reasoning and task unloading. Specifically, the timeline is divided into several equal-length cache periods. ; The internal structure is further subdivided into multiple task scheduling slots. ; During cache cycle Initially, the agent integrates prior knowledge and real-world interaction experience to generate semantic vectors for enhanced collaborative reasoning and task scheduling; these semantic vectors are written into the agent's memory cache for reuse in various scheduling slots within the cache window; during the cache period... At the end, a statistical summary of the accumulated interaction experience. It is used as experience feedback to the Agent for cognitive correction and dynamic updating of semantic vectors; In scheduling time slots Inside, the inference tasks generated by the vehicle terminal are uploaded to the base station; Combining network conditions, vehicle locations, and semantic vectors and reward signals output by the Agent, the DRL scheduler makes task offloading decisions, and finally the MEC controller executes batch task offloading decisions.

2. The task offloading method for Agent-Assisted DRL oriented to vehicle-side collaborative reasoning according to claim 1, characterized in that: The architecture of the LLM Agent consists of four modules: 1) Road network environment perception module: responsible for receiving external environment information, and is the input layer for Agent cognition; The road network environment perception module accesses three types of traffic knowledge: terrain communication descriptions in the road network knowledge base. Peak-hour traffic congestion communication description And information on emergencies that are sensed in real time by IoT sensors and roadside units (RSUs). These three types of traffic knowledge together constitute the agent's complete perceptual input of the current traffic environment; 2) Scheduling experience memory module: responsible for storing and maintaining the historical scheduling experience of the MEC controller; The scheduling experience memory module updates the MEC controller's real-time interaction memory with each collaborating node during historical scheduling processes on a rolling basis, using cache cycles as units. At the end of each cache cycle, the task scheduling results are statistically aggregated by the scheduling experience memory module to form an experience feedback summary for the inference module to read. ; 3) Scene Semantic Reasoning Module: Using traffic knowledge input from the road network environment perception module and historical experience maintained by the scheduling experience memory module as joint input, the module comprehensively assesses the risk status of the current traffic scene through the semantic understanding and chain reasoning capabilities of LLM. The scene semantic reasoning module follows a chain paradigm of "environmental understanding, risk quantification, and instruction generation" to output structured scene analysis results, providing a basis for decision-making for the semantic vector generation module. 4) Semantic Vector Generation Module: The reasoning results of the scene semantic reasoning module are converted into semantic vectors that are directly used by DRL, driving the controller to make more accurate and risk-adaptive task unloading decisions in complex dynamic scenarios.

3. The task offloading method for Agent-Assisted DRL oriented to vehicle-side collaborative reasoning according to claim 2, characterized in that: During the agent-assisted DRL task offloading and scheduling process: First, during the cache cycle At the outset, the LLM Agent integrates road network knowledge, traffic situation, and historical experience summaries from the memory module to generate two types of decision support information: ① traffic semantic vectors to enhance the state representation of the DRL, and ② dynamic reward weights to drive the reward function to adaptively adjust according to scenario risks. Then, in each scheduling slot within the cache period, DRL reads traffic semantic vectors and dynamic reward weights from the cache, combines them with environmental observations to form an enhanced state, and performs policy training and offloading decisions. Finally, during the cache cycle At the end, the interactive experiences accumulated by DRL are aggregated into a structured experience summary. The feedback is sent to the Agent memory module to correct the semantic vector generation for the next cache cycle, covering knowledge guidance, DRL execution, experience feedback and memory update, forming a closed-loop optimization chain.

4. The task offloading method for Agent-Assisted DRL oriented to vehicle-side collaborative reasoning according to claim 3, characterized in that: The methods used by the LLM Agent to generate traffic semantic vectors and dynamic reward weights are as follows: a. Traffic semantic vector generation During cache cycle Initially, the agent's perception module acquires three types of traffic knowledge. These correspond to terrain communication descriptions, peak-hour congestion descriptions, and real-time emergency information, respectively; this knowledge is then combined with the experience feedback summaries stored in the memory module. As a joint input to LLM, the edge nodes The traffic semantic vector is normalized to ， The set of edge node base stations is represented as follows: ; These represent the terrain influence factor, peak period sensitivity factor, and sudden event influence factor, respectively, which are used to quantify the degree of influence of terrain, congestion, and sudden events on collaborative reasoning. b. Generate dynamic reward weights Agent quantitatively assesses the current cache cycle. The overall risk level of the scenario in which the vehicle is located is represented by the generated scenario risk index. ， The prompt template guides LLM to assess the risks of the current scenario from three dimensions: ① the impact of terrain occlusion on communication stability; ② the pressure level of traffic density on base station queues during the current period; ③ the urgency of sudden events on task latency constraints. The evaluation results from the three dimensions are aggregated into a single risk index through internal reasoning within the agent's reasoning module. , The closer the value is to 1, the higher the risk level of the current scenario. based on Agent generation and edge nodes Dynamic performance weights that match scenario risks and loss weight The two weight generation functions are defined as follows: ， and ， in, and These represent the preset baseline performance and baseline loss weights, respectively; when In high-risk scenarios, Performance rewards are narrowed, and agents receive less incentive for successful tasks, making them more inclined to choose collaborators with low risk and high reliability; at the same time The penalties for failure are increased, and the tolerance for decision-making errors is reduced; when In low-risk scenarios, Performance rewards are restored to normal levels to encourage agents to actively pursue efficiency; The penalties are moderate, and a degree of decision-making exploration is allowed.

5. The task offloading method for Agent-Assisted DRL oriented to vehicle-side collaborative reasoning according to claim 2, characterized in that: The LLM Agent assists DRL training from two dimensions: expanding the state space and enriching the reward signal. During the cache period, the agent performs semantic vector generation and dynamic reward adjustment to continuously provide DRL with enhanced state representation and adaptive reward signal. Under the guidance of enhanced state and reward signal, the decision quality and risk adaptability of DRL in complex dynamic traffic scenarios are improved.

6. The task offloading method for Agent-Assisted DRL oriented to vehicle-side collaborative reasoning according to claim 5, characterized in that: The semantically enhanced DRL offline training method is as follows: based on the traffic semantic vector and reward weights output by the Agent, the task offloading problem is modeled as an extended Markov decision process. The training process of the DRL scheduler is enhanced from two dimensions: state representation and reward signal, forming an offline training mechanism that promotes mutual reinforcement between knowledge-driven and experience-driven approaches, as detailed below: State space S: rounds Corresponding state This includes real-time information on vehicles, tasks, and base station load, as well as traffic semantic vectors generated by the Agent. ; In the round The state is represented as ， in, This represents an autonomous driving task, which includes a set of characterization parameters. ,in Represents the size of the task data. Indicates task Delay constraints; Indicates base station The collection of autonomous driving tasks collected within the coverage area; Indicates vehicle speed; This indicates the service strength of the MEC controller's offload queue; Indicates cooperative base station The processing queue service intensity; Action Space A: The MEC controller filters the set of candidate collaborators that can continuously cover the vehicle during task execution; In the round The task unloading action is represented as ， in Representative round The set of task unloading decisions within; Cumulative Reward R: The Agent generates a scenario risk index at the beginning of each cache period. Based on this, dynamic performance weights are calculated. With loss weight ; In the round In this scenario, the reward for successfully completing a task within the time constraint is: ， The penalty is as follows if the task fails to be completed within the time constraint: ， In state Next action The total reward received is: ； Strategy: Use a dual-depth Q-network (DDQN) as the policy network; The cumulative expected reward for DDQN discounts is calculated as follows: ， in As a discount factor, To the total number of training rounds, For all possible unload sets; The training objective is to maximize the cumulative expected reward after discounts; the parameters maintained by DDQN are as follows: Policy Networks and The target network employs a dual-network mechanism to separate action selection and value assessment: Within each decision scheduling slot, the interaction quadruple is stored in the experience replay pool. During training Random sampling batch To break the temporal correlation between samples; The optimal action for selecting the next state by the policy network is represented as: ， The Q-value of this action is evaluated by the target network and calculated as follows: ， The mean square error between the target Q value and the predicted Q value is ， The loss function used to construct the policy network; by minimizing Gradient updates are performed on the policy network parameters, and the target network parameters are updated. Every Each step synchronizes with the policy network once to maintain training stability.

7. The task offloading method for Agent-Assisted DRL oriented to vehicle-side collaborative reasoning according to claim 2, characterized in that: LLM Agent's memory is continuously updated: At the end of each cache cycle, the offloading strategies generated by multiple rounds of DRL are aggregated into a structured experience summary and fed back to the agent; The Agent's scheduling experience memory module continuously integrates historical and latest experiences.

8. The task offloading method for Agent-Assisted DRL oriented to vehicle-side collaborative reasoning according to claim 7, characterized in that: The steps for continuous updating of the agent's memory with DRL assistance are as follows: S1, Construction of Experience Summaries Cache cycle The interactive data within the system was statistically aggregated into three types of interpretable statistics. , and Collaboration Nodes The summary of experience feedback is represented as follows: ， in, For period Inner selection edge nodes The average reward obtained when the node is used as an unloading target reflects the overall service quality actually provided by the node. The corresponding task failure rate is used to measure the node. Reliability in actual interaction; The difference between the average reward of this period and the average reward of the previous period represents the value of this node. Dynamic trends in performance; S2, Memory-weighted update Historical experience and the latest experience are merged and updated by the memory module using an exponentially weighted moving average (EMA) strategy. ， in It acts as a forgetting factor, controlling the degree to which historical experiences are retained.

Images