A method, system, and medium for inference offloading of large model agents in edge computing scenarios.

Abstract

This invention discloses a method, system, and medium for inference offloading of large model agents in edge computing scenarios. The method includes inputting the target large model task to be processed into a pre-constructed Transformer-PPO model to obtain a corresponding target offloading scheme for decomposing the target large model task into multiple target sub-tasks and determining the corresponding offloading decisions. The model building process includes using a Transformer network to obtain an inference chain of a directed acyclic graph of the large model task; generating an offloading scheme based on the PPO algorithm, which includes offloading strategies for each sub-task, where the offloading strategy is the offloading decision and the probability of the offloading decision for each sub-task; and obtaining the offloading scheme for the current training round by penalizing and / or rewarding the offloading strategy that increases and / or decreases the total latency in the previous training round. This invention can obtain a better inference offloading scheme for large model tasks, improve the processing efficiency of large model tasks, and reduce device energy consumption.

Description

Technical Field

[0001] This invention relates to the field of large model and edge computing technology, and in particular to a reasoning offloading method, system and medium for large model agents in edge computing scenarios. Background Technology

[0002] In recent years, the rapid increase in IoT devices has generated massive amounts of data. This data needs to be processed closer to the data source to reduce transmission latency and improve response speed. 5G communication technology can quickly transmit this data to edge computing nodes, and the low latency of the 5G network itself enables edge computing to respond more rapidly. [1] Against this backdrop, the field of mobile computing has also undergone a significant transformation, gradually shifting from centralized mobile cloud computing to mobile edge computing. Mobile edge computing integrates the technologies and theories of mobile computing and wireless communication, and its main function is to offload tasks such as mobile computing, network control, and storage to network edge nodes (e.g., base stations and routers). [2] This enables the running of computationally intensive and latency-sensitive inference chain programs on mobile devices with limited computing resources. Mobile edge computing will significantly reduce computational latency and the energy consumption of mobile devices. [3] .

[0003] Inference refers to the process of using a trained AI model to run real-time data to predict or solve tasks. Inference offloading is a specific type of task offloading. Large models refer to machine learning models with a large number of parameters (usually over a billion) and complex computational structures. These models are typically built from deep neural networks and can handle massive amounts of data to complete various complex tasks, such as natural language processing, computer vision, and speech recognition. A large model agent can be understood as a system capable of autonomous planning and decision-making, comprehensively utilizing multiple tools to complete complex tasks. In this system, the large model acts as a "core scheduler." This scheduler is responsible for interpreting the user's natural language input, planning a series of executable sub-tasks, and then calling appropriate models to complete these sub-tasks step by step. The process of calling models to complete all sub-tasks is called inference by the large model agent.

[0004] For a long time, humanity has been pursuing artificial intelligence (AI) that is equal to or surpasses human capabilities. Large language models are seen as a potential spark for artificial general intelligence (AGI), offering hope for building general large-scale model agents. [4]Many researchers have used large models as the foundation for building large-scale intelligent agents and have made significant progress. The English term for intelligent agent is "Agent," and the AI ​​industry has proposed various definitions for it. An intelligent agent is a general-purpose problem solver. From a software engineering perspective, an intelligent agent is a computer program based on a large language model, possessing planning and thinking abilities, memory capabilities, and the ability to use utility functions, capable of autonomously completing a given task. However, deploying large-scale intelligent agents locally presents problems of insufficient device computing power and energy consumption. How to offload multiple subtasks determined by LLM (Low-Level Model) decisions to the edge to reduce the total latency and energy consumption of the LLMA agent in completing user requests has become an unsolved problem. [5] .

[0005] Task offloading is a core technology in edge computing (inference offloading is a specific type of task offloading). It allows computationally intensive tasks to be moved from resource-constrained user servers and local devices, such as smartphones and sensors, to more powerful edge servers, or edge devices. The aim is to reduce device power consumption, improve task execution efficiency, and lower latency, thereby enhancing the overall user experience. [6] The decision-making process for task offloading needs to comprehensively consider various factors such as the task's computational requirements, data volume, network conditions, and the load and availability of edge devices, aiming to optimize multiple objectives such as task completion time, energy consumption, and cost. In terms of technical implementation, task offloading often uses a directed acyclic graph (DAG) to represent the dependencies between tasks. [7] Task offloading is determined through static rules or dynamic algorithms. These algorithms may include heuristics, machine learning, or reinforcement learning, making decisions dynamically based on the current environment and system state. While task offloading offers significant performance advantages, it also faces challenges such as network fluctuations, data security and privacy protection, and the heterogeneity of edge computing environments.

[0006] Existing large-scale AI agent deployment solutions, such as HuggingGPT, are deployed in the cloud. When HuggingGPT receives a user request, it calls the ChatGPT API for task planning. ChatGPT selects the appropriate AI model from the metadata of various models in HuggingFace to execute each sub-task and summarizes the response based on the execution results. By leveraging ChatGPT's powerful language capabilities and the rich AI models in HuggingFace, HuggingGPT can handle a variety of complex AI tasks across different modalities and domains, achieving impressive results in language, vision, speech, and other challenging tasks. [8] .

[0007] However, existing large-scale intelligent agent inference solutions heavily rely on cloud-based API services, resulting in issues such as long response times, service instability, and privacy concerns. First, requests and responses to cloud models require multiple round trips across the network, leading to significant latency and hindering the rapid model inference capabilities required for real-time inference chains, such as robot control and instant responses in voice dialogues. Second, cloud computing response speeds can decrease or even become unavailable during peak periods. Finally, cloud-based large models raise serious privacy concerns, especially when handling sensitive data from hospitals and banks, as well as personal data such as text input and photos on mobile phones. Currently, large models are not only becoming increasingly large but also increasingly smaller, enabling them to run on resource-constrained mobile and edge devices.

[0008] Existing inference offloading methods, such as deep reinforcement learning (DRL), have become popular directions for task offloading in edge computing in recent years due to their flexibility and adaptability to different environments. [9] DRL learns effective policies (i.e., mappings from environmental states to actions) by interacting with the environment in order to maximize rewards. Leveraging the powerful feature extraction capabilities of DNNs, DRL can effectively solve complex decision-making problems with large, high-dimensional state and action spaces. DRL is also used to address the task offloading problem in MEC (Multi-access Edge Computing).

[0009] However, existing inference-based offloading methods are computationally complex. In edge computing systems, time latency is a critical metric, and genetic algorithms, with their high computational complexity, may require lengthy search times and risk getting trapped in local optima. Such long-running algorithms could lead to system failure, as time latency is crucial for system stability and response speed.

[0010] [1]X.Kong,Y.Wu,H.Wang,F.Xia,Edge computing for internet ofeverything:Asurvey,IEEE Internet of Things Journal 9(23)(2022)23472–23485.

[0011] [2] W.Shi, J.Cao, Q.Zhang, Y.Li, L.Xu, Edge computing: Vision and challenges, IEEE Internet of Things Journal 3(5)(2016)637–646.

[0012] [3]Y.Siriwardhana,P.Porambage,M.Liyanage,M.Ylianttila,A survey onmobile augmented reality with 5g mobile edge computing:Architec-tures,applications,and technical aspects,IEEE Communications Surveys&Tutorials 23(2)(2021)1160–1192.

[0013] [4]L.Wang,C.Ma,X.Feng,Z.Zhang,H.Yang,J.Zhang,Z.Chen,J.Tang,X.Chen,Y.Lin,W.X.Zhao,Z.Wei,J.Wen,A survey onlarge language model based autonomousagents,Frontiers of Computer Science 18(6)(2024)186345.

[0014] [5]Y.Mao,C.You,J.Zhang,K.Huang,K.B.Letaief,A survey on mobile edgecomputing:The communication perspective,IEEE Communications Surveys&Tutorials19(4)(2017)2322–2358.

[0015] [6]J.Chen,X.Ran,Deep learning with edge computing:Areview,Proceedingsof the IEEE 107(8)(2019)1655–1674.

[0016] [7]W.Liang,J.Xiao,Y.Chen,C.Yang,K.Xie,K.-C.Li,B.Di Martino,Tmhd:Twin-bridge scheduling of multi-heterogeneous dependent tasks for edge computing,Future Generation Computer Systems 158(2024)60–72.

[0017] [8]Y.Shen, K.Song,

[0018] [9] J.Wang, J.Hu, G.Min, W.Zhan, AYZomaya, N.Georgalas, Dependent taskoffloading for edge computing based on deep reinforcement learning, IEEETransactions on Computers 71(10)(2022)2449–2461.

[0019] The above background information is provided only to assist in understanding the inventive concept and technical solution of this invention. It does not necessarily belong to the prior art of this patent application, nor does it necessarily provide technical teaching. In the absence of clear evidence that the above information was disclosed before the filing date of this patent application, the above background information should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0020] The purpose of this invention is to provide a method, system, and medium for inference offloading of large model agents in edge computing scenarios, which can obtain a better inference offloading scheme for large model tasks, thereby improving the processing efficiency of large model tasks and reducing device energy consumption.

[0021] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0022] A method for inference offloading of large model agents in edge computing scenarios, the method comprising the following steps:

[0023] The target large model task to be processed is input into a pre-built Transformer-PPO model to obtain a target unloading scheme to complete the target large model task. The target unloading scheme includes multiple target sub-tasks, the execution order of each target sub-task, and unloading decisions. The unloading decisions include unloading or not unloading. The target large model task is decomposed into multiple target sub-tasks.

[0024] The Transformer-PPO model is established in advance through the following steps:

[0025] Design a basic model based on Transformer network and PPO algorithm;

[0026] Using a large model task as the learning sample, a Transformer network is used to obtain an inference chain G(V,E) from the large model task. G(V,E) is a directed acyclic graph, where V is the set of vertices of the directed acyclic graph, representing the set of subtasks obtained by decomposing the large model task, and E is the set of directed edges in the directed acyclic graph, with each directed edge e(v...)... i ,v j )∈E represents subtask v i With subtask v j The dependency relationship between them, v i ∈V, v j ∈V, where i and j are both positive integers;

[0027] Generate an unloading scheme for the inference chain, the unloading scheme including an unloading strategy for each subtask, the unloading strategy being the unloading decision of the subtask and the probability of the unloading decision;

[0028] The base model is optimized and trained in multiple rounds until the trained model is obtained and used as the Transformer-PPO model: Based on the PPO algorithm, the unloading scheme in the current round of training is obtained by penalizing the unloading strategy that increases the total latency in the previous round of training and / or rewarding the unloading strategy that reduces the total latency in the previous round of training. The total latency is the total time used to complete the task vector according to the unloading scheme.

[0029] Furthermore, based on any one or a combination of the aforementioned technical solutions, the unloading strategy for the subtasks in the unloading scheme is expressed as follows:

[0030] π(a i |O i-1 G(V,E))=softmax(f Transformer (s i ))

[0031] Wherein, π(a i |O i-1 G(V,E)) represents the expression for the inference chain G(V,E) in the parent task vector O. i-1 Under the uninstallation scheme, subtask v i The uninstallation decision is a i The probability, softmax() represents the probability distribution generated by the softmax function, f Transformer (s i ) represents the output of the Transformer network.

[0032] Furthermore, following any one or a combination of the aforementioned technical solutions, based on the PPO algorithm, the basic model is subjected to multiple rounds of optimization training to obtain a better unloading scheme, which further includes:

[0033] For each time step t, calculate the strategy ratio r of the unloading scheme. t (θ), its calculation formula is:

[0034]

[0035] Where, π θ (a t |s t ) indicates that after the policy update, in state s t Select action a under the influence of t The probability, This indicates the state s before the policy update. t Select action a under the influence of t The probability of;

[0036] The extent of the uninstallation scheme update is limited by a preset clipping loss function, which is expressed as follows:

[0037]

[0038] Among them, L CLIP (θ) represents the editing loss function, E t The clip(r) represents the action value function. t (θ), 1-∈, 1+∈) are used to limit the magnitude of policy updates, where ∈ is a hyperparameter. It is the dominance function, which is expressed as: γ represents the discount factor, λ is a preset parameter, and r t (θ) represents the strategy proportion, V(s) t+k+1 ) represents state s t+k+1 The value of V(s) t+k ) represents state s t+k The value of θ represents the neural network parameters.

[0039] Furthermore, based on any one or a combination of the aforementioned technical solutions, an unloading scheme for the inference chain is generated, including the following steps:

[0040] A state space S is constructed based on the inference chain G(V,E). The state space S includes the computing power requirements corresponding to each subtask, the size of the sub-model used to process each subtask, the amount of data corresponding to completing each subtask, the topology of the inference chain G(V,E), the wireless transmission rate, the local device status, and the edge device status.

[0041] Using the state space S as input, the unloading strategy of the subtask is determined by using the unloading strategy calculation formula based on the softmax function.

[0042] Furthermore, following any one or a combination of the aforementioned technical solutions, and based on the PPO algorithm, performing multiple rounds of optimization training on the basic model to obtain a better unloading solution, the method further includes the following steps:

[0043] A preset reward function is defined by the following formula: in, Indicates task v i The impact of the unloading decision on the total latency. This indicates that task v is included in this round. i The total delay under the first uninstallation scheme of the uninstallation decision, This represents the total delay under the first unloading scheme in the previous round;

[0044] In the k-th round of training, for the first subtask where the unloading decision changes, the reward function value corresponding to the first subtask is calculated using the preset reward function;

[0045] If the reward function value is positive, then in the (k+1)th round of training, a positive reward is given to the unloading decision of the first subtask in the kth round of training to increase the probability that the first subtask maintains the unloading decision in the kth round of training.

[0046] If the reward function value is negative, then in the (k+1)th round of training, a negative reward is given to the unloading decision of the first subtask in the kth round of training to reduce the probability that the first subtask maintains the unloading decision in the kth round of training.

[0047] Furthermore, based on any one or a combination of the aforementioned technical solutions, the total delay is determined in the following manner:

[0048] Determine each sub-terminating task in the inference chain, wherein the sub-terminating task is the endpoint of the directed edge;

[0049] Determine the completion time of each of the sub-termination tasks, wherein the completion time of the sub-termination task is the time taken from the start of execution of the first sub-task in the inference chain to the completion of the sub-termination task;

[0050] The maximum value of the completion time of the multiple sub-termination tasks is determined to be the total delay.

[0051] Furthermore, following any one or a combination of the aforementioned technical solutions, the method further includes calculating the total delay using the following formula:

[0052]

[0053] Among them, AL On The total delay represents the total delay of the inference chain under the offloading scheme, v e This indicates that there is a one-to-one correspondence between the sub-terminating tasks in the inference chain, the endpoint of the directed edge, and the sub-terminating task. Indicates the termination of subtask v e Local execution completion time, indicating the termination of subtask v e The completion time corresponding to the partial / full execution on the local device. Indicates the termination of subtask v e Edge device execution completion time, indicating sub-terminated task v e The completion time corresponding to the partial / full execution on the edge device. This indicates the completion time of each sub-terminating task in the inference chain. This indicates that the maximum value is taken for the completion time of each sub-terminating task in the inference chain.

[0054] Furthermore, based on any one or a combination of the aforementioned technical solutions, the completion time of the sub-task is calculated in the following manner:

[0055] If subtask v i If the uninstallation decision is to uninstall, then its completion time is the time of the subtask v. i Local execution completion time on the local device

[0056] If subtask v i If the uninstallation decision is not to uninstall, then its completion time will be based on the upload completion time. Edge device execution completion time and download completion time Confirmed, wherein the upload completion time To make the subtask v i The completion time of offloading from the local device to the edge device, and the completion time of the edge device execution. The completion time for the edge device to execute the subtask, the download completion time. To make the subtask v i Completion time from edge device back to local device.

[0057] Furthermore, based on any one or a combination of the aforementioned technical solutions, the local execution completion time is calculated in the following manner.

[0058]

[0059] Wherein, pre(v i ) represents task v i All prerequisite tasks, where max represents finding the maximum value. Indicates task v i Take the maximum of the upload completion time and download completion time from all the preceding tasks. This indicates the preparation time of the local device, which represents the idle time before the task begins execution. This indicates a local execution delay;

[0060] The upload completion time is calculated using the following method.

[0061]

[0062] in, Indicates the preparation time for the upload link. This indicates an upload delay;

[0063] The edge device execution completion time is calculated in the following manner.

[0064]

[0065] in, Indicates the preparation time of the edge device. Indicates the execution latency of edge devices;

[0066] The download completion time is calculated using the following method.

[0067]

[0068] in, Indicates the preparation time for the download link. This indicates a download delay.

[0069] Furthermore, based on any one or a combination of the aforementioned technical solutions, the preparation time of the local device is calculated using the following formula. in, Subtask v i-1 The corresponding local device preparation time, Subtask v i-1 The corresponding local execution completion time;

[0070] The preparation time of the upload link is calculated using the following formula. in, Subtask v i-1The corresponding preparation time for the upload link, Subtask v i-1 The corresponding upload completion time;

[0071] The preparation time of the edge device is calculated using the following formula. in, Subtask v i-1 The corresponding preparation time for edge devices, Subtask v i-1 The corresponding edge device execution completion time;

[0072] The preparation time of the download link is calculated using the following formula. in, Subtask v i-1 The preparation time for the corresponding download link. Subtask v i-1 The corresponding download completion time.

[0073] Furthermore, based on any one or a combination of the aforementioned technical solutions, the upload delay... The calculation formula is: in, It is task v i Data size, R up Upload speed;

[0074] The execution delay The calculation formula is: Among them, C i It is a subtask v i Required computation, cs ct It refers to the computing power of edge servers (edge ​​device hosts);

[0075] The download delay The calculation formula is: in, It is a subtask v i The size of the returned data, R do It refers to the download speed;

[0076] The local delay The calculation formula is: Among them, cs ud This indicates the computing power of the user's server.

[0077] Furthermore, following any one or a combination of the aforementioned technical solutions, in the process of establishing the Transformer-PPO model, the unloading strategy is obtained using a value evaluation network with state space, environmental information, and unloading decision as inputs, and the basic model is trained with the goal of minimizing total latency.

[0078] According to another aspect of the present invention, the present invention provides an inference offloading system for large model agents in an edge computing scenario, including a local device and a plurality of edge devices communicatively connected to the local device;

[0079] The local device stores the Transformer-PPO model obtained using the reasoning offloading method for large model agents in edge computing scenarios, which utilizes any one or a combination of the above technical solutions.

[0080] According to another aspect of the present invention, a computer-readable storage medium is provided for storing program instructions configured to be invoked to perform the steps of the method described in any one of the above technical solutions or a combination of multiple technical solutions.

[0081] The beneficial effects of the technical solution provided by this invention are as follows:

[0082] a. This invention constructs a large model task as a reasoning chain of a directed acyclic graph based on the Transformer network, reasonably considers the dependencies between subtasks, and can better unload the reasoning task of the large model agent from a global perspective. It can avoid the existing unloading schemes that build each subtask in isolation, where the unclear dependencies between subtasks make the reasoning unloading process of the large model task more difficult.

[0083] b. This invention is based on the PPO algorithm. It uses the state space of factors including device computing power, device memory, model size, and model computational load, and combines reinforcement learning mechanism to determine the unloading decision using a pre-built policy proportion function. It optimizes each round of training according to the reward function, which can adapt to environmental changes, avoid local optima, and achieve optimal allocation of global resources. This improves the real-time performance, accuracy and overall efficiency of task scheduling, and ensures the high performance and reliability of the edge computing system in complex dynamic environments.

[0084] c. This invention uses a clipping loss function to limit the magnitude of unloading policy updates. By sampling trajectories multiple times and updating policy network parameters, the optimal unloading decision is finally output, thereby learning more complex unloading policies. Existing heuristic algorithms and rule-based algorithms cannot generalize well to unseen situations when dealing with continuous or high-dimensional state spaces. Attached Figure Description

[0085] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0086] Figure 1 A schematic diagram of the inference unloading process for a large model task provided as an exemplary embodiment of the present invention;

[0087] Figure 2 A schematic diagram illustrating the reasoning process of a specific reasoning chain provided as an exemplary embodiment of the present invention;

[0088] Figure 3 A schematic diagram illustrating the optimization principle of a reasoning unloading scheme provided as an exemplary embodiment of the present invention;

[0089] Figure 4 A flowchart illustrating the training process of the Transformer-PPO model provided as an exemplary embodiment of the present invention. Detailed Implementation

[0090] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0091] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0092] The technical terms involved in this invention and their meanings include the following:

[0093] 1. Reinforcement Learning: Reinforcement learning is a machine learning algorithm that trains a model to make optimal decisions in a specific environment. Once the model is trained, it becomes an algorithm for achieving the optimization objective.

[0094] 2. Heuristic Algorithms: Heuristic algorithms are problem-solving methods based on experience and intuition. Their purpose is to simplify the solution process of complex problems and improve the solution speed, but they do not guarantee that they are the optimal solution. They are suitable for problems with excessively high solution costs.

[0095] 3. Proximal Policy Optimization (PPO): This is a reinforcement learning algorithm used to train an agent to make decisions in its environment. It improves training stability by limiting the numerical values ​​updated in each policy update, thus avoiding excessively large updates.

[0096] 4. Image captioning is one of the main goals of computer vision, aiming to automatically generate natural descriptions of images. It requires not only identifying salient objects in an image and understanding their interactions, but also expressing them using natural language, making it extremely challenging.

[0097] 5. Among the many tasks in Natural Language Processing (NLP), semantic similarity algorithms are a core technology, which are mainly used to measure the semantic similarity between two texts.

[0098] Existing task offloading techniques consider the number of CPU cycles and memory required for a task, which is not suitable for inference offloading in large-scale intelligent agent inference scenarios. Similar research includes inference offloading of large models and inference offloading of deep neural network models. These studies focus more on the inference offloading of individual models, offloading the entire model as a task to edge devices, or splitting the model into pieces and offloading the pieces as subtasks to edge devices. However, the complex dependencies between subtasks make model inference offloading even more difficult.

[0099] Existing task offloading methods rely on heuristic algorithms like genetic algorithms, which may get stuck in local optima rather than global optima. In edge computing environments, if a genetic algorithm selects a local optimum, it may lead to inefficient resource allocation, impacting system performance. If an edge node is assigned too many computing tasks, it may become overloaded while other nodes remain idle, reducing overall system efficiency.

[0100] To address these shortcomings, this invention primarily focuses on the offloading of large model agent inference tasks deployed in edge environments, as well as the task scheduling optimization problem for combinations of multiple terminal devices and edge nodes. Offloading large model agent inference tasks to edge devices closer to the data source, such as edge devices, edge gateways, and mobile phones, is a promising solution to these problems. The optimization objectives are: 1) Reduce latency: By optimizing task allocation and edge node selection, data processing and transmission time is reduced, thereby improving system response speed; 2) Save energy: Reasonable task allocation reduces unnecessary computation and transmission, lowering energy consumption of terminal devices and edge nodes and extending device lifespan; 3) Protect user privacy: Data is not uploaded to the cloud, preventing enterprises from illegally using user data and preventing data theft during transmission; 4) Improve resource utilization: By dynamically adjusting task allocation, resources are ensured to be used most effectively, avoiding resource waste.

[0101] The purpose of this invention is to design a task scheduling algorithm / model for edge computing systems based on reinforcement learning for large-scale intelligent agent inference offloading. This algorithm / model aims to fully consider factors such as device computing power, device memory, model size, and computational load required for large-scale model inference, and incorporate reinforcement learning mechanisms to adapt to environmental changes, avoid local optima, and achieve optimal allocation of global resources. This improves the real-time performance, accuracy, and overall system efficiency of task scheduling, ensuring high performance and reliability of the edge computing system in complex and dynamic environments.

[0102] In one embodiment of the present invention, a method for inference offloading (LATO) of large model agents in an edge computing scenario is provided, see [link to relevant documentation]. Figure 4 The method includes the following steps:

[0103] The target large model task to be processed is input into a pre-built Transformer-PPO model to obtain a target offloading scheme to complete the target large model task. The target offloading scheme includes multiple target subtasks, the execution order of each target subtask, and an offloading decision. The offloading decision includes offloading or not offloading. The target large model task is decomposed into multiple target subtasks. Wherein, offloading means offloading the target subtask to an edge device for execution, and not offloading means not offloading the target subtask to an edge device and executing it on the user device.

[0104] The Transformer-PPO model is established in advance through the following steps:

[0105] Design a basic model based on Transformer network and PPO algorithm;

[0106] Using the large model task as the learning sample, an inference chain G(V,E) is obtained using a Transformer network. G(V,E) is a directed acyclic graph, where V is the set of vertices of the directed acyclic graph, representing the set of subtasks obtained from the decomposition of the large model task, and E is the set of directed edges in the directed acyclic graph, with each directed edge e(v...)... i ,v j )∈E represents subtask v i With subtask v j The dependency relationship between them, v i ∈V, v j ∈V, where i and j are both positive integers;

[0107] Generate an unloading scheme for the inference chain, the unloading scheme including an unloading strategy for each subtask, the unloading strategy being the unloading decision of the subtask and the probability of the unloading decision;

[0108] The base model is optimized and trained in multiple rounds until the trained model is obtained and used as the Transformer-PPO model: Based on the PPO algorithm, the unloading scheme in the current round of training is obtained by penalizing the unloading strategy that increases the total latency in the previous round of training and / or rewarding the unloading strategy that reduces the total latency in the previous round of training. The total latency is the total time used to complete the task vector according to the unloading scheme.

[0109] Unlike existing uninstallation methods, in this application, such as Figure 2 As shown, the large model agent performs task planning based on the user's input, i.e., the target large model task to be processed, using a Transformer network. The target large model task is decomposed into an inference chain consisting of multiple target sub-tasks. Each target sub-task in the inference chain is executed by its corresponding target sub-model. Then, the inference chain, information of each target sub-model, and device information are input into the algorithm provided in this application (the Transformer-PPO model). The algorithm provides an offloading decision for each target sub-task and, according to the offloading decision, offloads each target sub-task to a local device or edge device for inference. Finally, the inference / execution results of each target sub-task are aggregated on the large model agent and returned to the user.

[0110] Figure 3 A concrete example of an inference chain is provided. Images A, B, and C are each input into the same image captioning model. The output of the model inference, built on a Transformer network, is the natural description text of each image. Then, the texts are input pairwise into a semantic similarity search model. The output of the model inference is the text similarity value. Finally, the similarity is sent to a larger model to combine with the user's input for inference and answering.

[0111] In this invention, the inference chain is modeled as a directed acyclic graph (DAG) G = (V, E), where V is the set of vertices of the DAG, representing the set of subtasks obtained from the decomposition of the large model task, and E is the set of directed edges in the DAG, with directed edges e(v...). i ,v j )∈E represents subtask v i With subtask v j The dependency relationship between them, v i ∈V, v j ∈V, where i and j are both positive integers; where each vertex v i ∈V represents a model, where each directed edge e(v) i ,v j )∈E represents model v i With model v j Dependencies between them. Calculate the dependencies for each of the subtasks v. i The required computing power is C. i To express.

[0112] Uninstall Inference Task v i To edge device m j The process includes the following steps: First, the local device sends the task to the edge device via a wireless channel. Second, the edge device runs the model for inference. Finally, the edge device sends the inference results back to the local device.

[0113] The total latency is influenced by the requirements of all sub-models in the inference chain (model size and computational cost) and the state of the edge device system (the computing power and memory provided by the server). The goal of optimizing the offloading scheme is to minimize the end-to-end latency of the entire inference chain DAG. In practical applications, the offloading decision is dynamic and depends on the current state of the model and the execution status of previous models.

[0114] When developing an uninstallation plan, task priorities must be considered. The goal of uninstallation is to calculate an uninstallation plan O. n =(o1,o2,...,o n To minimize the delay of the inference chain (DAG) G = (V, E), where n = |V| represents the number of directed edges in the directed acyclic graph. Here, o i Indicates task v i The uninstallation decision, o i Using 0 to indicate no unloading (local inference) and 1 to indicate unloading (remote inference), the optimization objective is to minimize the total latency of the inference chain.

[0115] Based on the above analysis, in one embodiment of the present invention, the total delay is calculated using the following formula:

[0116]

[0117] Among them, AL On The total delay represents the total delay of the inference chain under the offloading scheme, v e This indicates that there is a one-to-one correspondence between the sub-terminating tasks in the inference chain, the endpoint of the directed edge, and the sub-terminating task. Indicates the termination of subtask v e Local execution completion time, indicating the termination of subtask v e The completion time corresponding to the partial / full execution on the local device. Indicates the termination of subtask v e Edge device execution completion time, indicating sub-terminated task v e The completion time corresponding to the partial / full execution on the edge device. This indicates the completion time of each sub-terminating task in the inference chain. This indicates that the maximum value is taken for the completion time of each sub-terminating task in the inference chain.

[0118] It should be noted that for a subtask, part of it may be completed locally (executed by the local device) and part may be completed remotely (executed by the edge device). The actual application latency will only occur after both parts of the subtask are completed. Furthermore, in this application, the starting point for the completion time is the start time of the first task in the inference chain.

[0119] The local execution completion time is determined based on the device status, the data size of the subtask, and the size of the submodel. Upload completion time Edge device execution completion time and download completion time The device status includes the device's computing power, idle time, upload rate of the upload link, and download speed of the download link. The device includes local devices, edge devices, receiving and sending devices on the upload link, and receiving and sending devices on the download link.

[0120] For a subtask, the completion time of the subtask is calculated in the following way:

[0121] If subtask v i If the uninstallation decision is to uninstall, then its completion time is the time of the subtask v. i Local execution completion time on the local device

[0122] If subtask v iIf the uninstallation decision is not to uninstall, then its completion time will be based on the upload completion time. Edge device execution completion time and download completion time Confirmed, wherein the upload completion time To make the subtask v i The completion time of offloading from the local device to the edge device, and the completion time of the edge device execution. The completion time for the edge device to execute the subtask, the download completion time. To make the subtask v i Completion time from edge device back to local device.

[0123] For the subtasks that need to be uninstalled, the uninstallation process involves three stages: data upload, inference on the edge device, and result download. Correspondingly, each stage has a completion time, i.e., a delay. The time required for the data upload stage is the upload delay, the time required for inference on the edge device is the execution delay, and the time required for result download is the result delay.

[0124] The upload delay The calculation formula is:

[0125]

[0126] in, It is task v i Data size, R up Upload speed.

[0127] The execution delay The calculation formula is:

[0128]

[0129] Among them, c i It is a subtask v i Required computation, cs ct It refers to the computing power of edge devices (edge ​​device hosts).

[0130] The download delay The calculation formula is:

[0131]

[0132] in, It is a subtask v i The size of the returned data, R do That's the download speed.

[0133] For subtasks that do not require uninstallation, i.e., subtasks that perform local inference, their completion time is the local latency. The calculation formula is:

[0134]

[0135] Among them, cs ud c represents the computing power of the local device. i It is a subtask v i The amount of computation required.

[0136] The inference offloading method for large model agents in edge computing scenarios provided by this invention, in calculating the total latency, not only considers the upload latency, execution latency, and download latency of each subtask, but also the preparation time RT before task execution during task processing. The preparation time represents the idle time of the device before the task begins execution, specifically including the local processor preparation time, i.e., the local device preparation time. Upload link preparation time Edge device preparation time and download link preparation time

[0137] The local processor preparation time The calculation formula is as follows:

[0138]

[0139] The upload link preparation time The calculation formula is as follows:

[0140]

[0141] Edge device preparation time The calculation formula is as follows:

[0142]

[0143] Download link preparation time The calculation formula is as follows:

[0144]

[0145] in, Subtask v i-1 The corresponding local device preparation time, Subtask v i-1 The corresponding local execution completion time; Subtask v i-1 The corresponding preparation time for the upload link, Subtask v i-1 The corresponding upload completion time; Subtask v i-1 The corresponding preparation time for edge devices, Subtask v i-1 The corresponding edge device execution completion time; Subtask v i-1 The preparation time for the corresponding download link. Subtask v i-1 The corresponding download completion time.

[0146] Based on the aforementioned upload latency, execution latency, download latency, local processor preparation time, upload link preparation time, edge device preparation time, and download link preparation time, for each subtask v i This determines the corresponding completion time for each resource (local device, edge device, upload link, and download link).

[0147] Local execution completion time The calculation formula is as follows:

[0148]

[0149] Wherein, pre(v i ) represents task v i All prerequisite tasks, where max represents finding the maximum value. Indicates task v i Take the maximum of the upload completion time and download completion time from all the preceding tasks;

[0150] Upload completion time The calculation formula is as follows:

[0151]

[0152] Edge device execution completion time The calculation formula is as follows:

[0153]

[0154] Download completion time The calculation formula is as follows:

[0155]

[0156] In one embodiment of the present invention, the inventive concept for optimizing the offloading scheme based on Transformer-PPO is as follows: Reinforcement learning (RL) interacts with the environment to learn a policy to take corresponding actions to maximize cumulative reward. The RL problem is represented using a Markov Decision Process (MDP), consisting of tuples (S, A, P, R, γ). Our goal is to find an offloading policy π(s) that maximizes the expected cumulative reward over time. The policy network π(A|S, θ) takes state S as input and outputs a probability distribution of action A, where θ represents the neural network parameters. Action A includes 0 and 1, where 0 represents no offloading and 1 represents offloading. The policy network parameters are optimized using the PPO algorithm, which updates the network weights using a batch of sampled trajectories during training.

[0157] The main elements of an MDP include the following:

[0158] State space S: includes the model's computing power requirements, model size, data size, DAG topology, wireless transmission rate, and the state of edge devices (computing power, memory capacity); the state space is represented as: S = {s} i |s i =(G(V,E),O i In this context, G(V,E) represents the encoded DAG, and O i This indicates a partial uninstallation plan;

[0159] Action space A: contains binary choices, if task v i If the task is executed locally on the local device, then A = 0. i If the process is offloaded to an edge device, then A = 1.

[0160] The reward function R is defined as minimizing the inference chain delay after the unloading decision. The reward function is calculated as follows: in, Indicates task v i The impact of the unloading decision on the total latency. This indicates that task v is included in this round. i The total delay under the first uninstallation scheme of the uninstallation decision, This represents the total delay under the first unloading scheme in the previous round.

[0161] Based on the aforementioned reward function, the model is optimized during the training process of the base model using the following methods:

[0162] In the k-th round of training, for the first subtask where the unloading decision changes, the reward function value corresponding to the first subtask is calculated using the preset reward function;

[0163] If the reward function value is positive, then in the (k+1)th round of training, a positive reward is given to the unloading decision of the first subtask in the kth round of training to increase the probability that the first subtask maintains the unloading decision in the kth round of training.

[0164] If the reward function value is negative, then in the (k+1)th round of training, a negative reward is given to the unloading decision of the first subtask in the kth round of training to reduce the probability that the first subtask maintains the unloading decision in the kth round of training.

[0165] We use Transformers to construct our policy network. Transformers process task embeddings of DAG sequences and refine the representation through successive Transformer layers. Based on the output, the executor makes an offloading decision for each task. Meanwhile, a critic evaluates the value function for each task.

[0166] Specifically, the training process of the base model also includes:

[0167] A state space S is constructed based on the inference chain G(V,E). State space S includes the computing power requirements for each subtask, the size of the sub-model used to process each subtask, the amount of data required to complete each subtask, the topology of the inference chain G(V,E), the wireless transmission rate, the local device status, and the edge device status. Using state space S as input, the offloading strategy for each subtask is determined using an offloading strategy calculation formula based on the softmax function.

[0168] In the basic model, the Transformer network is used to achieve the following:

[0169] Input embedding vector: The embedding vector includes a task feature vector, a parent task vector, and child task vectors. The task feature vector includes the task size and computational cost. The parent task vector consists of task vectors executed before and after the current task feature vector. For the embedding vector sequence [v1, v2, ..., v...], ... n The Transformer processes input through a multi-layer attention mechanism and outputs sequence embeddings. Sequence embeddings represent high-dimensional feature representations of task embedding vectors (task dependencies (parent and child task information) and task features) generated by the Transformer model's multi-layer attention mechanism. This representation encodes inter-task dependencies, execution priorities, and resource states into a high-dimensional space, providing input for subsequent decisions (such as generating task offloading strategies).

[0170] Action prediction: Based on the output of the Transformer, the decision in the action space is generated by the policy network through the softmax function to generate a probability distribution, thereby choosing whether to unload or execute locally.

[0171] The uninstallation strategy for the subtasks in the uninstallation scheme is expressed as follows:

[0172] π(a i |O i-1 G(V,E))=softmax(f Transformer (s i ))

[0173] Wherein, π(a i |O i-1 G(V,E)) represents the expression for the inference chain G(V,E) in the parent task vector O. i-1 Under the uninstallation scheme, subtask v i The uninstallation decision is a i The probability, softmax() represents the probability distribution generated by the softmax function, f Transformer (s i ) represents the output of the Transformer network.

[0174] The process of performing multiple rounds of optimization training on the base model to obtain a better unloading scheme also includes:

[0175] For each time step t, calculate the strategy ratio r of the unloading scheme. t (θ), its calculation formula is:

[0176]

[0177] Where, π θ (a t |s t ) indicates that after the policy is updated, in environment s t Select action a under the influence of t The probability, This indicates that the policy was updated in environment s before the update. t Select action a under the influence of t The probability of;

[0178] The extent of the uninstallation scheme update is limited by a preset clipping loss function, which is expressed as follows:

[0179]

[0180] Among them, L CLIP (θ) represents the optimization objective function, E t The clip(r) represents the action value function.t (θ), 1-∈, 1+∈) are used to limit the magnitude of policy updates, where ∈ is a hyperparameter. It is the dominance function, which is expressed as: γ represents the discount factor, λ is a preset parameter, and r t (θ) represents the strategy proportion, V(s) t+k+1 ) represents state s t+k+1 The value of V(s) t+k ) represents state s t+k The value of θ represents the neural network parameters.

[0181] In the process of establishing the Transformer-PPO model, the unloading strategy is obtained by using the state space, environmental information, and unloading decision as inputs, and the basic model is trained with the goal of minimizing total latency.

[0182] like Figure 3 As shown, the Large Model Agent Inference Offload (LATO) method (LATO) for edge computing scenarios provided by this invention uses a Transformer to process task-dependent and sequential data, generating candidate offload decisions, while the Advantage Function (PPO) optimizes these decisions to ensure long-term reward maximization. First, the Transformer receives observations of the environment and produces two results: policy logits and a value function. The policy logits are passed through a softmax function to obtain an appropriate probability distribution of available actions. Next, the actor network takes the output of the Transformer and generates the final policy, providing a probability distribution for available actions. Finally, the critic network takes the output of the Transformer and generates an estimate of the current state. The advantage function captures the difference between the actual reward, the estimated reward, and the estimate of the current state.

[0183] The reasoning offloading method (LATO) for large model agents in the edge computing scenario employs the PPO algorithm as the policy optimization method. For a given learning task T, the PPO algorithm creates a trajectory using the sample policy π(A|S, θ) and updates the target policy π(θ) over multiple epochs. The probability ratio r... t (θ) is used to update the executor's policy. The PPO algorithm uses clipped agent objectives to avoid excessive policy updates. The combined objectives include policy loss, value function loss, and entropy bonus to encourage exploration in the policy space.

[0184] In this way, LATO can effectively optimize inference offloading decisions in edge computing environments, reduce inference chain latency, and improve the computing efficiency of local devices and the resource utilization of edge devices.

[0185] For example, given a trained Transformer-PPO model, a large target model task is input, and a target inference chain is obtained based on the Transformer network and input into the PPO algorithm. The PPO algorithm directly outputs an optimal offloading solution. However, during training, the PPO algorithm evaluates and optimizes the offloading solutions it outputs in each training round. For instance, if the inference chain contains dependency information, model information, and device information for five tasks, the PPO algorithm will output an offloading solution consisting of five offloading decisions [0, 0, 0, 0, 0]. After evaluation, if the fourth decision is found to be unsuitable, the next training round will output [0, 0, 0, 1, 0]. Finally, after the model converges, an optimal solution [1, 1, 0, 1, 0] will be output.

[0186] Existing task offloading mechanisms rely on heuristic algorithms like genetic algorithms, which may get stuck in local optima rather than global optima. Specifically, in edge computing environments, if a genetic algorithm selects a local optimum, it may lead to inefficient resource allocation, impacting system performance. If an edge device (edge ​​node) is assigned too many computing tasks, it may become overloaded while other nodes remain idle, reducing overall system efficiency. This invention addresses this by introducing a Transformer network into the PPO algorithm using the Transformer-PPO model. The Transformer processes sequential data and generates decision candidates, while PPO optimizes these decisions to maximize long-term rewards. The trained inference offloading decision algorithm, i.e., the Transformer-PPO model, can dynamically generate large model task offloading schemes in real time based on the dependencies between models in the inference chain, model information, and device information. Therefore, the reasoning offloading method for large model agents in edge computing scenarios provided by this invention fully combines factors such as device computing power, device memory, model size, and model computation amount that need to be considered for large model reasoning, and combines reinforcement learning mechanism to adapt to environmental changes, avoid local optima, and achieve optimal allocation of global resources, thereby improving the real-time performance, accuracy and overall system efficiency of task scheduling, and ensuring the high performance and reliability of edge computing systems in complex dynamic environments.

[0187] In one embodiment of the present invention, an inference offloading system for large model agents in an edge computing scenario is provided, comprising a local device and multiple edge devices communicatively connected to the local device; the local device stores the Transformer-PPO model obtained using the inference offloading method for large model agents in an edge computing scenario as described above. When a target large model task to be processed is input into the local device, the Transformer-PPO model generates a target offloading scheme for the target large model task. The inference offloading system decomposes and offloads the target large model task to the edge devices for execution according to the target offloading scheme, or executes it on the local device without offloading.

[0188] The system utilizes the planning capabilities of large models to break down user tasks into multiple sub-tasks that can be completed by different models. Each sub-task on the inference chain of the large model agent is offloaded to the user device or edge device, thereby improving the response speed of the large model agent, reducing the energy consumption of the user device, and avoiding privacy issues caused by uploading user data to the cloud.

[0189] In one embodiment of the present invention, a computer-readable storage medium is provided for storing program instructions configured to be invoked to perform the steps of the method as described in any of the preceding embodiments.

[0190] It should be noted that the above-mentioned embodiments of the inference offloading system and computer-readable storage medium for large model intelligent agents in edge computing scenarios and the embodiments of the inference offloading method for large model intelligent agents in edge computing scenarios belong to the same inventive concept. All contents of the embodiments of the inference offloading method for large model intelligent agents in edge computing scenarios are incorporated into the embodiments of the computer-readable storage medium and the computer-readable storage medium by reference.

[0191] It should be noted that, in this document, relational terms are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0192] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for inference offloading of large model agents in edge computing scenarios, characterized in that, The method includes the following steps: The target large model task to be processed is input into a pre-built Transformer-PPO model to obtain a target unloading scheme to complete the target large model task. The target unloading scheme includes multiple target sub-tasks, the execution order of each target sub-task, and unloading decisions. The unloading decisions include unloading or not unloading. The target large model task is decomposed into multiple target sub-tasks. The Transformer-PPO model is established in advance through the following steps: Design a basic model based on Transformer network and PPO algorithm; Using a large model task as the learning sample, a Transformer network is used to obtain an inference chain G(V,E) from the large model task. G(V,E) is a directed acyclic graph, where V is the set of vertices of the directed acyclic graph, representing the set of subtasks obtained by decomposing the large model task, and E is the set of directed edges in the directed acyclic graph, with each directed edge e(v...)... i ,v j )∈E represents subtask v i With subtask v j The dependency relationship between them, v i ∈V, v j ∈V, where i and j are both positive integers; Generate an unloading scheme for the inference chain, the unloading scheme including an unloading strategy for each subtask, the unloading strategy being the unloading decision of the subtask and the probability of the unloading decision; The base model is optimized and trained in multiple rounds until the trained model is obtained and used as the Transformer-PPO model: Based on the PPO algorithm, the unloading scheme in the current round of training is obtained by penalizing the unloading strategy that increases the total latency in the previous round of training and / or rewarding the unloading strategy that reduces the total latency in the previous round of training. The total latency is the total time used to complete the task according to the unloading scheme.

2. The inference offloading method for large model agents in edge computing scenarios according to claim 1, characterized in that, The uninstallation strategy for the subtasks in the uninstallation scheme is expressed as follows: π(a i |O i-1 ,G(V,E))=softmax(f Transformer (s i )) Wherein, π(a i |O i-1 G(V,E)) represents the expression for the inference chain G(V,E) in the parent task vector O. i-1 Under the uninstallation scheme, subtask v i The uninstallation decision is a i The probability, softmax() represents the probability distribution generated by the softmax function, f Transformer (s i ) represents the output of the Transformer network.

3. The inference offloading method for large model agents in edge computing scenarios according to claim 2, characterized in that, Based on the PPO algorithm, the basic model is subjected to multiple rounds of optimization training to obtain a better unloading scheme, which also includes: For each time step t, calculate the strategy ratio r of the unloading scheme. t (θ), its calculation formula is: Where, π θ (a t |s t ) indicates that after the policy is updated, in environment s t Select action a under the influence of t The probability, This indicates that the policy was updated in environment s before the update. t Select action a under the influence of t The probability of; The extent of the uninstallation scheme update is limited by a preset clipping loss function, which is expressed as follows: Among them, L CLIP (θ) represents the editing loss function, E t The clip(r) represents the action value function. t (θ), 1-∈, 1+∈) are used to limit the magnitude of policy updates, where ∈ is a hyperparameter. It is the dominance function, which is expressed as: γ represents the discount factor, λ is a preset parameter, and r t (θ) represents the strategy proportion, V(s) t+k+1 ) represents state s t+k+1 The value of V(s) t+k ) represents state s t+k The value of θ represents the neural network parameters.

4. The inference offloading method for large model agents in edge computing scenarios according to claim 2, characterized in that, Generating the unloading scheme for the inference chain includes the following steps: A state space S is constructed based on the inference chain G(V,E). The state space S includes the computing power requirements corresponding to each subtask, the size of the sub-model used to process each subtask, the amount of data corresponding to completing each subtask, the topology of the inference chain G(V,E), the wireless transmission rate, the local device status, and the edge device status. Using the state space S as input, the unloading strategy of the subtask is determined by using the unloading strategy calculation formula based on the softmax function.

5. The inference offloading method for large model agents in edge computing scenarios according to claim 1, characterized in that, Based on the PPO algorithm, the basic model is subjected to multiple rounds of optimization training to obtain a better unloading scheme, which also includes the following steps: A preset reward function is defined by the following formula: in, Indicates task v i The impact of the unloading decision on the total latency. This indicates that task v is included in this round. i The total delay under the first uninstallation scheme of the uninstallation decision, This represents the total delay under the first unloading scheme in the previous round; In the k-th round of training, for the first subtask where the unloading decision changes, the reward function value corresponding to the first subtask is calculated using the preset reward function; If the reward function value is positive, then in the (k+1)th round of training, a positive reward is given to the unloading decision of the first subtask in the kth round of training to increase the probability that the first subtask maintains the unloading decision in the kth round of training. If the reward function value is negative, then in the (k+1)th round of training, a negative reward is given to the unloading decision of the first subtask in the kth round of training to reduce the probability that the first subtask maintains the unloading decision in the kth round of training.

6. The inference offloading method for large model agents in edge computing scenarios according to claim 1, characterized in that, The total delay is determined in the following manner: Determine each sub-terminating task in the inference chain, wherein the sub-terminating task is the endpoint of the directed edge; Determine the completion time of each of the sub-termination tasks, wherein the completion time of the sub-termination task is the time taken from the start of execution of the first sub-task in the inference chain to the completion of the sub-termination task; The maximum value of the completion time of the multiple sub-termination tasks is determined to be the total delay.

7. The inference offloading method for large model agents in edge computing scenarios according to claim 6, characterized in that, This also includes calculating the total delay using the following formula: Among them, AL On The total delay represents the total delay of the inference chain under the offloading scheme, v e This indicates that there is a one-to-one correspondence between the sub-terminating tasks in the inference chain, the endpoint of the directed edge, and the sub-terminating task. Indicates the termination of subtask v e Local execution completion time, indicating the termination of subtask v e The completion time corresponding to the partial / full execution on the local device. Indicates the termination of subtask v e Edge device execution completion time, indicating sub-terminated task v e The completion time corresponding to the partial / full execution on the edge device. This indicates the completion time of each sub-terminating task in the inference chain. This indicates that the maximum value is taken for the completion time of each sub-terminating task in the inference chain.

8. The inference offloading method for large model agents in edge computing scenarios according to claim 6, characterized in that, The completion time of the subtask is calculated in the following way: If subtask v i If the uninstallation decision is to uninstall, then its completion time is the time of the subtask v. i Local execution completion time on the local device If subtask v i If the uninstallation decision is not to uninstall, then its completion time will be based on the upload completion time. Edge device execution completion time and download completion time Confirmed, wherein the upload completion time To make the subtask v i The completion time of offloading from the local device to the edge device, and the completion time of the edge device execution. The completion time for the edge device to execute the subtask, the download completion time. To make the subtask v i Completion time from edge device back to local device.

9. The inference offloading method for large model agents in edge computing scenarios according to claim 8, characterized in that, The local execution completion time is calculated in the following manner. Wherein, pre(v i ) represents task v i All prerequisite tasks, where max represents finding the maximum value. Indicates task v i Take the maximum of the upload completion time and download completion time from all the preceding tasks. This indicates the preparation time of the local device, which represents the idle time before the task begins execution. This indicates a local execution delay; The upload completion time is calculated using the following method. in, Indicates the preparation time for the upload link. This indicates an upload delay; The edge device execution completion time is calculated in the following manner. in, Indicates the preparation time of the edge device. Indicates the execution latency of edge devices; The download completion time is calculated using the following method. in, Indicates the preparation time for the download link. This indicates a download delay.

10. The inference offloading method for large model agents in edge computing scenarios according to claim 9, characterized in that, The preparation time of the local device is calculated using the following formula. in, Subtask v i-1 The corresponding local device preparation time, Subtask v i-1 The corresponding local execution completion time; The preparation time of the upload link is calculated using the following formula. in, Subtask v i-1 The corresponding preparation time for the upload link, Subtask v i-1 The corresponding upload completion time; The preparation time of the edge device is calculated using the following formula. in, Subtask v i-1 The corresponding preparation time for edge devices, Subtask v i-1 The corresponding edge device execution completion time; The preparation time of the download link is calculated using the following formula. in, Subtask v i-1 The preparation time for the corresponding download link. Subtask v i-1 The corresponding download completion time.

11. The inference offloading method for large model agents in edge computing scenarios according to claim 9, characterized in that, The upload delay The calculation formula is: in, It is task v i Data size, r up Upload speed; The edge device execution delay The calculation formula is: Among them, c i It is a subtask v i Required computation, cs ct It is the computing power of edge devices; The download delay The calculation formula is: in, It is a subtask v i The size of the returned data, r do It refers to the download speed; Local execution delay The calculation formula is: Among them, cs ud This indicates the computing power of the user's server.

12. The inference offloading method for large model agents in edge computing scenarios according to claim 1, characterized in that, In the process of establishing the Transformer-PPO model, the unloading strategy is obtained by using the state space, environmental information, and unloading decision as inputs, and the basic model is trained with the goal of minimizing total delay.

13. A reasoning offloading system for large model agents in edge computing scenarios, characterized in that, This includes a local device and multiple edge devices that are communicatively connected to the local device; The Transformer-PPO model, obtained using the reasoning offloading method for large model agents in edge computing scenarios as described in any one of claims 1 to 12, is stored on the local device.

14. A computer-readable storage medium for storing program instructions, characterized in that, The program instructions are configured to be invoked to perform the steps of the method as described in any one of claims 1 to 12.

Images