A method and system for adaptive task offloading of user demand

Abstract

The application provides a user demand adaptive task offloading method and system, and relates to the field of deep reinforcement learning.The method comprises the following steps: constructing a network model for determining an inference task offloading node; establishing a delay model to calculate the total inference delay of an inference application on a terminal device; establishing an energy consumption model to determine the total inference energy consumption of the inference task on the terminal device; combining the total inference delay and the total inference energy consumption to determine an offloading strategy generated by an MEC task scheduling server, and establishing a multi-objective optimization function of the offloading strategy based on entropy; taking the multi-objective optimization function value as a reward function, and constructing an intelligent reinforcement learning model based on historical offloading strategies; obtaining real-time inference tasks; inputting the real-time inference tasks into the intelligent reinforcement learning model to output an optimal offloading strategy.Through the application, the generalization ability of the calculation offloading scheme and the adaptability of the offloading strategy are improved.

Description

Technical Field

[0001] The present invention relates to the field of deep reinforcement learning, and in particular to a method and system for adaptive task offloading based on user needs. Background Art

[0002] Computation offloading is a technology that offloads computational tasks from resource-constrained devices (such as smartphones and IoT devices) to more powerful computing platforms (such as cloud servers, edge servers, or other high-performance devices). This approach can significantly increase device computing power, save battery power, reduce latency, improve processing efficiency, and reduce the burden on local devices. Computation offloading is widely used in Mobile Edge Computing (MEC) and Mobile Cloud Computing (MCC), and has become a key technology for improving network performance and computational efficiency.

[0003] The deep integration of artificial intelligence and industry can significantly improve production efficiency, optimize resource allocation, and reduce production costs, providing strong technical support and impetus for the development of industrial production. The recently emerged Large Language Models (LLMs) are mainly used to process and generate natural language text. They are an advanced form of artificial intelligence technology. LLMs are good at understanding complex languages ​​and can generate customized responses and content according to the needs of different users or enterprises. In the Industrial Internet, LLMs have many advantages in improving efficiency, optimizing decision-making processes and increasing automation. For example, they enable operators to interact with machines using natural language instructions. In addition, as intelligent assistants, LLMs can provide real-time help, guide operating procedures and troubleshooting suggestions in complex environments, thereby improving employee work efficiency.

[0004] However, large language models typically contain billions to trillions of parameters, and running these models requires a large amount of computing resources and dedicated hardware. The computing power and energy of industrial equipment are limited, making it difficult to meet the high-performance requirements of LLMs. Industrial equipment cannot directly run large language models. Traditional RL algorithms require precise definition of reward functions. Suitable reward functions are very complex and require careful adjustment of the weights of different factors. Moreover, reward functions are only applicable to specific scenarios or needs. When the scenarios or needs change, the reward function needs to be redesigned, which limits the adaptability of the strategy. Due to the limitations and biases of human cognition, the designed reward function may have imperfections, making it difficult for the intelligent agent to find the optimal strategy. Usually, it can only choose suboptimal strategies, affecting overall performance. Summary of the Invention

[0005] In order to solve the technical problems that large language models in the prior art generally contain tens of billions to hundreds of billions of parameters, running these models requires a large amount of computing resources and special hardware, the computing power and energy of industrial equipment are limited, it is difficult to meet the high performance requirements of LLMs, industrial equipment is difficult to directly run large language models, traditional RL algorithms require precise definition of reward functions, suitable reward functions are very complex, the weights of different factors need to be carefully adjusted, and the reward function is only applicable to specific scenarios or requirements, when the scenario or requirement changes, the reward function needs to be redesigned, which limits the adaptability of the strategy, due to the limitations and biases of human cognition, the designed reward function may have imperfections, making it difficult for the agent to find the optimal strategy, and usually only a suboptimal strategy can be selected, affecting the overall performance, the present application provides a user demand adaptive task offloading method and system.

[0006] The technical scheme provided by the embodiments of the present application is as follows:

[0007] The first aspect

[0008] The user demand adaptive task offloading method provided by the embodiments of the present application is applied to an adaptive task offloading platform, wherein the adaptive task offloading platform comprises a cloud center, a plurality of terminal devices and a plurality of MEC servers, wherein each terminal device is connected with the cloud center through a MEC server, and the MEC servers are connected with each other, wherein the MEC server comprises a MEC task scheduling server, a MEC task computing server and a MEC task forwarding server; the method comprises:

[0009] S1: constructing a network model for determining an inference task offloading node, defining a MEC processing server composed of a MEC task computing server and a MEC task forwarding server, and an inference application generated by each terminal device, wherein the inference application is used to issue an inference task;

[0010] S2: establishing a delay model to calculate the total inference delay of the inference application on the terminal device;

[0011] S3: establishing an energy consumption model to determine the total inference energy consumption of the inference task on the terminal device;

[0012] S4: combining the total inference delay and the total inference energy consumption, determining an offloading strategy generated by the MEC task scheduling server, and establishing a multi-objective optimization function of the offloading strategy based on entropy, wherein the offloading strategy comprises an offloading node and a resource allocation amount;

[0013] S5: taking the multi-objective optimization function value as a reward function, and constructing an intelligent reinforcement learning model based on historical offloading strategies;

[0014] S6: obtaining real-time inference tasks;

[0015] S7: Input the real-time reasoning task into the intelligent reinforcement learning model and output the optimal offloading strategy.

[0016] Second aspect

[0017] An embodiment of the present invention provides a user-demand adaptive task offloading system, comprising:

[0018] processor;

[0019] The memory stores computer-readable instructions, which, when executed by the processor, implement the adaptive task offloading method according to user needs in the first aspect.

[0020] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0021] In the present invention, by constructing a network model, the reasoning tasks are layered and offloaded to the terminal devices, edge computing servers and cloud centers. The delay model and energy consumption model are combined to accurately calculate the time and energy of each offloading strategy, which greatly reduces the computing burden of the terminal devices and reduces the dependence on high-performance hardware. The task processing method with the lowest energy consumption is selected, the life of the terminal devices is extended, and the energy efficiency of the system is improved. The adaptive task offloading platform is used to comprehensively consider the delay and energy consumption through multi-objective optimization, dynamically determine the offloading nodes and resource allocation amount of the task, and intelligently allocate computing resources, so that the reasoning tasks can be efficiently allocated and processed, avoiding resource waste, and supporting the allocation of different reasoning tasks to different The computing layer flexibly adapts to the resource requirements of the task and improves the load handling capacity of the system. By establishing an intelligent reinforcement learning model, it no longer relies on a single linear weighted reward function, but directly optimizes the unloading strategy through multi-dimensional indicators, which can more accurately adapt to the actual task requirements. The loss function based on KL divergence guides the model to approach the real strategy, improves the strategy quality and optimality of the intelligent agent, and enables it to find the best solution in complex environments, making the system more flexible and adaptable. The intelligent reinforcement learning model automatically learns historical data, avoids the complexity of manually designed reward functions, improves the generalization ability of the model, is applicable to a variety of industrial scenarios, and increases the application breadth of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1A flowchart of a method for adaptively offloading tasks based on user needs provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of the structure of an adaptive task offloading platform provided by an embodiment of the present invention;

[0025] Figure 3 A schematic structural diagram of a user-demand adaptive task offloading system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0027] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0028] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0029] Reference Manual Figure 1 , shows a flow chart of a method for adaptive task offloading based on user needs provided by an embodiment of the present invention.

[0030] Reference Manual Figure 2 , which shows a structural diagram of an adaptive task offloading platform provided by an embodiment of the present invention.

[0031] like Figure 2The adaptive task offloading platform includes a cloud center, multiple terminal devices and multiple MEC servers, wherein each terminal device is connected to the cloud center through a MEC server, and each MEC server is connected to each other. The MEC server includes a MEC task scheduling server, a MEC task computing server and a MEC task forwarding server. The MEC task scheduling server (MTSS) processes the LLMs inference requests from the device and decides whether to offload these tasks to the MEC task computing server (MTCS) or the cloud center based on factors such as environmental conditions and resource requirements. At the same time, MEC also manages the allocation of wireless resources within its coverage area to optimize network utilization. The MEC task forwarding server (MTFS) is responsible for forwarding the LLMs inference requests from the device to the cloud.

[0032] An embodiment of the present invention provides a user-demand adaptive task offloading method, which is applied to an adaptive task offloading platform. The adaptive task offloading platform includes a cloud center, multiple terminal devices, and multiple MEC servers. Each terminal device is connected to the cloud center via an MEC server, and the MEC servers are connected to each other. The processing flow of the user-demand adaptive task offloading method may include the following steps:

[0033] S1: Build a network model for determining the inference task offloading node, define the MEC processing server composed of the MEC task computing server and the MEC task forwarding server, and the inference application generated by each terminal device, where the inference application is used to issue the inference task.

[0034] Among them, the MEC processing server includes a combination of the MEC task computing server and the MEC task forwarding server, which processes the task offloading request of the terminal device as a whole. The inference application is an application running on the terminal device, which generates inference tasks that need to be processed, such as language model inference or image recognition and other complex computing tasks. The inference task is a specific computing task generated by the inference application.

[0035] In a possible implementation, S1 specifically includes:

[0036] S101: Define MEC task processing server:

[0037] M * ={1,2,...,M+1}

[0038] Among them, M *It represents an MEC task processing server consisting of M MEC task computing servers and a single MEC task forwarding server. M+1 represents the tag sequence number of the MEC task forwarding server.

[0039] S102: Define the terminal device for the inference task:

[0040] u={1,2,…,U}

[0041] Wherein, u represents the terminal device tag, and U represents the total number of terminal devices.

[0042] S103: Define the inference application generated by each terminal device:

[0043]

[0044] in, Represents the nth inference application on the uth terminal device.

[0045] S104: Construction To define the dependency relationship between task nodes:

[0046]

[0047] in, express A directed acyclic graph of Represents a collection of task nodes, Represents the adjacency matrix used to represent the relationship between task nodes.

[0048] S105: Define reasoning tasks:

[0049]

[0050] in, Representation reasoning application The i-th reasoning task issued, Indicates that the number of processing programs required for the i-th reasoning task is k, i.e., k types, where the type of reasoning task is K = {1, 2, ..., K}, Representing reasoning tasks The input data size, Representing reasoning tasks The output data size.

[0051] S105: Analyze the reasoning task types supported by the MEC task server based on the obtained network model and determine the reasoning task offloading node, where the network model consists of M, u, composition.

[0052] It should be noted that by building a network model that includes MEC task computing and forwarding servers, efficient allocation of reasoning tasks can be achieved, making the selection of task offloading nodes more reasonable, thereby optimizing the utilization of computing resources and task processing efficiency.

[0053] S2: Build a latency model and calculate the total inference latency of the inference application on the terminal device.

[0054] Among them, the delay model is a mathematical model used to evaluate the time required for a task to be generated and completed, covering factors such as computing delay, transmission delay, and queuing delay. The total inference delay is the total time spent on the inference task on the terminal device, including all delay factors from generating the task to receiving the result.

[0055] In a possible implementation, the delay model is specifically:

[0056]

[0057] When the inference task is offloaded to the MEC task computing server:

[0058]

[0059] When inference tasks are offloaded to the cloud center:

[0060]

[0061] in, Representing reasoning tasks The total delay, Representation reasoning application The i-th reasoning task issued, max means taking the maximum value, Representing reasoning tasks The set of all predecessor tasks, Representation reasoning application The jth reasoning task issued, Representing reasoning tasks The total delay, Representing reasoning tasks and reasoning tasks The data transmission delay between Representing reasoning tasks The queue delay, Representing reasoning tasks The execution delay, Representing reasoning tasks The output data size of m1 and m2 represents the MEC task computing server and cloud center respectively. Representing reasoning tasks Transfer from m1 to inference task The transmission rate between nodes m2, Represents the total inference latency of the inference application on the terminal device, Representing reasoning tasks and the transmission delay between the terminal device, Representing reasoning tasks Queue delay when offloading to the MEC task processing server, ρ1 represents the inference task In MEC task computing server utilization, P Q Indicates the probability that the terminal device needs to wait, represents the inference task offloaded to the cloud center, Representing reasoning tasks The queue delay when offloading to the cloud center, λ1 represents the inference task In the MEC task calculation server arrival rate, λ2 represents the inference task The arrival rate at the cloud center, μ1 represents the inference task The service rate of the task offloaded to the MEC computing server, μ2 represents the inference task The service rate offloaded to the cloud center, M represents the total number of MEC servers, m represents the total number of running MEC servers, M! represents the factorial of M, and m! represents the factorial of m.

[0062] Specifically, if the terminal device transmits the inference task to the MEC task processing server, the channel power gain is calculated as:

[0063]

[0064] Among them, g u,m represents the channel power gain between the terminal device and the MEC task processing server, β0 represents the channel power gain at a reference distance of 1 meter, represents the distance between the terminal device and the MEC task processing server, p m and q u They represent the horizontal coordinates of the MEC task processing server and the terminal device, h m Indicates the height of the signal antenna of the MEC task processing server from the ground;

[0065] Reasoning Task The transmission delay can be expressed as:

[0066]

[0067] If m=M+1, it means the reasoning task The MEC task forwarding server forwards the data to the cloud center for execution. Since an inference application consists of multiple inference tasks, the inference tasks may be processed on different nodes. Therefore, during the execution of the inference application, data interaction may occur between MEC task computing servers or between MEC task computing servers and the cloud center. and reasoning tasks The data transmission delay between can be calculated as:

[0068]

[0069] It should be noted that the delay model provides a comprehensive assessment of the total inference time by accurately calculating various delays, enabling the system to select low-latency offloading solutions, thereby improving the real-time performance and response speed of the task.

[0070] S3: Establish an energy consumption model to determine the total inference energy consumption of the inference task on the terminal device.

[0071] Among them, the energy consumption model is a mathematical model used to calculate the energy consumption of inference tasks during the operation of the device, covering aspects such as transmission energy consumption and computing energy consumption. The total inference energy consumption is the total energy consumed to execute the entire inference task on the terminal device, including multiple energy consumptions such as computing and transmission.

[0072] In a possible implementation, the energy consumption model is specifically:

[0073]

[0074] When the reasoning task is the first task, and the reasoning task When offloaded to the MEC computing server:

[0075]

[0076] When the reasoning task is the first task, and the reasoning task When offloaded to the cloud center:

[0077]

[0078] When the reasoning task There are pre-tasks and reasoning tasks When offloaded to the cloud center:

[0079]

[0080] in, represents the total inference energy consumption of the inference application on the terminal device, Representing reasoning tasks The total energy consumption, represents the transmission energy consumption of the last inference task transmitted to the terminal device, Indicates the transmission power of the MEC server, Representing reasoning tasks The maximum output data, Representing reasoning tasks and reasoning tasks The transmission energy consumption between Representing reasoning tasks Energy consumption generated by the inference task offloading node, Indicates the transmission power of the terminal equipment, represents the transmission rate between the terminal device and the inference task offloading node, represents the transmission energy consumption between the inference task offloading node and the cloud center, κ m,c represents the transmission delay coefficient between the inference task offloading node and the cloud center, Represents the transmission energy consumption from the cloud center to the MEC task processing server.

[0081] The energy consumption model can comprehensively evaluate the energy consumption of inference tasks on terminal devices, enabling the system to give priority to low-energy consumption solutions when allocating tasks, thereby extending the battery life of the device and improving resource utilization efficiency. This model optimizes the power management of the device, is suitable for energy-sensitive industrial Internet scenarios, and helps to achieve green and energy-saving computing.

[0082] S4: Combined with the total inference latency and total inference energy consumption, determine the offloading strategy generated by the MEC task scheduling server, and establish a multi-objective optimization function of the offloading strategy based on entropy, where the offloading strategy includes the offloading nodes and resource allocation amount.

[0083] Among them, the offloading strategy is a task processing plan generated by the MEC task scheduling server, which determines whether the inference task is calculated on the local device, MEC node or cloud center, and also includes the amount of resource allocation. Entropy is an information theory metric used to measure uncertainty. The multi-objective optimization function is an objective function that includes multiple performance indicators (such as latency, energy consumption, etc.), which is used to optimize the offloading strategy and ensure that the system achieves a balance between multiple objectives.

[0084] In a possible implementation, the multi-objective optimization function is specifically:

[0085]

[0086] in, Indicates the amount of task offloading and the wireless resource allocation amount W C Unloading strategy, δ represents the adjustment factor for adjusting KL divergence, represents the accuracy index of task processing, represents the evaluation distribution of the reasoning task, p Φ (·) represents the true distribution of the reasoning task, represents the KL divergence of the evaluation distribution of the reasoning task relative to the true distribution of the reasoning task, represents the entropy of the evaluation distribution of the inference task, represents the total inference latency, Indicates the maximum inference latency allowed by the terminal device. Indicates the minimum accuracy index of task processing, represents the total inference energy consumption, Indicates the maximum energy consumption allowed by the terminal device. represents the offloading strategy of the i-th task of the inference task, Indicates the i-th task type on the n-th terminal device, Indicates whether the MEC task computing server can support the task type

[0087] Specifically, in this way, KL divergence, as a metric for distribution comparison, can directly indicate the quality of the distribution with respect to the key variable, which is the total delay in the multi-objective optimization function. Accuracy metrics for each inference application and total energy consumption The present invention characterizes user needs as a multidimensional random variable, thereby recognizing the distribution of these variables, making it possible to directly and multidimensionally evaluate performance criteria. In constraint C1, the total inference delay for solving the inference application shall not exceed the maximum delay allowed by the terminal device. In constraint C2, the accuracy of solving the inference task on the terminal device shall exceed the minimum accuracy. In constraint C3, the total energy consumption for solving the inference task on the terminal device shall not exceed the maximum energy consumption. For each inference task and MEC task computing server, constraint C4 must hold. These constraints ensure that if the task is offloaded to a certain MEC task computing server, the MEC task computing server must have the ability to process a specific task type.

[0088] It should be noted that by combining the total inference latency and total inference energy consumption to establish a multi-objective optimization function, it is possible to achieve the best balance between different performance indicators and generate the optimal offloading strategy. This method effectively improves the flexibility and accuracy of task scheduling. It can not only meet the requirements of low latency and low energy consumption, but also dynamically adapt to resource changes in different scenarios, thereby improving the overall computing efficiency and stability of the system.

[0089] S5: Use the multi-objective optimization function value as the reward function and build an intelligent reinforcement learning model based on the historical unloading strategy.

[0090] Wherein, the reward function is a function used to measure the effect of the agent's action in reinforcement learning, the agent learns the optimal strategy by maximizing the reward, the historical offloading strategy is the strategy data used by the system in the past task offloading process, containing successful and failed experiences, used to guide the training of the intelligent reinforcement learning model, the intelligent reinforcement learning model is an intelligent model based on reinforcement learning algorithm, which continuously learns and optimizes the task offloading strategy through reward and punishment mechanism to achieve more efficient task processing.

[0091] In a possible implementation, the intelligent reinforcement learning model includes an input layer module, a transformer block sequence module, an action prediction module, an intelligent acquisition module, and a neural network update module.

[0092] The input layer module is used to receive input data including time steps, states, and actions in the trajectory, and perform self-attention operation within each independent trajectory.

[0093] Wherein, in reinforcement learning, a trajectory is a sequence of states and rewards observed by an agent after performing actions in an environment, each trajectory records the interaction process of the agent within a period of time, containing time steps, states, actions, etc., a time step refers to the time unit of each step in the reinforcement learning process, each time step represents an interaction between the agent and the environment, including observing the state of the environment, performing actions, and obtaining rewards, etc., the state is the information description of the environment at each time step, which is used to represent the current environment, the state helps the agent understand the current environment and provides a basis for selecting the next action, the action is the action taken by the agent in a certain state, which affects the state change of the environment, and the self-attention can give different importance weights to each data point, helping the model understand the key information and correlation in the trajectory.

[0094] The transformer block sequence module is used to extract features from the input data, ensuring the integrity of the data order, wherein the transformer block sequence module includes a causal attention unit and a multi-layer perceptron unit.

[0095] The action prediction module is used to extract information from the output of the last causal self-attention unit with mask, and the output includes the action prediction result of each time step in the sequence.

[0096] The intelligent acquisition module and the neural network update module are used to calculate the increment of the multi-objective optimization function value.

[0097] In a possible implementation, S5 specifically includes:

[0098] S501: Obtain trajectory data with optimal offloading strategy, wherein the trajectory data includes states, actions representing offloading strategies, and total reasoning delay, task processing accuracy, and total reasoning energy consumption.

[0099] S502: Input the trajectory data into the intelligent reinforcement learning model for training, and determine the hyperparameters of the intelligent reinforcement learning model, where the hyperparameters include the embedding length of the intelligent reinforcement learning model, the input sequence length, the maximum number of training iterations, the number of updates per training iteration, the maximum number of test iterations, the number of updates per test iteration, the sample set, the sample batch size, and the learning rate.

[0100] S503: Calculate the loss function value of the intelligent reinforcement learning model, where the loss function is the KL divergence value between the actual state generated based on the action and the predicted state of the intelligent reinforcement learning model.

[0101] S504: When the loss function value is less than the preset loss function value and the increment of the multi-objective optimization function value is less than the preset increment, output the current action; otherwise, change the hyperparameters and return to step S502.

[0102] S505: Output the intelligent reinforcement learning model under the current hyperparameters.

[0103] In one possible implementation, the loss function value is calculated as follows:

[0104] loss = D KL (a preds ,a target )

[0105] Among them, loss represents the loss function value, a preds represents the predicted action of the uninstallation strategy, a target Indicates the actual action of uninstalling the policy.

[0106] It should be noted that by using the multi-objective optimization function value as the reward function, the intelligent reinforcement learning model can dynamically adapt to different task requirements, automatically optimize the unloading strategy, reduce manual intervention, and improve the accuracy and efficiency of decision-making. By utilizing historical unloading strategy data, the model can converge quickly and enhance generalization capabilities, thereby achieving efficient task allocation and resource management in a complex industrial Internet environment, which helps to improve the system's adaptability and robustness.

[0107] S6: Get real-time reasoning tasks.

[0108] It should be noted that by obtaining real-time inference tasks, it is possible to respond to the latest computing needs in a timely manner, dynamically adjust and optimize offloading strategies, and thus significantly improve the real-time performance and accuracy of task processing.

[0109] S7: Input the real-time reasoning task into the intelligent reinforcement learning model and output the optimal offloading strategy.

[0110] It should be noted that after inputting the real-time reasoning task into the intelligent reinforcement learning model, the system can quickly generate the optimal offloading strategy, realize the optimization of dynamic resource allocation and task scheduling, and significantly improve the processing efficiency and system response speed.

[0111] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0112] In the present invention, by constructing a network model, the reasoning tasks are layered and offloaded to the terminal devices, edge computing servers and cloud centers. The delay model and energy consumption model are combined to accurately calculate the time and energy of each offloading strategy, which greatly reduces the computing burden of the terminal devices and reduces the dependence on high-performance hardware. The task processing method with the lowest energy consumption is selected, the life of the terminal devices is extended, and the energy efficiency of the system is improved. The adaptive task offloading platform is used to comprehensively consider the delay and energy consumption through multi-objective optimization, dynamically determine the offloading nodes and resource allocation amount of the task, and intelligently allocate computing resources, so that the reasoning tasks can be efficiently allocated and processed, avoiding resource waste, and supporting the allocation of different reasoning tasks to different The computing layer flexibly adapts to the resource requirements of the task and improves the load handling capacity of the system. By establishing an intelligent reinforcement learning model, it no longer relies on a single linear weighted reward function, but directly optimizes the unloading strategy through multi-dimensional indicators, which can more accurately adapt to the actual task requirements. The loss function based on KL divergence guides the model to approach the real strategy, improves the strategy quality and optimality of the intelligent agent, and enables it to find the best solution in complex environments, making the system more flexible and adaptable. The intelligent reinforcement learning model automatically learns historical data, avoids the complexity of manually designed reward functions, improves the generalization ability of the model, is applicable to a variety of industrial scenarios, and increases the application breadth of the algorithm.

[0113] Reference Manual Figure 3 , showing a structural diagram of a user demand-based adaptive task offloading system provided by the present invention.

[0114] The present invention further provides a user demand-based adaptive task offloading system 20, which is applied to the above-mentioned user demand-based adaptive task offloading method, comprising:

[0115] Processor 201.

[0116] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201 , the method for adaptive task offloading based on user needs in the method embodiment is implemented.

[0117] The user demand adaptive task offloading system 20 provided by the present invention can execute the above-mentioned user demand adaptive task offloading method and achieve the same or similar technical effects. To avoid repetition, the present invention will not elaborate on it again.

[0118] It should be understood that the processor in the embodiments of the present invention may be a central processing unit (CPU), but may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0119] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0120] The above embodiments can be implemented in whole or in part through software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function according to the embodiments of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired method (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available media can be magnetic media (such as floppy disks, hard disks, tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0121] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0122] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0123] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0124] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0125] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0126] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical, or other forms.

[0127] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0128] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0129] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program codes.

[0130] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for adaptively offloading tasks according to user needs in the method embodiment is implemented.

[0131] The computer-readable storage medium provided by the present invention can implement the steps and effects of the user-demand adaptive task offloading method of the above method embodiment. To avoid repetition, the present invention will not elaborate on them.

[0132] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0133] There are a few points to note:

[0134] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0135] (2) For the sake of clarity, the thickness of layers or regions in the drawings used to describe the embodiments of the present invention are exaggerated or reduced, that is, these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or intervening elements may be present.

[0136] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to form new embodiments.

[0137] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for adaptive task offloading based on user needs, characterized in that: Applied to an adaptive task offloading platform, wherein the adaptive task offloading platform includes a cloud center, multiple terminal devices and multiple MEC servers, wherein each terminal device is connected to the cloud center through a MEC server, and each MEC server is connected to each other, wherein the MEC server includes a MEC task scheduling server, a MEC task computing server and a MEC task forwarding server; the method includes: S1: Build a network model for determining the inference task offloading node, define the MEC processing server composed of the MEC task computing server and the MEC task forwarding server, and the inference application generated by each terminal device, wherein the inference application is used to issue the inference task; S2: Establish a delay model and calculate the total inference delay of the inference application on the terminal device; S3: Establish an energy consumption model to determine the total inference energy consumption of the inference task on the terminal device; S4: Determine the offloading strategy generated by the MEC task scheduling server based on the total inference delay and the total inference energy consumption, and establish a multi-objective optimization function of the offloading strategy based on entropy, wherein the offloading strategy includes an offloading node and a resource allocation amount; S5: Using the multi-objective optimization function value as a reward function, and building an intelligent reinforcement learning model based on the historical unloading strategy; S6: Get real-time reasoning tasks; S7: Inputting the real-time reasoning task into the intelligent reinforcement learning model and outputting an optimal offloading strategy; The intelligent reinforcement learning model includes an input layer module, a transformer block sequence module, an action prediction module, an intelligent acquisition module and a neural network update module; The input layer module is used to receive input data including time steps, states and actions in trajectories and perform self-attention operations within each independent trajectory; The transformer block sequence module is used to extract features from the input data to ensure the sequential integrity of the data, wherein the transformer block sequence module includes a causal attention unit and a multi-layer perceptron unit; The action prediction module is used to extract information from the output of the last masked causal self-attention unit, and the output includes the action prediction result for each time step in the sequence; The intelligent acquisition module and the neural network update module are used to calculate the increment of the multi-objective optimization function value; The S5 specifically includes: S501: Acquire trajectory data with an optimal offloading strategy, wherein the trajectory data includes a state, an action representing the offloading strategy, and a total inference delay, a task processing accuracy, and a total inference energy consumption; S502: Input the trajectory data into the intelligent reinforcement learning model for training, and determine the hyperparameters of the intelligent reinforcement learning model, wherein the hyperparameters include the embedding length of the intelligent reinforcement learning model, the input sequence length, the maximum number of training iterations, the number of updates per training iteration, the maximum number of test iterations, the number of updates per test iteration, the sample set, the sample batch size, and the learning rate; S503: Calculating a loss function value of the intelligent reinforcement learning model, wherein the loss function is a KL divergence value between a real state generated based on the action and a predicted state of the intelligent reinforcement learning model; S504: If the loss function value is less than a preset loss function value and the increment of the multi-objective optimization function value is less than a preset increment, output the current action; otherwise, change the hyperparameter and return to step S502; S505: Output the intelligent reinforcement learning model under the current hyperparameters.

2. The method for adaptive task offloading based on user needs according to claim 1, characterized in that: Said S1 specifically includes: S101: Define MEC task processing server: M * ={1,2,...,M+1} Among them, M * Represents a MEC task processing server consisting of M MEC task computing servers and a single MEC task forwarding server. M+1 represents the tag sequence number of the MEC task forwarding server. S102: Define the terminal device for the reasoning task: u={1,2,…,U} Wherein, u represents the terminal device tag, and U represents the total number of terminal devices; S103: Define the inference application generated by each terminal device: in, represents the nth inference application on the uth terminal device; S104: Construction To define the dependency relationship between task nodes: in, express A directed acyclic graph of Represents a collection of task nodes, Represents the adjacency matrix used to represent the relationship between each task node; S105: Define reasoning tasks: in, Representation reasoning application The i-th reasoning task issued, Indicates that the number of processing programs required for the i-th reasoning task is k, i.e., k types, where the type of reasoning task is K = {1, 2, ..., K}, Representing reasoning tasks The input data size, Representing reasoning tasks The output data size; S106: Analyze the reasoning task type supported by the MEC task server based on the obtained network model, and determine the reasoning task offloading node, wherein the network model is composed of M, u, composition.

3. The method for adaptive task offloading based on user needs according to claim 2, characterized in that: The delay model is specifically: When the inference task is offloaded to the MEC task computing server: When the inference task is offloaded to the cloud center: in, Representing reasoning tasks The total delay, Representation reasoning application The i-th reasoning task issued, max means taking the maximum value, Representing reasoning tasks The set of all predecessor tasks, Representation reasoning application The jth reasoning task issued, Representing reasoning tasks The total delay, Representing reasoning tasks and reasoning tasks The data transmission delay between Representing reasoning tasks The queue delay, Representing reasoning tasks The execution delay, Representing reasoning tasks The output data size of m1 and m2 represents the MEC task computing server and cloud center respectively. Representing reasoning tasks Transfer from m1 to inference task The transmission rate between nodes m2, Represents the total inference latency of the inference application on the terminal device, Representing reasoning tasks and transmission delay between the terminal device, Representing reasoning tasks Queue delay when offloading to the MEC task processing server, ρ1 represents the inference task In MEC task computing server utilization, P Q Indicates the probability that the terminal device needs to wait, represents the inference task offloaded to the cloud center, Representing reasoning tasks The queue delay when offloading to the cloud center, λ1 represents the inference task In the MEC task calculation server arrival rate, λ2 represents the inference task The arrival rate at the cloud center, μ1 represents the inference task The service rate of the task offloaded to the MEC computing server, μ2 represents the inference task The service rate offloaded to the cloud center, M represents the total number of MEC servers, m represents the total number of running MEC servers, M! represents the factorial of M, and m! represents the factorial of m.

4. The method for adaptive task offloading based on user needs according to claim 1, characterized in that: The energy consumption model is specifically: When the reasoning task is the first task, and the reasoning task When offloaded to the MEC computing server: When the reasoning task is the first task, and the reasoning task When offloaded to the cloud center: When the reasoning task There are pre-tasks and reasoning tasks When offloaded to the cloud center: in, represents the total inference energy consumption of the inference application on the terminal device, Representing reasoning tasks The total energy consumption, represents the transmission energy consumption of the last inference task transmitted to the terminal device, represents the transmission power of the MEC server marked as m, Representing reasoning tasks The maximum output data, Representing reasoning tasks and reasoning tasks The transmission energy consumption between Representing reasoning tasks Energy consumption generated by the inference task offloading node, Indicates the transmission power of the terminal equipment, represents the transmission rate between the terminal device and the inference task offloading node, represents the transmission energy consumption between the inference task offloading node and the cloud center, κ m,c represents the transmission delay coefficient between the inference task offloading node and the cloud center, Represents the transmission energy consumption from the cloud center to the MEC task processing server.

5. The method for adaptive task offloading based on user needs according to claim 1, characterized in that: The multi-objective optimization function is specifically: in, Indicates the amount of task offloading and the wireless resource allocation amount W C Unloading strategy, δ represents the adjustment factor for adjusting KL divergence, represents the accuracy index of task processing, represents the evaluation distribution of the reasoning task, p Φ (·) represents the true distribution of the reasoning task, represents the KL divergence of the evaluation distribution of the reasoning task relative to the true distribution of the reasoning task, represents the entropy of the evaluation distribution of the inference task, represents the total inference latency, Indicates the maximum inference latency allowed by the terminal device. Indicates the minimum accuracy index of task processing, represents the total inference energy consumption, Indicates the maximum energy consumption allowed by the terminal device. represents the offloading strategy of the i-th task of the inference task, Indicates the i-th task type on the n-th terminal device, Indicates whether the MEC task computing server can support the task type 6. The method for adaptive task offloading based on user needs according to claim 1, characterized in that: The calculation method of the loss function value is: loss=D KL (a preds ,a target ) Among them, loss represents the loss function value, a preds represents the predicted action of the uninstallation strategy, a target Indicates the actual action of uninstalling the policy.

7. A user-demand adaptive task offloading system, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method for adaptive task offloading according to user needs as claimed in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for adaptive task offloading based on user needs according to any one of claims 1 to 6 is implemented.

Images