Self-adaptive task unloading method and system according to user requirements

Abstract

The invention provides a user-required adaptive task unloading method and system, and relates to the field of deep reinforcement learning, and the method comprises the steps: constructing a network model used for determining reasoning task unloading nodes; establishing a delay model, and calculating the total reasoning delay of the reasoning application on the terminal equipment; establishing an energy consumption model, and determining the total reasoning energy consumption of the reasoning task on the terminal equipment; determining an unloading strategy generated by the MEC task scheduling server in combination with the total reasoning delay and the total reasoning energy consumption, and establishing a multi-objective optimization function of the unloading strategy based on entropy; taking a multi-objective optimization function value as a reward function, and constructing an intelligent reinforcement learning model based on a historical unloading strategy; acquiring a real-time reasoning task; and inputting the real-time reasoning task into the intelligent reinforcement learning model, and outputting an optimal unloading strategy. According to the invention, the generalization ability of the calculation unloading scheme and the adaptability of the unloading strategy are improved.

Description

Technical Field

[0001] The present invention relates to the field of deep reinforcement learning, and in particular to a method and system for adaptive task offloading based on user needs. Background Art

[0002] Computation offloading is a technology that transfers computing tasks from resource-constrained devices (such as smartphones, IoT devices, etc.) to more powerful computing platforms (such as cloud servers, edge servers, or other high-performance devices). This approach can significantly improve the computing power of devices, save battery power, reduce latency, improve processing efficiency, and reduce the burden on local devices. Computation offloading is widely used in Mobile Edge Computing (MEC) and Mobile Cloud Computing (MCC), and has become an important technology to improve network performance and computing efficiency.

[0003] The deep integration of artificial intelligence and industry can greatly improve production efficiency, optimize resource allocation, and reduce production costs, providing strong technical support and impetus for the development of industrial production. The recently emerged Large Language Model (LLMs) is mainly used to process and generate natural language text. It is an advanced form of artificial intelligence technology. LLMs are good at understanding complex languages ​​and can generate customized responses and content according to the needs of different users or enterprises. In the industrial Internet, LLMs have many advantages in improving efficiency, optimizing decision-making processes and increasing automation. For example, it enables operators to interact with machines using natural language instructions. In addition, as an intelligent assistant, LLMs can provide real-time help, guide operating procedures and troubleshooting suggestions in complex environments, thereby improving employee work efficiency.

[0004] However, large language models usually contain billions to trillions of parameters. Running these models requires a lot of computing resources and dedicated hardware. The computing power and energy of industrial equipment are limited, and it is difficult to meet the high performance requirements of LLMs. It is difficult for industrial equipment to directly run large language models. Traditional RL algorithms require precise definition of reward functions. Suitable reward functions are very complex and require careful adjustment of the weights of different factors. Moreover, reward functions are only applicable to specific scenarios or needs. When the scenarios or needs change, the reward function needs to be redesigned, which limits the adaptability of the strategy. Due to the limitations and biases of human cognition, the designed reward function may be imperfect, which makes it difficult for the intelligent agent to find the optimal strategy. Usually, it can only choose suboptimal strategies, affecting overall performance. Summary of the invention

[0005] In order to solve the problem that large language models in the prior art usually contain billions to trillions of parameters, running these models requires a lot of computing resources and dedicated hardware, the computing power and energy of industrial equipment are limited, it is difficult to meet the high performance requirements of LLMs, and it is difficult for industrial equipment to directly run large language models. Traditional RL algorithms require precise definition of reward functions, and suitable reward functions are very complex, requiring careful adjustment of the weights of different factors, and reward functions are only applicable to specific scenarios or needs. When the scenarios or needs change, the reward function needs to be redesigned, which limits the adaptability of the strategy. Due to the limitations and biases of human cognition, the designed reward function may be imperfect, which makes it difficult for the intelligent agent to find the optimal strategy and usually can only choose suboptimal strategies, affecting the overall performance. The present invention provides a user-demand adaptive task offloading method and system.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect

[0008] An adaptive task offloading method for user needs provided by an embodiment of the present invention is applied to an adaptive task offloading platform, wherein the adaptive task offloading platform includes a cloud center, multiple terminal devices and multiple MEC servers, wherein each terminal device is connected to the cloud center through a MEC server, and each MEC server is connected to each other, wherein the MEC server includes a MEC task scheduling server, a MEC task computing server and a MEC task forwarding server; the method includes:

[0009] S1: Build a network model for determining the inference task offloading node, define the MEC processing server composed of the MEC task computing server and the MEC task forwarding server, and the inference application generated by each terminal device, where the inference application is used to issue the inference task;

[0010] S2: Establish a latency model and calculate the total inference latency of the inference application on the terminal device;

[0011] S3: Establish an energy consumption model to determine the total inference energy consumption of the inference task on the terminal device;

[0012] S4: Determine the offloading strategy generated by the MEC task scheduling server based on the total inference delay and the total inference energy consumption, and establish a multi-objective optimization function of the offloading strategy based on entropy, where the offloading strategy includes offloading nodes and resource allocation amounts;

[0013] S5: Use the multi-objective optimization function value as the reward function and build an intelligent reinforcement learning model based on the historical unloading strategy;

[0014] S6: Get real-time reasoning tasks;

[0015] S7: Input the real-time reasoning task into the intelligent reinforcement learning model and output the optimal offloading strategy.

[0016] Second aspect

[0017] An embodiment of the present invention provides a user demand-based adaptive task offloading system, comprising:

[0018] processor;

[0019] A memory having computer-readable instructions stored therein, wherein when the computer-readable instructions are executed by the processor, the adaptive task offloading method according to the user's needs as in the first aspect is implemented.

[0020] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0021] In the present invention, by constructing a network model, the reasoning tasks are hierarchically offloaded to the terminal devices, edge computing servers and cloud centers. The delay model and energy consumption model are combined to accurately calculate the time and energy of each offloading strategy, which greatly reduces the computing burden of the terminal devices and reduces the dependence on high-performance hardware. The task processing method with the lowest energy consumption is selected, the life of the terminal devices is extended, and the energy efficiency of the system is improved. The adaptive task offloading platform is used to comprehensively consider the delay and energy consumption through multi-objective optimization, dynamically determine the offloading nodes and resource allocation amount of the task, and intelligently allocate computing resources, so that the reasoning tasks can be efficiently allocated and processed, avoiding resource waste, and supporting the allocation of different reasoning tasks to different The computing layer flexibly adapts to the resource requirements of the task and improves the load handling capacity of the system. By establishing an intelligent reinforcement learning model, it no longer relies on a single linear weighted reward function, but directly optimizes the unloading strategy through multi-dimensional indicators, which can more accurately adapt to the actual task requirements. The loss function based on the KL divergence guides the model to approach the real strategy, improves the strategy quality and optimality of the intelligent agent, and enables it to find the best solution in a complex environment, making the system more flexible and adaptable. The intelligent reinforcement learning model automatically learns historical data, avoids the complexity of artificially designed reward functions, improves the generalization ability of the model, is suitable for a variety of industrial scenarios, and improves the application breadth of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1A flowchart of a method for adaptive task offloading based on user needs provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of the structure of an adaptive task offloading platform provided by an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of the structure of a user-demand adaptive task offloading system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0027] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0028] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0029] Reference Manual Attached Figure 1 , showing a flow chart of a method for adaptive task offloading based on user needs provided by an embodiment of the present invention.

[0030] Reference Manual Attached Figure 2 , showing a schematic diagram of the structure of an adaptive task offloading platform provided in an embodiment of the present invention.

[0031] like Figure 2The adaptive task offloading platform includes a cloud center, multiple terminal devices and multiple MEC servers, wherein each terminal device is connected to the cloud center through a MEC server, and each MEC server is connected to each other. The MEC server includes a MEC task scheduling server, a MEC task computing server and a MEC task forwarding server. The MEC task scheduling server (MTSS) processes the LLMs reasoning requests from the device and decides whether to offload these tasks to the MEC task computing server (MTCS) or the cloud center based on factors such as environmental conditions and resource requirements. At the same time, MEC also manages the allocation of wireless resources within its coverage area to optimize network utilization. The MEC task forwarding server (MTFS) is responsible for forwarding the LLMs reasoning requests from the device to the cloud.

[0032] The embodiment of the present invention provides an adaptive task offloading method for user needs, which is applied to an adaptive task offloading platform, wherein the adaptive task offloading platform includes a cloud center, multiple terminal devices and multiple MEC servers, wherein each terminal device is connected to the cloud center through the MEC server, and each MEC server is connected to each other; the processing flow of the adaptive task offloading method for user needs may include the following steps:

[0033] S1: Build a network model for determining the inference task offloading node, define the MEC processing server composed of the MEC task computing server and the MEC task forwarding server, and the inference application generated by each terminal device, where the inference application is used to issue the inference task.

[0034] Among them, the MEC processing server includes a combination of a MEC task computing server and a MEC task forwarding server, which processes the task offloading request of the terminal device as a whole. The inference application is an application running on the terminal device, which generates inference tasks that need to be processed, such as language model inference or image recognition and other complex computing tasks. The inference task is a specific computing task generated by the inference application.

[0035] In a possible implementation, S1 specifically includes:

[0036] S101: Define MEC task processing server:

[0037] M * ={1,2,...,M+1}

[0038] Among them, M *It represents a MEC task processing server consisting of M MEC task computing servers and a single MEC task forwarding server. M+1 represents the mark number of the MEC task forwarding server.

[0039] S102: Define the terminal device for the reasoning task:

[0040] u={1,2,…,U}

[0041] Among them, u represents the terminal device tag, and U represents the total number of terminal devices.

[0042] S103: Define the inference application generated by each terminal device:

[0043]

[0044] in, Represents the nth inference application on the uth terminal device.

[0045] S104: Build To define the dependencies between task nodes:

[0046]

[0047] in, express A directed acyclic graph of Represents a collection of task nodes, Represents the adjacency matrix used to represent the relationship between task nodes.

[0048] S105: Define reasoning tasks:

[0049]

[0050] in, Representation Reasoning Application The i-th reasoning task issued, Indicates that the number of processing programs required for the i-th reasoning task is k, i.e., k types, where the type of reasoning task is K = {1, 2, …, K}, Representing reasoning tasks The input data size is Representing reasoning tasks The output data size.

[0051] S105: Analyze the inference task types supported by the MEC task server based on the obtained network model, and determine the inference task offloading node, where the network model consists of M, u, composition.

[0052] It should be noted that by building a network model that includes MEC task computing and forwarding servers, efficient allocation of reasoning tasks can be achieved, making the selection of task offloading nodes more reasonable, thereby optimizing the utilization of computing resources and task processing efficiency.

[0053] S2: Establish a latency model and calculate the total inference latency of the inference application on the terminal device.

[0054] Among them, the delay model is a mathematical model used to evaluate the time required for a task to be generated and completed, covering factors such as computing delay, transmission delay, and queuing delay. The total inference delay is the total time spent on the inference task on the terminal device, including all delay factors from generating the task to receiving the result.

[0055] In a possible implementation, the delay model is specifically:

[0056]

[0057] When the inference task is offloaded to the MEC task computing server:

[0058]

[0059] When the inference task is offloaded to the cloud center:

[0060]

[0061] in, Representing reasoning tasks The total delay, Representation Reasoning Application The i-th reasoning task issued, max means taking the maximum value, Representing reasoning tasks The set of all predecessor tasks, Representation Reasoning Application The jth reasoning task issued, Representing reasoning tasks The total delay, Representing reasoning tasks and reasoning tasks The data transmission delay between Representing reasoning tasks The queue delay, Representing reasoning tasks The execution delay of Representing reasoning tasks Output data size, m 1 and m 2 They represent the MEC task computing server and cloud center respectively. Representing reasoning tasks From m1 Transfer to inference task Node m 2 The transmission rate between represents the total inference latency of the inference application on the terminal device, Representing reasoning tasks and the transmission delay between the terminal device, Representing reasoning tasks Queue delay when offloading to the MEC task processing server, ρ 1 Representing reasoning tasks In MEC task computing server utilization, P Q represents the probability that the terminal device needs to wait, represents the inference task offloaded to the cloud center, Representing reasoning tasks Queue delay when offloading to the cloud center, λ 1 Representing reasoning tasks In the MEC task, the arrival rate of the server is calculated, λ 2 Representing reasoning tasks The arrival rate at the cloud center, μ 1 Representing reasoning tasks The service rate of the task offloaded to the MEC computing server, μ 2 Representing reasoning tasks The service rate offloaded to the cloud center, M represents the total number of MEC servers, m represents the total number of running MEC servers, M! represents the factorial of M, and m! represents the factorial of m.

[0062] Specifically, if the terminal device transmits the inference task to the MEC task processing server, the channel power gain is calculated as:

[0063]

[0064] Among them, g u,m represents the channel power gain between the terminal device and the MEC task processing server, β 0 Indicates the channel power gain at a reference distance of 1 meter, represents the distance between the terminal device and the MEC task processing server, p m and q u They represent the horizontal coordinates of the MEC task processing server and the terminal device, respectively. m Indicates the height of the signal antenna of the MEC task processing server from the ground;

[0065] Reasoning Task The transmission delay can be expressed as:

[0066]

[0067] If m = M + 1, it means the reasoning task The MEC task forwarding server forwards the data to the cloud center for execution. Since an inference application consists of multiple inference tasks, the inference tasks may be processed on different nodes. Therefore, during the execution of the inference application, data interaction may occur between MEC task computing servers or between MEC task computing servers and the cloud center. and reasoning tasks The data transmission delay between can be calculated as:

[0068]

[0069] It should be noted that the delay model provides a comprehensive assessment of the total inference time by accurately calculating various delays, enabling the system to select a low-latency offloading solution, thereby improving the real-time performance and response speed of the task.

[0070] S3: Establish an energy consumption model to determine the total inference energy consumption of the inference task on the terminal device.

[0071] Among them, the energy consumption model is a mathematical model used to calculate the energy consumption of reasoning tasks during the operation of the device, covering aspects such as transmission energy consumption and computing energy consumption. The total reasoning energy consumption is the total energy consumed to execute the entire reasoning task on the terminal device, including calculation, transmission and other energy consumption.

[0072] In a possible implementation, the energy consumption model is specifically:

[0073]

[0074] When the reasoning task is the first task, and the reasoning task When offloaded to the MEC computing server:

[0075]

[0076] When the reasoning task is the first task, and the reasoning task When offloaded to the cloud center:

[0077]

[0078] When the reasoning task There are pre-tasks and reasoning tasks When offloaded to the cloud center:

[0079]

[0080] in, represents the total inference energy consumption of the inference application on the terminal device, Representing reasoning tasks The total energy consumption, represents the transmission energy consumption of the last inference task transmitted to the terminal device, Indicates the transmission power of the MEC server, Representing reasoning tasks The maximum output data, Representing reasoning tasks and reasoning tasks The transmission energy consumption between Representing reasoning tasks The energy consumption generated by the inference task offloading node, represents the transmission power of the terminal equipment, represents the transmission rate between the terminal device and the inference task offloading node, represents the transmission energy consumption between the inference task offloading node and the cloud center, κ m,c represents the transmission delay coefficient between the inference task offloading node and the cloud center, Represents the transmission energy consumption from the cloud center to the MEC task processing server.

[0081] The energy consumption model can comprehensively evaluate the energy consumption of reasoning tasks on terminal devices, enabling the system to give priority to low-energy consumption solutions when allocating tasks, thereby extending the battery life of the device and improving resource utilization efficiency. The model optimizes the power management of the device, is suitable for energy-sensitive industrial Internet scenarios, and helps to achieve green and energy-saving computing.

[0082] S4: Determine the offloading strategy generated by the MEC task scheduling server based on the total inference delay and the total inference energy consumption, and establish a multi-objective optimization function of the offloading strategy based on entropy, where the offloading strategy includes offloading nodes and resource allocation amounts.

[0083] Among them, the offloading strategy is a task processing plan generated by the MEC task scheduling server, which determines whether the inference task is calculated on the local device, MEC node or cloud center, and also includes the amount of resource allocation. Entropy is an information theory metric used to measure uncertainty. The multi-objective optimization function is an objective function that contains multiple performance indicators (such as latency, energy consumption, etc.), which is used to optimize the offloading strategy and ensure that the system achieves a balance between multiple objectives.

[0084] In a possible implementation, the multi-objective optimization function is specifically:

[0085]

[0086] in, Indicates the amount of task offloading and the wireless resource allocation amount W C The unloading strategy, δ represents the adjustment factor for adjusting the KL divergence, Represents the accuracy index of task processing, represents the evaluation distribution of the reasoning task, p Φ (·) represents the true distribution of the reasoning task, represents the KL divergence of the evaluation distribution of the reasoning task relative to the true distribution of the reasoning task, represents the entropy of the evaluation distribution of the inference task, represents the total inference latency, Indicates the highest inference latency allowed by the terminal device. Represents the minimum accuracy index of task processing, represents the total inference energy consumption, Indicates the maximum energy consumption allowed by the terminal device. represents the offloading strategy of the i-th task of the reasoning task, Indicates the i-th task type on the n-th terminal device, Indicates whether the MEC task computing server can support the task type

[0087] Specifically, in this way, KL divergence, as a metric for distribution comparison, can directly indicate the quality of the distribution with respect to the key variable, which is the total delay in the multi-objective optimization function. Accuracy metrics for each inference application and total energy consumption The present invention recognizes the distribution of these variables by characterizing user needs as a multidimensional random variable, so that performance standards can be evaluated directly and multidimensionally. In constraint C1, the total reasoning delay for solving reasoning applications shall not exceed the maximum delay allowed by the terminal device. In constraint C2, the accuracy of solving reasoning tasks on the terminal device should exceed the minimum accuracy. In constraint C3, the total energy consumption for solving reasoning tasks on the terminal device should not exceed the maximum energy consumption. For each reasoning task and MEC task computing server, constraint C4 must hold. These constraints ensure that if the task is offloaded to a MEC task computing server, the MEC task computing server must have the ability to process specific task types.

[0088] It should be noted that by combining the total inference delay and the total inference energy consumption to establish a multi-objective optimization function, we can achieve the best balance between different performance indicators and generate the optimal offloading strategy. This method effectively improves the flexibility and accuracy of task scheduling. It can not only meet the needs of low latency and low energy consumption, but also dynamically adapt to resource changes in different scenarios, thereby improving the overall computing efficiency and stability of the system.

[0089] S5: The multi-objective optimization function value is used as the reward function to build an intelligent reinforcement learning model based on the historical unloading strategy.

[0090] Among them, the reward function is a function used to measure the effectiveness of the agent's actions in reinforcement learning. The agent learns the optimal strategy by maximizing the reward. The historical unloading strategy is the strategy data used by the system in the past task unloading process, which contains the experience of success and failure and is used to guide the training of the intelligent reinforcement learning model. The intelligent reinforcement learning model is an intelligent model based on the reinforcement learning algorithm. It continuously learns and optimizes the task unloading strategy through the reward and punishment mechanism to achieve more efficient task processing.

[0091] In one possible implementation, the intelligent reinforcement learning model includes an input layer module, a transformer block sequence module, an action prediction module, an intelligent acquisition module, and a neural network update module.

[0092] The input layer module is used to receive input data including time steps, states, and actions in trajectories, and perform self-attention operations within each individual trajectory.

[0093] Among them, in reinforcement learning, a trajectory is a sequence of states and rewards observed after an agent performs an action in the environment. Each trajectory records the interaction process of the agent over a period of time, including information such as time step, state, and action. The time step refers to the time unit of each step in the reinforcement learning process. Each time step represents an interaction between the agent and the environment, including observing the state of the environment, performing actions, and obtaining rewards. The state is the information description of the environment at each time step, which is used to represent the current state of the environment. The state helps the agent understand the current environment and provides a basis for selecting the next action. The action is the action taken by the agent in a certain state, which affects the state change of the environment. Self-attention can assign different importance weights to each data point to help the model understand the key information and associations in the trajectory.

[0094] The transformer block sequence module is used to extract features from input data and ensure the sequential integrity of the data, wherein the transformer block sequence module includes a causal attention unit and a multi-layer perceptron unit.

[0095] The action prediction module is used to extract information from the output of the last masked causal self-attention unit, and the output includes the action prediction results for each time step in the sequence.

[0096] The intelligent acquisition module and the neural network update module are used to calculate the increment of the multi-objective optimization function value.

[0097] In a possible implementation, S5 specifically includes:

[0098] S501: Acquire trajectory data with an optimal offloading strategy, wherein the trajectory data includes a state, an action representing the offloading strategy and a total reasoning delay, a task processing accuracy, and a total reasoning energy consumption.

[0099] S502: Input the trajectory data into the intelligent reinforcement learning model for training, and determine the hyperparameters of the intelligent reinforcement learning model, wherein the hyperparameters include the embedding length of the intelligent reinforcement learning model, the input sequence length, the maximum number of training iterations, the number of updates for each training iteration, the maximum number of test iterations, the number of updates for each test iteration, the sample set, the sample batch size, and the learning rate.

[0100] S503: Calculate the loss function value of the intelligent reinforcement learning model, where the loss function is a KL divergence value between a real state generated based on the action and a predicted state of the intelligent reinforcement learning model.

[0101] S504: When the loss function value is less than the preset loss function value and the increment of the multi-objective optimization function value is less than the preset increment, output the current action; otherwise, change the hyperparameters and return to step S502.

[0102] S505: Output the intelligent reinforcement learning model under the current hyperparameters.

[0103] In a possible implementation, the loss function value is calculated as follows:

[0104] loss = D KL (a preds ,a target )

[0105] Among them, loss represents the loss function value, a preds represents the predicted action of the uninstallation strategy, a target Indicates the actual action of uninstalling the policy.

[0106] It should be noted that by taking the multi-objective optimization function value as the reward function, the intelligent reinforcement learning model can dynamically adapt to different task requirements, automatically optimize the unloading strategy, reduce human intervention, and improve the accuracy and efficiency of decision-making. By using historical unloading strategy data, the model can converge quickly and enhance generalization capabilities, thereby achieving efficient task allocation and resource management in a complex industrial Internet environment, which helps to improve the system's adaptability and robustness.

[0107] S6: Get real-time reasoning tasks.

[0108] It should be noted that by obtaining real-time reasoning tasks, it is possible to respond to the latest computing needs in a timely manner, dynamically adjust and optimize the offloading strategy, thereby significantly improving the real-time and accuracy of task processing.

[0109] S7: Input the real-time reasoning task into the intelligent reinforcement learning model and output the optimal offloading strategy.

[0110] It should be noted that after inputting the real-time reasoning task into the intelligent reinforcement learning model, the system can quickly generate the optimal offloading strategy, realize the optimization of dynamic resource allocation and task scheduling, and significantly improve the processing efficiency and system response speed.

[0111] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0112] In the present invention, by constructing a network model, the reasoning tasks are hierarchically offloaded to the terminal devices, edge computing servers and cloud centers. The delay model and energy consumption model are combined to accurately calculate the time and energy of each offloading strategy, which greatly reduces the computing burden of the terminal devices and reduces the dependence on high-performance hardware. The task processing method with the lowest energy consumption is selected, the life of the terminal devices is extended, and the energy efficiency of the system is improved. The adaptive task offloading platform is used to comprehensively consider the delay and energy consumption through multi-objective optimization, dynamically determine the offloading nodes and resource allocation amount of the task, and intelligently allocate computing resources, so that the reasoning tasks can be efficiently allocated and processed, avoiding resource waste, and supporting the allocation of different reasoning tasks to different The computing layer flexibly adapts to the resource requirements of the task and improves the load handling capacity of the system. By establishing an intelligent reinforcement learning model, it no longer relies on a single linear weighted reward function, but directly optimizes the unloading strategy through multi-dimensional indicators, which can more accurately adapt to the actual task requirements. The loss function based on the KL divergence guides the model to approach the real strategy, improves the strategy quality and optimality of the intelligent agent, and enables it to find the best solution in a complex environment, making the system more flexible and adaptable. The intelligent reinforcement learning model automatically learns historical data, avoids the complexity of artificially designed reward functions, improves the generalization ability of the model, is suitable for a variety of industrial scenarios, and improves the application breadth of the algorithm.

[0113] Reference Manual Attached Figure 3 , showing a structural schematic diagram of a user demand-based adaptive task offloading system provided by the present invention.

[0114] The present invention further provides a user demand-based adaptive task offloading system 20, which is applied to the above-mentioned user demand-based adaptive task offloading method, comprising:

[0115] Processor 201.

[0116] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201 , the method for adaptive task offloading based on user needs in the method embodiment is implemented.

[0117] The user demand adaptive task offloading system 20 provided by the present invention can execute the above-mentioned user demand adaptive task offloading method and achieve the same or similar technical effects. To avoid repetition, the present invention will not elaborate on it.

[0118] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0119] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0120] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0121] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0122] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0123] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0124] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0126] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0127] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0129] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0130] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for adaptively offloading tasks according to user needs in the method embodiment is implemented.

[0131] A computer-readable storage medium provided by the present invention can implement the steps and effects of the user-demand adaptive task offloading method of the above method embodiment, and to avoid repetition, the present invention will not go into details.

[0132] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0133] There are a few points to note:

[0134] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0135] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0136] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0137] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An adaptive task offloading method based on user needs, characterized in that: Applied to an adaptive task offloading platform, wherein the adaptive task offloading platform includes a cloud center, multiple terminal devices and multiple MEC servers, wherein each terminal device is connected to the cloud center through a MEC server, and each MEC server is connected to each other, wherein the MEC server includes a MEC task scheduling server, a MEC task computing server and a MEC task forwarding server; the method includes: S1: Construct a network model for determining the inference task offloading node, define the MEC processing server composed of the MEC task computing server and the MEC task forwarding server, and the inference application generated by each terminal device, wherein the inference application is used to issue the inference task; S2: Establish a delay model and calculate the total inference delay of the inference application on the terminal device; S3: Establishing an energy consumption model to determine the total inference energy consumption of the inference task on the terminal device; S4: Determine the offloading strategy generated by the MEC task scheduling server in combination with the total inference delay and the total inference energy consumption, and establish a multi-objective optimization function of the offloading strategy based on entropy, wherein the offloading strategy includes an offloading node and a resource allocation amount; S5: Using the multi-objective optimization function value as a reward function, and building an intelligent reinforcement learning model based on the historical unloading strategy; S6: Get real-time reasoning tasks; S7: Input the real-time reasoning task into the intelligent reinforcement learning model, and output the optimal offloading strategy.

2. The method for adaptive task offloading based on user needs according to claim 1, characterized in that: The S1 specifically includes: S101: Define MEC task processing server: M * ={1,2,...,M+1} Among them, M * It represents a MEC task processing server consisting of M MEC task computing servers and a single MEC task forwarding server. M+1 represents the mark number of the MEC task forwarding server. S102: Define the terminal device for the reasoning task: u={1,2,…,U} Wherein, u represents the terminal device tag, and U represents the total number of terminal devices; S103: Define the inference application generated by each terminal device: in, represents the nth inference application on the uth terminal device; S104: Build To define the dependencies between task nodes: in, express A directed acyclic graph of Represents a collection of task nodes, represents the adjacency matrix used to represent the relationship between each task node; S105: Define reasoning tasks: in, Representation Reasoning Application The i-th reasoning task issued, Indicates that the number of processing programs required for the i-th reasoning task is k, i.e., k types, where the type of reasoning task is K = {1, 2, …, K}, Representing reasoning tasks The input data size is Representing reasoning tasks The output data size of S106: Analyze the reasoning task type supported by the MEC task server based on the obtained network model, and determine the reasoning task offloading node, wherein the network model is composed of M, u, composition.

3. The method for adaptive task offloading based on user needs according to claim 2, characterized in that: The delay model is specifically: When the reasoning task is offloaded to the MEC task computing server: When the reasoning task is offloaded to the cloud center: in, Representing reasoning tasks The total delay, Representation Reasoning Application The i-th reasoning task issued, max means taking the maximum value, Representing reasoning tasks The set of all predecessor tasks of Representation Reasoning Application The jth reasoning task issued, Representing reasoning tasks The total delay, Representing reasoning tasks and reasoning tasks The data transmission delay between Representing reasoning tasks The queue delay, Representing reasoning tasks The execution delay of Representing reasoning tasks The output data size of m1 and m2 represent the MEC task computing server and cloud center respectively. Representing reasoning tasks Transfer from m1 to inference task The transmission rate between nodes m2, represents the total inference latency of the inference application on the terminal device, Representing reasoning tasks and the transmission delay between the terminal device, Representing reasoning tasks Queue delay when offloading to the MEC task processing server, ρ1 represents the inference task In MEC task computing server utilization, P Q represents the probability that the terminal device needs to wait, represents the inference task offloaded to the cloud center, Representing reasoning tasks Queue delay when offloading to the cloud center, λ1 represents the inference task In the MEC task, the arrival rate of the server is calculated, and λ2 represents the inference task. The arrival rate at the cloud center, μ1 represents the inference task The service rate of the task offloaded to the MEC computing server, μ2 represents the inference task The service rate offloaded to the cloud center, M represents the total number of MEC servers, m represents the total number of running MEC servers, M! represents the factorial of M, and m! represents the factorial of m.

4. The method for adaptive task offloading based on user needs according to claim 1, characterized in that: The energy consumption model is specifically: When the reasoning task is the first task, and the reasoning task When offloaded to the MEC computing server: When the reasoning task is the first task, and the reasoning task When offloaded to the cloud center: When the reasoning task There are pre-tasks and reasoning tasks When offloaded to the cloud center: in, represents the total inference energy consumption of the inference application on the terminal device, Representing reasoning tasks The total energy consumption, represents the transmission energy consumption of the last inference task transmitted to the terminal device, represents the transmission power of the MEC server marked as m, Representing reasoning tasks The maximum output data, Representing reasoning tasks and reasoning tasks The transmission energy consumption between Representing reasoning tasks The energy consumption generated by the inference task offloading node, represents the transmission power of the terminal equipment, represents the transmission rate between the terminal device and the inference task offloading node, represents the transmission energy consumption between the inference task offloading node and the cloud center, κ m,c represents the transmission delay coefficient between the inference task offloading node and the cloud center, Represents the transmission energy consumption from the cloud center to the MEC task processing server.

5. The method for adaptive task offloading based on user needs according to claim 1, characterized in that: The multi-objective optimization function is specifically: in, Indicates the amount of task offloading and the wireless resource allocation amount W C The unloading strategy, δ represents the adjustment factor for adjusting the KL divergence, Represents the accuracy index of task processing, represents the evaluation distribution of the reasoning task, p Φ (·) represents the true distribution of the reasoning task, represents the KL divergence of the evaluation distribution of the reasoning task relative to the true distribution of the reasoning task, represents the entropy of the evaluation distribution of the inference task, represents the total inference latency, Indicates the highest inference latency allowed by the terminal device. Represents the minimum accuracy index of task processing, represents the total inference energy consumption, Indicates the maximum energy consumption allowed by the terminal device. represents the offloading strategy of the i-th task of the reasoning task, Indicates the i-th task type on the n-th terminal device, Indicates whether the MEC task computing server can support the task type 6. The method for adaptive task offloading based on user needs according to claim 1, characterized in that: The intelligent reinforcement learning model includes an input layer module, a transformer block sequence module, an action prediction module, an intelligent acquisition module and a neural network update module; The input layer module is used to receive input data including time steps, states and actions in trajectories, and perform self-attention operations within each independent trajectory; The transformer block sequence module is used to extract features from the input data to ensure the sequential integrity of the data, wherein the transformer block sequence module includes a causal attention unit and a multi-layer perceptron unit; The action prediction module is used to extract information from the output of the last masked causal self-attention unit, and the output includes the action prediction result for each time step in the sequence; The intelligent acquisition module and the neural network update module are used to calculate the increment of the multi-objective optimization function value.

7. The method for adaptive task offloading based on user needs according to claim 1, characterized in that: The S5 specifically includes: S501: Acquire trajectory data with an optimal offloading strategy, wherein the trajectory data includes a state, an action representing the offloading strategy and a total reasoning delay, a task processing accuracy, and a total reasoning energy consumption; S502: Input the trajectory data into the intelligent reinforcement learning model for training, and determine the hyperparameters of the intelligent reinforcement learning model, wherein the hyperparameters include the embedding length, input sequence length, maximum number of training iterations, number of updates for each training iteration, maximum number of test iterations, number of updates for each test iteration, sample set, sample batch size, and learning rate of the intelligent reinforcement learning model; S503: Calculating a loss function value of the intelligent reinforcement learning model, wherein the loss function is a KL divergence value between a real state generated based on the action and a predicted state of the intelligent reinforcement learning model; S504: if the loss function value is less than a preset loss function value and the increment of the multi-objective optimization function value is less than a preset increment, output the current action; otherwise, change the hyperparameter and return to step S502; S505: Output the intelligent reinforcement learning model under the current hyperparameters.

8. The method for adaptive task offloading based on user needs according to claim 7, characterized in that: The calculation method of the loss function value is: loss=D KL (a preds ,a target ) Among them, loss represents the loss function value, a preds represents the predicted action of the uninstallation strategy, a target Indicates the actual action of uninstalling the policy.

9. A user-demand adaptive task offloading system, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method for adaptive task offloading according to user needs as claimed in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for adaptive task offloading according to user needs as claimed in any one of claims 1 to 8 is implemented.

Images