Task collaborative reasoning methods, devices, electronic equipment and media

Abstract

This invention provides a task collaborative reasoning method, apparatus, electronic device, and medium, relating to the field of computer technology. The method includes: dividing a task arriving on a mobile device into multiple subtasks and constructing an offloading strategy for each subtask to characterize execution on the mobile device or MEC server; constructing an average overhead minimization problem for all mobile devices within a time slot for each mobile device; reconstructing the average overhead minimization problem using Lyapunov optimization theory to obtain an independent time slot policy optimization problem corresponding to the average overhead minimization problem; transforming the independent time slot policy optimization problem into a Markov decision process and solving the Markov decision process using a deep reinforcement learning algorithm to obtain the optimal offloading strategy, optimal mobile device transmission power, and optimal number of threads allocated to the MEC server for each mobile device. The invention effectively utilizes distributed computing resources and significantly improves reasoning efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a task collaborative reasoning method, apparatus, electronic device, and medium. Background Technology

[0002] With the rapid development and widespread application of deep learning technology, especially with the continuous emergence of successful cases in fields such as computer vision and natural language processing, deep neural networks (DNNs) have become a major tool for solving complex tasks. However, with the increase in model size and the growing popularity of real-time applications, traditional methods of performing inference on a single device are facing severe challenges.

[0003] Deep neural networks typically employ a Directed Acyclic Graph (DAG) structure to model complex features and relationships. In such networks, collaborative inference at the offloading layer becomes particularly important to meet the growing demands for real-time performance and efficiency. While technologies such as distributed deep learning and edge computing have made significant progress, several key challenges remain in the collaborative inference process at the offloading layer. Furthermore, communication bottlenecks between servers and uneven resource allocation hinder the efficiency of the inference process, thus requiring urgent improvement. Summary of the Invention

[0004] This invention provides a task collaborative reasoning method, apparatus, electronic device, and medium to solve the defects in the prior art, such as communication bottlenecks between servers and uneven resource allocation, which hinder the efficiency of the reasoning process, and realize the full utilization of distributed computing resources to perform task collaborative reasoning.

[0005] According to a first aspect of the present invention, a task collaborative reasoning method is provided, applicable to a complex heterogeneous scenario including multiple mobile devices and multiple MEC servers, comprising:

[0006] Based on the DNN task model structure deployed on each mobile device, the tasks arriving on the corresponding mobile device are divided into multiple sub-tasks, and an offloading strategy is constructed for each sub-task to characterize the execution on the mobile device or MEC server, wherein each mobile device corresponds to a task queue.

[0007] With the goals of a joint offloading strategy, mobile device transmission power, and the number of threads allocated to the MEC server for executing offloaded subtasks, and with constraints such as mobile device CPU cycle frequency requirements, task queue length requirements, long-term energy consumption requirements, and task inference latency requirements, we construct the problem of minimizing the average overhead of all mobile devices within a time slot.

[0008] The strong coupling problem between variables in different time slots in the average cost minimization problem is reconstructed using Lyapunov optimization theory to obtain the independent time slot policy optimization problem corresponding to the average cost minimization problem.

[0009] The independent time slot policy optimization problem is transformed into a Markov decision process, and a deep reinforcement learning algorithm is used to solve the Markov decision process to obtain the optimal offloading policy, the optimal mobile device transmission power, and the optimal number of threads allocated to the MEC server for each mobile device.

[0010] In some possible implementations, the DNN task model structure deployed on each mobile device divides the task arriving on the corresponding mobile device into multiple subtasks, including:

[0011] The mobile devices in the complex and heterogeneous scenario are identified as target mobile devices through a traversal approach, and the following operations are performed on the target mobile devices:

[0012] The DNN task model deployed on the target mobile device is modeled as a directed acyclic graph, wherein the directed acyclic graph includes nodes representing each inference layer and edges connecting each node.

[0013] Subtasks are determined based on the computational tasks of the inference layer corresponding to each node in the directed acyclic graph.

[0014] The execution order and dependencies between subtasks are determined based on the edges in the directed acyclic graph.

[0015] In some possible implementations, the construction of an offloading strategy for each of the subtasks to characterize the execution on the mobile device or MEC server includes:

[0016] The unloading strategy for the inference layer in the directed acyclic graph corresponding to each mobile device is defined as follows:

[0017]

[0018] In the formula, 0≤x≤L u L u γ represents the maximum number of inference layers of the DNN task model deployed on the u-th mobile device. u,0 (t) represents the input layer of the DNN task model deployed on the u-th mobile device, γ u,0 (t) = 0, v u,x This represents the x-th inference layer of the DNN task model deployed on the u-th mobile device, where inference layer v is the x-th inference layer. u,0 γ represents the input layer of the DNN task model. u,x (t) represents v u,x Uninstallation strategy, γ u,0(t) represents the offloading strategy of the input layer of the u-th mobile device, γ u,0 (t) = 0;

[0019] Define the uninstallation policy constraint for each mobile device as follows:

[0020] γ u,y (t)≤γ u,y+1 (t);

[0021] In the formula, 0≤y≤L u -1, γ u,y (t) represents the y-th inference layer of the DNN task model deployed on the u-th mobile device, γ u,y+1 (t) represents γ u,y The next inference layer after (t);

[0022] The size of the input matrix of each inference layer in the directed acyclic graph corresponding to each mobile device is defined as follows:

[0023]

[0024] In the formula, I u,x (t) represents the size of the input matrix of the u-th mobile device in time slot t, φ(v u,x ) represents the reasoning layer v u,x With reasoning layer v u,0 The ratio of the input matrix, φ(v) u,x D is obtained through historical information requested via reasoning during model training. u (t) represents the initial input data of the u-th mobile device.

[0025] In some possible implementations, the problem of minimizing the average overhead of all mobile devices within a time slot is constructed, with the objectives of a joint offloading strategy, mobile device transmission power, and the number of threads allocated to the MEC server for executing offloaded subtasks, and constraints such as mobile device CPU cycle frequency requirements, task queue length requirements, long-term energy consumption requirements, and task inference latency requirements. This includes:

[0026] Define a task queue for each mobile device to store tasks arriving within time slot t, and build a task queue backlog for each task queue;

[0027] For each mobile device, the computation latency of the mobile device, the transmission latency of data from the mobile device to the MEC server, the computation latency of the MEC server, and the download latency of data from the MEC server to the mobile device are constructed, and the sum is taken as the total latency.

[0028] For each mobile device, the computing power consumption of the mobile device and the data transmission power consumption from the mobile device to the MEC server are constructed, and the sum is taken as the total energy consumption;

[0029] For each mobile device, the total latency and the total energy consumption are weighted and summed based on preset latency and energy consumption allocation weights to obtain the total cost within time slot t;

[0030] The problem of minimizing the average cost is defined as follows:

[0031]

[0032] In the formula, N = {1, 2, ..., N} represents the set of mobile devices, N represents the number of elements in the set of mobile devices, M = {1, 2, ..., M} represents the set of MEC servers, M represents the number of elements in the set of MEC servers, t ∈ {0, 1, ..., T-1} represents the time-slot system, and T represents the given duration of the time-slot system. Let represent the set of unloading decisions. Denotes the set of transmitted power, B = {k u,m (t)} N×M A set representing the number of threads to be allocated; Let be the set of layers on the u-th mobile device within time slot t where the inference layer offload policy is 0. The set of layers whose output data from the DNN inference layer on the u-th mobile device is uploaded to the m-th MEC server. Let $C$ be the set of layers on the $u$-th mobile device whose inference layer unload policy is 1. u (t) represents the total cost of the u-th mobile device; γ u,y (t)≤γ u,y+1 (t) represents the offloading policy constraint for each mobile device; Let the queue average constraint be E{Q} corresponding to the u-th move. u Q(t) represents the average backlog of the task queue for the u-th mobile device across all time slots. u (t) represents the task queue product of the u-th mobile device. This represents the maximum value of the task queue backlog for the u-th mobile device; This represents the CPU cycle frequency constraint for the u-th mobile device. This represents the maximum CPU cycle frequency of the u-th mobile device; Constraints on the transmission power of the u-th mobile device This represents the maximum transmission power of the u-th mobile device; 1 ≤ k u,m (t)≤K represents the constraint on the number of threads allocated to the m-th MEC server, where K represents the maximum number of threads for the MEC server; E represents the long-term average energy consumption constraint. u (t) represents u mobile devices processing A u (t) Total energy consumption of the task, A u (t) represents the number of tasks to be processed by the u-th mobile device. It is assumed that all mobile devices have the same energy consumption threshold in each time slot, denoted by e(t); T u (t)≤β u T represents the inference delay requirement constraint. u (t) represents the processing A of the u-th mobile device. u (t) Total delay of the task, β u This represents the inference latency threshold for the DNN task on the u-th mobile device.

[0033] In some possible implementations, the Lyapunov optimization theory is used to reconstruct the strongly coupled problem of variables between different time slots in the average cost minimization problem, resulting in an independent time slot policy optimization problem corresponding to the average cost minimization problem, including:

[0034] Introduce N energy queues Here, the energy queue is the queue backlog of energy consumption within each time slot, and the initial energy queue backlog H u (t) = 0;

[0035] The energy queue backlog of the u-th mobile device is H. u The dynamic change of (t) can be expressed as:

[0036] H u (t+1)=max{H u (t)-e(t),0}+E u (t);

[0037] In the formula, H u (t) represents the energy queue backlog of the u-th mobile device in time slot t, H u (t+1) represents the u-th mobile device in H. u The energy queue backlog in the next time slot of (t) is represented by the energy consumption threshold e(t), which represents the service rate of the queue. Under the condition that the energy queue is stable, the long-term average energy consumption can be kept below e(t) to meet the constraints.

[0038] Based on the energy queue backlog H u The dynamic change of (t) is reconstructed using Lyapunov optimization theory to solve the long-term energy consumption constraint and average cost minimization problem, resulting in the independent time slot policy optimization problem expressed as:

[0039]

[0040] In the formula, Q u(t) represents the backlog in the task queue of the u-th mobile device, D u (t) Initial input data of the u-th mobile device, H u (t) represents the energy queue backlog of the u-th mobile device, E u (t) represents u mobile devices processing A u (t) represents the total energy consumption of the task, where V is the Lyapunov tradeoff factor, used to characterize the tradeoff between the system's long-run average cost and queue backlog. u (t) represents the total cost of the u-th mobile device, γ u,y (t)≤γ u,y+1 (t) represents the offloading policy constraint for each mobile device. This represents the CPU cycle frequency constraint for the u-th mobile device. This represents the maximum CPU cycle frequency of the u-th mobile device; Constraints on the transmission power of the u-th mobile device This represents the maximum transmission power of the u-th mobile device; 1 ≤ k u,m (t)≤K represents the constraint on the number of threads allocated to the m-th MEC server, and K represents the maximum number of threads for the MEC server; T u (t)≤β u T represents the inference delay requirement constraint. u (t) represents the processing A of the u-th mobile device. u (t) Total delay of the task, β u This represents the inference latency threshold for the DNN task on the u-th mobile device.

[0041] In some possible implementations, the transformation of the independent time-slot policy optimization problem into a Markov decision process includes:

[0042] Define a state space s(t). Within time slot t, state s(t) consists of the amount of tasks arriving on the mobile device, the CPU cycle frequency of the mobile device, the distance between the mobile device and the MEC server, the task queue backlog, and the energy queue backlog. The state space s(t) is expressed as:

[0043] s(t)={D(t),f(t),d(t),Q(t),H(t)};

[0044] In the formula, D(t)={D u (t)} N Let f(t) represent the set of task data arriving on the mobile device. u (t)} N Let d(t) represent the set of CPU cycle frequencies of a mobile device. u,m (t)} N×MLet Q(t) represent the set of distances between the u-th mobile device and the m-th MEC server. u (t)} N H(t) represents the set of backlogged task queues on a mobile device. u (t)} N The set representing the energy queue backlog of a mobile device;

[0045] Define an action space a(t). Within time slot t, action a(t) consists of offloading decisions, transmission power, and the number of allocated threads. The action space a(t) is expressed as:

[0046] a(t) = {X(t), A(t), B(t)};

[0047] In the formula, Represents the set of unloading decisions. Denotes the set of transmitted power, B = {k u,m (t)} N×M A set representing the number of threads to be allocated, the transmission power, and the number of threads to be allocated.

[0048] Define the cumulative reward function R(t), which represents the sum of rewards obtained by all mobile devices within a given time period T:

[0049]

[0050] In the formula, α represents the discount coefficient, which reflects the degree of influence of historical rewards on current rewards; r(t) represents the value of the u-th mobile device in state s within time slot t, according to the objective optimization function. u (t) take action a u The reward generated at time (t) N represents the number of elements in the collection of mobile devices; r u (t) indicates that within time slot t, the u-th mobile device will be positioned in state s according to the objective optimization function. u (t) take action a u The reward generated at time (t), r u (t)=-Q u (t)D u (t)-H u (t)E u (t)-Vcost u (t), Q u (t) represents the backlog in the task queue of the u-th mobile device, D u (t) represents the initial input data of the u-th mobile device, H u (t) represents the energy queue backlog of the u-th mobile device, E u (t) represents u mobile devices processing A. u(t) Total energy consumption of the task, A u (t) represents the number of tasks to be processed on the u-th mobile device, V is the Lyapunov tradeoff factor, used to characterize the tradeoff between the long-run average overhead of the system and the queue backlog, and cost is... u (t) represents the total cost of the u-th mobile device.

[0051] In some possible implementations, the use of deep reinforcement learning algorithms to solve the Markov decision process includes:

[0052] A deep reinforcement learning model is built on each mobile device to obtain an agent corresponding to each mobile device. The agent is used to learn the mapping from the current state to the action of the corresponding mobile device to obtain the policy.

[0053] A two-layer competitive deep neural network is introduced for each agent. At the bottom layer of the two-layer competitive deep neural network, each agent's individual network learns its own policy. At the top layer of the two-layer competitive deep neural network, a global network is introduced, and the agents adjust their respective policies through competitive learning.

[0054] In each time slot, each agent selects an action based on its current state, obtains a reward according to the cumulative reward function, and updates the weights of the deep reinforcement learning model based on the reward.

[0055] The dual-layer competitive deep neural network is used to learn task migration strategies at the global level to determine the optimal unloading strategy, optimal mobile device transmission power, and optimal number of threads allocated to the MEC server for each agent in each time slot.

[0056] By periodically detecting whether the long-term average overhead of the system of each of the intelligent agents' mobile devices tends to a minimum, the parameters of the deep reinforcement learning model corresponding to each of the intelligent agents are adjusted based on the detection results.

[0057] According to a second aspect of the present invention, a task collaborative reasoning apparatus is also provided, applied to a complex heterogeneous scenario including multiple mobile devices and multiple MEC servers, the apparatus comprising:

[0058] The task segmentation module is used to segment the tasks arriving on the corresponding mobile device into multiple sub-tasks based on the DNN task model structure deployed on each mobile device, and to construct an offloading strategy for each sub-task to characterize the execution on the mobile device or MEC server, wherein each mobile device corresponds to a task queue.

[0059] The problem construction module is used to construct the problem of minimizing the average overhead of all mobile devices within a time slot, with the objectives of a joint optimal offloading strategy, mobile device transmission power, and the number of threads allocated to the MEC server for executing offloaded subtasks, and with constraints of mobile device CPU cycle frequency requirements, task queue length requirements, long-term energy consumption requirements, and task inference latency requirements.

[0060] The problem reconstruction module is used to reconstruct the strongly coupled problem of variables between different time slots in the average cost minimization problem using Lyapunov optimization theory, so as to obtain the independent time slot policy optimization problem corresponding to the average cost minimization problem.

[0061] The solution module is used to transform the independent time slot policy optimization problem into a Markov decision process, and to solve the Markov decision process using a deep reinforcement learning algorithm to obtain the optimal offloading policy, the optimal mobile device transmission power, and the optimal number of threads allocated to the MEC server for each mobile device.

[0062] According to a third aspect of the present invention, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the task collaborative reasoning method as described above.

[0063] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the task collaborative reasoning method as described above.

[0064] The task collaborative reasoning method provided by this invention has at least the following beneficial effects:

[0065] (1) Comprehensive performance optimization: By jointly optimizing the DNN task offloading strategy, transmission power and the number of MEC server threads, this invention provides a more comprehensive performance optimization method; this not only considers task execution efficiency, but also covers network transmission and server processing capabilities, thereby improving the overall system performance.

[0066] (2) Improved resource utilization efficiency: By intelligently allocating computing tasks and adjusting transmission power, the present invention can more effectively utilize the resources of mobile devices and MEC servers, reducing resource waste.

[0067] (3) Energy consumption optimization: Considering the CPU cycle frequency and long-term energy consumption of local mobile devices, the present invention helps to reduce energy consumption, which is especially important for mobile devices, as they are usually limited by battery life.

[0068] (4) Balance between latency and accuracy: By taking into account the latency requirements of task inference, the present invention can reduce processing and transmission latency while ensuring accuracy, which is particularly important for real-time or near-real-time application scenarios.

[0069] (5) Dynamic adaptability: Due to the consideration of task queue length and different operating environment constraints, the present invention can better adapt to dynamically changing environments and needs, such as automatically adjusting strategies when network conditions change or device load changes.

[0070] (6) Long-term cost benefits: By minimizing the long-term average overhead of the system, this invention not only focuses on short-term performance optimization, but also considers long-term cost benefits, which is especially important for commercial applications.

[0071] (7) Better user experience: By optimizing latency and energy consumption, the present invention can provide users with a smoother and more durable user experience, which is crucial in user-centric application design.

[0072] Furthermore, the task collaborative reasoning device, electronic device, and non-transitory computer-readable storage medium provided by this invention can also achieve the above-mentioned technical effects, and will not be elaborated further here. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0074] Figure 1 This is a schematic diagram of a task collaborative reasoning application scenario provided by the present invention;

[0075] Figure 2 This is a flowchart illustrating the task collaborative reasoning method provided by the present invention;

[0076] Figure 3 It is the directed acyclic graph corresponding to the local model of GoogleNet provided by this invention;

[0077] Figure 4 This is a schematic diagram of the structure of the task collaborative reasoning device provided by the present invention;

[0078] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0080] The following is combined Figures 1 to 5 This invention describes a task collaborative reasoning method, a task collaborative reasoning device, an electronic device, and a non-transitory computer-readable storage medium.

[0081] Figure 1 This is a schematic diagram of a task collaborative reasoning application scenario provided by the present invention. This complex heterogeneous scenario consists of multiple mobile devices (MDs) and multiple MEC servers. The MEC servers are deployed near base stations to provide computation offloading services to the multiple mobile devices. Each mobile device is equipped with a trained deep neural network to compute its task. The MEC server has a trained DNN library covering all DNN types used by all mobile devices. Therefore, the MEC server can handle tasks offloaded from any mobile device. Let N = {1,2,...,N} represent the set of all mobile devices, and M = {1,2,....,M} represent the set of all MEC servers. A time-slot system is considered, t ∈ {0,1,...,T-1}, with each time slot having a length of τ.

[0082] Please refer to Figure 2 As shown, this embodiment provides a task collaborative reasoning method that can be implemented through steps S201 to S204. The following will provide a detailed explanation of each step:

[0083] Step S201: Based on the DNN task model structure deployed on each mobile device, the tasks arriving on the corresponding mobile device are divided into multiple sub-tasks, and an offloading strategy is constructed for each sub-task to characterize the execution on the mobile device or MEC server, wherein each mobile device corresponds to a task queue.

[0084] In this embodiment, task partitioning using the DNN task model structure can be achieved through network layers and nodes. For example, a DNN task model with three network layers can be divided into three sub-tasks. For DNN task models with complex inter-layer structures, sub-task partitioning can be based on nodes. Let's assume a DNN task model with 40 network nodes; the processing task corresponding to each node can be considered a sub-task. It should be noted that sub-task partitioning is performed on a per-task basis. When multiple tasks arrive on a mobile device, each task needs to be sub-partitioned separately. These sub-tasks obtained after partitioning can be executed on the local mobile device or offloaded to the MEC server for processing.

[0085] Step S202: With the joint offloading strategy, mobile device transmission power, and the number of threads allocated to the MEC server for executing the offloaded subtasks as objectives, and with constraints such as the CPU cycle frequency requirements of mobile devices, task queue length requirements, long-term energy consumption requirements, and task inference latency requirements, construct the problem of minimizing the average overhead of all mobile devices within the time slot.

[0086] In this embodiment, the MEC server used to execute subtasks can be any MEC server in a complex heterogeneous scenario. The mobile device can select the optimal server based on its distance from the MEC server or the performance of the MEC server. For the same DNN task, the mobile device needs to assign the unloaded part of the subtasks to the same MEC server. However, for different subtasks arriving on the mobile device, the selected MEC server may be the same or different.

[0087] Step S203: The strong coupling problem of variables between different time slots in the average cost minimization problem is reconstructed using Lyapunov optimization theory to obtain the independent time slot policy optimization problem corresponding to the average cost minimization problem.

[0088] Step S204: The independent time slot policy optimization problem is transformed into a Markov decision process, and a deep reinforcement learning algorithm is used to solve the Markov decision process to obtain the optimal offloading policy, the optimal mobile device transmission power, and the optimal number of threads allocated to the MEC server for each mobile device.

[0089] This embodiment of the task collaborative reasoning method aims to jointly optimize task offloading strategies, transmission power, and the number of threads allocated to the MEC server. It fully considers constraints such as the CPU cycle frequency of the local mobile device, task queue length, long-term energy consumption, and task reasoning latency requirements, constructing a long-term average overhead minimization problem. Due to the long-term constraints of the task queue and energy queue, and the mutual coupling of variables in different time slots, the long-term objective is decomposed into independent time-slot policy optimization problems based on Lyapunov optimization theory. Considering the special properties of independent time-slot policy optimization problems, traditional optimization methods may be difficult to solve effectively. To effectively address this challenge, a deep reinforcement learning algorithm is chosen. This algorithm excels in handling complex, nonlinear, and high-dimensional problems, and improves policy performance gradually through simulating the interaction between the agent and the environment and learning. Markov decision processes are also introduced to better model the problem. The aim is to address the challenges of current technology by optimizing resource allocation, reducing communication latency and energy consumption, and opening up broader prospects for the application of deep learning in mobile edge computing and other fields. This task collaborative reasoning method can effectively utilize distributed computing resources and significantly improve reasoning efficiency.

[0090] In some possible implementations, the DNN task model structure deployed on each mobile device in step S201 above divides the task arriving on the corresponding mobile device into multiple sub-tasks, specifically including:

[0091] The mobile devices in the complex and heterogeneous scenario are identified as target mobile devices through a traversal approach, and the following operations are performed on the target mobile devices:

[0092] The DNN task model deployed on the target mobile device is modeled as a directed acyclic graph, wherein the directed acyclic graph includes nodes representing each inference layer and edges connecting each node.

[0093] Subtasks are determined based on the computational tasks of the inference layer corresponding to each node in the directed acyclic graph.

[0094] The execution order and dependencies between subtasks are determined based on the edges in the directed acyclic graph.

[0095] Specifically, the segmentation process for the DNN task is as follows: First, the DNN used on the u-th mobile device is modeled as a directed acyclic graph, using G... u =(V u ,Y u ) indicates that V u Y represents the set of inference layers in a DNN model. u This represents the set of computation order and dependencies between layers of a DNN. Assume the DNN model used on the u-th mobile device has L...u One reasoning layer, then Where v u,0 This represents the virtual input layer of the DNN. And each edge (v) in the DAG... u,i ,v u,j )∈Y u , where 0≤i<j≤L u Specifically, the edge (v) u,i ,v u,j ) represents the DNN inference layer v on the u-th mobile device. u,j For inference layer v u,i The dependency, that is, only after the inference layer v is completed. u,i Only through calculation can the inference layer v be analyzed. u,j Perform calculations. Taking the GoogleNet model as an example, its local model corresponds to the DAG, such as... Figure 3 As shown.

[0096] Based on the characteristics of the DAG-based DNN model, all DNN inference layers except the virtual input layer can be further divided into multiple sub-layers. These sub-layers can be offloaded to the MEC server and executed using a multi-threaded approach to meet the requirements of latency-sensitive DNN tasks. When multiple threads are used to perform matrix multiplication calculations, the system can process the matrix input and output in parallel, which improves computational efficiency. During task offloading, the sub-channels employ advanced multiple access technology, namely Sparse Code Multiple Access (SCMA). Each mobile terminal device uses a unique codebook for task offloading, ensuring good anti-interference performance between devices while improving the utilization efficiency of wireless channel resources.

[0097] In some possible implementations, to fully utilize the computing power of local mobile devices and allocated MEC servers to accelerate the DNN inference process, this embodiment defines γ. u,x (t) represents the DNN inference layer v on the u-th mobile device within time slot t. u,x The unloading strategy, in step S201 above, involves constructing an unloading strategy for each subtask to characterize the execution on the mobile device or MEC server, specifically including:

[0098] The unloading strategy for the inference layer in the directed acyclic graph corresponding to each mobile device is defined as follows:

[0099]

[0100] In the formula, 0≤x≤L u L u γ represents the maximum number of inference layers of the DNN task model deployed on the u-th mobile device. u,0(t) represents the input layer of the DNN task model deployed on the u-th mobile device, γ u,0 (t) = 0, v u,x This represents the x-th inference layer of the DNN task model deployed on the u-th mobile device, where inference layer v is the x-th inference layer. u,0 γ represents the input layer of the DNN task model. u,x (t) represents v u,x Uninstallation strategy, γ u,0 (t) represents the offloading strategy of the input layer of the u-th mobile device, γ u,0 (t) = 0;

[0101] Define the uninstallation policy constraint for each mobile device as follows:

[0102] γ u,y (t)≤γ u,y+1 (t) Formula 2;

[0103] In the formula, 0≤y≤L u -1, γ u,y (t) represents the y-th inference layer of the DNN task model deployed on the u-th mobile device, γ u,y+1 (t) represents γ u,y The next inference layer after (t);

[0104] At the beginning of time slot t, the DNN task size of the u-th mobile device arriving is D. u (t), and is modeled as independent and identically distributed, using the set D = {D1(t), D2(t), ..., D...} N Let A(t) represent all tasks arriving at the mobile device within time slot t, and assume that the number of tasks to be processed by the u-th mobile device during that time slot is A. u (t). Since different DNNs can use different activation functions to process their neurons, the input matrix of each layer will vary depending on its initial input data D. u The values ​​of (t) vary depending on the input matrix. Furthermore, the ratio of the input matrix to the original data size for each layer is typically bounded and given. Presentation layer v u,x The ratio of v. In particular, since v u,0 It is a virtual input layer, so and

[0105] The size of the input matrix of each inference layer in the directed acyclic graph corresponding to each mobile device is defined as follows:

[0106]

[0107] In the formula, I u,x(t) represents the size of the input matrix of the u-th mobile device in time slot t, φ(v u,x ) represents the reasoning layer v u,x With reasoning layer v u,0 The ratio of the input matrix, φ(v) u,x D is obtained through historical information requested via reasoning during model training. u (t) represents the initial input data of the u-th mobile device.

[0108] The task collaborative reasoning method in this embodiment divides the DAG-type DNN tasks into hierarchical subtasks, dividing the tasks arriving on each mobile device into multiple subtasks. These subtasks can be executed on the local mobile device or offloaded to the MEC server for processing. In other words, by combining the structure of the DNN, the tasks are divided into a reasonable number, which facilitates the offloading and parallel processing of subsequent tasks and further improves reasoning efficiency.

[0109] In some possible implementations, step S202, with the joint offloading strategy, mobile device transmission power, and the number of threads allocated to the MEC server for executing the offloaded subtasks as objectives, and constrained by the mobile device's CPU cycle frequency requirements, task queue length requirements, long-term energy consumption requirements, and task inference latency requirements, constructs a problem to minimize the average overhead of all mobile devices within a time slot, specifically including:

[0110] Define a task queue for each mobile device to store tasks arriving within time slot t, and build a task queue backlog for each task queue;

[0111] In the specific implementation, each mobile device has a buffer queue for storing arriving DNN tasks, denoted by Q(t); the tasks arriving at the u-th mobile device within time slot t are stored in queue Q. u In (t), the task queues are waiting for task assignment, and the set of DNN task queues on all mobile devices within this time slot is represented as Q(t):

[0112] Q(t) = {Q1(t), Q2(t), ..., Q} N Formula 4 for (t)};

[0113] The task queue on the u-th mobile device has a backlog of Q. u The dynamic change of (t) can be expressed as:

[0114] Q u (t+1)=max{Q u (t)-A u (t), 0}+D u (t) Formula 5;

[0115] And the initial task queue has a backlog of Q.u (t) = 0.

[0116] To ensure system stability and avoid the problem of latency-sensitive DNN task queues failing due to excessive length, this example introduces the concept of an average queue. This indicates an effort to reduce queuing delays. To ensure controllable task response times while simultaneously limiting queue length, therefore... It can be represented as:

[0117]

[0118] In the formula, E{Q u (t)} represents the mean of the DNN task queue for the u-th mobile device across all time slots. It is the maximum value of its task queue length.

[0119] For each mobile device, the computation latency of the mobile device, the transmission latency of data from the mobile device to the MEC server, the computation latency of the MEC server, and the download latency of data from the MEC server to the mobile device are constructed, and the sum is taken as the total latency.

[0120] In the system model of the embodiment, processing latency includes the computation latency of the local mobile device, the transmission latency of data uploaded from the mobile device to the MEC server, the computation latency of the MEC server, and the download latency of data downloaded from the MEC server to the mobile device. First, define f... u (t) represents the CPU cycle frequency of the u-th mobile device, C u Let v be the CPU cycle of the u-th mobile device. Then, the u-th mobile device executes the DNN layer v. u,x The resulting computational delay It can be modeled as:

[0121]

[0122] In the formula, f u The magnitude of (t) satisfies the following constraints:

[0123]

[0124] in, Let represent the maximum CPU cycle frequency of the u-th mobile device. To more conveniently represent computation latency, define a set... v is the DNN inference layer on the u-th mobile device within time slot t. u,x Define a set of layers with an unloading strategy of 0. This refers to the set of layers for which the offloading strategy is set to 1. Therefore, within time slot t, the local computation latency of the u-th mobile device is... It can be modeled as:

[0125]

[0126] definition Let the upload rate of data from the u-th mobile device to the m-th MEC server be expressed as follows according to Shannon's theorem:

[0127]

[0128] In the formula, B u,m h represents the channel bandwidth. u,m Let Nu represent the channel power gain between the u-th mobile device and the m-th MEC server, and N0 represent the noise power spectral density. u,m (t) represents the distance between the u-th mobile device and the m-th MEC server. This represents the transmission power of the u-th mobile device. And... The following constraints must be met:

[0129]

[0130] in, This represents the maximum transmission power of the u-th mobile device.

[0131] On the u-th mobile device, the DNN inference layer v u,x The transmission delay for uploading the output data to the m-th MEC server is defined as... It can be modeled as:

[0132]

[0133] Among them, O u,x (t) represents the DNN inference layer v on the u-th mobile device within time slot t. u,x The output data size. Then, define. For the u-th mobile device, the DNN inference layer v u,x The output data is uploaded to the layer set of the m-th MEC server, and satisfies... Therefore, within time slot t, the transmission delay caused by the u-th mobile device transmitting data... It can be modeled as:

[0134]

[0135] Assumption set The inference latency of the offloaded layer is proportional to the number of threads allocated to the MEC server. Furthermore, it is assumed that each offloaded DNN part can be allocated at most K threads to the MEC server, and that these K threads have the same processing rate, using g... m Representation. Define k.u,m (t) represents the number of threads allocated to the m-th MEC server, satisfying the following constraints:

[0136] 1≤k u，m (t)≤K (Formula 14)

[0137] Therefore, the processing speed of the DNN offloading part of the u-th mobile device on the m-th MEC server can be expressed as k. u,m (t)·g m .use Indicates the inference layer v u,x The computation latency on the m-th MEC server is formulated as follows:

[0138]

[0139] Therefore, the DNN offload layer set In having k u,m Computation latency on a MEC server with (t) threads It can be modeled as:

[0140]

[0141] use This represents the size of the data result that the u-th mobile device needs to download, and sets the data inference results that all mobile devices can download within time slot t. This indicates that the size of the downloadable data result on the u-th mobile device is related to the size of its last layer L in the DNN. u Output data size Equal, that is In particular, if An empty set indicates that all tasks are processed locally on the mobile device, meaning the mobile device does not need to download the results returned by the MEC server. use The transmission delay generated during the download process can be modeled as follows:

[0142]

[0143] in, The download speed, representing the data rate from the m-th MEC server to the u-th mobile device, is formulated as follows:

[0144]

[0145] In the formula, This represents the transmission power of the m-th MEC server.

[0146] Since the latency and energy consumption incurred during task partitioning are very small compared to the total latency and energy consumption of the entire DNN task processing, they can be ignored. Within time slot t, DNN task A is processed... u (t) The total delay T required u (t) can be modeled as:

[0147]

[0148] In the formula, See Formula 9 for details. See Formula 13 for details. See Formula 16 for details. See Formula 17 for details.

[0149] For each mobile device, the computing power consumption of the mobile device and the data transmission power consumption from the mobile device to the MEC server are constructed, and the sum is taken as the total energy consumption;

[0150] Since the MEC server is grid-powered and has a continuous power supply, the computational energy consumption of the MEC server and the energy consumption of downloading inference results from the MEC server to the mobile device are not considered. Therefore, this example only considers the computational energy consumption of the local mobile device and the energy consumption of data transmission from the mobile device to the MEC server. The computational energy consumption of the DNN task on the u-th mobile device can be modeled as:

[0151]

[0152] In the formula, w is the energy coefficient based on the CPU chip architecture.

[0153] The power consumption of each mobile device depends on the channel transmission rate and data transmission time. Therefore, the u-th mobile device will have a set of DNN inference layers. The energy consumed during the process of uploading the output data to the MEC server. It can be modeled as:

[0154]

[0155] Total energy consumption E u (t) can be modeled as:

[0156]

[0157] For each mobile device, the total latency and the total energy consumption are weighted and summed based on preset latency and energy consumption allocation weights to obtain the total cost within time slot t;

[0158] Since the system's latency and energy consumption are dynamic, this example focuses on the system's latency and energy consumption metrics. The total cost (cost) of the u-th mobile device within time slot t is defined. u (t) by Delay and Energy consumption The weighted sum composition is represented as follows:

[0159] cost u (t)=λ·T u (t)+(1-λ)·E u (t) Formula 23;

[0160] In the formula, λ is the weight of processing delay, and 1-λ is the weight of energy consumption.

[0161] Based on this, we establish the minimization problem P1 as the long-run average cost of the system, and define the average cost minimization problem as follows:

[0162]

[0163] In the formula, N = {1, 2, ..., N} represents the set of mobile devices, N represents the number of elements in the set of mobile devices, M = {1, 2, ..., M} represents the set of MEC servers, M represents the number of elements in the set of MEC servers, t ∈ {0, 1, ..., T-1} represents the time-slot system, and T represents the given duration of the time-slot system. Let represent the set of unloading decisions. Denotes the set of transmitted power, B = {k u,m (t)} N×M A set representing the number of threads to be allocated; Let be the set of layers on the u-th mobile device within time slot t where the inference layer offload policy is 0. The set of layers whose output data from the DNN inference layer on the u-th mobile device is uploaded to the m-th MEC server. Let $C$ be the set of layers on the $u$-th mobile device whose inference layer unload policy is 1. u (t) represents the total cost of the u-th mobile device; γ u,y (t)≤γ u,y+1 (t) represents the offloading policy constraint for each mobile device; Let the queue average constraint be E{Q} corresponding to the u-th move. u Q(t) represents the average backlog of the task queue for the u-th mobile device across all time slots. u (t) represents the task queue product of the u-th mobile device. This represents the maximum value of the task queue backlog for the u-th mobile device; This represents the CPU cycle frequency constraint for the u-th mobile device. This represents the maximum CPU cycle frequency of the u-th mobile device; Constraints on the transmission power of the u-th mobile device This represents the maximum transmission power of the u-th mobile device; 1 ≤ k u,m (t)≤K represents the constraint on the number of threads allocated to the m-th MEC server, where K represents the maximum number of threads for the MEC server; E represents the long-term average energy consumption constraint. u (t) represents u mobile devices processing A u (t) Total energy consumption of the task, A u (t) represents the number of tasks to be processed by the u-th mobile device. It is assumed that all mobile devices have the same energy consumption threshold in each time slot, denoted by e(t); T u (t)≤β u T represents the inference delay requirement constraint. u (t) represents the processing A of the u-th mobile device. u (t) Total delay of the task, β u This represents the inference latency threshold for the DNN task on the u-th mobile device.

[0164] The task collaborative reasoning method in this embodiment is designed for a complex heterogeneous scenario with multiple mobile devices and multiple MEC servers. It aims to jointly optimize the task offloading strategy, transmission power and the number of threads allocated to the MEC server. It fully considers the constraints such as the CPU cycle frequency of the local mobile device, the task queue length, long-term energy consumption and task reasoning latency requirements, and constructs a problem to minimize the long-term average overhead of the system.

[0165] In some possible implementations, due to the large amount of uncertain state information in wireless channel conditions and task queues, and the unknown future environment and state of the system, it is difficult for the local mobile device to make task offloading decisions. Furthermore, system decisions within different time slots are mutually influential, making it difficult to meet long-term energy consumption constraints. Moreover, the objective of problem P1 is a long-term goal, so problem P1 cannot be solved directly. To solve problem P1, this example employs Lyapunov optimization techniques to decouple the original problem into a policy optimization problem P2 within an independent time slot, thereby ensuring system stability.

[0166] In step S203 above, Lyapunov optimization theory is used to reconstruct the strongly coupled problem of variables between different time slots in the average cost minimization problem, resulting in an independent time slot policy optimization problem corresponding to the average cost minimization problem, specifically including:

[0167] Introduce N energy queues Here, the energy queue is the queue backlog of energy consumption within each time slot, and the initial energy queue backlog Hu (t) = 0;

[0168] The energy queue backlog of the u-th mobile device is H. u The dynamic change of (t) can be expressed as:

[0169] H u (t+1)=max{H u (t)-e(t),0}+E u (t) Formula 25;

[0170] In the formula, H u (t) represents the energy queue backlog of the u-th mobile device in time slot t, H u (t+1) represents the u-th mobile device in H. u The energy queue backlog in the next time slot of (t) is represented by the energy consumption threshold e(t), which represents the service rate of the queue. Under the condition that the energy queue is stable, the long-term average energy consumption can be kept below e(t) to meet the constraints.

[0171] Then, the Lyapunov function L(Θ(t)) is defined as follows:

[0172]

[0173] In the formula, Θ(t) represents the queue backlog matrix, and L(Θ(t)) represents the queue backlog state. The Lyapunov drift function Δ(Θ(t)) is defined as follows:

[0174] Δ(Θ(t))=E{L(Θ(t+1))-L(Θ(t))|Θ(t)} Formula 27;

[0175] Based on Lyapunov optimization theory, and considering the system's long-term average cost and queue backlog, the drift plus penalty function is defined in this example as follows:

[0176]

[0177] In the formula, V≥0 serves as the Lyapunov tradeoff factor, representing the compromise parameter between the system's long-run average cost and queue backlog. Specifically, the larger the value of V, the greater the weight imposed on the system's long-run average cost.

[0178] To ensure the stability of L(Θ(t)), this example needs to obtain Δ V The minimum value of (Θ(t)). The upper bound of the drift plus penalty function is determined by Lemma 1.

[0179] Lemma 1: Within each time slot t, the Lyapunov drift plus penalty function Δ V The upper bound of (Θ(t)) is:

[0180]

[0181] Proof: According to the inequality The following formula can be obtained:

[0182]

[0183] Similarly, the following formula can be obtained:

[0184]

[0185] Next, use Let represent the number of DNN tasks actually processed by the u-th mobile device within time slot t. Then, we can obtain the following formula:

[0186]

[0187] Based on the above formula, we can obtain the following formula:

[0188]

[0189] use Let the energy consumed each time be represented, then we can obtain the following formula:

[0190]

[0191] Based on the above formula, we can obtain the following formula:

[0192]

[0193] Then, restating the above formula 30, we get:

[0194]

[0195] because and D u The values ​​of (t) are all non-negative, so we can obtain the following formula:

[0196]

[0197] Similarly, and E u The values ​​of (t) are all non-negative, so we can obtain the following formula:

[0198]

[0199] Then, the following formula can be obtained:

[0200]

[0201] Because the amount of tasks that the u-th mobile device needs to process in time slot t is A u (t) will not exceed the amount of data in the current task queue, so and Therefore, the following formula can be obtained:

[0202]

[0203] Where, the value of B is equal to

[0204] Therefore, the Lyapunov drift plus penalty function Δ V The upper bound of (Θ(t)) can be expressed as:

[0205]

[0206] The proof is valid.

[0207] The goal of this example is to minimize queue backlog while simultaneously minimizing the above formula 41, and by removing the constant terms in the formula. The transformed independent time-slot policy optimization problem is then denoted as P2 based on the energy queue backlog H. u The dynamic change of (t) is reconstructed using Lyapunov optimization theory to solve the long-term energy consumption constraint and average cost minimization problem, resulting in the independent time slot policy optimization problem expressed as:

[0208]

[0209] In the formula, Q u (t) represents the backlog in the task queue of the u-th mobile device, D u (t) Initial input data of the u-th mobile device, H u (t) represents the energy queue backlog of the u-th mobile device, E u (t) represents u mobile devices processing A u (t) represents the total energy consumption of the task, where V is the Lyapunov tradeoff factor, used to characterize the tradeoff between the system's long-run average cost and queue backlog. u (t) represents the total cost of the u-th mobile device, γ u,y (t)≤γ u,y+1 (t) represents the offloading policy constraint for each mobile device. This represents the CPU cycle frequency constraint for the u-th mobile device. This represents the maximum CPU cycle frequency of the u-th mobile device; Constraints on the transmission power of the u-th mobile device This represents the maximum transmission power of the u-th mobile device; 1 ≤ k u,m(t)≤K represents the constraint on the number of threads allocated to the m-th MEC server, and K represents the maximum number of threads for the MEC server; T u (t)≤β u T represents the inference delay requirement constraint. u (t) represents the processing A of the u-th mobile device. u (t) Total delay of the task, β u This represents the inference latency threshold for the DNN task on the u-th mobile device.

[0210] The task collaborative reasoning method in this embodiment uses Lyapunov optimization theory to reconstruct the strongly coupled problem of variables between different time slots in the long-term average cost minimization problem of the system, and transform it into a policy optimization problem with independent time slots.

[0211] In some possible implementations, by observing the above formulas and all constraints, it can be found that the transformed independent time-slot policy optimization problem P2 involves a combination of discrete and continuous variables, belonging to the typical NP-hard problem with extremely high computational complexity in mixed-integer nonlinear programming. Due to the special properties of the independent time-slot policy optimization problem, traditional optimization methods may be difficult to solve effectively. To effectively address this challenge, this paper chooses to use a deep reinforcement learning algorithm, which performs excellently in handling complex, nonlinear, and high-dimensional problems. Furthermore, it improves policy performance gradually through simulating the interaction between the agent and the environment and through learning. To better model the problem, this example introduces a Markov decision process.

[0212] The aforementioned step S204, which transforms the independent time-slot policy optimization problem into a Markov decision process, specifically includes:

[0213] Define a state space s(t). Within time slot t, state s(t) consists of the amount of tasks arriving on the mobile device, the CPU cycle frequency of the mobile device, the distance between the mobile device and the MEC server, the task queue backlog, and the energy queue backlog. The state space s(t) is expressed as:

[0214] s(t)={D(t),f(t),d(t),Q(t),H(t)} Formula 43;

[0215] In the formula, D(t)={D u (t)} N Let f(t) represent the set of task data arriving on the mobile device. u (t)} N Let d(t) represent the set of CPU cycle frequencies of a mobile device. u,m (t)} N×M Let Q(t) represent the set of distances between the u-th mobile device and the m-th MEC server.u (t)} N H(t) represents the set of backlogged task queues on a mobile device. u (t)} N The set representing the energy queue backlog of a mobile device;

[0216] Define an action space a(t). Within time slot t, action a(t) consists of offloading decisions, transmission power, and the number of allocated threads. The action space a(t) is expressed as:

[0217] a(t)={X(t), A(t), B(t)} Formula 44;

[0218] In the formula, Represents the set of unloading decisions. Denotes the set of transmitted power, B = {k u,m (t)} N×M A set representing the number of threads to be allocated, the transmission power, and the number of threads to be allocated.

[0219] Define the cumulative reward function R(t), which represents the sum of rewards obtained by all mobile devices within a given time period T:

[0220]

[0221] In the formula, α represents the discount coefficient, which reflects the degree of influence of historical rewards on current rewards; r(t) represents the value of the u-th mobile device in state s within time slot t, according to the objective optimization function. u (t) take action a u The reward generated at time (t) N represents the number of elements in the collection of mobile devices; r u (t) indicates that within time slot t, the u-th mobile device will be positioned in state s according to the objective optimization function. u (t) take action a u The reward generated at time (t), r u (t)=-Q u (t)D u (t)-H u (t)E u (t)-Vcost u (t), Q u (t) represents the backlog in the task queue of the u-th mobile device, D u (t) represents the initial input data of the u-th mobile device, H u (t) represents the energy queue backlog of the u-th mobile device, E u (t) represents u mobile devices processing A. u (t) Total energy consumption of the task, A u(t) represents the number of tasks to be processed on the u-th mobile device, V is the Lyapunov tradeoff factor, used to characterize the tradeoff between the long-run average overhead of the system and the queue backlog, and cost is... u (t) represents the total cost of the u-th mobile device. As the system cost decreases, the task queue shrinks, and the energy queue shrinks, the agent will receive a greater reward value.

[0222] In some possible implementations, the aforementioned step S204, which employs a deep reinforcement learning algorithm to solve the Markov decision process, specifically includes:

[0223] A deep reinforcement learning model is built on each mobile device to obtain an agent corresponding to each mobile device. The agent is used to learn the mapping from the current state to the action of the corresponding mobile device to obtain the policy.

[0224] A two-layer competitive deep neural network is introduced for each agent. At the bottom layer of the two-layer competitive deep neural network, each agent's individual network learns its own policy. At the top layer of the two-layer competitive deep neural network, a global network is introduced, and the agents adjust their respective policies through competitive learning.

[0225] In each time slot, each agent selects an action based on its current state, obtains a reward according to the cumulative reward function, and updates the weights of the deep reinforcement learning model based on the reward.

[0226] The dual-layer competitive deep neural network is used to learn task migration strategies at the global level to determine the optimal unloading strategy, optimal mobile device transmission power, and optimal number of threads allocated to the MEC server for each agent in each time slot.

[0227] By periodically detecting whether the long-term average overhead of the system of each of the intelligent agents' mobile devices tends to a minimum, the parameters of the deep reinforcement learning model corresponding to each of the intelligent agents are adjusted based on the detection results.

[0228] The task collaborative reasoning method in this embodiment addresses the non-convexity of the independent time slot policy optimization problem by using a Markov decision process for modeling and introducing a multi-agent deep reinforcement learning model to solve the task allocation process. This yields the optimal task offloading strategy, transmission power, and the number of allocated threads to adapt to the non-convexity of the problem, thereby ensuring system stability while minimizing the total system cost.

[0229] The task collaborative reasoning apparatus provided by the present invention is described below. The task collaborative reasoning apparatus described below and the task collaborative reasoning apparatus method described above can be referred to in correspondence.

[0230] Please refer to Figure 4 As shown, this embodiment provides a task collaborative reasoning device applied to a complex heterogeneous scenario including multiple mobile devices and multiple MEC servers. The device includes: a task segmentation module 410, a problem construction module 420, a problem reconstruction module 430, and a solution module 440. The above modules will be described in detail below:

[0231] The task segmentation module 410 is used to segment the tasks arriving on the corresponding mobile device into multiple sub-tasks based on the DNN task model structure deployed on each mobile device, and to construct an offloading strategy for each sub-task to characterize the execution on the mobile device or MEC server, wherein each mobile device corresponds to a task queue.

[0232] Problem construction module 420 is used to construct the problem of minimizing the average overhead of all mobile devices within a time slot, with the joint offloading strategy, mobile device transmission power and the number of threads allocated to the MEC server for executing offloaded subtasks as objectives, and the CPU cycle frequency requirements, task queue length requirements, long-term energy consumption requirements and task inference latency requirements of mobile devices as constraints.

[0233] Problem reconstruction module 430 is used to reconstruct the strongly coupled problem of variables between different time slots in the average cost minimization problem using Lyapunov optimization theory, so as to obtain the independent time slot policy optimization problem corresponding to the average cost minimization problem.

[0234] The solution module 440 is used to convert the independent time slot policy optimization problem into a Markov decision process, and to solve the Markov decision process using a deep reinforcement learning algorithm to obtain the optimal offloading policy, the optimal mobile device transmission power, and the optimal number of threads allocated to the MEC server for each mobile device.

[0235] The task collaborative inference device in this embodiment aims to jointly optimize the task offloading strategy, transmission power, and the number of threads allocated to the MEC server. It fully considers constraints such as the CPU cycle frequency of the local mobile device, task queue length, long-term energy consumption, and task inference latency requirements, constructing a long-term average overhead minimization problem. Due to the long-term constraints of the task queue and energy queue, and the mutual coupling of variables in different time slots, the long-term objective is decomposed into independent time-slot policy optimization problems based on Lyapunov optimization theory. Considering the special nature of independent time-slot policy optimization problems, traditional optimization methods may be difficult to solve effectively. To effectively address this challenge, a deep reinforcement learning algorithm is chosen. This algorithm excels in handling complex, nonlinear, and high-dimensional problems, and improves policy performance gradually through simulating the interaction between the agent and the environment and learning. Markov decision processes are also introduced to better model the problem. The aim is to address the challenges of current technology by optimizing resource allocation, reducing communication latency and energy consumption, and opening up broader prospects for the application of deep learning in mobile edge computing and other fields. This task collaborative inference method can effectively utilize distributed computing resources and significantly improve inference efficiency.

[0236] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a task-based collaborative reasoning method.

[0237] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0238] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the task collaborative reasoning method provided by the above methods.

[0239] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the task cooperative reasoning methods provided by the above methods.

[0240] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0241] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0242] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task-based collaborative reasoning method, characterized in that, Applied to complex heterogeneous scenarios involving multiple mobile devices and multiple MEC servers, the method includes: Based on the DNN task model structure deployed on each mobile device, the tasks arriving on the corresponding mobile device are divided into multiple sub-tasks, and an offloading strategy is constructed for each sub-task to characterize the execution on the mobile device or MEC server, wherein each mobile device corresponds to a task queue. With the goals of a joint offloading strategy, mobile device transmission power, and the number of threads allocated to the MEC server for executing offloaded subtasks, and with constraints such as mobile device CPU cycle frequency requirements, task queue length requirements, long-term energy consumption requirements, and task inference latency requirements, we construct the problem of minimizing the average overhead of all mobile devices within a time slot. The strongly coupled problem of variables between different time slots in the average cost minimization problem is reconstructed using Lyapunov optimization theory to obtain an independent time slot policy optimization problem corresponding to the average cost minimization problem. This reconstruction includes: Introduction Energy queues The energy queue is the queue backlog of energy consumption within each time slot, and the initial energy queue backlog is... ; The first Mobile device power queue backlog The dynamic changes can be represented as: ； In the formula, Indicates the first Mobile devices in time slots Energy queue backlog, For the first Mobile devices The energy queue backlog in the next time slot, energy consumption threshold The service rate of the queue represents the long-term average energy consumption under stable energy queue conditions. The following conditions are met; Based on the energy queue backlog The dynamic changes are reconstructed using Lyapunov optimization theory to solve the long-term energy consumption constraint and average cost minimization problem, resulting in the independent time-slot policy optimization problem expressed as: ； In the formula, Indicates the first There is a backlog of task queues on mobile devices. No. Initial input data for each mobile device Indicates the first Energy queue backlog for individual mobile devices express Mobile device processing Total energy consumption of the mission Lyapunov tradeoff factor, used to characterize the tradeoff between the system's long-run average cost and queue backlog. Indicates the first Total cost of each mobile device This represents the uninstallation policy constraints for each mobile device. Indicates the first CPU cycle frequency constraints for individual mobile devices Indicates the first The maximum CPU cycle frequency of a mobile device. No. Constraints on the transmission power of individual mobile devices Indicates the first Maximum transmission power of a mobile device; Indicates the first The number of threads allocated to each MEC server is constrained. This indicates the maximum number of threads on the MEC server; This indicates a reasoning delay requirement constraint. Indicates the first Processing of mobile devices Total task delay Indicates the first Inference latency threshold for DNN tasks on a mobile device; The independent time slot policy optimization problem is transformed into a Markov decision process, and a deep reinforcement learning algorithm is used to solve the Markov decision process to obtain the optimal offloading policy, the optimal mobile device transmission power, and the optimal number of threads allocated to the MEC server for each mobile device.

2. The task collaborative reasoning method according to claim 1, characterized in that, The DNN task model structure deployed on each mobile device divides the tasks arriving on the corresponding mobile device into multiple sub-tasks, including: The mobile devices in the complex and heterogeneous scenario are identified as target mobile devices through a traversal approach, and the following operations are performed on the target mobile devices: The DNN task model deployed on the target mobile device is modeled as a directed acyclic graph, wherein the directed acyclic graph includes nodes representing each inference layer and edges connecting each node. Subtasks are determined based on the computational tasks of the inference layer corresponding to each node in the directed acyclic graph. The execution order and dependencies between subtasks are determined based on the edges in the directed acyclic graph.

3. The task collaborative reasoning method according to claim 2, characterized in that, The step of constructing an offload strategy for each of the subtasks to characterize the execution on the mobile device or MEC server includes: The unloading strategy for the inference layer in the directed acyclic graph corresponding to each mobile device is defined as follows: ； In the formula, , Indicates the first The maximum number of inference layers for a DNN task model deployed on a mobile device. Indicates the first The input layer of a DNN task model deployed on a mobile device. , Indicates the first The first DNN task model deployed on the mobile device One reasoning layer, reasoning layer This represents the input layer of the DNN task model. express Uninstallation strategy Indicates the first The offloading strategy for the input layer of a mobile device. ; Define the uninstallation policy constraint for each mobile device as follows: ； In the formula, , Indicates the first The first DNN task model deployed on the mobile device One reasoning layer, express The next level of reasoning; The size of the input matrix of each inference layer in the directed acyclic graph corresponding to each mobile device is defined as follows: ； In the formula, Indicates time slot Inner The size of the input matrix for each mobile device Indication of the reasoning layer With reasoning layer The ratio of the input matrix, Historical information from inference requests is obtained during model training. Indicates the first Initial input data for each mobile device.

4. The task collaborative reasoning method according to claim 2, characterized in that, The problem of minimizing the average overhead of all mobile devices within a time slot is constructed, with the objectives of a joint offloading strategy, mobile device transmission power, and the number of threads allocated to the MEC server for executing offloaded subtasks, and constraints of mobile device CPU cycle frequency requirements, task queue length requirements, long-term energy consumption requirements, and task inference latency requirements. This includes: Define a storage time slot for each mobile device The task queues that arrive within the task queue are processed, and a task queue backlog is built for each task queue. For each mobile device, the computation latency of the mobile device, the transmission latency of data from the mobile device to the MEC server, the computation latency of the MEC server, and the download latency of data from the MEC server to the mobile device are constructed, and the sum is taken as the total latency. For each mobile device, the computing power consumption of the mobile device and the data transmission power consumption from the mobile device to the MEC server are constructed, and the sum is taken as the total energy consumption; For each mobile device, the total latency and total energy consumption are weighted and summed based on preset latency and energy consumption allocation weights to obtain the time slot within the time slot. Total cost; The problem of minimizing the average cost is defined as follows: ； In the formula, Represents a collection of mobile devices. This indicates the number of elements in the collection for mobile devices. Represents a collection of MEC servers. This indicates the number of elements in the MEC server's collection. Indicates a time-slot system, This represents the given duration of the time slot system. Let represent the set of unloading decisions. Represents the set of transmitted power. A set representing the number of threads to be allocated; For time slots Inner A set of inference layer unload policy values ​​of 0 on a mobile device. For the first The output data of the DNN inference layer on the mobile device is uploaded to the first... A set of layers for MEC servers. For the first A set of inference layer unloading policies of 1 on a mobile device. Indicates the first Total cost of each mobile device; This represents the uninstallation policy constraints for each mobile device; Indicates the first Each move corresponds to an average queue constraint. Indicates the first The average backlog of task queues for each mobile device across all time slots. Indicates the first The product of mobile device task queues Indicates the first The maximum backlog in the task queue of a single mobile device; Indicates the first CPU cycle frequency constraints for individual mobile devices Indicates the first The maximum CPU cycle frequency of a mobile device. No. Constraints on the transmission power of individual mobile devices Indicates the first Maximum transmission power of a mobile device; Indicates the first The number of threads allocated to each MEC server is constrained. This indicates the maximum number of threads on the MEC server; This represents the long-term average energy consumption constraint. express Mobile device processing Total energy consumption of the mission Indicates the first The number of tasks to be processed by each mobile device, assuming that the energy consumption threshold of all mobile devices is equal in each time slot, is used... express; This indicates a reasoning delay requirement constraint. Indicates the first Processing of mobile devices Total task delay Indicates the first Inference latency threshold for DNN tasks on mobile devices.

5. The task collaborative reasoning method according to claim 4, characterized in that, The process of transforming the independent time-slot policy optimization problem into a Markov decision process includes: Define state space In the time slot Internal state The state space is composed of the amount of tasks arriving on the mobile device, the CPU cycle frequency of the mobile device, the distance between the mobile device and the MEC server, the task queue backlog, and the energy queue backlog. Represented as: ； In the formula, This represents the collection of task data arriving on the mobile device. A set representing the CPU cycle frequencies of mobile devices. Indicates the first The mobile device and the first The set of distances between MEC servers This represents the set of backlogged tasks in the mobile device's task queue. The set representing the energy queue backlog of a mobile device; Define action space In the time slot Inside, action The action space consists of unloading decisions, transmission power, and the number of allocated threads. Represented as: ； In the formula, Represents the set of unloading decisions. Represents the set of transmitted power. A set representing the number of transmission power and the number of allocated threads; Define cumulative reward function Given duration The sum of rewards earned by all mobile devices within the country Represented as: ； In the formula, Represents the discount factor. It can reflect the degree to which historical rewards have an impact on current rewards; Indicates in time slot Inside, according to the objective optimization function, the first... A mobile device in status Take action below The rewards generated at that time , Indicates the number of elements in the collection of mobile devices; Indicates in time slot The inner optimization function will be based on the first A mobile device in status Take action below The rewards generated at that time , Indicates the first There is a backlog of task queues on mobile devices. Indicates the first Initial input data for each mobile device Indicates the first Energy queue backlog for individual mobile devices Indicate Mobile device processing Total energy consumption of the mission Indicates the first Number of tasks pending on each mobile device Lyapunov tradeoff factor, used to characterize the tradeoff between the system's long-run average cost and queue backlog. Indicates the first The total cost of each mobile device.

6. The task collaborative reasoning method according to claim 5, characterized in that, The method of solving the Markov decision process using a deep reinforcement learning algorithm includes: A deep reinforcement learning model is built on each mobile device to obtain an agent corresponding to each mobile device. The agent is used to learn the mapping from the current state to the action of the corresponding mobile device to obtain the policy. A two-layer competitive deep neural network is introduced for each agent. At the bottom layer of the two-layer competitive deep neural network, each agent's individual network learns its own policy. At the top layer of the two-layer competitive deep neural network, a global network is introduced, and the agents adjust their respective policies through competitive learning. In each time slot, each agent selects an action based on its current state, obtains a reward according to the cumulative reward function, and updates the weights of the deep reinforcement learning model based on the reward. The dual-layer competitive deep neural network is used to learn task migration strategies at the global level to determine the optimal unloading strategy, optimal mobile device transmission power, and optimal number of threads allocated to the MEC server for each agent in each time slot. By periodically detecting whether the long-term average overhead of the system of each of the intelligent agents' mobile devices tends to a minimum, the parameters of the deep reinforcement learning model corresponding to each of the intelligent agents are adjusted based on the detection results.

7. A task collaborative reasoning device, characterized in that, The device, applicable to complex heterogeneous scenarios involving multiple mobile devices and multiple MEC servers, comprises: The task segmentation module is used to segment the tasks arriving on the corresponding mobile device into multiple sub-tasks based on the DNN task model structure deployed on each mobile device, and to construct an offloading strategy for each sub-task to characterize the execution on the mobile device or MEC server, wherein each mobile device corresponds to a task queue. The problem construction module is used to construct the problem of minimizing the average overhead of all mobile devices within a time slot, with the objectives of a joint optimal offloading strategy, mobile device transmission power, and the number of threads allocated to the MEC server for executing offloaded subtasks, and with constraints of mobile device CPU cycle frequency requirements, task queue length requirements, long-term energy consumption requirements, and task inference latency requirements. The problem reconstruction module is used to reconstruct the strongly coupled problem of variables between different time slots in the average cost minimization problem using Lyapunov optimization theory, thereby obtaining an independent time slot policy optimization problem corresponding to the average cost minimization problem. The process of reconstructing the strongly coupled problem of variables between different time slots in the average cost minimization problem using Lyapunov optimization theory to obtain the independent time slot policy optimization problem corresponding to the average cost minimization problem includes: Introduction Energy queues The energy queue is the queue backlog of energy consumption within each time slot, and the initial energy queue backlog is... ; The first Mobile device power queue backlog The dynamic changes can be represented as: ； In the formula, Indicates the first Mobile devices in time slots Energy queue backlog, For the first Mobile devices The energy queue backlog in the next time slot, energy consumption threshold The service rate of the queue represents the long-term average energy consumption under stable energy queue conditions. The following conditions are met; Based on the energy queue backlog The dynamic changes are reconstructed using Lyapunov optimization theory to solve the long-term energy consumption constraint and average cost minimization problem, resulting in the independent time-slot policy optimization problem expressed as: ； In the formula, Indicates the first There is a backlog of task queues on mobile devices. No. Initial input data for each mobile device Indicates the first Energy queue backlog for individual mobile devices express Mobile device processing Total energy consumption of the mission Lyapunov tradeoff factor, used to characterize the tradeoff between the system's long-run average cost and queue backlog. Indicates the first Total cost of each mobile device This represents the uninstallation policy constraints for each mobile device. Indicates the first CPU cycle frequency constraints for individual mobile devices Indicates the first The maximum CPU cycle frequency of a mobile device. No. Constraints on the transmission power of individual mobile devices Indicates the first Maximum transmission power of a mobile device; Indicates the first The number of threads allocated to each MEC server is constrained. This indicates the maximum number of threads on the MEC server; This indicates a reasoning delay requirement constraint. Indicates the first Processing of mobile devices Total task delay Indicates the first Inference latency threshold for DNN tasks on a mobile device; The solution module is used to transform the independent time slot policy optimization problem into a Markov decision process, and to solve the Markov decision process using a deep reinforcement learning algorithm to obtain the optimal offloading policy, the optimal mobile device transmission power, and the optimal number of threads allocated to the MEC server for each mobile device.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the task collaborative reasoning method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the task collaborative reasoning method as described in any one of claims 1 to 6.

Images