Task precoding and resource allocation optimization method in wireless edge computing power network

Abstract

The invention provides a task precoding and resource allocation optimization method in a wireless edge computing power network. The method comprises the following steps: constructing a wireless edge computing power network architecture model, establishing a task execution time delay and energy consumption model, and establishing a joint optimization problem of computing power scheduling and resource allocation by taking task execution time delay minimization as a target in a computing power network; further, the joint optimization problem is decomposed into a task precoding sub-problem and a joint task division, computing power association and computing power distribution sub-problem. Through joint design of a convex optimization-based method and a multi-agent reinforcement learning algorithm, challenges of communication overhead, insufficient utilization of computing power, privacy security of tasks and the like of task transmission in a dynamic complex environment are effectively handled. According to the method disclosed by the invention, the intelligent agent can be effectively guided to obtain an optimal strategy of task precoding, computing power association and resource optimization at a higher convergence speed while the efficiency and the stable performance are ensured.

Description

Technical Field

[0001] The present invention relates to the field of edge computing network technology, and in particular to a method for optimizing task precoding and resource allocation in a wireless edge computing network. Background Art

[0002] The advent of the digital age has driven the booming development of computing-intensive wireless services, resulting in a continuous increase in demand for computing resources and posing major challenges to traditional computing architectures. CPN (Computing Power Network) has been proposed as a promising new architecture that can effectively cope with the explosive growth in computing power demand. Through ubiquitous network connections, the computing power network integrates multi-level computing power and storage resources, connects geographically dispersed computing resources, coordinates the allocation and scheduling of computing tasks, and provides users with on-demand, comprehensive computing services based on the deep integration of communication, computing, and network resources.

[0003] The construction of computing power network relies on edge intelligence as a key supporting technology. Edge intelligence is achieved by introducing computing and caching resources to the edge of mobile networks, deploying artificial intelligence services on edge networks, and establishing dynamic and adaptive edge network management and maintenance to significantly reduce communication overhead and latency, and provide diversified, distributed, low-latency, and reliable intelligent services. The concept of edge intelligence includes the following two aspects: first, deploying artificial intelligence services through edge computing to achieve the combination of edge and intelligence; second, integrating artificial intelligence technology into edge computing networks, establishing dynamic and adaptive edge network management and maintenance, and forming an integration of intelligence and edge technology.

[0004] By integrating edge intelligence with computing power networks, computing power networks can provide flexible computing resource scheduling and optimization for edge intelligence. At the same time, edge intelligence expands computing power services and enables computing power networks to be deployed intelligently, making unparalleled progress in processing power and real-time decision-making. The enhanced overall scheduling capabilities of computing power networks and the enhanced computing performance of MEC (Mobile Edge Computing) have also spawned a series of computing-intensive innovative applications such as autonomous driving, VR (Virtual Reality) / AR (Augmented Reality), and smart cities. In addition to the above-mentioned application characteristics, CPN still has significant advantages from the perspective of implementation and deployment. In the future mobile communication system, any terminal device with computing power can be integrated into the edge network as a computing power node, providing great flexibility and compatibility through the optimization and efficient use of edge networks and computing resources.

[0005] With the development of computing power networks and edge networks, the joint optimization of computing power scheduling and resource allocation has become a hot topic in current research, which is complex and challenging. However, considering the large number of heterogeneous and ubiquitous computing power nodes in computing power networks, how to efficiently utilize, match and deploy computing resources in edge networks is an urgent problem to be solved. The progress of existing computing paradigms such as edge computing provides a variety of solutions, but fails to solve the current pain points, including high communication overhead in computing power scheduling, underutilization of computing power resources, privacy and security of task data, and low latency requirements of intelligent tasks. Computing power is task-oriented. For specific computing tasks, it is necessary to adjust the scheduling and networking strategies of computing power according to the requirements and complexity of the task. In complex environments such as high mobility, limited resources and time-varying nature of mobile networks, heterogeneous resources of different devices also aggravate resource shortages. In real scenarios (such as vehicle-mounted networks), the probability of dynamic time-varying channel states and unstable network topologies is very high. Implementing computing power scheduling will inevitably bring high communication overhead, and may face the dilemma of excessive upload traffic and unbearable upload time. The computing resources of computing power nodes with limited computing power may be difficult to meet the diverse needs of the network. What is more serious is that when coordinating distributed computing resources during computing power scheduling, there is a high probability that task data will be leaked, leading to privacy threats.

[0006] In addition, the environment in the computing network is complex and changeable. The existing resource optimization scheme is not suitable for the computing network architecture. The interaction between distributed computing nodes increases the randomness and complexity of the algorithm. The enhanced requirements of 6G applications make the existing computing offloading methods no longer suitable for delay performance. The limited resources of heterogeneous devices and the time-varying channel environment require flexible scheduling and elastic arrangement of ubiquitous computing resources, making joint optimization more complicated. Summary of the invention

[0007] An embodiment of the present invention provides a method for optimizing task precoding and resource allocation in a wireless edge computing network to achieve joint optimization of computing power scheduling and resource allocation, minimize task execution delay and improve learning convergence speed and efficiency.

[0008] In order to achieve the above object, the present invention adopts the following technical scheme.

[0009] A method for optimizing task precoding and resource allocation in a wireless edge computing network, comprising:

[0010] Build a wireless edge computing network architecture model;

[0011] Based on the wireless edge computing network architecture model, a joint optimization problem of computing power scheduling and resource allocation in the wireless edge computing network with the goal of minimizing task execution delay is established;

[0012] Design a two-layer optimization algorithm to decompose the joint optimization problem into task precoding subproblems and joint task partitioning, computing power association and computing power allocation subproblems;

[0013] After an equivalence analysis of the task precoding subproblem, the precoding strategy is obtained by designing an algorithm based on convex optimization; by designing a multi-agent deep reinforcement learning algorithm, the joint task division, computing power association and computing power allocation subproblems are solved to obtain the optimization strategy of task division, computing power association and resource allocation.

[0014] Preferably, the construction of the wireless edge computing network system architecture model, initialization of the task nodes, computing nodes and computing scheduling system environment in the network, and establishment of the task execution delay and energy consumption model include:

[0015] Construct a wireless edge computing network WECPN architecture including a wireless edge access layer, a computing power adaptation layer, and an application layer. In the wireless edge access layer, nodes that generate computing tasks are set as task nodes. In the computing power adaptation layer, computing power nodes that use computing power resources for computing and communication are set. In the WECPN architecture, it is assumed that there are N heterogeneous and ubiquitous computing power nodes, and the computing power node set is used. Indicates that N = K + M, the computing power node set Divided into terminal service device computing node collection and a collection of server compute nodes

[0016] Assume that the indexes of computing nodes and task nodes are represented as i and n respectively, and the status of computing nodes is represented as in and f represent the current computing power and CPU computing frequency of the CPU of computing power node i, χ i Indicates the location of computing power node i; Set triplet Γ n = <D n , ψ n , τ n > represents the generated computing task, where D n is the computational task size, ψ n is the computational density, i.e., the number of CPU cycles required to complete a 1-bit computational task, τ n Indicates the maximum tolerable delay in completing a computational task.

[0017] Preferably, the joint optimization problem of computing power scheduling and resource allocation in the wireless edge computing power network with the goal of minimizing task execution delay is established based on the wireless edge computing power network system architecture model, including:

[0018] The task execution delay in the wireless edge computing network includes task precoding overhead, task calculation and communication delay overhead. The average task expression rate of task n is expressed as ε n , ε n ∈(0, 1], the task size processed by the user in the tth time slot in the precoding stage is expressed as: For each user u n , the computation time to extract the task representation from the original task n is:

[0019]

[0020] in is the amount of computation required to extract the task representation. n ,The execution performance of the task and the extraction rate of the task expression are modeled as:

[0021]

[0022] When the local computing mode is selected, the computing subtask will be executed on the terminal device u n The local execution time is the computation time. Task u is executed in time slot t. n The estimated time required is expressed as:

[0023]

[0024] When the service device is selected for calculation, task u will be executed n The subtask is offloaded to the service device u at time slot t via wireless communication. d , and execute remotely;

[0025] u n -u d Uplink transmission: in

[0026] u d calculate:

[0027] u d -u n Downlink transmission:

[0028] The total task execution time of the service device computing power node is expressed as:

[0029]

[0030] Assume that there is no overlap between any of the subtasks that satisfy

[0031] u n -u s Uplink transmission: in

[0032] u s calculate:

[0033] The collaborative computing time of the server nodes is expressed as:

[0034]

[0035] T n The task execution delay corresponding to the computing task is expressed as:

[0036]

[0037] Computational energy consumption of executing tasks: Then the energy consumption of all computing nodes executing the entire computing task is

[0038] Communication energy consumption: Then the non-link task transmission energy consumption of all computing nodes is expressed as:

[0039] Total energy consumption:

[0040] The joint optimization problem P1 of computing power scheduling and resource allocation in the wireless edge computing network with the goal of minimizing task execution delay is:

[0041]

[0042] Preferably, the two-layer optimization algorithm is used to decompose the joint optimization problem into task precoding sub-problems and joint task division, computing power association and computing power allocation sub-problems, including:

[0043] Design a two-layer optimization algorithm to decompose the joint optimization problem into task precoding subproblems and joint task partitioning, computing power association and computing power allocation subproblems;

[0044] Given the task division, computing power association and computing power allocation, the task precoding subproblem P2 is expressed as:

[0045]

[0046] The joint task division, computing power association and computing power allocation sub-problem P3 is expressed as:

[0047]

[0048] Preferably, after equivalent analysis of the task precoding subproblem, the precoding strategy is obtained by designing an algorithm based on convex optimization, including:

[0049] The changing trends of task performance and computational amount functions are analyzed, and expressions of the relationships between computational amount, computational performance and task expression rate are given. For the additional computational amount and task performance functions, a fitting algorithm is used to solve them. An equivalent analysis is performed on the task precoding subproblem, and the task precoding subproblem is converted into a convex optimization problem. The Carlo-Kuhn-Tucker (KKT) condition is used to solve the convex optimization problem, and the optimal solution ε of the precoding strategy, i.e., the task expression rate, is obtained.

[0050] Preferably, the method of solving the joint task division, computing power association and computing power allocation sub-problems by designing a multi-agent deep reinforcement learning algorithm to obtain an optimization strategy for task division, computing power association and resource allocation includes:

[0051] Convert the joint task division, computing power association and computing power allocation sub-problem P3 into a Markov decision process, and establish the state space, action space and reward function of the multi-agent reinforcement learning algorithm;

[0052] (1) Determine the state space;

[0053] In each time slot, the agent divides the tasks and computing resources. represents the joint observation space of all agents, and each agent obtains partial observations o from the environment n (t) is expressed as χ n (t), f i (t), d n (t), h n (t)}, Represents the global state of the environment, expressed in each time slot as:

[0054] represents the joint action space of all agents. Each agent needs to make decisions on task division, computing power association, and computing power allocation. The actions of the agent in the time slot are expressed as The actions of all agents are represented as

[0055] Design the reward function as a hybrid reward function that is consistent with the constraints and optimization objectives;

[0056] The joint task division, computing power association and computing power allocation sub-problems are solved through a multi-agent deep reinforcement learning algorithm to obtain task division, computing power association and resource allocation strategies, that is, which computing power nodes the tasks are associated with, the proportion of computing power nodes executing tasks, and the computing resources allocated when computing power resources are used to execute tasks.

[0057] Preferably, the reward function includes: an immediate reward and a long-term reward. The immediate reward is a reward for evaluating the result after each task is completed, involving a mixed reward consistent with the constraint and optimization goal; the long-term reward uses a discount factor to strike a balance between the immediate reward and the long-term reward. The agent continuously interacts with the environment to maximize the long-term discounted reward.

[0058] State normalization: State normalization is used to scale the states and put them in the range of [0, 1];

[0059] Action mask: constrain all continuous variables such as task partitioning, computing power association, and computing power allocation to [0, 1], and take the value of the task partition variable from the set (0, 0.01, 0.02, ..., 1);

[0060] The total reward for all computing task nodes is:

[0061]

[0062] The long-term cumulative reward R is calculated as:

[0063]

[0064] It can be seen from the technical solutions provided by the above embodiments of the present invention that the method of the present invention integrates computing power network with edge intelligence, and proposes a wireless edge computing power network model driven by edge intelligence and based on task perception by converting computing power scheduling into task migration, so as to support users with limited available resources to collaboratively perform various intelligent tasks through ubiquitous computing power nodes. The two-layer optimization task precoding and resource allocation algorithm proposed in the present invention can effectively guide the intelligent agent to approach the optimal strategy of computing power scheduling and resource optimization while ensuring efficiency and stable performance, and obtain the optimized solution at a faster convergence speed.

[0065] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0067] Figure 1 A processing flow chart of a method for optimizing the design of task precoding and resource allocation in a wireless edge computing network provided by an embodiment of the present invention;

[0068] Figure 2 A schematic diagram of the architecture of a wireless edge computing network provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0070] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0071] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0072] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0073] The embodiment of the present invention considers the wireless computing power network driven by edge intelligence, and proposes a two-layer optimization task precoding and resource optimization algorithm to support users with limited available resources to collaboratively perform various intelligent tasks through ubiquitous computing power nodes. In response to the challenges brought by channel uncertainty and limited resources, task precoding schemes and task division, computing power association and computing power resource allocation schemes are jointly designed. At the computing power task node, a task precoding method based on semantic perception is proposed to obtain the semantic expression of the task; based on this, the task division, computing power association and resource allocation are further dynamically jointly designed and arranged, and a multi-agent reinforcement learning algorithm is proposed to enhance the efficiency and robustness of the system and minimize the system delay overhead.

[0074] The embodiment of the present invention proposes a two-layer optimization algorithm to realize the joint optimization of task precoding, computing task division, computing association and resource allocation in the wireless edge computing network, so as to minimize the task execution delay of all computing nodes under the constraints of limited resources and task performance. In the inner layer, a closed-form expression is used to derive the optimal task representation rate using a convex optimization-based method. In the outer layer, based on the obtained optimal task semantic expression, a multi-agent deep reinforcement learning algorithm is proposed to efficiently solve the joint optimization sub-problem. The proposed algorithm aims to achieve rapid convergence while effectively shortening the task execution time and enhancing the efficiency and robustness of the system.

[0075] The processing flow chart of a method for optimizing task precoding and resource allocation in a wireless edge computing network provided by an embodiment of the present invention is as follows: Figure 1 As shown, the process includes the following steps:

[0076] Step S10: Build a wireless edge computing network architecture model.

[0077] Step S20: Based on the above wireless edge computing network system architecture model, a joint optimization problem of computing power scheduling and resource allocation in the wireless edge computing network is established with the goal of minimizing task execution delay.

[0078] Step S30: Using a two-layer optimization algorithm, the above joint optimization problem is decomposed into task precoding sub-problems, joint task division, computing power association and computing power allocation optimization sub-problems.

[0079] Step S40: After an equivalent analysis of the task precoding sub-problem, the precoding strategy is obtained by designing an algorithm based on convex optimization; the above-mentioned joint task division, computing power association and computing power allocation optimization sub-problems are solved by an algorithm based on multi-agent deep reinforcement learning to obtain the task division, computing power association and resource allocation strategy.

[0080] The structure of the wireless edge computing network model proposed in the embodiment of the present invention is as follows: Figure 2As shown in the figure, a method for optimizing task precoding and resource allocation in wireless edge computing networks, namely a two-layer optimization algorithm, is further proposed to achieve joint optimization of task precoding, task division, computing power association and resource allocation to minimize task execution delay. The algorithm is based on the joint design of convex optimization method and a multi-agent reinforcement learning algorithm, which effectively copes with challenges such as expensive communication overhead of task transmission in dynamic and complex environments, underutilized computing power, and privacy and security of computing power task data. It further effectively guides the agent to obtain the optimal strategy for task precoding, computing power association and resource optimization, and obtains the optimized solution at a faster convergence speed.

[0081] The above step S10 specifically includes:

[0082] Step 1: Build a wireless edge computing network system architecture model.

[0083] Construct a WECPN (Wireless Edge Computing Power Networks) network architecture consisting of a wireless edge access layer, a computing power adaptation layer, and an application layer. Figure 2 A schematic diagram of the architecture of a WECPN provided for an embodiment of the present invention. At the wireless edge access layer, some users need to perform computing tasks, namely computing task nodes. Therefore, users connect to nearby edge servers through wireless communication. Computing resources are allocated according to demand using the proposed learning algorithm. The computing adaptation layer is deployed at the edge of the network, including interconnected heterogeneous and ubiquitous computing power nodes, such as service devices, edge servers, cloud servers, etc. The computing adaptation layer uses computing power resources to promote the integration of the entire network, enhance the computing power of nodes, and promote active, intelligent integration and collaboration between nodes. The application layer provides support for applications such as intelligent transportation, VR, remote interactive services, and big data AI services. Unlike the traditional vertically scheduled MEC system, WECPN is based on ubiquitous algorithm resources and forms a computing resource pool for computing resources such as terminals, edge computing servers, and cloud servers that are heterogeneous and geographically distributed in the network, thereby coordinating and scheduling service requests and comprehensively determining the computing and communication between ubiquitous computing nodes.

[0084] In WECPN, it is assumed that there are N heterogeneous and ubiquitous nodes, using the set In other words, the computing power node set can be divided into terminal service equipment computing power and server computing power All of them have the computing power to perform computing tasks. Through the coordinated adaptation of computing resources, WECPN provides on-demand and efficient computing services for various applications. Note that the terminal device can be both a task initiator and a task executor. Assume that the indexes of the computing power node and the task node are denoted as i and n respectively. In addition, the state of the computing power node is represented as in and f i Indicates the current computing power and CPU computing frequency of the CPU of computing power node i. i Indicates the location of the calculation power node i. In addition, let the triple Γ n = <D n , ψ n , τ n > represents the user's computing task, where D n is the computational task size, ψ n is the computational density, i.e., the number of CPU cycles required to complete a 1-bit computational task, τ n Indicates the maximum tolerable delay in completing a computing task. In actual scenarios, computing power is difficult to schedule directly. The present invention considers converting computing power scheduling into task migration. At the same time, computing tasks can be decomposed into multiple subtasks to make fuller use of heterogeneous computing power resources.

[0085] Step 2: Establish the delay model and energy consumption model of task execution.

[0086] Step 2.1 Establishment of delay model

[0087] This section will analyze the system latency in detail, including task precoding overhead, task calculation, and communication delay overhead.

[0088] Task-aware precoding

[0089] Denote the average task expression rate of task n as ε n , ε n ∈(0, 1], the task size processed by the user in the tth time slot during the task-aware precoding phase is expressed as: As mentioned above, in order to reduce the transmission burden, each device equipped with a task precoding processing unit will extract the task representation information instead of the original data. Note that each user performing the extraction will bring additional workload for the extraction. n , the computation time to extract the task representation from the original task n is:

[0090]

[0091] in is the amount of computation required to extract the task representation, which is a function of the task representation rate. n , the execution performance of the task is expressed as:

[0092]

[0093] Local computing time

[0094] When the local computing mode is selected, the computing subtask will be executed on the terminal device u n The total local execution time is the computation time. Task u is executed in time slot t. n The estimated time required is expressed as:

[0095]

[0096] in Indicates the ratio of subtasks assigned to itself.

[0097] Service Equipment Computing

[0098] When selecting service device calculation, u n The subtask is offloaded to the service device u at time slot t via wireless communication. d , And execute remotely.

[0099] u n -u d Uplink transmission: in B nd (t) and P nd (t) is the bandwidth and transmission power allocated for time slot t, h nd For user u n With service equipment d The current channel coefficient between . Indicates the proportion of subtasks assigned to the service device.

[0100] u d calculate:

[0101] u d -u n Downlink transmission:

[0102] In summary, the total execution time of the service device computing power node can be expressed as:

[0103]

[0104] Server Node Computing

[0105] Due to resource and latency constraints, users may further offload computing tasks to servers with more powerful computing capabilities. For ease of discussion, we assume that there is no overlap between any subtasks that satisfy

[0106] u n -us Uplink transmission: in h ns (t) represents user u n With server u s The current channel coefficient between .

[0107] u s calculate:

[0108] In this case, the collaborative computation time can be expressed as:

[0109]

[0110] T ni As the overall execution time corresponding to the computing task, it is expressed as:

[0111]

[0112] Step 2.2: Energy consumption model establishment

[0113] This part mainly includes computing and communication energy consumption, as follows:

[0114] Calculate energy consumption: Then, the energy consumption of all computing nodes executing the entire computing task is

[0115] Communication energy consumption: The non-link task transmission energy consumption of all computing nodes can be expressed as

[0116] Total energy consumption:

[0117] The above step S20 specifically includes:

[0118] System delay minimization problem establishment

[0119] The present invention takes into account resource limitations, energy consumption constraints, task performance constraints, etc., and minimizes the task completion time by jointly optimizing task precoding, task division, computing power division, and computing power resource allocation, which can be expressed as:

[0120]

[0121] The above step S30 specifically includes:

[0122] Due to the existence of binary variables and the coupling of multidimensional optimization variables, it can be seen that the optimization problem is a mixed integer programming problem. Due to the heterogeneity of computing power nodes and the time-varying nature of the environment, the amount of calculation of the optimization model is extremely large, so it is a challenge to ensure the migration and calculation of tasks. It is usually difficult to come up with a direct solution. In order to improve the processing efficiency, a two-layer optimization algorithm framework is proposed. In the inner layer algorithm, given the remaining variables, a convex optimization method is used to optimize the task representation rate ε n Then, in the outer algorithm, using the obtained optimal task representation rate, a multi-agent reinforcement learning algorithm is designed to optimize the remaining variables A, Υ, and F.

[0123] 1) Inner layer: Optimize computing power task precoding:

[0124] Given the task division, computing power association and computing power allocation, the corresponding computing power task-aware precoding problem can be expressed as:

[0125]

[0126] 2) Outer layer: Joint task division, computing power association and computing power allocation sub-problems:

[0127] For a given task-aware representation extraction, problem P1 can be simplified as:

[0128]

[0129] The above step S40 specifically includes:

[0130] Step 1: Task precoding sub-problem solving

[0131] The thorny challenge in solving P2 is the computational complexity of task expression extraction. and task execution performance The implicit expression of the additional calculation introduced relative to the task expression rate. To solve this problem, we first analyze the changing trends of the task performance and the computational amount function, and give an expression for the computational amount. For task performance, the corresponding expression is given. For the additional computational amount and task performance function, a fitting algorithm can be used to solve it. Further, an equivalent analysis is performed on the problem, which is converted into a convex optimization problem. The KKT (Karush-Kuhn-Tucker) condition is used to solve the transformed convex problem, and the optimal solution ε of the precoding strategy is obtained, that is, the task expression rate. The specific solution steps are as follows:

[0132] 1. Define the Lagrange multiplier and obtain the corresponding Lagrange function;

[0133] 2. Based on the Lagrangian function, the first-order optimality condition, the original feasibility condition, the dual feasibility condition and the steady-state condition are obtained;

[0134] 3. Combining the above original feasible conditions, steady-state conditions and first-order optimality conditions, the optimal precoding strategy, that is, the ratio of the task amount extracted before the task is transmitted to the computing power node to the original task size, can obtain the optimal solution.

[0135] Step 2: Solve the sub-problems of joint task division, computing power association and computing power allocation optimization

[0136] Considering the coupling relationship between the sub-problems formulated by P3 and the lack of a central controller, a multi-agent reinforcement learning algorithm is used to solve the problem efficiently and accurately. Therefore, problem P3 is formulated as an MDP that maximizes the cumulative discounted reward.

[0137] State Space: Represents the global state of the environment, expressed as

[0138] Observation Space: represents the joint observation space of all agents, where each agent obtains a partial observation value o from the environment n (t) can be expressed as where Γ n (t) and χ n (t) is the intrinsic parameter associated with each agent, and the rest are perception parameters, which represent the local parameters collected by each agent from the environment. Based on this,

[0139] Action Space: represents the joint action space of all agents. Based on local observations, each agent needs to make decisions on task division, computing power association, and computing power allocation. The actions of an agent in a time slot can be expressed as The actions of all agents are represented as

[0140] Reward: The goal of each agent is to learn an optimal strategy, effectively utilize computing resources, reduce task execution delays, and ensure successful task execution. The present invention designs the reward as a hybrid reward function that is consistent with constraints and optimization goals. It is divided into immediate rewards and long-term rewards. The immediate reward is a reward for evaluating the results after each task is completed, involving a hybrid reward consistent with constraints and optimization goals; the long-term reward uses a discount factor to strike a balance between immediate rewards and long-term rewards. The agent continuously interacts with the environment to maximize the long-term discounted reward, which effectively solves the sparsity challenge observed in a single-layer reward function based only on the final goal, ensuring fast and stable convergence during training.

[0141] Furthermore, to improve the algorithm efficiency and adapt the output to our formulation problem, state normalization and action masking are used.

[0142] ① State normalization: In order to speed up the convergence, enhance the training stability, and make the strategy more generalizable,

[0143] State normalization is used to scale the values ​​in the state space to put them in the range [0, 1].

[0144] ② Action mask: To reduce the dimensionality of the action space, all continuous variables such as task division, computing power association, and computing power allocation are constrained to [0, 1]. In addition, considering that in real-world scenarios, tasks are not arbitrarily divisible, can only take a limited number of values. At the same time, in order to reduce the dimension of the action space, for the task partition variable, we consider it to be from the set (0, 0.01, 0.02, … , take the value in 1).

[0145] Finally, the total reward for all computing task nodes is:

[0146]

[0147] The long-term cumulative reward R is calculated as:

[0148]

[0149] The designed multi-agent deep reinforcement learning algorithm is used to solve the joint task division, computing power association and computing power allocation sub-problems to obtain task division, computing power association and resource allocation strategies, that is, which computing power nodes the tasks are associated with, the proportion of computing power nodes executing tasks, and the computing resources allocated when computing power resources are used to execute tasks.

[0150] In summary, the embodiments of the present invention are committed to solving the pain points of expensive communication overhead of task transmission, underutilized computing power, privacy and security of computing power task data, and low latency requirements of intelligent tasks. A computing power framework based on edge intelligence, a wireless edge computing network model, is proposed. Under the constraints of resource limitations and task performance, task precoding, computing power association, task division and resource allocation are jointly optimized to minimize task execution delay. Considering the complexity of the problem and the dynamics of the system, the problem is decomposed into task precoding subproblems and joint task division, computing power association and resource allocation subproblems. Then, a two-layer optimization algorithm is proposed to formalize the problem. Specifically, in the inner layer, a task precoding method based on semantic perception is proposed, and the optimal task representation rate is derived by a closed-form expression using a convex optimization-based method. In the outer layer, based on the obtained optimal task representation rate, a multi-agent deep reinforcement learning algorithm is proposed to efficiently solve the joint optimization subproblem.

[0151] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0152] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0153] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0154] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for optimizing task precoding and resource allocation in a wireless edge computing network, characterized in that: include: Build a wireless edge computing network architecture model; Based on the wireless edge computing network architecture model, a joint optimization problem of computing power scheduling and resource allocation in the wireless edge computing network with the goal of minimizing task execution delay is established; Design a two-layer optimization algorithm to decompose the joint optimization problem into task precoding subproblems and joint task partitioning, computing power association and computing power allocation subproblems; After equivalent analysis of the task precoding subproblem, a precoding strategy is obtained by designing an algorithm based on convex optimization; By designing a multi-agent deep reinforcement learning algorithm to solve the joint task division, computing power association and computing power allocation sub-problems, the optimization strategy of task division, computing power association and resource allocation is obtained.

2. The method according to claim 1, characterized in that The construction of the wireless edge computing network system architecture model, initialization of the task nodes, computing nodes and computing scheduling system environment in the network, and establishment of the task execution delay and energy consumption model include: Construct a wireless edge computing network WECPN architecture including a wireless edge access layer, a computing power adaptation layer, and an application layer. In the wireless edge access layer, nodes that generate computing tasks are set as task nodes. In the computing power adaptation layer, computing power nodes that use computing power resources for computing and communication are set. In the WECPN architecture, it is assumed that there are N heterogeneous and ubiquitous computing power nodes, and the computing power node set is used. Indicates that N = K + M, the computing power node set Divided into terminal service device computing node collection and a collection of server compute nodes Assume that the indexes of computing nodes and task nodes are represented as i and n respectively, and the status of computing nodes is represented as where ζ(f i ) and f i represents the current computing power and CPU computing frequency of the CPU of computing power node i, χ i Indicates the location of computing power node i; Set triplet Γ n = <D n , ψ n , τ n > represents the generated computing task, where D n is the computational task size, ψ n is the computational density, i.e., the number of CPU cycles required to complete a 1-bit computational task, τ n Indicates the maximum tolerable delay in completing a computational task.

3. The method according to claim 2, characterized in that The joint optimization problem of computing power scheduling and resource allocation in the wireless edge computing power network with the goal of minimizing task execution delay is established based on the wireless edge computing power network system architecture model, including: The task execution delay in the wireless edge computing network includes task precoding overhead, task calculation and communication delay overhead. The average task expression rate of task n is expressed as ε n , ε n ∈(0, 1], the task size processed by the user in the tth time slot in the precoding stage is expressed as: For each user u n , the computation time to extract the task representation from the original task n is: in is the amount of computation required to extract the task representation. n ,The execution performance of the task and the extraction rate of the task expression are modeled as: When the local computing mode is selected, the computing subtask will be executed on the terminal device u n The local execution time is the computation time. Task u is executed in time slot t. n The estimated time required is expressed as: When the service device is selected for calculation, task u will be executed n The subtask is offloaded to the service device u at time slot t via wireless communication. d , and execute remotely; u n -u d Uplink transmission: in u d calculate: u d -u n Downlink transmission: The total task execution time of the service device computing power node is expressed as: Assume that there is no overlap between any of the subtasks that satisfy u n -u s Uplink transmission: in u s calculate: The collaborative computing time of the server nodes is expressed as: T n The task execution delay corresponding to the computing task is expressed as: Computational energy consumption of executing tasks: Then the energy consumption of all computing nodes executing the entire computing task is Communication energy consumption: Then the non-link task transmission energy consumption of all computing nodes is expressed as: Total energy consumption: The joint optimization problem P1 of computing power scheduling and resource allocation in the wireless edge computing network with the goal of minimizing task execution delay is:

4. The method according to claim 3, characterized in that The two-layer optimization algorithm is used to decompose the joint optimization problem into task precoding sub-problems and joint task division, computing power association and computing power allocation sub-problems, including: Design a two-layer optimization algorithm to decompose the joint optimization problem into task precoding subproblems and joint task partitioning, computing power association and computing power allocation subproblems; Given the task division, computing power association and computing power allocation, the task precoding subproblem P2 is expressed as: The joint task division, computing power association and computing power allocation sub-problem P3 is expressed as:

5. The method according to claim 4, characterized in that After equivalent analysis of the task precoding subproblem, the precoding strategy is obtained by designing an algorithm based on convex optimization, including: The changing trends of task performance and computational amount functions are analyzed, and expressions of the relationships between computational amount, computational performance and task expression rate are given. For the additional computational amount and task performance functions, a fitting algorithm is used to solve them. An equivalent analysis is performed on the task precoding subproblem, and the task precoding subproblem is converted into a convex optimization problem. The Carlo-Kuhn-Tucker (KKT) condition is used to solve the convex optimization problem, and the optimal solution ε of the precoding strategy, i.e., the task expression rate, is obtained.

6. The method according to claim 5, characterized in that The method of solving the joint task division, computing power association and computing power allocation sub-problems by designing a multi-agent deep reinforcement learning algorithm to obtain an optimization strategy for task division, computing power association and resource allocation includes: Convert the joint task division, computing power association and computing power allocation sub-problem P3 into a Markov decision process, and establish the state space, action space and reward function of the multi-agent reinforcement learning algorithm; (1) Determine the state space; In each time slot, the agent divides the tasks and computing resources. represents the joint observation space of all agents, and each agent obtains partial observations o from the environment n (t) is expressed as Represents the global state of the environment, expressed in each time slot as: represents the joint action space of all agents. Each agent needs to make decisions on task division, computing power association, and computing power allocation. The actions of the agent in the time slot are expressed as The actions of all agents are represented as Design the reward function as a hybrid reward function that is consistent with the constraints and optimization objectives; The joint task division, computing power association and computing power allocation sub-problems are solved through a multi-agent deep reinforcement learning algorithm to obtain task division, computing power association and resource allocation strategies, that is, which computing power nodes the tasks are associated with, the proportion of computing power nodes executing tasks, and the computing resources allocated when computing power resources are used to execute tasks.

7. The method according to claim 6, characterized in that The reward function includes: immediate reward and long-term reward. The immediate reward is a reward for evaluating the result after each task is completed, involving a mixed reward consistent with the constraints and optimization goals. The long-term reward uses a discount factor to strike a balance between the immediate reward and the long-term reward. The intelligent agent continuously interacts with the environment to maximize the long-term discounted reward. State normalization: State normalization is used to scale the states and put them in the range of [0, 1]; Action mask: constrain all continuous variables such as task partitioning, computing power association, and computing power allocation to [0, 1], and take the value of the task partition variable from the set (0, 0.01, 0.02, ..., 1); The total reward for all computing task nodes is: The long-term cumulative reward R is calculated as:

Images