Multi-dimensional resource management joint optimization method based on wireless edge network

Abstract

The invention relates to the technical field of wireless communication, and discloses a multi-dimensional resource management joint optimization method based on a wireless edge network. The method comprises the following steps: acquiring multi-dimensional resource state information of each node in the wireless edge network in real time; calculating a corresponding resource index based on each piece of resource state information; based on each resource index, determining an initial resource allocation strategy of each node through reinforcement learning, so that each node in each node constructs and trains a local first strategy model based on the initial resource allocation strategy, and generates a local resource allocation strategy of the node; and based on the local resource allocation strategy and the emergency task queue length of each node, updating the local resource allocation strategy through federated learning, and performing multi-dimensional resource allocation based on the local resource allocation strategy. By adopting the method, collaborative scheduling of multi-dimensional resources such as calculation, storage, frequency spectrum and power in the wireless edge network can be realized, and the overall efficiency of the network is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wireless communication, in particular to a multi-dimensional resource management joint optimization method based on a wireless edge network. BACKGROUND

[0002] As a key infrastructure supporting emerging applications such as vehicle networking, industrial Internet and augmented reality, the core challenge of the wireless edge network is how to efficiently coordinate multi-dimensional resources such as computing, storage, spectrum and power to meet the service requirements of high reliability and low latency. Currently, the resource management optimization technology for edge networks mainly optimizes single resource dimension independently. For example, by allocating computing resources through static rules, dynamically adjusting spectrum resources based on channel state, etc.

[0003] However, this resource optimization method only establishes a simple mapping relationship between limited dimensions, lacks global coordination of multiple resources, and is difficult to adapt to the coupling constraints of tasks. For example, when a compute-intensive task is offloaded to a node, if the spectrum bandwidth allocation is not synchronized and coordinated, the transmission delay may be too high to offset the time advantage of local computing. Based on this, how to realize the collaborative management and joint optimization of multiple resources of the wireless edge network to improve the overall efficiency of the wireless edge network is a problem to be solved at present. SUMMARY

[0004] Therefore, the present application aims to provide a multi-dimensional resource management joint optimization method based on a wireless edge network to realize the collaborative management and joint optimization of multiple resources and improve the overall efficiency of the wireless edge network.

[0005] To achieve the above-mentioned purpose, the technical scheme of the present application is as follows: The first aspect of the embodiment of the present application provides a multi-dimensional resource management joint optimization method based on a wireless edge network, which is applied to a processor deployed in a terminal, a server or a base station, and includes: real-time acquisition of multi-dimensional resource state information of each node in the wireless edge network; calculation of corresponding resource indicators based on each resource state information, including calculation of load, storage fragmentation rate, spectrum efficiency and reliability indicators; based on each resource indicator, determining an initial resource allocation strategy of each node through reinforcement learning, so that each node in the each node performs the following operation: constructing a local first strategy model based on a spatio-temporal convolution long short-term memory network, training the first strategy model based on the initial resource allocation strategy, historical demand data and task execution data of the node, and generating a local resource allocation strategy of the node through the first strategy model; The local resource allocation strategy of each node is updated through federated learning based on the local resource allocation strategy of each node and the length of the emergency task queue. Multi-dimensional resource allocation is performed for each node based on the updated local resource allocation strategy of each node.

[0006] Optionally, a corresponding resource indicator is calculated based on each resource state information, including: A multi-dimensional resource dynamic model is constructed based on each resource state information; each dimension of the resource dynamic model respectively performs the following steps: The computing load of each node is calculated based on the task computing amount and the computing power ratio of each node; The storage fragmentation rate of each node is calculated based on the storage unit allocation capacity, the used capacity and the total capacity of each node; The spectral efficiency of each node is calculated based on the transmit power, the channel gain and the noise power of each node for the target user; The reliability indicator of each node is calculated based on the service rate, the task arrival rate, the maximum tolerable delay and the actual delay of each node performing different tasks.

[0007] Optionally, the initial resource allocation strategy of each node is determined through reinforcement learning based on each resource indicator, including: The state space is defined through a deep reinforcement learning algorithm based on each resource indicator, and state space parameters are generated; The action space is defined based on each state space parameter, and a corresponding resource adjustment amount is generated as an action space parameter; the resource adjustment amount includes a computing power allocation adjustment amount, a storage allocation adjustment amount, a bandwidth allocation adjustment amount and a power adjustment amount; A policy network is constructed based on the state space parameters and the corresponding action space parameters; A multi-objective reward function is constructed based on the state space parameters, the action space parameters and the overall performance indicators of the wireless edge network; the overall performance indicators of the wireless edge network include the network total delay, the network maximum allowed delay, the network computing power upper limit, the data volume of tasks in the network and the storage access rate; The parameters of the policy network are updated based on the multi-objective reward function; the initial resource allocation strategy of each node is generated based on the updated policy network.

[0008] Optionally, updating the parameters of the policy network further includes: A user utility function is constructed by introducing the competition relationship and uncertainty between users through Bayesian game theory based on the bandwidth allocation adjustment amount and the power adjustment amount of the action space parameters to simulate resource competition behavior in a multi-user environment; The multi-objective reward function and the user utility function are optimized by an alternating iteration manner to update parameters of the policy network.

[0009] Optionally, the user utility function is constructed, including: A transmission rate term of a node i is constructed based on a bandwidth allocation adjustment amount, a power adjustment amount, a spectrum efficiency, and a power compensation reference value of the node i; the transmission rate term is used to reflect a balance between a transmission rate and power consumption of a user; A resource competition term of the node i is constructed based on a difference between a ratio of a power adjustment amount to a bandwidth allocation adjustment amount of the node i and another node j, and a resource competition penalty coefficient; A user utility function of the node i is constructed based on the transmission rate term and the resource competition term of the node i.

[0010] Optionally, the local resource allocation strategy of each node is updated through federated learning based on the local resource allocation strategy of each node and the emergency task queue length, including: A weight corresponding to each node is determined based on a current emergency task queue length of each node; A global model parameter is calculated according to a model parameter of a first strategy model of each node and the weight corresponding to the node; The global model parameter is sent to each node, so that each node updates a corresponding first strategy model based on the global model parameter; A local resource allocation strategy of the node is generated based on the first strategy model after the parameter is updated.

[0011] Optionally, after the local resource allocation strategy of each node is adjusted through federated learning, the method further includes: A Lyapunov function is constructed and a corresponding drift term is calculated based on a total length of a task queue of the wireless edge network and a real-time total power consumption; A first objective function is constructed based on a total power consumption budget of the wireless edge network, a current real-time total power consumption, and the drift term; The allocation weight of each dimension resource in the local resource allocation strategy of each node is adjusted to minimize the first objective function, and the local resource allocation strategy of each node is updated.

[0012] Optionally, the resource state information further includes resource demand information of each node; and the method further includes: The network state information of the wireless edge network is acquired at a first time interval, including a node topology structure and device parameters of each node; constructing, by simulation, a network mirror model corresponding to the wireless edge network to map behaviors and performances of each node in the wireless edge network based on the local resource allocation strategy of each node, the current resource state information of the node, and the network state information; After the local resource strategy of each node is updated, behaviors and performances of each node in the wireless edge network are simulated synchronously by an incremental learning and adaptive adjustment algorithm.

[0013] Optionally, the method further comprises: In a high-reliability low-latency scenario, behaviors and performances of each node are obtained at a second time interval according to the network mirror model, and current latency and reliability indicators are calculated; The current latency and reliability indicators are compared with latency and reliability thresholds corresponding to the high-reliability low-latency scenario; In a case where the latency is higher than the latency threshold or the reliability indicator is lower than the reliability threshold, the local resource allocation strategy of each node is updated by federated learning.

[0014] Optionally, the method further comprises: Task execution data of each node are obtained from the network mirror model at a third time interval; the task execution data include latency indicator data, energy consumption indicator data, and task progress indicator data; A resource state report of the wireless edge network is generated based on the task execution data of each node; The first strategy model of each node is updated based on the task execution data of each node.

[0015] The wireless edge network-based multi-dimensional resource management joint optimization method provided in the application can realize accurate monitoring of dynamic changes of network state by acquiring multi-dimensional resource state information of each node in the wireless edge network in real time, calculating the current resource index, and dynamically perceiving network state changes to quickly reflect and timely adjust the resource allocation strategy. Based on the current resource index of each dimension, the initial resource allocation strategy of the nodes in the network is determined through reinforcement learning, and each node trains a local first strategy model based on the initial resource allocation strategy, and generates a local resource allocation strategy of the node through the first strategy model. Since the initial allocation strategy in the present scheme is determined based on the real-time multi-dimensional resource index of all nodes in the edge network, the initial resource allocation strategy reflects the collaborative optimization result of resource allocation among nodes and local multi-dimensional resource allocation of the nodes in the process of executing tasks in the current network. Compared with the traditional single-dimensional resource independent optimization scheme, the present scheme cooperatively schedules the multi-dimensional resources such as computing, storage, spectrum and power in the wireless edge network, so that the network better adapts to the coupling constraint of multi-dimensional resources in actual application, and improves the overall efficiency of the network.

[0016] On this basis, based on the local resource allocation strategy of each node and the length of the emergency task queue, the local resource allocation strategy of each node is further optimized through federated learning to take into account the resource allocation demand of the network for emergency task processing, and to enhance the network's ability to respond to emergency tasks and sudden situations, thereby realizing the improvement of the overall efficiency of the edge network while improving the network's ability to respond to emergency tasks. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed to be used in the description of the embodiments of the application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a flowchart of the wireless edge network-based multi-dimensional resource management joint optimization method according to an embodiment of the application; Figure 2 is a flowchart of updating the local resource allocation strategy through federated learning according to an embodiment of the application; Figure 3 is a flowchart of determining the multi-dimensional resource allocation strategy for the wireless edge network according to an embodiment of the application. DETAILED DESCRIPTION

[0019] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of the present application.

[0020] It should be understood that, throughout the specification, reference to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Therefore, appearances of "in one embodiment" or "in an embodiment" in various places throughout the specification are not necessarily referring to the same embodiment. Furthermore, various specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0021] In various embodiments of the present application, it should be understood that the size of the serial number of the following processes does not mean the order of execution, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0022] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is presented with reference to the drawings, wherein the same reference numbers can indicate same or similar elements throughout the description. The embodiments described in the following exemplary embodiments are not meant to represent all implementations in accordance with the present application. Rather, they are merely examples in accordance with some aspects of the present application as detailed herein.

[0023] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0024] The traditional wireless edge network resource allocation scheme has the following defects: (1) In real-time interactive scenarios, such as autonomous driving, AR (Augmented Reality), VR (Virtual Reality), etc., a single-dimensional resource optimization strategy is difficult to adapt to the coupling constraints of task requirements. For example, if the spectrum bandwidth allocation is not synchronized and coordinated when a computing-intensive task is offloaded to an edge server, the transmission delay may be too high to offset the time advantage of local computing; (2) Static allocation strategies cannot adapt to network dynamics, especially when there are sudden task requirements in the network (such as sudden task requests, channel time-varying, and position drift of mobile terminals, etc.), which can easily cause task queue congestion and affect the overall efficiency of the network.

[0025] In view of the problems existing in the current resource allocation scheme of wireless edge network, the application proposes a multi-dimensional resource cooperative scheduling method which can better adapt to the coupling constraints of tasks and quickly adapt to the dynamic changes of network resource management optimization method. Through real-time monitoring of the multi-dimensional resource allocation and task execution of each node in the network, the method realizes the cooperative optimization of multi-node and multi-dimensional resource allocation, improves the overall efficiency of the network, and also enhances the response ability of the network to sudden task demand.

[0026] The application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0027] Figure 1 is a flow chart of a multi-dimensional resource management joint optimization method based on a wireless edge network according to an embodiment of the application. As shown in Figure 1 , the method is applied to a processor, which is deployed in a terminal, a server or a base station; the method comprises: real-time acquisition of multi-dimensional resource state information of each node in the wireless edge network; based on each resource state information, calculating the corresponding resource index, including: calculating the load, storage fragmentation rate, spectrum efficiency and reliability index; based on each resource index, determining the initial resource allocation strategy of the each node through reinforcement learning, so that each node in the each node performs the following operation: based on the initial resource allocation strategy, the historical demand data and the task execution data of the node, training the first strategy model based on the space-time convolution long short-term memory network to construct the local first strategy model; generating the local resource allocation strategy of the node through the first strategy model; based on the local resource allocation strategy of each node and the length of the emergency task queue, updating the local resource allocation strategy of each node through federated learning; based on the updated local resource allocation strategy of each node, performing multi-dimensional resource allocation for the each node.

[0028] In the embodiments of the application, the multi-dimensional resource management joint optimization method is applied to a processor, which can be a server, a terminal, a base station or other devices deployed in a wireless edge network. In the embodiments of the application, the processor is taken as a terminal for illustration, and the method is realized through the interaction of the terminal, the server and the base station.

[0029] In the embodiments of the present application, various sensors and monitoring devices deployed in the wireless edge network collect multi-dimensional resource state information of each node (i.e., edge server) in real time, including: computing resource state data, storage resource state data, spectrum resource state data, and power resource state data. Among them, the computing resource state data includes: CPU utilization, memory occupation, task queue length, etc.; the storage resource state data includes: the remaining capacity of the storage device, read-write speed, data fragmentation distribution, etc.; the spectrum resource state data includes: channel quality, bandwidth occupation, interference level, etc.; the power resource state data includes: the current power level of the base station and terminal device, battery status, energy consumption rate, etc.

[0030] After collecting the resource state information of each node through various sensors and detection devices, it is sent to the centralized data processing node of the core network layer in the network, i.e., the central data processing unit, at a preset time interval. The central data processing unit cleans and standardizes the received original resource state data, removes outliers, and fills in missing data. Optionally, data normalization is performed on each dimension of data to convert data of different dimensions and orders of magnitude to a unified numerical range for subsequent modeling algorithm processing and analysis.

[0031] According to the resource state information of each dimension, the corresponding resource indicators are calculated, specifically including: computing load, storage fragmentation rate, spectrum efficiency, and reliability indicators. Based on the resource indicators of each dimension, the initial resource allocation strategy of all nodes in the target network is determined through reinforcement learning, realizing the collaborative optimization of resource allocation in each dimension, achieving dynamic balance of multi-dimensional performance indicators, and improving the overall performance of the network. Each node in the network constructs and trains a local first strategy model based on the initial allocation strategy, and generates a local resource allocation strategy for each dimension through the first strategy model of each node, i.e., a local resource allocation strategy.

[0032] In addition, in order to take into account the emergency task demand of each node in the network, the local resource allocation strategy of each node is further optimized through federated learning based on the emergency task queue length of each node and the parameters of the first strategy model, and finally each node is allocated multi-dimensional resources based on the local resource allocation strategy of each node obtained through federated learning optimization. Through federated learning, the resource allocation strategy of each node is further optimized, so that the wireless edge network can be globally optimized in resource allocation, and the network can better meet the emergency task demand, improve the overall efficiency and service quality of the network.

[0033] As an embodiment of the present application, based on each resource state information, the corresponding resource indicators are calculated, including: Based on each resource state information, a multi-dimensional resource dynamic model is constructed; each dimension of the resource dynamic model respectively performs the following steps: Based on the task calculation amount and the proportion of computing power of each node, the computing load of the node is calculated. Based on the storage unit allocation capacity, used capacity and total capacity of each node, the storage fragmentation rate of the node is calculated. Based on the transmission power, channel gain and noise power of each node for the target user, the spectral efficiency of the node is calculated. Based on the service rate, task arrival rate, maximum tolerable delay and actual delay of each node performing different tasks, the reliability index of the node is calculated.

[0034] In an embodiment, a resource dynamic model is constructed for real-time mapping of resource indicators of each dimension resource. The resource dynamic model is represented in the form of a vector or a matrix, and each dimension maps corresponding resource state information, including: computing resource state data, storage resource state data, spectrum resource state data and power resource state data. The input of the model is the pre-processed resource state information of each node obtained in real time, and the corresponding resource indicators are output after calculation.

[0035] Specifically, the resource dynamic model is represented as: ; Wherein, represents a multi-source dynamic resource model; represents the real-time computing load; represents the storage fragmentation rate of the node; represents the spectral efficiency of the node; represents the reliability index of the node.

[0036] The resource indicator calculation method of each dimension is as follows: (1) Calculate the real-time computing load: based on the task arrival rate, task complexity and server processing capacity of the node (edge server), use the queuing theory model or task scheduling algorithm to calculate the computing load of the current network. In the embodiment of the present application, the real-time load is calculated by using the task scheduling algorithm. It should be noted that the M / G / 1 queuing model (queuing theory model) can also be used, in which the task arrival process conforms to the Poisson distribution, and the service time obeys the general distribution. By solving the queuing theory model, the utilization rate and waiting time of the computing resource and other load indicators can be obtained. The specific calculation steps are as follows: 1) Task computation amount perception: Deploy a computation resource monitoring module at the edge network node to obtain the computation amount of each task in real time. The computation amount is usually measured in CPU (Central Processing Unit) cycles. By analyzing the code structure, data processing requirements, and historical running records of the task, the number of CPU cycles required by each task is determined. For example, for an image processing task, the estimated CPU cycles required by the task are calculated based on factors such as image resolution and processing algorithm complexity. 2) Algorithmic power allocation rate determination: Based on the current resource state and task priority, the node allocates a corresponding algorithmic power allocation rate to each task. Specifically, the algorithmic power allocation rate ranges from 0 to 1, representing the proportion of the total computing resources allocated to the task by the server. For example, if the server has a total of 10 CPU cores and allocates 8 cores to a high-priority task, the algorithmic power allocation rate of the task is 0.8. 3) Based on the node's task computation amount and algorithmic power allocation rate, the node's computation load is calculated as follows: wherein is the computation amount of the i-th task, i is the algorithmic power allocation rate allocated by the edge server to the i-th task, and N is the current total number of tasks on the node. i For example, there are currently two tasks on the node, where the computation amount of task 1 is 1000 CPU cycles and the algorithmic power allocation rate is 0.5; the computation amount of task 2 is 1500 CPU cycles and the algorithmic power allocation rate is 0.5, then the real-time load

[0037] is: CPU cycles.

[0038] (2) Calculate the storage fragmentation rate: By analyzing the data storage and access patterns of the node's storage device, the number and size distribution of storage space fragments are counted, and the storage fragmentation rate is dynamically calculated by a pre-defined formula, such as the ratio of total fragment area to total storage capacity, combined with the growth trend of data storage and deletion strategy, to reflect the utilization efficiency and availability of storage resources. The specific calculation steps are as follows: 1) Monitor storage unit allocation and usage: Monitor the storage resources of each node in detail to obtain the allocated capacity and actual used capacity of each storage unit. The allocated capacity refers to the size of the storage space allocated to a specific data storage task in advance, and the actual used capacity refers to the part actually occupied in the allocated space.

[0039] ​​​​For example, in a distributed network storage system, user 1 is allocated 100 GB of storage space (allocated capacity), but user 1 is currently only using 60 GB (actual usage capacity).

[0040] 2) Calculate the storage fragmentation rate: based on the allocated capacity and actual usage capacity of each storage unit in the node, calculate the storage fragmentation rate of the node : ; wherein, is the allocated capacity of the mth storage unit, is the actual usage capacity, is the total storage capacity, and M is the total number of storage units.

[0041] For example, in a case where a node contains 3 storage units (i.e. M = 3) and the total storage capacity is 500 GB, the allocated capacity of storage unit 1 is 150 GB and the actual usage capacity is 130 GB; the allocated capacity of storage unit 2 is 200 GB and the actual usage capacity is 180 GB; the allocated capacity of storage unit 3 is 100 GB and the actual usage capacity is 70 GB. Then the storage fragmentation rate of the node is: .

[0042] (3) Calculate the spectral efficiency : based on the characteristics of the wireless channel, modulation and demodulation mode, multi-user access situation and other factors, calculate the spectral efficiency of the node using Shannon formula or similar communication theory model. Alternatively, according to the signal-to-noise ratio, bandwidth and transmission rate and other parameters, determine the maximum data transmission rate that can be supported per unit spectral width under the current spectral resource allocation, to measure the utilization efficiency of spectral resources. The specific calculation steps are as follows: 1) Obtain spectral resource parameters: obtain the transmit power, channel gain and noise power parameters of each user in real time through the wireless interface monitoring module. For example, in an AR / VR application scenario, multiple users connect to the node in the wireless edge network through wireless devices, and the wireless interface monitoring module of the node obtains the spectral resource parameters of each user in real time.

[0043] 2) Calculate the spectral efficiency of the user based on the obtained transmit power, channel gain and noise power parameters : ; wherein, is the transmit power of the nth user, is the channel gain of the nth user; is the transmit power of the jth user, is the channel gain of the jth user; is the noise power. The spectral efficiency reflects the effectiveness of the frequency spectrum resources occupied by each user in the presence of interference, and is used to evaluate the utilization efficiency of the frequency spectrum resources.

[0044] For example, assuming that at a certain moment, the transmission power of user 1 is 0.5 W, the channel gain is 2; the transmission power of user 2 is 0.3 W, the channel gain is 1.5; the noise power is 0.1 W, then the spectral efficiency of user 1 is calculated to be about 2.58.

[0045] The product of the transmission power and the channel gain of the target user represents the strength of the useful signal, while the sum of the interference signal powers of other users and the sum of the noise powers constitute the interference to the useful signal. The greater the ratio obtained by dividing the useful signal power by the total power of the interference and the noise, the higher the data transmission rate that can be achieved under the same frequency spectrum resources, and the higher the spectral efficiency. For example, in a wireless edge network in which multiple users share the same frequency band, by accurately calculating the spectral efficiency of each user, the frequency spectrum resources can be reasonably allocated to avoid interference and improve the spectral utilization rate of the overall network.

[0046] (4) Calculate the reliability index : The reliability of each component in the edge network is considered comprehensively, such as the failure rate of the server, the stability of the communication link, the data integrity of the storage device, etc. The reliability engineering method, such as failure mode and effect analysis or reliability block diagram method, is used to build a "delay-reliability" joint model to calculate the overall reliability index, such as mean time between failures (MTBF) or reliability probability, to ensure that the resource management strategy meets the performance requirements while having sufficient reliability guarantee.

[0047] Specifically, the service reliability index in the "delay-reliability" joint model is defined as The expression is as follows: ; ; Wherein, is the service rate of the kth task, is the task arrival rate, is the maximum tolerable delay, is the actual delay of the kth task; is the computing load of the node, is the storage fragmentation rate of the node, is the user's spectrum efficiency; α, β, and γ represent the contribution of computing resource load, storage fragmentation rate, and spectrum efficiency to task latency, respectively (i.e., the weight coefficients of each coefficient), satisfying α + β + γ = 1, and α, β, and γ are all greater than 0. In practical applications, the weight coefficients should be determined based on the specific application scenario and resource characteristics, using appropriate weighting methods such as the analytic hierarchy process (AHP) and the entropy method to ensure the rationality and accuracy of the formula.

[0048] The service rate and task arrival rate describe the service and arrival process of tasks in the network, while the maximum tolerable delay is the threshold for determining whether task delays are acceptable. The reliability index is calculated by calculating the difference between the maximum tolerable delay and the actual delay, and combining it with the service rate and arrival rate. A reliability index value closer to 1 indicates higher service reliability, meaning the probability of a task completing within the specified delay is greater. Conversely, a value closer to 0 indicates lower service reliability, meaning the probability of a task completing within the specified delay is lower.

[0049] For example, in Internet of Vehicles applications, the wireless edge network updates the vehicle's real-time location information by calculating its reliability index, evaluating the reliability of the service under current network conditions, and thus providing a reference for driving decisions.

[0050] In one embodiment, based on the updated local resource allocation policy of each node, multi-dimensional resource allocation is performed for each node. The specific steps are as follows: (1) Based on the optimized local resource allocation strategy, establish a mapping relationship between physical devices and resource allocation strategies in the edge network. Generate corresponding task instructions based on the local resource allocation strategy. For example, generate computing task offloading instructions based on the computing resource allocation strategy, determine the tasks that need to be processed locally or offloaded to other edge servers, and determine the specific offloading path and transmission parameters. Generate bandwidth allocation instructions based on the spectrum resource allocation strategy, clarify the spectrum resource allocation ratio and bandwidth size of each communication link or user. Generate power control instructions based on power resources, set the transmission power of base stations and terminal devices to meet communication quality and energy consumption requirements; (2) The generated control instructions for resource allocation in each dimension are formatted according to the device and network communication protocols in the wireless edge network to ensure that the instructions can be correctly parsed and executed. Then, the instructions are sent to the corresponding devices and system components, such as edge servers (nodes), base stations, and terminal devices, through the corresponding communication channels to achieve real-time control and adjustment of resource allocation.

[0051] As an implementation method of the present application, based on various resource indicators, the initial resource allocation strategy of each node is determined by reinforcement learning, including: Based on each resource index, a state space is defined through a deep reinforcement learning algorithm, and state space parameters are generated; Based on each state space parameter, an action space is defined, and corresponding resource adjustment amounts are generated as action space parameters; the resource adjustment amounts include: computing power allocation adjustment amount, storage allocation adjustment amount, bandwidth allocation adjustment amount, and power adjustment amount; Based on the state space parameters and corresponding action space parameters, a policy network is constructed; Based on the state space parameters, the action space parameters, and the overall performance indicators of the wireless edge network, a multi-objective reward function is constructed; the overall performance indicators of the wireless edge network include: network total latency, network maximum allowed latency, network computing power upper limit, data volume of tasks in the network, and storage access rate; Based on the multi-objective reward function, the parameters of the policy network are updated; based on the updated policy network, an initial resource allocation strategy for each node is generated.

[0052] In an embodiment, based on a resource dynamic model, an initial resource allocation strategy for a wireless edge network is generated through a hierarchical intelligent decision engine. In this embodiment, the hierarchical intelligent decision engine is constructed based on deep reinforcement learning. Specifically, the initial resource allocation strategy for the node is determined through deep reinforcement learning, and the specific steps are as follows: (1) Based on a deep reinforcement learning algorithm, each resource index is obtained from the resource dynamic model as a state space parameter, including: computing load , storage fragmentation rate , spectrum efficiency , and reliability index at the previous moment . These parameters comprehensively reflect the resource state and service quality of the edge network at time t. Based on each state space parameter, a state space vector is constructed as follows: ; The state space vector, as the input of the deep reinforcement learning agent to perceive the state of the environment, provides comprehensive environmental information for subsequent resource allocation decisions.

[0053] (2) Each state space parameter is defined in an action space to generate a multi-dimensional resource adjustment amount. Specifically, first, the action space parameters are determined, and the adjustable resource allocation parameters in each dimension are taken as the action space parameters, specifically including: computing power allocation adjustment amount , storage allocation adjustment amount , bandwidth allocation adjustment amount , and power adjustment amount The action space parameters are associated with the dynamic adjustment of various resources in the wireless edge network, which can flexibly change the resource allocation strategy to adapt to different network states. The expressions of each action space parameter are as follows: ; ; ; ; wherein, , , , are the weight matrices of the corresponding parameters; , , , are the bias vectors of the respective corresponding parameters; is the state space vector; is the sigmoid activation function, which is used to limit the output within a reasonable range; is the hyperbolic tangent function, which is used to limit the output within the range of [−1, 1]; ReLU is the rectified linear unit function, which is used to ensure that the bandwidth allocation adjustment amount is non-negative.

[0054] (3) Based on each action space parameter, the action space vector is constructed as follows: .

[0055] The agent realizes real-time adjustment of resource allocation in each dimension of the network by outputting the action space vector.

[0056] For example, when the computing load is high, the agent adjusts the computing power allocation adjustment amount to relieve the computing pressure; when the storage fragmentation rate is too high, the storage allocation adjustment amount is adjusted to optimize the utilization of storage resources.

[0057] In order to balance the service reliability and delay of the node and other targets, the improvement effect of the action space parameters on the state space parameters and the overall performance index of the network are comprehensively considered, and a multi-objective reward function is constructed, which is as follows: ; wherein, , , are the weight coefficients, satisfying ; the weight coefficients are used to balance each optimization target, so as to ensure that the reward function can comprehensively reflect the pros and cons of the resource allocation strategy; is the total network delay; is the maximum allowed network delay, L is the data volume of the task in the network, R is the storage access rate; ΔL is the computing power allocation adjustment amount, ΔB is the bandwidth allocation adjustment amount, ΔS is the storage allocation adjustment amount, L is the computing load of the node, R is the spectrum efficiency of the user, S is the storage fragmentation rate of the node.

[0058] The weight coefficient is used to balance each optimization target, and the correlation term of the state space parameter and the action space parameter (including the network total delay, the maximum allowed delay, the computing power upper limit, the data volume of the task and the storage access rate) is introduced to enhance the sensitivity of the reward function to the state and the action.

[0059] The policy network of the agent is dynamically updated by optimizing the target reward function, so as to obtain the initial resource allocation strategy of all nodes in the network.

[0060] The embodiment can accurately perceive the network state change and timely adjust the resource allocation strategy through dynamic modeling and intelligent decision-making, so as to ensure that the network can still maintain low-delay communication performance when facing sudden task requests, channel time-varying and mobile terminal position drift and other dynamic changes. The multi-dimensional resource collaborative optimization breaks the limitations of hierarchical management in the traditional resource allocation scheme, and realizes efficient collaboration across resource dimensions.

[0061] For example, in the case of wireless edge network applied to the intelligent medical scene, the computing resource, the storage resource, the spectrum resource and the power resource in the network can be flexibly allocated and dynamically adjusted according to the actual task demand and the network state, so as to improve the resource utilization rate of the medical image real-time analysis task, and can reduce the time delay of the medical resource utilization, thereby supporting the real-time interaction of remote surgery.

[0062] As an embodiment of the application, updating the parameters of the policy network further includes: Based on the bandwidth allocation adjustment amount and the power adjustment amount of the action space parameter, the competition relationship and uncertainty between users are introduced through Bayesian game theory to simulate the resource competition behavior in the multi-user environment, and the user utility function is constructed; The multi-objective reward function and the user utility function are optimized in an alternating iterative manner to update the parameters of the policy network.

[0063] In an embodiment, the competition relationship and uncertainty among user tasks in the network are also taken into consideration to ensure efficient and fair resource allocation, so that in addition to updating the policy network according to the multi-objective reward function, the bandwidth allocation adjustment amount and the power adjustment amount of the action space parameter are also analyzed for game equilibrium based on Bayesian game theory to generate a user utility function. The competition relationship and uncertainty among users are introduced by using the user utility function to simulate multi-user competition behavior and generate a reasonable competition strategy, thereby achieving fair and efficient allocation of resources.

[0064] On this basis, based on the multi-objective reward function and the user utility function, common optimization is performed through alternating iteration to achieve an efficient allocation strategy that takes into account resource fairness and multi-user competition fairness.

[0065] As an embodiment of the present application, a user utility function is constructed, including: For a node i , a bandwidth allocation adjustment amount, a power adjustment amount, a spectrum efficiency, and a power compensation reference value are constructed to generate a transmission rate term of the node i ; the transmission rate term is used to reflect the balance between the transmission rate and the power consumption of the user; Based on the difference between the ratio of the power adjustment amount and the bandwidth allocation adjustment amount of the node i and another node j , and a resource competition penalty coefficient, a resource competition term of the node i is constructed; Based on the transmission rate term and the resource competition term of the node i , a user utility function of the node i is constructed.

[0066] In an embodiment, the construction of the user utility function comprehensively considers the resource allocation obtained by each user and the cost paid by each user, as well as the expectation of the behavior of other users, thereby generating a reasonable competition strategy and achieving fair and efficient allocation of resources. Specifically, the user utility function is composed of the communication efficiency improvement brought by bandwidth and power adjustment, and the penalty brought by resource competition. The communication efficiency improvement reflects the benefits obtained by the user through adjusting the bandwidth and power, and the resource competition penalty inhibits the user from excessively occupying resources, prompting the user to consider the impact on other users in the competition, thereby maintaining the overall fairness and stability of the network.

[0067] In an embodiment of the present application, the expression of the user utility function is as follows: ; Wherein, is a power compensation reference value; k is a resource competition penalty coefficient. The transmission rate item is used to reflect the balance between the user's transmission rate and power consumption; It is a resource competition item used to suppress excessive competition among users; By comparing the ratio of the power adjustment amount to the bandwidth allocation adjustment amount of user i with the ratio of the power adjustment amount to the bandwidth allocation adjustment amount of other users j, it is used to measure the user i the degree of competition for resources faced; Adjust the amount for bandwidth allocation, Spectral efficiency for users, is the power adjustment amount; 、 For two users i 、 j The bandwidth allocation adjustment amount; 、 For two users i 、 j The power adjustment amount.

[0068] In practical applications, calculating the user utility function requires considering each user's action space parameters as well as the status and actions of other users in the network. When adjusting a user's bandwidth and power allocation, the user utility function is calculated based on the current network state and the actions of other users. Furthermore, solving the Bayesian equilibrium yields the optimal user competition strategy for the current network state.

[0069] For example, when the user i Increasing the bandwidth allocation adjustment or decreasing the power adjustment increases resource utilization efficiency, thereby improving the user's utility function. At the same time, if the power-to-bandwidth ratio difference with other users increases, the resource competition penalty increases accordingly, reducing the user's utility function. Therefore, it is necessary to find a balance between resource utilization efficiency and competition fairness to maximize user utility.

[0070] In the embodiment of the present application, the multi-objective reward function and the user utility function of the deep reinforcement learning policy network are updated by alternating iterations to generate an initial resource allocation strategy. The specific update process is as follows: (1) Based on the reward value and action value function of the multi-objective reward function, the parameters of the agent strategy network are updated by gradient descent method. : ; in, is the learning rate; is the discount factor; is the action-value function; Represents the policy network parameters Find the gradient; The immediate reward represents a reward signal fed back by the environment after the agent performs an action at time t, and is used to guide the direction of policy optimization. (2) Solving the Bayesian equilibrium, by optimizing the combination of the user utility function and the reward value of the multi-objective reward function, the best response strategy of the user is solved: ; wherein, is the collaborative optimization weight, is the power adjustment amount in the best response strategy; is the bandwidth allocation adjustment amount in the best response strategy; is the immediate reward; is the user utility function; (3) Set the convergence condition to judge whether the policy network of the agent and the user strategy converge, and the judgment condition is as follows: ; wherein, and are convergence thresholds, and the process is repeated until the convergence condition is met; , are the computing power allocation adjustment amounts at time t and time t+1 respectively; , are the bandwidth allocation adjustment amounts at time t and time t+1 respectively.

[0071] The repeated iteration process includes updating the agent policy network, solving the Bayesian game equilibrium solution, and convergence judgment. In the policy network update, the reward value and the action value function are used to guide the adjustment of the network parameters; in the Bayesian game equilibrium solving, the optimal user competition strategy is found by combining the user utility function and the collaborative optimization weight; whether to stop updating is determined by defining the convergence judgment condition (such as the change amplitude threshold of the resource adjustment amount). In the case of convergence of the policy network and the user utility function, the initial resource allocation strategy is output by the policy network. The initial resource allocation strategy will serve as a benchmark for subsequent resource optimization, guiding the actual allocation and adjustment of various resources in the edge network, and the final output of the initial resource allocation strategy is: ; wherein, represents the computing power allocation adjustment amount at the next time (time t+1) in the best response strategy, represents the storage allocation adjustment amount in the best response strategy, represents the bandwidth allocation adjustment amount in the best response strategy, represents the power adjustment amount in the best response strategy.

[0072] In the embodiments of the present application, the multi-objective reward function and the user utility function are updated alternately to enable the agent to optimize the resource allocation strategy step by step in consideration of user competition, thereby achieving the dual optimization of network performance and user fairness. Further, each node constructs and trains a local first strategy network based on the initial resource allocation strategy, thereby generating a local resource allocation strategy according to the local resources, tasks and user competition of the node.

[0073] As an embodiment of the present application, the resource state information further includes resource demand information of each node; and the method further includes: The network state information of the wireless edge network is acquired at a first time interval, including node topology structure, device parameters of each node; Based on the local resource allocation strategy of each node, the current resource state information of the node and the network state information, a network mirror model corresponding to the wireless edge network is constructed by simulation to map the behavior and performance of each node in the wireless edge network; After the local resource strategy of each node is updated, the behavior and performance of each node in the wireless edge network are simulated synchronously by an incremental learning and adaptive adjustment algorithm.

[0074] In an embodiment, a digital twin technology is adopted to construct a network mirror model based on an initial resource allocation strategy to support verification and dynamic optimization of the resource allocation strategy. The network mirror model is based on the physical components and logical architecture of the actual wireless edge network, and uses physical modeling, data-driven modeling and simulation technology to map real-time network state information (including network topology structure, node device parameters) and resource state information into the network mirror model. The network mirror model accurately maps the behavior and performance of each node in the actual network. The behavior of the node in the network refers to the dynamic operation logic and interaction rules of the network node in the resource scheduling process, including: (1) task scheduling behavior, including: edge server offloading decision for computing tasks, priority queue management; (2) resource competition behavior, including: power adjustment under Bayesian equilibrium, fragmentation reorganization trigger threshold; (3) fault response behavior, including: task migration path selection when the node is down, spectrum switching strategy (FMEA reliability model) when the channel is interrupted.

[0075] The performance of the node in the network refers to the quantifiable service indicators generated after the resource allocation strategy acts on the network, including: (1) key latency indicators, including: end-to-end task processing latency; (2) resource efficiency indicators, including: storage fragmentation rate, spectrum utilization rate, computing resource idle rate; (3) System reliability indicators, including: task packet loss rate, service availability.

[0076] Moreover, the network mirror model has a dynamic data synchronization mechanism. As the wireless edge network changes dynamically, such as the addition of new devices, dynamic changes in tasks, consumption and replenishment of resources, etc., the network mirror model updates the network state information dynamically through incremental learning and self-adaptive adjustment algorithm, ensures that the model is consistent with the real-time state of the actual wireless edge network, and realizes real-time simulation of the running of the real wireless edge network under different resource allocation strategies in the virtual environment.

[0077] As an embodiment of the present application, the method further comprises: In the high-reliability low-latency scenario, the behavior and performance of each node are obtained according to a second time interval through the network mirror model, and the current latency and reliability indicators are calculated; The current latency and reliability indicators are compared with the latency threshold and reliability threshold corresponding to the high-reliability low-latency scenario; In the case where the latency is higher than the latency threshold or the reliability indicator is lower than the reliability threshold, the local resource allocation strategy of each node is updated through federated learning.

[0078] In this embodiment, the network mirror model is also used to simulate the network performance under different resource allocation strategies, replacing the real network trial and error to verify the effectiveness of the resource decision. Optionally, in the high-reliability low-latency scenario, the behavior and performance of each node in the network are mapped in real time through the network mirror model, and the latency and reliability indicators of the network when performing emergency tasks in this scenario are simulated. Further, based on the simulated latency and reliability indicators, in combination with the latency threshold and reliability threshold required by the high-reliability low-latency scenario, the local resource allocation strategy of the node is updated to improve the ability of the wireless edge network to respond to emergency tasks in the high-reliability low-latency scenario, so that the network has timely and efficient multi-dimensional resource dynamic allocation performance, and can better meet the user's demand for low-latency and high-reliability communication, provide full-link performance guarantee for front-end digital services, and improve the user's experience.

[0079] In this embodiment, the high-reliability low-latency scenario can be: an autonomous driving scenario, an AR / VR video frame real-time rendering scenario, a control scenario requiring microsecond-level synchronization of instructions in smart industry, a zero-jitter interaction scenario for remote operation of smart medical treatment, a scenario for balancing real-time access of a large number of terminals in smart city, etc.

[0080] For example, in an automatic driving scenario, the network mirror model maps the state data of vehicles, roadside units and edge servers in real time, simulates the end-to-end latency and reliability of an emergency obstacle avoidance task, and updates the local resource allocation strategy of the nodes based on the latency threshold and reliability threshold in the emergency obstacle avoidance scenario through federated learning.

[0081] In this embodiment, the network mirror model also records the historical demand data of each node in the network, which is also used to predict the future resource demand of the node. The prediction result is used to train the first strategy model locally. In this embodiment, based on the ConvLSTM (Convolutional Long Short-Term Memory, Convolutional Long Short-Term Memory Network) model and the fully connected layer, an adaptive optimization model for node characteristics, i.e. the first strategy model, is constructed.

[0082] The ConvLSTM model combines convolutional neural networks and long short-term memory networks and can process spatial and temporal data at the same time. In a wireless edge network, resource demand often has spatiotemporal correlation, for example, the demand for computing resources at different time points and different nodes may exhibit certain patterns. In this embodiment, the ConvLSTM model is used to capture the spatiotemporal characteristics of resource demand and improve the accuracy of future resource demand prediction. The structure of the ConvLSTM model includes an input layer, a CNN (Convolutional Neural Network) layer, an LSTM (Long Short-Term Memory) layer, and an output layer.

[0083] The network mirror model records the historical resource demand data of the node, including the historical demand data of computing resources, storage resources and spectrum resources. These historical resource demand data are normalized and time series constructed and input into the ConvLSTM model. The input layer of the model receives the historical resource demand data (a multi-dimensional spatiotemporal matrix) of the node recorded by the network mirror model in the previous time period, extracts the spatial features (such as the resource demand distribution pattern between different edge nodes) of the historical resource demand data through the convolution layer, and then captures the long-term dependence relationship (such as the periodic change and trend of resource demand) of the time series through the LSTM layer, thereby generating the prediction result of the future resource demand of the node, and outputting the future resource demand sequence. The prediction result of the ConvLSTM model The following expression is obtained: ; wherein, is a spatiotemporal convolution kernel weight matrix; is an LSTM unit parameter matrix; is a historical sequence input with a time window (a time series slice) representing the set of continuous spatio-temporal feature tensors from time instant to time instant, written as . .

[0084] In actual application, the historical resource demand data recorded by the network mirror model is used to train the ConvLSTM model. During the training, the loss function (such as mean square error) is used to measure the difference between the predicted value and the true value, and the model parameters are adjusted through the back propagation algorithm and . After the training is completed, the model predicts the resource demand data (resource demand sequence ) at future time instant t+1 according to the current time instant (time instant t) of the wireless edge network and the historical resource demand data of the historical time period, which provides a basis for subsequent global-to-single-node adjustment of the resource allocation strategy by federated learning. Optionally, the future resource demand sequence can be a scalar or a vector.

[0085] For example, in the industrial internet scenario, by analyzing the computing resource and storage resource demand of different production periods in the past week, the spatio-temporal convolution LSTM model is trained to predict the resource demand of the next production period, thereby providing a basis for the scheduling of production tasks and the pre-allocation of resources.

[0086] Optionally, the future resource demand sequence output by the ConvLSTM model is used to assist the training of the first strategy model of the node, accelerate the model convergence, and improve the generation efficiency of the local resource allocation strategy.

[0087] By constructing a network mirror model to map the dynamically changing network state information in the wireless edge network in real time, and using a ConvLSTM model to predict future resource demand data, the local first strategy network is updated to achieve the effect of dynamically adapting the node's strategy to the spatio-temporal variation characteristics of the network, thereby improving the network's ability to adapt to dynamic changes. Whether it is a short-term burst task or a long-term network topology change, it can be effectively dealt with through real-time sensing and intelligent decision-making, so that the edge network can still maintain good performance and service quality in a complex and changing environment.

[0088] As an embodiment of the present application, based on the local resource allocation strategy of each node and the length of the emergency task queue, the local resource allocation strategy of each node is updated through federated learning, including: determining the weight corresponding to each node based on the current length of the emergency task queue of each node; calculating the global model parameters according to the model parameters of the first strategy model of each node and the weight corresponding to the node; send the global model parameters to each node, so that each node updates the corresponding first strategy model based on the global model parameters; generate the local resource allocation strategy of the node based on the first strategy model updated based on the parameters.

[0089] In the traditional resource allocation strategy of the wireless edge network, the resource competition among distributed nodes lacks a global coordination mechanism. For example, in a large-scale user scenario such as a smart city, a user selfish strategy easily leads to a Nash equilibrium deviating from an optimal state, causing a decline in overall resource allocation fairness and efficiency and affecting the overall performance of network services.

[0090] In order to realize global resource allocation strategy adjustment considering the competition among nodes, in the embodiment, the local resource allocation strategies of each node are updated through federated learning. Federated learning allows multiple participants to collaboratively train a shared model without sharing raw data. In the embodiment, each node in the wireless edge network is regarded as a participant, and each participant has a corresponding local resource allocation strategy. Through federated learning, the parameters of the local first strategy model of each node are globally adjusted to generate global model parameters, and the local resource allocation strategies of each node are adjusted based on the global model parameters, so as to aggregate the local resource allocation strategies of each node while protecting the data privacy of each node, thereby globally adjusting the local resource allocation strategies of each node and improving the overall resource allocation efficiency.

[0091] Specifically, the federated learning framework in the embodiment includes participants, a coordinator, a communication mechanism, and a privacy protection mechanism. The participants are all nodes, and in the embodiment, each node trains a local first strategy model using local resource demand data and task execution data. The resource demand data includes demand data for computing resources, storage resources, and spectrum resources; the task execution data includes time delay data such as task processing time delay of computing / storage access, transmission time delay affected by spectrum allocation; energy consumption data such as actual power consumption of devices under power control instructions such as base station transmission power and server CPU energy consumption; task success rate, task interruption frequency, etc. These data reflect the resource usage mode and task characteristics specific to the node.

[0092] The coordinator is a central server of the wireless edge network, responsible for coordinating the training process of each node, aggregating the parameters of the local first strategy model, and generating global model parameters. The communication mechanism defines the communication mode and data transmission format between the participants and the coordinator, and the privacy protection mechanism is used to ensure the data privacy of the participants is protected during the training process to prevent data leakage.

[0093] In the embodiment, according to the emergency task queue length of each node Determine the corresponding weight of the node. For nodes with longer emergency task queues, the greater the impact of their local policies on the global model. The emergency task queue length refers to the number of high-priority tasks backlogged within a single node. In actual applications, the node's emergency task queue length is obtained in real time through the local task scheduler of the edge server node. Specifically, when a task arrives at a node, the task scheduler marks the priority of different tasks according to preset rules. High-priority tasks are added to the emergency task queue, while tasks of other priority levels are added to the normal task processing queue.

[0094] For example, AR / VR frame rendering tasks are marked as normal priority, and vehicle collision warning tasks are marked as emergency priority. Preset rules can be set based on latency sensitivity, business type, etc. The scheduler keeps track of the number of tasks marked as emergency and not yet processed, generating real-time value.

[0095] Using the weights corresponding to each node and the local model parameters of the node , calculate the global model parameters The details are as follows: ; in, For the i The length of the node's urgent task queue; For the j The length of the node's urgent task queue; N is the total number of nodes.

[0096] Federated learning is used to calculate weights based on the weights of each node and local model parameters, thereby ensuring that the network's resource allocation strategy can focus more on the needs of urgent tasks in the nodes, thereby improving the overall response speed and service quality of the network.

[0097] Figure 2 This is a flow chart of updating the local resource allocation strategy through federated learning in one embodiment of the present application. Figure 2 As shown in the figure, based on the ConvLSTM model and the fully connected layer, the first node-local strategy model is constructed. Based on the initial resource allocation strategy and the historical resource demand data of each node recorded by the network mirror model, each node (Note: the network includes n nodes, Figure 2The first strategy model is trained according to the local task load and resource usage to obtain corresponding local model parameters. The ConvLSTM model predicts the future resource demand data of the node based on the historical resource demand data of the node, and updates the parameters of the first strategy model based on the predicted resource demand data and the actual resource demand data at the next moment. The central server aggregates the parameters of the first strategy model of each node, determines the weight of the model parameters of the corresponding node according to the emergency task queue length of each node, and then calculates the global model parameters of the wireless edge network based on the weight. The central server distributes the global model parameters to each node, and each node updates the local resource allocation strategy using the global model parameters, so as to realize the resource allocation strategy optimization considering the global optimization of the network and the local emergency task demand of the node.

[0098] As an embodiment of the present application, after adjusting the local resource allocation strategy of each node through federated learning, it further includes: Based on the total length of the task queue and the real-time total power consumption of the wireless edge network, a Lyapunov function is constructed and a corresponding drift term is calculated; Based on the total power consumption budget of the wireless edge network, the current real-time total power consumption and the drift term, a first objective function is constructed; Adjust the allocation weight of each dimension resource in the local resource allocation strategy of each node to minimize the first objective function, and update the local resource allocation strategy of each node.

[0099] In an embodiment, after adjusting the local resource allocation strategy of the node through federated learning, the network has the ability to quickly respond to emergency tasks, and further through the Lyapunov drift optimization method, the stability of the task queue and the resource consumption of the network are optimized to guarantee the long-term performance of the wireless edge network.

[0100] Specifically, the resource weight is adjusted through the Lyapunov drift optimization method to optimize the long-term stability of the task queue and the power consumption in the network, and to ensure efficient allocation of resources under the premise of meeting the power budget. First, the state and control variables (total length of task queue, total power consumption of network, etc.) of the Lyapunov queue dynamic model are defined, and the Lyapunov queue dynamic model at the next moment (i.e. at time t+1) is constructed: ; Wherein, is the service capacity of the network, which refers to the upper limit of the number of tasks that the edge network can actually handle within time slot t; is the task arrival amount before the network; is the total length of the task queue of the network at the next moment.

[0101] In the embodiment of the present application, the future queue length of the node is calculated based on the prediction result of the future resource demand of the node output by the ConvLSTM model. Further, based on the future queue length of each node, the total length of the task queue of the wireless edge network is calculated. .

[0102] Based on this dynamic model, the resource weight vector adjust ,in, To calculate the load The optimization weight of Storage fragmentation rate The control weight of Spectral efficiency Its role is to abstract queue stability into an optimizable object. It has no parameter sharing with the resource dynamic model and reinforcement learning strategy model, and independently drives resource weight adjustment.

[0103] In this embodiment, the constraints of the Lyapunov drift optimization (i.e., the purpose of the optimization) are: (1) Total power consumption of the network Do not exceed the total power budget ; (2) Minimize the growth of the total backlog of the network task queue to ensure the stability of the network queue.

[0104] Based on the Lyapunov dynamic model and constraints, construct the Lyapunov function and the corresponding drift term : ; in, is the task queue length of the kth edge server at time t.

[0105] The first objective function of Lyapunov drift optimization includes the following parts: (1) Lyapunov drift term: measures the changing trend of network status, such as the change of queue length; (2) Penalty terms: terms related to network constraints, such as the difference between power consumption and budget; (3) Weight adjustment item: Dynamically adjust resource weights to balance the allocation of different resource dimensions.

[0106] Specifically, the expression of the first objective function is as follows: ; Wherein, V is the stability control parameter; is the total power consumption of the network at time t; is the total power budget of the network; is a Lyapunov drift term, and its expression is: ; wherein E is a mathematical expectation operator, representing a statistical average value of the change of the queue length under the condition of random task arrival; is a Lyapunov function.

[0107] In this embodiment, by adjusting the allocation weight of each dimension resource (such as computing resource, storage resource, bandwidth resource and power resource, etc.), the first objective function is minimized, that is, the weighted sum of the Lyapunov drift term and the network power consumption exceeding the total power budget is minimized. Further, under the premise of meeting the network power budget constraint , the growth of the task queue backlog in the network is minimized, and the stability of the task queue length is guaranteed. The stability control parameter V is used to balance the relationship between the queue stability and the power consumption, and by adjusting the value of V, different optimization target focuses can be realized. For example, in the case of more strict control of power consumption, the value of V can be increased to increase the weight of the power cost term in the optimization target.

[0108] The embodiments of the present application realize the stability constraint of the total network power consumption and the task queue length by Lyapunov optimization, guarantee the stability of the network queue, avoid the communication interruption or data loss caused by queue congestion, improve the stability and reliability of the network providing long-term service, and improve the overall performance and resource utilization of the network. Taking the automatic driving scene as an example, the present scheme can realize end-to-end extremely low latency and high reliability of tasks, and meet the stringent requirements of vehicle cooperative decision and emergency braking.

[0109] As an embodiment of the present application, the method further comprises: acquiring task execution data of each node from the network mirror model according to a third time interval; the task execution data includes latency index data, energy consumption index data and task progress index data; generating a resource state report of the wireless edge network based on the task execution data of each node; updating the first strategy model of each node based on the task execution data of each node.

[0110] In this embodiment, after each node executes the corresponding control instruction based on the local local resource allocation strategy, the behavior and performance of each node are continuously monitored. Specifically, according to a third time interval, task execution data of the nodes are obtained from the network mirror model, including latency index data, energy consumption index data, and task progress index data. Among them, the latency index data includes task processing latency, data transmission latency, etc.; the energy consumption index data includes power consumption of devices in the network, battery power change, etc.; the task progress index data includes task execution state (including completion progress, stage), dependency relationship (including subtask order), resource matching information (i.e. whether the current resources are sufficient), and environmental dynamics (including node movement information, channel quality), etc. The task execution data of each node is aggregated, sorted and analyzed to generate a resource state report of the network. The report content can be presented in the form of intuitive charts, curves or data tables, so as to clearly show the latency and energy consumption of multi-dimensional resources in actual operation, facilitate users to understand the resource allocation effect and network performance, provide basis for subsequent strategy adjustment and further optimization, and improve user experience. In addition, the newly generated task execution data in each node is used for incremental training of the first strategy model of the node locally, so as to dynamically adapt to the change of the node and maintain the optimal overall performance of the network.

[0111] Figure 3 is a flowchart of determining a multi-dimensional resource allocation strategy for a wireless edge network in an embodiment of the present application. As shown in Figure 3As shown, first, the resource state information of each node in the network is collected in real time by sensors and monitoring devices deployed in the wireless edge network, including state data of computing resources, storage resources, spectrum resources, and power resources. The collected multi-dimensional data is cleaned and standardized, and then a dynamic resource model that dynamically maps multi-dimensional resource indicators is constructed based on a dynamic modeling algorithm to quantify the real-time state of the network. Then, through a hierarchical intelligent decision engine, the state space parameters (including computing load, storage fragmentation rate, spectrum efficiency, reliability indicators, etc.) and action space parameters (including computing power allocation adjustment, storage allocation adjustment, bandwidth allocation adjustment, power adjustment, etc.) are defined using deep reinforcement learning, and a multi-objective reward function is constructed based on the state space parameters and action space parameters to jointly optimize latency, energy consumption, and resource utilization. At the same time, a user utility function is introduced to balance the competition among multiple users in a single node, and an alternating iteration method is used to jointly optimize the multi-objective reward function and the user utility function to generate an initial resource allocation strategy that takes into account resource utilization efficiency and user fairness. On this basis, a network mirror model is constructed using digital twinning technology to map the network dynamics in real time, and the network performance under different resource allocation strategies is simulated through the network mirror model to replace the real network trial and error to verify the effectiveness of the resource decision. With changes in the actual network, such as the addition of new devices, dynamic changes in tasks, resource consumption and replenishment, etc., the network mirror model is dynamically updated in real time through incremental learning and adaptive adjustment algorithms. Through real-time sensing modeling, hierarchical intelligent decision-making, and digital twinning dynamic optimization mechanisms, the collaborative scheduling of computing, storage, spectrum, and power resources is achieved.

[0112] On the basis of the initial resource allocation strategy, a local first strategy model is constructed based on the ConvLSTM model, the historical resource demand data obtained from the network mirror model is predicted through the ConvLSTM model, and the local first strategy model is trained using the historical resource demand data and task execution data of the node to obtain local model parameters. The local model parameters of each node are aggregated through federated learning, and the weight is adjusted according to the length of the emergency task queue to determine the global model parameters, and the local first strategy model of each node is updated based on the global model parameters. Further, the local resource allocation strategy of each node is updated through Lyapunov drift optimization to dynamically adjust the allocation weight of each dimension of resources in the network, so as to constrain the stability of the total power consumption and the task queue length of the network, improve the stability and reliability of the network in providing long-term services, and improve the overall performance and resource utilization of the network.

[0113] The control instructions are generated based on the resource allocation strategies of the nodes, including: calculating task offloading instructions, spectrum bandwidth allocation instructions and power control instructions, and executing by the controllers of the nodes. After the nodes execute the control instructions, the newly generated task execution data (such as time delay data, energy consumption indicators, etc.) of the nodes are continuously monitored and fed back to generate resource state reports, forming a closed-loop resource management link of "perception -> modeling -> decision -> execution -> feedback", continuously improving the accuracy of resource management allocation and the robustness of the network. In addition, the newly generated task execution data of the nodes is used as new training data to incrementally train the first strategy model of the nodes, thereby further improving the accuracy of resource management allocation.

[0114] Based on the same inventive concept, an embodiment of the present application provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps in the multi-dimensional resource management joint optimization method based on a wireless edge network according to any one of the above embodiments of the present application.

[0115] Based on the same inventive concept, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the multi-dimensional resource management joint optimization method based on a wireless edge network according to any one of the above embodiments of the present application.

[0116] Based on the same inventive concept, an embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, which, when executed by the processor, implements the steps in the multi-dimensional resource management joint optimization method based on a wireless edge network according to any one of the above embodiments of the present application.

[0117] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0118] For the method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and components involved are not necessarily required by the present application.

[0119] Those skilled in the art will appreciate that embodiments of the application can be supplied as a method, a device, or a computer program product. Thus, embodiments of the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0120] Embodiments of the application are described herein with reference to the drawings, in which are shown flowcharts and / or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the application. It will be understood that each flow and / or block of the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowcharts and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart or block diagram block or blocks.

[0121] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart or block diagram block or blocks.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer-implemented process such that the instructions which execute on the computer or other programmable terminal device provide steps for implementing the flowchart or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart or block diagram block or blocks.

[0123] While preferred embodiments of the application have been described, modifications and variations can be apparent to those skilled in the art once aware of the general underlying concepts. Therefore, the application is to be interpreted in the broadest sense and encompasses all modifications and variations falling within the scope of the application.

[0124] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0125] The above describes in detail the multi-dimensional resource management joint optimization method based on wireless edge network provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for joint optimization of multi-dimensional resource management based on wireless edge network, characterized in that, The application is applied to a processor arranged in a terminal, a server or a base station; the method comprises: Real-time acquisition of multi-dimensional resource state information of each node in a wireless edge network; Calculation of corresponding resource indicators based on each resource state information, including calculation of load, storage fragmentation rate, spectral efficiency and reliability indicators; Based on each resource indicator, an initial resource allocation strategy of each node is determined through reinforcement learning, so that each node in the each node performs the following operations: based on a space-time convolution long short-term memory network, a local first strategy model is constructed, and the first strategy model is trained based on the initial resource allocation strategy, historical demand data and task execution data of the node; through the first strategy model, a local resource allocation strategy of the node is generated; Based on the local resource allocation strategy of each node and the length of the emergency task queue, the local resource allocation strategy of each node is updated through federated learning; Based on the updated local resource allocation strategy of each node, multi-dimensional resource allocation is performed for the each node.

2. The method of claim 1, wherein, Based on each resource state information, corresponding resource indicators are calculated, including: Based on each resource state information, a multi-dimensional resource dynamic model is constructed; each dimension of the resource dynamic model respectively performs the following steps: Based on the task calculation amount and the computing power proportion of each node, the computing load of the node is calculated; Based on the storage unit allocation capacity, the used capacity and the total capacity of each node, the storage fragmentation rate of the node is calculated; Based on the transmit power, channel gain and noise power of each node to the target user, the spectral efficiency of the node is calculated; Based on the service rate, task arrival rate, maximum tolerance delay and actual delay of each node executing different tasks, the reliability indicator of the node is calculated. 3.The method of claim 1, wherein, Based on each resource indicator, an initial resource allocation strategy of each node is determined through reinforcement learning, including: Based on each resource indicator, a state space is defined through a deep reinforcement learning algorithm to generate state space parameters; Based on each state space parameter, an action space is defined to generate corresponding resource adjustment amounts as action space parameters; the resource adjustment amounts include: computing power allocation adjustment amount, storage allocation adjustment amount, bandwidth allocation adjustment amount and power adjustment amount; Based on the state space parameters and the corresponding action space parameters, a strategy network is constructed; Based on the state space parameters, the action space parameters and the overall performance indicators of the wireless edge network, a multi-objective reward function is constructed; the overall performance indicators of the wireless edge network include: network total delay, network maximum allowed delay, network computing power upper limit, task data volume in the network and storage access rate; Based on the multi-objective reward function, the parameters of the strategy network are updated; based on the updated strategy network, an initial resource allocation strategy of each node is generated.

4. The method of claim 3, wherein, Updating the parameters of the strategy network also includes: Based on the bandwidth allocation adjustment amount and the power adjustment amount of the action space parameters, the competition relationship and uncertainty between users are introduced through Bayesian game theory to simulate resource competition behavior in a multi-user environment, and a user utility function is constructed; The multi-objective reward function and the user utility function are optimized by alternating iteration to update parameters of the policy network.

5. The method of claim 4, wherein, The user utility function is constructed, including: A transmission rate term of a node i is constructed based on a bandwidth allocation adjustment amount, a power adjustment amount, a spectrum efficiency and a power compensation reference value of the node i; the transmission rate term is used to reflect a balance between a transmission rate and power consumption of a user; A resource competition term of the node i is constructed based on a difference between a ratio of the power adjustment amount to the bandwidth allocation adjustment amount of the node i and another node j, and a resource competition penalty coefficient; The user utility function of the node i is constructed based on the transmission rate term and the resource competition term of the node i.

6. The method of claim 1, wherein, The local resource allocation strategy of each node is updated through federated learning based on the local resource allocation strategy of each node and the length of the emergency task queue of each node, including: The weight corresponding to each node is determined based on the current length of the emergency task queue of each node; Global model parameters are calculated according to the model parameters of the first strategy model of each node and the weight corresponding to the node; The global model parameters are sent to each node to enable each node to update the corresponding first strategy model based on the global model parameters; The local resource allocation strategy of the node is generated based on the first strategy model after the parameters are updated.

7. The method of claim 1 or 6, wherein, After adjusting the local resource allocation strategy of each node through federated learning, it further includes: A Lyapunov function is constructed and a corresponding drift term is calculated based on the total length of the task queue and the real-time total power consumption of the wireless edge network; A first objective function is constructed based on the total power consumption budget of the wireless edge network, the current real-time total power consumption and the drift term; The allocation weight of each dimension resource in the local resource allocation strategy of each node is adjusted to minimize the first objective function, and the local resource allocation strategy of each node is updated. 8.The method of claim 1, wherein, The resource state information further includes resource demand information of each node; the method further includes: The network state information of the wireless edge network is obtained at a first time interval, including node topology structure, device parameters of each node; A network mirror model corresponding to the wireless edge network is constructed through simulation to map the behavior and performance of each node in the wireless edge network based on the local resource allocation strategy of each node, the current resource state information of the node and the network state information; After the local resource strategy of each node is updated, the behavior and performance of each node in the wireless edge network are simulated synchronously through an incremental learning and adaptive adjustment algorithm. 9.The method of claim 8, wherein, The method further includes: In a high-reliability low-latency scenario, the behavior and performance of each node are obtained at a second time interval through the network mirror model, and the current latency and reliability indicators are calculated; The current latency and reliability indicators are compared with the latency threshold and reliability threshold corresponding to the high-reliability low-latency scenario; In the case where the latency is higher than the latency threshold or the reliability indicator is lower than the reliability threshold, the local resource allocation strategy of each node is updated through federated learning. 10.The method of claim 8, wherein, Further include: According to a third time interval, obtain task execution data of each node from the network mirror model; The task execution data comprises: time delay index data, energy consumption index data and task progress index data; Based on the task execution data of each node, generate a resource state report of the wireless edge network; Based on the task execution data of each node, update the first strategy model of each node.

Images