Low-altitude edge network resource coordination method and electronic device

Abstract

The embodiment of the present specification provides a low-altitude edge network resource coordination method and electronic equipment, and relates to the technical field of communication. The method is applied to a target edge node in a plurality of edge nodes of a low-altitude edge network system, and the plurality of edge nodes include a UAV node. The resource coordination method comprises: acquiring a node state vector of the target edge node; the node state vector can represent situation awareness information of the edge node; inputting the node state vector into a resource decision model to output a decision action; the decision action is used to indicate scheduling at least one of the following resources: a link access mode, a communication bandwidth allocation, a task offloading target node, and a computing power allocation ratio; the task offloading target node includes a cloud and / or any edge node in the plurality of edge nodes; in the case that the decision action meets a feasibility constraint condition, the decision action is executed to perform resource coordination scheduling on the target edge node. In this way, the resource scheduling reliability of the low-altitude edge network system can be improved.

Description

Technical Field

[0001] The embodiments described in this specification relate to the field of communication technology, specifically to a low-altitude edge network resource coordination method and electronic device. Background Technology

[0002] With the rapid development of urban low-altitude applications (such as inspection, monitoring, logistics, and emergency response), the scale of terminals, data, and services accessed by low-altitude drone swarms is constantly growing. The explosive data volume places higher demands on the performance of drone nodes or their cloud-based systems. Edge computing, as a distributed computing model, can offload tasks to nearby edge nodes for execution, enabling timely and effective processing.

[0003] In related technologies, when performing edge computing in low-altitude drone swarms, the allocation of network resources is relatively simple and passive, and there is a lack of effective coordination mechanisms between various edge computing nodes. As a result, network resources are difficult to utilize effectively, leading to insufficient edge computing service processing capabilities.

[0004] Therefore, there is an urgent need for a resource coordination method for low-altitude edge networks to improve the reliability of resource scheduling in low-altitude edge network systems. Summary of the Invention

[0005] In view of this, various embodiments of this specification aim to provide a low-altitude edge network resource coordination method and electronic device to improve the resource scheduling reliability of low-altitude edge network systems.

[0006] This specification provides a resource coordination method for a low-altitude edge network, applied to a target edge node among multiple edge nodes in a low-altitude edge network system, wherein the multiple edge nodes include at least one UAV node. The resource coordination method includes: acquiring a node state vector of the target edge node; wherein the node state vector can characterize the situational awareness information of the edge node; the situational awareness information includes at least one of node environment information, task-related information, and node self-information; inputting the node state vector into a pre-trained resource decision model and outputting a decision action; wherein the resource decision model is a reinforcement learning model including a policy function and an objective function, the objective function is used to determine the policy function with the objective of minimizing the overall cost of the low-altitude edge network system, and the policy function is used to output the decision action; the decision action is used to indicate the scheduling of at least one of the following resources: link access mode, communication bandwidth allocation, task offloading target node, and computing power allocation ratio; the task offloading target node includes the cloud and / or any of the multiple edge nodes; and executing the decision action when the decision action meets feasibility constraints to perform resource coordination scheduling on the target edge node.

[0007] In some implementations, the target edge node is any one of the plurality of edge nodes, and the plurality of edge nodes further includes at least one ground base station. The low-altitude edge network also includes a cloud and a satellite node for relay communication. The low-altitude edge network system is a multi-mode communication network based on terrestrial 4G / 5G base station mobile communication, satellite communication, and UAV ad hoc networking. Multi-mode link communication is possible between any two edge nodes and between any edge node and the cloud. The cloud or any edge node can execute assigned tasks based on its available computing power. The node's own information includes at least one of the following: the target edge node's node location, node speed, node type, remaining power, current load, available computing power, and task load. The node environment information includes at least one of the following: the target edge node's link status information, the available computing power of neighboring edge nodes, and task load. The task-related information includes at least one of the following: task type, task priority, and task-required computing power.

[0008] In some implementations, the link access mode is at least one of single-link access, multi-link parallel access, and multi-link hybrid backup access; wherein, the single-link access uses any one of terrestrial base station mobile communication links, satellite communication links, and self-organizing network communication links; the multi-link parallel access uses at least two of terrestrial base station mobile communication links, satellite communication links, and self-organizing network communication links; and the multi-link hybrid backup access uses any one of terrestrial base station mobile communication links, satellite communication links, and self-organizing network communication links as the priority link, and at least one of the other two of terrestrial base station mobile communication links, satellite communication links, and self-organizing network communication links as the alternative link.

[0009] In some implementations, the feasibility constraints include at least one of the total link bandwidth constraint and the total node computing power constraint;

[0010] The total link bandwidth constraint is expressed by the following formula: Where u represents any target edge node, t represents any time, lowercase letter l represents any link, b represents the bandwidth requirement of target edge node u on link l, and B represents the maximum bandwidth of link l. The total computing power constraint of the nodes is expressed by the following formula: Where e represents any task unloading target node of any target edge node u, t represents any time, lowercase c represents the amount of computing power that task unloading target node e needs to allocate to target edge node u, and uppercase C represents the maximum computing power of task unloading target node e.

[0011] In some implementations, the resource decision model is a reinforcement learning model that includes a policy function and an objective function; the objective function is used to determine the policy function with the goal of minimizing the overall cost of the low-altitude edge network system, and the policy function is used to output the decision action; The objective function is expressed by the following formula: ; in, Represents the policy function; Let represent the objective function, which aims to minimize the overall cost of the policy function π; Represents the policy function The expected value of the overall cost; t represents any time; γ is a discount factor used to measure the impact of time on the expected value of the overall cost; Indicates end-to-end delay. These are the corresponding weighting factors; This represents a link loss metric. These are the corresponding weighting factors; This indicates energy consumption or computing power expenditure. These are the corresponding weighting factors; This represents the overload penalty term for the link or edge node. These are the corresponding weighting factors.

[0012] In some implementations, the resource coordination method further includes: obtaining an immediate reward and a new node state vector after the target edge node executes the decision action; wherein the immediate reward is an immediate feedback signal after executing the decision action, which can measure the quality of the decision action; uploading the decision action, the immediate reward, and the new node state vector to the experience pool in the cloud, so that the cloud performs periodic offline joint retraining and model aggregation based on the experience pool to generate new model parameters; receiving the new model parameters issued by the cloud, fine-tuning the new model parameters, and updating the local resource decision model based on the fine-tuned model parameters to achieve cloud-edge collaborative optimization.

[0013] In some implementations, the immediate reward is determined by an immediate reward function; the immediate reward function is expressed by the following formula: ; Where r represents the instantaneous reward and t represents any time point; Indicates the average delay. This corresponds to the time delay weighting coefficient; This represents the average packet loss rate. This is the corresponding packet loss rate weighting coefficient; Indicates the switching frequency. This is the corresponding handover frequency weighting coefficient; Indicates average energy consumption. This corresponds to the energy consumption weighting coefficient; Indicates overload penalty item, This refers to the weighting coefficient of the corresponding overload penalty term; This indicates the reward for a successful task. This refers to the weighting coefficient for the corresponding task success reward item.

[0014] In some implementations, before executing the decision action, the resource coordination method further includes: assessing the confidence level of the decision action, so as to execute the decision action if the decision action satisfies the feasibility constraint and the confidence level threshold; if the decision action does not satisfy the confidence threshold or the feasibility constraint, abandoning the execution of the decision action and executing a safety rollback strategy to conservatively coordinate resource scheduling.

[0015] In some implementations, the security fallback strategy includes at least one of the following: maintaining the current link state, reducing the data transmission rate, or caching or executing the task locally.

[0016] This specification provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the resource coordination method described in any of the above embodiments.

[0017] In the various implementations provided in this specification, firstly, the node state vector of the target edge node in the low-altitude edge network system, which can represent its situational awareness information, is obtained, and the node state vector is input into a pre-trained resource decision model to output a decision action. Then, if the decision action meets the feasibility constraints, the decision action is executed to instruct the scheduling of at least one of the following resources: link access mode, communication bandwidth allocation, task offloading target node, and computing power allocation ratio. This enables the coordinated scheduling of resources for the target edge node and improves the reliability of resource scheduling in the low-altitude edge network system. Attached Figure Description

[0018] Figure 1a This is a schematic diagram of a low-altitude edge network system provided in the embodiments of this specification; Figure 1b This is yet another schematic diagram of the low-altitude edge network system provided in the embodiments of this specification; Figure 2 This is a flowchart illustrating the low-altitude edge network resource collaboration method provided in the embodiments of this specification. Figure 3A schematic diagram of a low-altitude edge network resource coordination device provided for embodiments of this specification; Figure 4 A schematic diagram of an electronic device provided for an embodiment of this specification. Detailed Implementation

[0019] To enable those skilled in the art to better understand the solutions described in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0020] This specification provides an example application scenario for a low-altitude edge network resource coordination method. This application scenario can be a low-altitude edge network system, which is a multi-access network system. Please refer to [link to documentation]. Figure 1a A low-altitude edge network system may include multiple edge nodes and a cloud, with the multiple edge nodes including at least one drone node. For example, a drone node may include... Figure 1a The drones shown are 1, 2, ..., N, forming a heterogeneous drone swarm. These drones can be of different types. The cloud can refer to a data center or high-performance server cluster located remotely on the internet, possessing powerful global data storage, processing, and model training capabilities. The cloud and edge nodes constitute a cloud-edge collaborative architecture.

[0021] In this scenario example, the low-altitude edge network system may also include one or more satellite nodes for relay communication; exemplarily, the satellite node could be a Tiantong satellite. Additionally, the multiple edge nodes in the low-altitude edge network system may also include one or more ground base stations; exemplarily, the ground base station could be a 4G / 5G base station. In other words, an edge node in the low-altitude edge network system can refer to any drone node or any base station.

[0022] In this scenario example, the low-altitude edge network system is a multi-mode communication network based on terrestrial 4G / 5G base station mobile communication, satellite communication, and drone ad hoc networks. Utilizing base stations and satellite nodes, multi-mode link communication can be established between any two edge nodes and between any edge node and the cloud. Any edge node can communicate with the cloud or other edge nodes through one or more of the following: terrestrial base station mobile communication links, satellite communication links, and ad hoc network communication links. The cloud or any edge node can execute assigned tasks based on its available computing power. These assigned tasks can be computational tasks, which can be assigned by the node itself or by other nodes.

[0023] In this scenario example, please refer to Figure 1b Any edge node in a low-altitude edge network system, or any UAV node or base station in a low-altitude edge network system, can select an appropriate link access mode based on communication link redundancy, obtain communication link link quality feedback, and estimate communication link latency; it can also perform bandwidth slicing positioning based on bandwidth redundancy to allocate communication bandwidth; and it can also identify the type of communication data as data stream or control stream, and select the corresponding link access mode for control stream or data stream.

[0024] In this scenario example, the cloud can aggregate historical operational data from all edge nodes, perform large-scale offline model training and optimization, and distribute the trained model parameters to each edge node. For example, the cloud can run a complete deep reinforcement learning training framework, periodically update the resource decision model based on the experience data reported by the edge nodes, and distribute the updated model parameters to each drone node and each base station among multiple edge nodes.

[0025] In this scenario example, any edge node can be deployed with a pre-trained resource decision-making model. This model can output an optimal or near-optimal decision based on the input node state vector. This resource decision-making model can be built on a Deep Reinforcement Learning (DRL) framework. DRL frameworks can include Actor-Critic architectures, Deep Q-networks (DQN), twin delayed deep deterministic policy gradient (TD3), Proximal Policy Optimization (PPO), and Asynchronous Advantage Actor-Critic (A3C), etc. The resource decision-making model can be deployed to the edge node as a parameter file after the main training process is completed in the cloud. For example, the resource decision-making model can be a deep neural network that uses the node state vector as the input layer, processes it through several hidden layers, and finally provides a decision action regarding resource collaborative scheduling at the output layer.

[0026] For example, a low-altitude edge network system can include various types of resources or network resources, such as link access modes, communication bandwidth, edge nodes, and computing power. When scheduling different types of resources, different types of decision actions can be output by the policy function in the resource decision model to execute the scheduling. Specifically, the policy function in the resource decision model... It constructs a mapping relationship from the high-dimensional state space to the action space, and the policy function. Also known as a policy network, it includes an input layer, hidden layers, and an output layer. The hidden layer is also the feature extraction layer. The process of the policy function outputting decision actions can include state vector mapping and action generation steps. 1) State vector mapping: In the resource decision model, the policy function receives the input node state vector (such as link signal-to-noise ratio, task queue length, etc.) through the input layer, and maps the node state vector to hidden layer features through the internal feature extraction layer; 2) Action generation logic: The output layer of the policy function can output a probability distribution vector (for example, processed by the Softmax function). Each element in the probability distribution vector corresponds to the probability of selecting an optional decision action. The output layer of the policy function can sample or directly select the decision action with the highest probability (Argmax) as the final output decision action based on the probability distribution vector; or, the output layer of the policy function can also generate a scalar value (for example, processed by the Tanh or Sigmoid function). This scalar value is directly mapped to the physical quantity of the resource to be scheduled. Through the above process, the policy function can transform the abstract "situational awareness information" into specific, executable "physical control parameters".

[0027] This specification provides a method for resource coordination in low-altitude edge networks. Please refer to [link / reference]. Figure 2 , Figure 2 This is a flowchart illustrating a low-altitude edge network resource coordination method provided in this specification. This embodiment provides the method operation steps shown in the flowchart, but based on conventional or non-creative work, it may include more or fewer operation steps. The order of steps listed in the embodiment is merely one possible execution order among many, and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially as shown in the embodiment or in parallel (e.g., in a parallel processor or multi-threaded processing environment). This low-altitude edge network resource coordination method can be applied to target edge nodes in a low-altitude edge network system, specifically as follows... Figure 2 As shown, the low-altitude edge network resource coordination method may include the following steps.

[0028] Step S210: Obtain the node state vector of the target edge node; wherein, the node state vector can represent the situational awareness information of the edge node; the situational awareness information includes at least one of the following: node self-information, node environment information, and task-related information.

[0029] In some cases, low-altitude edge network systems exhibit significant communication and computing characteristics. For example, the Doppler and transient fading caused by dynamic changes in the speed, heading, and relative position of UAV nodes to base stations result in rapid time-varying communication links in low-altitude edge networks. Instantaneous link interruptions caused by buildings, bridges, overpasses, and temporary obstacles lead to sudden spatial obstruction and occlusion in low-altitude edge network systems. The availability of multiple link access methods, including ground base station mobile communication links, satellite links, and UAV self-organizing networks, also provides a high degree of selectivity in link access modes. Furthermore, the communication data, ranging from low-bandwidth telemetry and control flows to high-bandwidth data flows such as video backhaul and point cloud synchronization, exhibits high heterogeneity in task types and communication data. These characteristics make it difficult for traditional mobile communication and joint computing strategies designed for ground users to directly adapt to the needs of heterogeneous low-altitude UAV clusters. At the protocol and system level, edge nodes in low-altitude edge network systems can support various multi-connection and redundancy mechanisms, such as concurrent multi-links, fragmented parallel transmission, and master-slave replication. They can also allocate different network resource levels to control and data flows through bandwidth slicing, task prioritization, and Quality of Service (QoS) policies. Furthermore, while satellite communication links offer wide coverage, they are typically accompanied by significant latency and windowed availability issues. Ad hoc network communication has advantages in proximity communication and coverage gaps, but it falls short in large-scale QoS management. Therefore, based on the situational awareness information of edge nodes, a comprehensive evaluation and trade-off can be made regarding latency, bandwidth, reliability, and cost, and dynamic collaborative scheduling of network resources can be achieved through online decision-making.

[0030] In this context, a target edge node refers to a specific edge node among multiple edge nodes that requires resource scheduling decisions and operations. For example, when a low-altitude edge network system needs to allocate communication bandwidth to a drone node performing a high-definition video capture task and determine whether to offload the video analysis task, this drone node needs to be designated as the target edge node to trigger resource collaborative scheduling. The target edge node can be any edge node among multiple edge nodes in the low-altitude edge network system; for instance, it could be any drone in a heterogeneous drone cluster or any base station.

[0031] Specifically, situational awareness information of target edge nodes can be acquired, and node state vectors can be determined based on the acquired situational awareness information to serve as input to the resource decision-making model. The node state vectors of target edge nodes can characterize the situational awareness of the target edge node regarding the environment, the task, and the node itself.

[0032] For example, the target edge node can be any UAV node among multiple edge nodes. Its situational awareness information can include the current link quality of the UAV node, the available computing power of the UAV node and neighboring edge nodes, the task priority of computing tasks, and short-term environmental changes. Here, short-term environmental changes can refer to environmental changes over a period from the current moment to a certain historical moment. Neighboring edge nodes refer to edge nodes located near the target edge node, such as nearby ground base stations or UAV nodes. For example, neighboring edge nodes can refer to ground base stations, or UAV nodes in a heterogeneous UAV cluster other than the target edge node.

[0033] Step S220: Input the node state vector into the pre-trained resource decision model and output the decision action; wherein, the resource decision model can be a reinforcement learning model including a policy function and an objective function, the objective function is used to determine the policy function with the goal of minimizing the overall cost of the low-altitude edge network system, and the policy function is used to output the decision action; the decision action is used to indicate the scheduling of at least one of the following resources: link access mode, communication bandwidth allocation, task offloading target node, and computing power allocation ratio; the task offloading target node includes any edge node in the cloud and / or multiple edge nodes.

[0034] Specifically, the target edge nodes can be deployed with pre-trained resource decision-making models. By using the node state vector as input to the resource decision-making model, decision actions can be obtained to instruct on scheduling relevant resources. These decision actions can refer to resource allocation schemes, resource allocation strategies, or resource scheduling instructions.

[0035] The policy function maps node state vectors to decision actions. Specifically, the physical execution logic of the decision actions is as follows: 1) Communication bandwidth allocation: The decision action output by the resource decision model includes the bandwidth slice ratio coefficient of the corresponding link. For example, when the node state vector indicates that a task is a high-priority task and the link channel quality is good, the policy function will output a larger bandwidth slice ratio coefficient, indicating that more spectrum resources will be allocated to this task during resource scheduling to reduce transmission latency. 2) Task offloading target node: The decision action output by the resource decision model is the index of the task offloading target node, which can trigger a task migration mechanism to offload the computation task to the task offloading target node for execution. 3) Link access mode: The decision action output by the resource decision model also includes a link switching indication for multi-mode communication, used to select the link access mode to ensure link continuity.

[0036] For example, the objective function, also known as the value function, is used. In other words, the policy function can refer to the policy network, and the objective function can refer to the value network.

[0037] For example, the link access mode in the decision-making action is used to instruct the target edge node which link access mode to select to connect to the network, or in other words, which communication link to select.

[0038] For example, the communication bandwidth allocation in the decision-making action is used to indicate the specific bandwidth slice size or proportion allocated on the selected access link or communication link for the computing tasks or different service flows (such as control commands, telemetry data, video streams, etc.) of the target edge node.

[0039] For example, the task offloading target node in the decision-making action is used to instruct the target edge node to send all or part of its computational tasks (especially computationally intensive tasks such as target recognition and path planning) to a neighboring edge node or the cloud. For instance, the target edge node can offload the image recognition task to a ground base station, to the cloud, to a nearby drone node, or to its local machine.

[0040] For example, the computing power allocation ratio in the decision-making action is used to indicate how much computing resources to allocate to process the task when it is offloaded to an edge node or the cloud. For instance, it may request that 30% of the GPU computing power of the target base station node be allocated locally to process the task. The request can be made to any base station or cloud to allocate a specified amount of computing power to process the task.

[0041] Step S230: If the decision action meets the feasibility constraints, execute the decision action to perform resource collaborative scheduling on the target edge node.

[0042] In the above implementation, by acquiring the node state vector of the target edge node that can characterize its situational awareness information, and inputting the node state vector into a pre-trained resource decision model, a decision action is output. Then, if the decision action meets the feasibility constraints, the decision action is executed to instruct the scheduling of at least one of the following resources: link access mode, communication bandwidth allocation, task offloading target node, and computing power allocation ratio. This enables the coordinated scheduling of resources for the target edge node, thereby achieving smooth switching and adaptive resource allocation of UAV multi-access communication links without increasing system complexity, reducing communication latency and switching interruption risks, and improving the resource utilization and task execution continuity and reliability of the low-altitude edge network system.

[0043] In some implementations, decision-making actions may include action hierarchy information. Through action hierarchy information, the link access mode can be selected first, and then communication and computing resources can be allocated to balance the flexibility of decision-making and the real-time inference of resource decision-making models.

[0044] In some implementations, the node's own information may include at least one of the following: the target edge node's location, node speed, node type, remaining power, current load, available computing power, and task load.

[0045] For example, taking any drone node as the target edge node, the node position of the target edge node can refer to the node coordinates of the drone node.

[0046] In some implementations, node environment information may include at least one of the following: link status information of the target edge node, available computing power and task load of neighboring edge nodes, and queuing latency of neighboring edge nodes.

[0047] For example, link status information may include the real-time signal quality and / or historical stability of the communication link of the target edge node. The real-time quality of the communication link can be characterized at least by the signal-to-interference plus noise ratio (SINR) and / or packet loss rate, while historical stability can be characterized by the link interruption frequency or average signal quality within a preset time window. The link interruption frequency also refers to the link switching frequency.

[0048] For example, the link state information may also include the probability of communication obstruction based on the node location prediction of digital maps and drone nodes.

[0049] For example, the link status information may also include predictions of the visible time window for the satellite communication link.

[0050] In some implementations, task-related information includes at least one of the following: task type, task priority, and computational power required by the task. For example, a task, or computational task, may include computationally intensive tasks such as aerial video processing, intelligent sensing, and path optimization.

[0051] In some implementations, the link access mode can be at least one of single-link access, multi-link parallel access, and multi-link mixed backup access.

[0052] In single-link access mode, access can be achieved using any one of the following: terrestrial 4G / 5G base station mobile communication links, satellite communication links, and self-organizing network communication links.

[0053] Multi-link parallel access mode, also known as parallel access mode, can use at least two of the following for access: terrestrial 4G / 5G base station mobile communication links, satellite communication links, and self-organizing network communication links.

[0054] Multi-link hybrid access mode, also known as primary and backup hybrid access, can use any one of the terrestrial 4G / 5G base station mobile communication links, satellite communication links, and self-organizing network communication links as the primary link or priority link, and use at least one of the other two of the terrestrial base station mobile communication links, satellite communication links, and self-organizing network communication links as the backup link.

[0055] In the above embodiments, by generating a node state vector from situational awareness information including at least one of node self-information, node environment information, and task-related information, and using it as input to the resource decision model, it is possible to flexibly select single-link, parallel, or primary / backup hybrid access modes according to the node's own situation, the priority of different tasks, and the link status. This enables the low-altitude edge network system to achieve comprehensive optimization and dynamic allocation of communication bandwidth and computing resources.

[0056] In some implementations, each edge node may be deployed with a resource decision model. This means that multiple target edge nodes may simultaneously perform resource coordination scheduling. Therefore, after a target edge node obtains a decision based on its deployed resource decision model, a feasibility constraint determination is needed to determine whether the action can be executed. Feasibility constraints may include at least one of the following: total link bandwidth constraint and total node computing power constraint.

[0057] For example, the total link bandwidth constraint can refer to the total bandwidth allocated to all target edge nodes not exceeding the maximum bandwidth limit of each communication link. The total node computing power constraint can refer to the total amount of computing tasks unloaded to each target node not exceeding its total computing power limit.

[0058] For example, the total link bandwidth constraint can be expressed by Equation 1.

[0059] Formula 1; Where u represents any target edge node, t represents any time, lowercase l represents any link, b represents the bandwidth requirement of target edge node u on link l, and B represents the maximum bandwidth of link l.

[0060] The total computing power constraint of a node is represented by Formula 2.

[0061] Formula 2; Where e represents any task unloading target node of any target edge node u, t represents any time, lowercase c represents the amount of computing power that task unloading target node e needs to allocate to target edge node u, and uppercase C represents the maximum computing power of task unloading target node e.

[0062] In some implementations, the objective function can be represented by the following formula 3.

[0063] Formula 3; in, Represents the policy function; The objective function is denoted by π, which aims to minimize the overall cost of the policy function π; the double-capital letter E represents the expectation. Represents the policy function The expected value of the overall cost; t represents any time point; γ is a discount factor used to measure the impact of time on the expected value of the overall cost; or, in other words, to measure the impact of immediate rewards at future moments.

[0064] Indicates end-to-end delay. These are the corresponding weighting factors; This represents a link loss metric. These are the corresponding weighting factors; This indicates energy consumption or computing power expenditure. These are the corresponding weighting factors; This represents the overload penalty term for the link or edge node. These are the corresponding weighting factors.

[0065] For example, link loss metrics It can characterize the physical loss of the link.

[0066] For example, the value of γ can be greater than 1, that is, as time increases, such as the operating time of a low-altitude edge network system or the operating time of a heterogeneous drone swarm, the impact of time cost will become higher and higher.

[0067] In some implementations, the low-altitude edge network resource coordination method may further include the following steps S310-S330.

[0068] Step S310: Obtain the immediate reward and new node state vector obtained after the target edge node performs the decision action.

[0069] Instant rewards are immediate feedback signals after a decision is made, which can measure the quality of the decision.

[0070] Step S320: Upload the decision action, immediate reward, and new node state vector to the cloud experience pool so that the cloud can perform periodic offline joint retraining and model aggregation based on the experience pool to generate new model parameters.

[0071] For example, since the objective function is to determine the policy function with the goal of minimizing the overall cost of the low-altitude edge network system, the immediate reward function can be designed based on the factors affecting the overall cost, or in other words, the immediate reward function can be designed based on the objective function or the factors affecting the objective function.

[0072] For example, the end-to-end delay in the objective function Link loss metrics Energy consumption or computing power overhead Overload penalty for links or edge nodes Factors such as end-to-end delay in the objective function all affect the overall cost of the low-altitude edge network system. Therefore, the immediate reward function can be based on the end-to-end delay in the objective function. Link loss metrics Energy consumption or computing power overhead Overload penalty for links or edge nodes One or more of these can be set. It is understandable that the immediate reward function can also be based on the end-to-end delay in the objective function. Link loss metrics Energy consumption or computing power overhead Overload penalty for links or edge nodes The settings can be configured using one or more variations or related influencing factors.

[0073] For example, since immediate rewards can measure the quality of decision-making actions, the objective function can be continuously optimized by accumulating all immediate rewards to achieve the goal of minimizing the overall cost of the low-altitude edge network system. In other words, the objective function in the resource decision model is not to maximize the immediate reward of a single step, but to maximize the long-term cumulative reward. Therefore, during the periodic offline joint retraining based on the experience pool, the optimal objective function can be obtained by accumulating the maximum value of immediate rewards, so that the objective function can minimize the overall cost of the low-altitude edge network system, determine the new policy function or policy network in the resource decision model, and thus obtain the new model parameters.

[0074] As an example, a loss function incorporating immediate rewards can be constructed. The gradient of this loss function is then calculated, and the neural network weights of the policy and objective functions are updated using gradient descent through backpropagation. Alternatively, the immediate reward function acts as a feedback signal, guiding the resource decision-making model parameters to converge in a direction that yields higher cumulative rewards. This allows the resource decision-making model to output better decisions when encountering similar states in the future.

[0075] Step S330: Receive new model parameters from the cloud, fine-tune the new model parameters, and update the local resource decision model based on the fine-tuned model parameters to achieve cloud-edge collaborative optimization.

[0076] For example, new model parameters can be fine-tuned based on real-time situational awareness information, thus maintaining the stability and generalization ability of the low-altitude edge network system under different regions and different task loads.

[0077] In the above embodiments, single-link, parallel, and hybrid backup access mechanisms are implemented for multi-link collaboration in low-altitude multi-access edge networks. Taking the terrestrial 5G base station mobile communication link and satellite link as an example, the control flow prioritizes the high-reliability, low-latency bandwidth slice of the 5G base station mobile communication link, while the data flow is transmitted in parallel or in layers between the two. When a risk of 5G signal attenuation or obstruction is detected, it can automatically switch to hybrid backup access mode, with the satellite link carrying critical frame redundancy or large-capacity data segments to ensure uninterrupted communication. The target edge node can dynamically adjust the communication link allocation and data mapping to ensure transmission continuity and bandwidth utilization.

[0078] During the learning and updating process of the resource decision-making model, the target edge nodes continuously collect node state vectors, action decisions, immediate rewards, and result samples, and store them in a local experience buffer. They periodically report these data to the cloud, or periodically report priority samples or statistical summaries to the cloud. After joint retraining and model aggregation in the cloud, the updated model parameters are distributed to each edge node, realizing a cloud-edge collaborative continuous learning mechanism.

[0079] In some implementations, the immediate reward can be determined by an immediate reward function. The immediate reward function is expressed by the following formula 4.

[0080] Formula 4; Where r represents the instantaneous reward and t represents any time point; Indicates the average delay. This corresponds to the time delay weighting coefficient; This represents the average packet loss rate. This is the corresponding packet loss rate weighting coefficient; Indicates the switching frequency. This is the corresponding switching frequency weighting coefficient; This represents average energy consumption. This corresponds to the energy consumption weighting coefficient; Indicates overload penalty item, This refers to the weighting coefficient of the corresponding overload penalty term; This indicates the reward for a successful task. This refers to the weighting coefficient for the corresponding task success reward item.

[0081] As another example, in Formula 4, the overload penalty term and the task success reward term can be combined into the task success rate, and a corresponding task success rate weight coefficient can be assigned to this task success rate.

[0082] For example, average packet loss rate It can characterize or measure the condition of a communication network.

[0083] In some implementations, the low-altitude edge network resource coordination method may further include: adjusting at least one of the following weight coefficients in the immediate reward function based on the task priority in the task-related information in the situational awareness information: latency weight coefficient, packet loss rate weight coefficient, handover frequency weight coefficient, energy consumption weight coefficient, overload penalty weight coefficient, and task success reward weight coefficient, so as to effectively guide the resource decision model to learn low-latency and high-reliability scheduling decisions during retraining by comprehensively considering latency, packet loss rate, handover frequency, energy consumption, and task success rate.

[0084] In some implementations, the low-altitude edge network resource coordination method may further include the following steps S410-S420 before executing the decision action.

[0085] Step S410: Evaluate the confidence level of the decision action so that the decision action is executed if it meets the feasibility constraints and the confidence threshold.

[0086] Among them, the confidence level of a decision action can characterize the reliability of a decision action.

[0087] Step S420: If the decision action does not meet the confidence threshold or the feasibility constraint, abandon the decision action and execute a safety rollback strategy to conservatively coordinate resource scheduling.

[0088] In some implementations, the safety fallback strategy includes at least one of the following: maintaining the current link state, reducing the data transmission rate, or caching or executing the task locally.

[0089] Local caching or execution of tasks can be applied to high-priority, critical computational tasks.

[0090] Maintaining the current link status can include maintaining resource coordination scheduling strategies based on the previous or historical decision actions. For example, in addition to maintaining the link access mode, it can also include maintaining the task offloading target node, maintaining communication bandwidth allocation, and maintaining the computing power allocation ratio.

[0091] For example, a security rollback strategy may also include resource downsizing or task priority rollback strategies.

[0092] In the above implementation, by assessing the confidence level of the decision-making action and combining it with feasibility constraints, the possibility of communication interruption or resource conflict caused by the resource decision-making model in an unknown state can be reduced.

[0093] In the above embodiments, by jointly optimizing multiple access communication links, bandwidth and computing power, and combining engineered confidence detection and security fallback strategies, the reliability and deployability of low-altitude edge network systems in complex low-altitude environments can be improved, effectively supporting continuous communication and real-time task execution of heterogeneous UAV clusters under multi-source heterogeneous network conditions.

[0094] This specification provides a low-altitude edge network resource coordination device, applied to a target edge node among multiple edge nodes in a low-altitude edge network system. Please refer to... Figure 3 The low-altitude edge network resource coordination device may include a status acquisition module 510, an action output module 520, and a coordination scheduling module 530.

[0095] The state acquisition module 510 is used to acquire the node state vector of the target edge node; wherein, the node state vector can represent the situational awareness information of the edge node; The action output module 520 is used to input the node state vector into the pre-trained resource decision model and output a decision action; wherein, the decision action is used to indicate the scheduling of at least one of the following resources: link access mode, communication bandwidth allocation, task offloading target node, and computing power allocation ratio; the task offloading target node includes any edge node in the cloud and / or multiple edge nodes; The collaborative scheduling module 530 is used to execute decision actions when the decision actions meet the feasibility constraints, so as to perform resource collaborative scheduling on the target edge nodes.

[0096] The specific functions and effects of the low-altitude edge network resource coordination device can be explained by referring to other embodiments in this specification, and will not be repeated here. Each module in the low-altitude edge network resource coordination device can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in the processor of the electronic device in hardware form or independent of it, or it can be stored in the memory of the electronic device in software form, so that the processor can call and execute the operations corresponding to each module.

[0097] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, implements the low-altitude edge network resource coordination method in any of the above embodiments.

[0098] This specification also provides a computer program product containing instructions that, when executed by a computer, cause the computer to implement the low-altitude edge network resource coordination method in any of the above embodiments.

[0099] This specification also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the low-altitude edge network resource coordination method in any of the above embodiments.

[0100] In some implementations, please refer to Figure 4 The electronic device can be a terminal, and its internal structure diagram can be as follows: Figure 4 As shown, the electronic device includes a processor, memory, and a communication interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a low-altitude edge network resource collaboration method.

[0101] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments described herein, and are not intended to limit the scope of the invention.

[0102] It is understood that in the various embodiments described in this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments described in this specification.

[0103] It is understood that the various implementation methods described in this specification can be implemented individually or in combination, and the implementation methods in this specification are not limited in this respect.

[0104] Unless otherwise stated, all technical and scientific terms used in the embodiments of this specification have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0105] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this specification can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0106] It is understood that the memory in the embodiments of this specification may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0107] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0108] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0109] The above description is merely a specific embodiment of this specification, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this specification should be included within the scope of protection of this specification. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A method for resource coordination in low-altitude edge networks, characterized in that, A target edge node is applied to multiple edge nodes in a low-altitude edge network system, wherein the multiple edge nodes include at least one UAV node; the resource coordination method includes: Obtain the node state vector of the target edge node; wherein the node state vector can characterize the situational awareness information of the edge node; the situational awareness information includes at least one of node environment information, task-related information, and node self-information; The node state vector is input into a pre-trained resource decision model, which outputs a decision action. The resource decision model is a reinforcement learning model comprising a policy function and an objective function. The objective function is used to determine the policy function with the goal of minimizing the overall cost of the low-altitude edge network system. The policy function is used to output the decision action. The decision action indicates the scheduling of at least one of the following resources: link access mode, communication bandwidth allocation, task offloading target node, and computing power allocation ratio. The task offloading target node includes the cloud and / or any of the multiple edge nodes. If the decision action satisfies the feasibility constraints, the decision action is executed to perform resource collaborative scheduling on the target edge node.

2. The low-altitude edge network resource coordination method according to claim 1, characterized in that, The target edge node is any one of the multiple edge nodes, and the multiple edge nodes also include at least one ground base station. The low-altitude edge network system also includes a cloud and a satellite node for relay communication. The low-altitude edge network system is a multi-mode communication network based on ground 4G / 5G base station mobile communication, satellite communication and drone self-organizing network. Multimode link communication can be performed between any two edge nodes and between any edge node and the cloud. The cloud or any of the edge nodes can execute the assigned tasks based on their available computing power; The node's own information includes at least one of the following: the target edge node's node location, node speed, node type, remaining power, current load, available computing power, and task load. The node environment information includes at least one of the following: the link status information of the target edge node, the available computing power of neighboring edge nodes, and the task load; The task-related information includes at least one of the following: task type, task priority, and computing power required by the task.

3. The low-altitude edge network resource coordination method according to claim 2, characterized in that, The link access mode is at least one of single-link access, multi-link parallel access, and multi-link mixed backup access; Specifically, the single-link access uses any one of the terrestrial base station mobile communication links, satellite communication links, and self-organizing network communication links; the multi-link parallel access uses at least two of the terrestrial base station mobile communication links, satellite communication links, and self-organizing network communication links; and the multi-link mixed backup access uses any one of the terrestrial base station mobile communication links, satellite communication links, and self-organizing network communication links as the priority link, and at least one of the other two of the terrestrial base station mobile communication links, satellite communication links, and self-organizing network communication links as the alternative link.

4. The low-altitude edge network resource coordination method according to claim 1, characterized in that, The feasibility constraints include at least one of the total link bandwidth constraints and the total node computing power constraints. The total link bandwidth constraint is expressed by the following formula: ； Where u represents any target edge node, t represents any time, lowercase letter l represents any link, b represents the bandwidth requirement of target edge node u on link l, and B represents the maximum bandwidth of link l. The total computing power constraint of the nodes is expressed by the following formula: ； Where e represents any task unloading target node of any target edge node u, t represents any time, lowercase c represents the amount of computing power that task unloading target node e needs to allocate to target edge node u, and uppercase C represents the maximum computing power of task unloading target node e.

5. The low-altitude edge network resource coordination method according to claim 1, characterized in that, The objective function is expressed by the following formula: ； in, Represents the strategy function; Let represent the objective function, which aims to minimize the overall cost of the policy function π; Represents the policy function The expected value of the overall cost; t represents any time point; γ is a discount factor used to measure the impact of time on the expected value of the overall cost; Indicates end-to-end delay. These are the corresponding weighting factors; This represents a link loss metric. These are the corresponding weighting factors; This indicates energy consumption or computing power expenditure. These are the corresponding weighting factors; This represents the overload penalty term for the link or edge node. These are the corresponding weighting factors.

6. The low-altitude edge network resource coordination method according to claim 1, characterized in that, The resource coordination method also includes: Obtain the immediate reward and new node state vector obtained after the target edge node executes the decision action; wherein, the immediate reward is an immediate feedback signal after executing the decision action, which can measure the quality of the decision action execution; The decision action, the immediate reward, and the new node state vector are uploaded to the experience pool in the cloud, so that the cloud can perform periodic offline joint retraining and model aggregation based on the experience pool to generate new model parameters. The system receives the new model parameters sent from the cloud, fine-tunes the new model parameters, and updates the local resource decision model based on the fine-tuned model parameters to achieve cloud-edge collaborative optimization.

7. The low-altitude edge network resource coordination method according to claim 6, characterized in that, The instant reward is determined by an instant reward function; the instant reward function is expressed by the following formula: ； Where r represents the instantaneous reward and t represents any time point; Indicates the average delay. This corresponds to the time delay weighting coefficient; This represents the average packet loss rate. This is the corresponding packet loss rate weighting coefficient; Indicates the switching frequency. This is the corresponding handover frequency weighting coefficient; Indicates average energy consumption. This corresponds to the energy consumption weighting coefficient; Indicates overload penalty item, This refers to the weighting coefficient of the corresponding overload penalty term; This indicates the reward for a successful task. This refers to the weighting coefficient for the corresponding task success reward item.

8. The low-altitude edge network resource coordination method according to claim 1, characterized in that, Before executing the decision-making action, the resource coordination method further includes: Assess the confidence level of the decision action so that the decision action is executed if it satisfies the feasibility constraints and the confidence threshold. If the decision action does not meet the confidence threshold or the feasibility constraint, the decision action is abandoned and a safety rollback strategy is implemented to conservatively coordinate resource scheduling.

9. The low-altitude edge network resource coordination method according to claim 8, characterized in that, The security rollback strategy includes at least one of the following: maintaining the current link state, reducing the data transmission rate, or caching or executing the task locally.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the low-altitude edge network resource coordination method according to any one of claims 1 to 9.

Images