A Resource Allocation Method for Minimizing Error Information Age in Multi-UAV Assisted Edge Networks

By introducing an error information age optimization index and a multi-agent deep Q-network algorithm, a frame-level perception control and time slot-level resource adjustment mechanism is constructed, which solves the problems of state accuracy deviation and resource mismatch in multi-UAV edge networks and achieves more efficient resource allocation and collaborative scheduling.

CN122909618BActive Publication Date: 2026-11-17CHINA JILIANG UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611392186.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-09-09
Publication Date
2026-11-17
Estimated Expiration
2046-09-09

AI Technical Summary

Technical Problem

Existing multi-UAV edge network resource optimization methods are unable to accurately characterize state accuracy deviations, lack multi-timescale resource coordination mechanisms, and have insufficient multi-agent coordination efficiency, resulting in a mismatch between the frequency of perception data and real-time communication resources, which affects system stability and decision reliability.

Method used

Using error information age as the optimization metric, and combining the Lyapunov optimization method and the multi-agent deep Q-network algorithm, a collaborative scheduling mechanism of frame-level perceptual control and time slot-level resource adjustment is constructed. Resource allocation is performed through the multi-agent dual-Critic proximal strategy optimization algorithm to reduce the duration and degree of error state of edge servers.

Benefits of technology

It improves the effectiveness of edge server status updates and resource scheduling efficiency, enables more accurate evaluation of information timeliness and status accuracy, alleviates the mismatch between the frequency of sensing data and real-time communication resources, and enhances the efficiency of multi-UAV collaborative resource scheduling.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122909618B_ABST
    Figure CN122909618B_ABST
Patent Text Reader

Abstract

This scheme provides a resource allocation method that minimizes the age of error information in multi-UAV assisted edge networks. First, the network is constructed, and the age of error information, reflecting both timeliness and accuracy, is calculated. A resource allocation optimization model is established with the goal of minimizing the long-term average age of error information, and this is transformed into a short-term problem using Lyapunov optimization. The scheme employs a dual-timescale collaborative framework: at the frame level, a multi-agent deep Q-network is used to determine the perception cycle; at the time slot level, a multi-agent dual-Critic proximal policy optimization algorithm is used to determine energy allocation, transmission scheduling, and flight angle. By updating model parameters through reward feedback, this scheme effectively addresses the problem of insufficient efficiency in multi-UAV collaborative scheduling, significantly reduces the age of error information, and improves network performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) assisted edge networks, and in particular to a resource allocation method for minimizing the age of error information in multi-UAV assisted edge networks. Background Technology

[0002] Drone-assisted edge networks are widely used in critical scenarios such as low-altitude environmental monitoring, emergency inspections, and intelligent traffic patrols. In these scenarios, multiple drones collaborate to periodically perceive the ground environment and upload the collected status update data to an edge server to support subsequent real-time alarms, task scheduling, and control decisions. To ensure the reliability of these decisions, the information stored on the edge server not only needs to be highly timely but also needs to be highly consistent with the current real-world environmental state. If the server status deviates from the actual environment, it may lead to erroneous alarms, delayed task scheduling, or even resource allocation failures.

[0003] Currently, resource optimization methods for multi-UAV edge networks typically use information age, transmission latency, task completion time, or system energy consumption as optimization targets, improving network performance by coordinating the UAVs' perception, communication, energy, and mobility control resources. However, the aforementioned existing technologies still have the following significant drawbacks: 1. Existing metrics struggle to characterize state accuracy deviations: Current evaluation metrics often focus on the timeliness of data arrival, such as the Age of Information (AoI) metric, which fails to accurately describe the deviation between the storage state of edge servers and the current true state of the monitored area. In scenarios where the state of monitored objects evolves randomly, even if the AoI of a data packet is small, if its content does not match the current true state, the system may still make decisions based on erroneous information.

[0004] 2. Lack of multi-timescale resource coordination mechanisms: In complex low-altitude scenarios, there is a close coupling relationship between data sensing frequency, wireless transmission capability, UAV movement status, and energy supply. Existing methods often place different types of resource variables within the same time scale for unified adjustment, lacking consideration of the coordination mechanism between frame-level sensing control and time-slot-level communication, energy, and trajectory adjustment. This single-scale scheduling approach can easily lead to a mismatch between the frequency of sensing data generation and the real-time communication resource status and energy constraints, affecting the overall stability of the system.

[0005] 3. Single evaluation network limits the efficiency of multi-agent cooperation: Existing multi-agent resource scheduling methods typically employ a single evaluation network to assess the value of policy actions. While centralized evaluation networks can utilize global state information to promote cooperation, they struggle to fully characterize the individual action differences among different UAVs in local environments. Distributed evaluation networks, although capable of evaluating individual actions based on local observations, lack a global perspective, resulting in insufficient consideration of the cooperative relationships among multiple UAVs and thus limiting the efficiency of cooperative scheduling.

[0006] Therefore, existing technologies still lack a collaborative resource allocation method for multiple UAVs in low-altitude scenarios that can simultaneously characterize information timeliness and status accuracy, and solve the problems of mismatch between sensing data frequency and real-time communication resources, as well as insufficient efficiency of multi-UAV collaborative scheduling. Summary of the Invention

[0007] The purpose of this invention is to provide a resource allocation method for minimizing the age of error information in multi-UAV assisted edge networks. The method uses the age of error information as an optimization index, and jointly considers the UAV perception cycle, transmission scheduling, flight angle, and edge server energy allocation. Combined with data queue and energy queue constraints, a scheduling mechanism that coordinates frame-level perception control and time slot-level resource adjustment is constructed. This reduces the duration and degree of deviation of edge server error states, and improves the effectiveness of state updates and resource scheduling efficiency.

[0008] To achieve the above objectives, this technical solution provides a resource allocation method for minimizing the age of error information in multi-UAV assisted edge networks, comprising the following steps: S1. Deploy multiple drones and edge servers within the target service area to build a multi-drone-assisted edge network; S2. Calculate the age of the error information of the corresponding drone based on the current real state of the monitoring area and the storage state of the edge server; S3. Construct a resource allocation optimization model, in which the optimization objective is to minimize the long-term average age of all UAVs, and the optimization variables are edge server energy allocation, UAV perception cycle, UAV transmission scheduling, and UAV flight angle. S4. Based on the Lyapunov optimization method, the resource allocation optimization model is transformed, and the optimization objective is transformed into a short-term resource allocation problem within each time slot; S5. Divide multiple consecutive time slots into a frame-level scheduling cycle. At the beginning of each frame-level scheduling cycle, update the UAV perception cycle and keep the UAV perception cycle unchanged within the corresponding frame-level scheduling cycle. In each time slot, update the edge server energy allocation, UAV transmission scheduling, and UAV flight angle according to the real-time status. A frame-level perception control model and a time-slot-level resource adjustment model are constructed. The frame-level perception control model outputs the UAV perception cycle through a multi-agent deep Q-network algorithm within the frame-level scheduling cycle. The time-slot-level resource adjustment model outputs the edge server energy allocation, UAV transmission scheduling, and UAV flight angle through a multi-agent dual-Critic proximal strategy optimization algorithm within each time slot. S6. Perform resource allocation based on the output results of the time slot-level resource adjustment model and the frame-level perception control model, and update the model parameters of the frame-level perception decision network and the time slot-level resource adjustment network based on the reward feedback generated during the execution process, so as to obtain a multi-UAV collaborative resource allocation strategy.

[0009] Compared with existing technologies, this technical solution has the following characteristics and beneficial effects: 1. It enables more accurate evaluation of the effectiveness of edge state information. By introducing error information age as a resource optimization metric, it incorporates the duration of inconsistency between the state stored by the edge server and the current actual state, as well as the state deviation, into the evaluation. Compared to schemes that only use AoI, transmission latency, or task completion time, this metric can simultaneously reflect information timeliness and state accuracy, making it more suitable for scenarios with high state accuracy requirements, such as low-altitude environment monitoring, emergency inspections, and intelligent traffic patrols.

[0010] 2. The problem can be reduced under queue stability constraints. This invention combines the states of the data queue and the energy queue, and uses Lyapunov optimization to transform the problem of minimizing the long-term average error information age into a short-term resource scheduling problem that can be handled within each time slot. This approach ensures that the resource allocation process not only focuses on the effect of the current state update, but also takes into account the stability of the data queue and the energy queue, thereby improving the reliability of resource scheduling in long-term operating scenarios.

[0011] 3. This invention alleviates the mismatch between the frequency of sensing data and real-time communication resources. It uses the UAV sensing cycle as a frame-level control variable and transmission scheduling, flight angle, and edge server energy allocation as time-slot-level adjustment variables, ensuring that the frequency of sensing data generation matches the real-time channel state, queue state, UAV position, and energy state. This approach reduces data queue backlog caused by excessively rapid sensing, transmission failures due to insufficient communication resources, and edge server error state retention caused by untimely state updates.

[0012] 4. Improves the efficiency of multi-UAV collaborative resource scheduling. Under the condition of local observation by multiple UAVs, this invention combines global collaborative evaluation and local action evaluation, enabling the resource allocation strategy to both utilize the global state to coordinate multiple UAVs to share limited uplink and energy resources, and optimize individual actions based on the local state of each UAV, thereby improving the efficiency of resource utilization in complex dynamic environments. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the architecture of a multi-drone-assisted edge network system.

[0014] Figure 2 This is a schematic diagram of the collaborative scheduling framework.

[0015] Figure 3 This is a schematic diagram of the training architecture for the frame-level perceptual control model and the slot-level resource adjustment model.

[0016] Figure 4 This is a graph showing the convergence comparison results of different algorithms.

[0017] Figure 5 This is a comparison of the convergence of different weight ratios in centralized and distributed Critic networks.

[0018] Figure 6 This is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the application. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0020] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0021] Example 1 This solution provides a method for constructing a resource allocation model that minimizes the age of error information for multi-UAV assisted edge networks, including the following steps: S1. Deploy multiple drones and edge servers within the target service area to build a multi-drone-assisted edge network; S2. Calculate the age of the error information of the corresponding drone based on the current real state of the monitoring area and the storage state of the edge server; S3. Construct a resource allocation optimization model. The resource allocation optimization model takes the minimum long-term average error information age of all UAVs as the optimization objective, and takes edge server energy allocation, UAV perception cycle, UAV transmission scheduling and UAV flight angle as optimization variables. Set UAV safe distance constraints, flight angle constraints, perception cycle constraints, data queue stability constraints, energy queue stability constraints, upload resource quantity constraints and energy allocation constraints. S4. Based on the Lyapunov optimization method, the resource allocation optimization model is transformed, and the optimization objective is transformed into a short-term resource allocation problem within each time slot; S5. Divide multiple consecutive time slots into a frame-level scheduling cycle. At the beginning of each frame-level scheduling cycle, update the UAV perception cycle and keep the UAV perception cycle unchanged within the corresponding frame-level scheduling cycle. In each time slot, update the edge server energy allocation, UAV transmission scheduling, and UAV flight angle according to the real-time status. A frame-level perception control model and a time-slot-level resource adjustment model are constructed. The frame-level perception control model outputs the UAV perception cycle through a multi-agent deep Q-network algorithm within the frame-level scheduling cycle. The time-slot-level resource adjustment model outputs the edge server energy allocation, UAV transmission scheduling, and UAV flight angle through a multi-agent dual-Critic proximal strategy optimization algorithm within each time slot. S6. Perform resource allocation based on the output results of the time slot-level resource adjustment model and the frame-level perception control model, and update the model parameters of the frame-level perception decision network and the time slot-level resource adjustment network based on the reward feedback generated during the execution process, so as to obtain a multi-UAV collaborative resource allocation strategy.

[0022] Regarding step S1 of this plan: like Figure 1 As shown, the multi-drone assisted edge network includes multiple drones and an edge server positioned within the target service area. Each drone periodically senses its corresponding monitoring area and caches the sensed status update data in a local data buffer. The edge server receives the status update data uploaded by each drone and processes and evaluates the data. Specifically, each drone is equipped with a local data buffer and a rechargeable battery. The local data buffer caches the status update data generated by the drone's periodic sensing, and the rechargeable battery stores the energy required for the drone's operation.

[0023] In some embodiments, the UAV operates in discrete time slots, performing state perception operations according to the corresponding perception cycle within each time slot. After completing perception, it generates fixed-length state update data, stores the perceived state update data in a local cache, and sends the state update data to the edge server in an appropriate time slot. The edge server receives the data uploaded by the UAV, processes and evaluates the state update data, and allocates energy to the UAV according to its state.

[0024] Furthermore, the edge server and the drone transmit status update data via a wireless communication link. The edge server is configured with multiple orthogonal uplink resource blocks, and each time slot allows a maximum number of drones corresponding to the number of resource blocks to upload status update data.

[0025] Furthermore, the status update data is determined to be successfully transmitted based on the wireless channel status between the drone and the edge server. When the received signal-to-noise ratio meets the preset threshold, the status update data is considered to have successfully reached the edge server.

[0026] Specifically, the drone group is set as Where I represents the number of drones, and the drones perform actions according to discrete time slots, with the time slot index being... .

[0027] definition For drones The perception cycle, To sense state variables, every One perception cycle, the drone performs perception. = 1. Generate a fixed-length state update data; otherwise, do not generate new sensing data. In any time slot drones i The position is represented as ,in For flight altitude, For drones x-coordinate For drones The ordinate; Let the flight speed of the drone be... Flight angle is The drone coordinates will be updated in the following way: .

[0028] Assume there is a connection between the edge server and the drone. A maximum of one orthogonal uplink resource block is allowed in each time slot. Multiple drones simultaneously upload data, defining Indicates drone In the time slot Whether it is scheduled to upload, among which Indicates uploading. This indicates that no upload is being performed. Successful transmission is determined by the channel gain and received signal-to-noise ratio (SNR) from the drone to the edge server. Not lower than the threshold At that point, it is considered that the data packet has successfully reached the edge server.

[0029] Edge servers are used to receive, process, and evaluate data uploaded by drones, and allocate energy to each drone based on its status and energy harvesting capabilities. For time slots Total energy collected by the edge servers Time slot Internal allocation to drones Energy, drones in time slots Internal flight energy consumption Hovering energy consumption Sensing energy consumption and transmission energy consumption The total energy consumption is obtained by combining the results. .

[0030] Regarding step S2 of this plan: Since traditional AoI metrics are mainly used to describe the time interval between data generation and data reception, they cannot reflect the deviation between the state saved by the edge server and the current real state of the monitoring area. Therefore, this solution introduces error information age for the UAV's periodic perception, uploading, and successful reception mechanism to evaluate the effectiveness of the edge server's state information, so as to simultaneously characterize the timeliness and accuracy of the state information.

[0031] In step S2, the calculation process of the error information age includes: determining the edge server storage state based on the state update data received by the edge server; determining the state mismatch penalty item based on the difference between the edge server storage state and the current real state of the monitoring area; determining the time penalty item based on the duration of inconsistency between the edge server storage state and the current real state of the monitoring area; and combining the state mismatch penalty item and the time penalty item to obtain the error information age corresponding to the UAV.

[0032] In some embodiments, the current true state of the monitored area is the real-time state data collected by the drone.

[0033] In some embodiments, unlike traditional AoI, the state mismatch penalty term is used to describe the instantaneous deviation between the stored state of the edge server and the current real state of the monitoring area, and the time penalty term is used to represent the duration of state inconsistency from the last time the edge server obtained correct state update data to the current moment, that is, the duration of state mismatch between the state of the monitoring area and the state at the edge server.

[0034] Specifically, the state mismatch penalty term is defined as:

[0035] in For drones In the time slot Real-time status data of the monitoring area Cutoff time slots for edge servers Cache values ​​for status data of the same monitoring area.

[0036] Due to the existence of a periodic sensing mechanism, success was achieved in time slots. Data packets delivered to the edge server may not be detected in the same time slot. The state change process contained in the data received by the edge server, i.e., the storage state of the edge server, is represented as follows:

[0037] in This represents the index of the most recent sensing slot. To perceive state variables, This indicates that the indicator was successfully received. This indicates a drone. In the time slot The data transmission was scheduled and the data packets were successfully transmitted. This indicates a drone. In the time slot The detected data packet was not successfully received; Indicates drone In the time slot Perform the sensing and generate a fixed-length state update data. This indicates a drone. In the time slot No new sensory data was generated. For drones In the time slot Real-time status update data of the monitored area Cutoff time slots for edge servers Cache values ​​of status data for the same monitoring area Cutoff time slots for edge servers Cache values ​​for status data of the same monitoring area.

[0038] To characterize the duration of the state mismatch, let , indicating as of The edge server receives status data from the drone. The latest time slot for synchronizing the status of the monitoring area is defined as the time penalty is defined as follows: The corresponding time penalty terms are as follows:

[0039] in, For time slots Time penalty items, For time slots Time-based penalty items.

[0040] Therefore, the pattern of change in time penalty is as follows:

[0041] drones In the time slot Error message age definition:

[0042] It should be noted that the error information age is zero when the state is correct; when the state is incorrect, the error information age accumulates over time; and as the state deviation increases or the duration of the incorrect state increases, the error information age further increases. Therefore, the error information age can simultaneously reflect both the duration and magnitude of the error, and this method fundamentally explores how key variables affect the penalty term. Although the state mismatch penalty term... Essentially dependent on the random evolution of the state of the monitored area, the system can minimize the age of erroneous information by optimizing the sensing cycle and transmission scheduling, as well as the location of the drone.

[0043] Regarding step S3 of this plan: After obtaining the age of the error information, this solution further constructs a resource allocation optimization model, which uses the minimum long-term average age of error information for all UAVs as the optimization objective.

[0044] The optimization variables include edge server energy allocation. The perception cycle of drones UAV transmission scheduling and the flight angle of the drone , Indicates the first One drone, It is a collection of drones. Indicates the first Each time slot Represents a set of time slots. Time slot drones at the time The error message is about age.

[0045] Furthermore, the resource allocation optimization model is subject to constraints including UAV safe distance, flight angle, perception cycle, data queue stability, energy queue stability, uploaded resource quantity, and energy allocation. Specifically, the resource allocation optimization model is subject to these constraints, where the UAV safe distance constraint requires the distance between any two UAVs to be within a safe range, and the flight angle constraint requires the UAV's flight angle to be within a certain range. Within the specified range, the number of uploaded resources is constrained to no more than the number of uplink resources per time slot. The energy allocation constraint is that the sum of the energy allocated by the edge server to all drones does not exceed their available energy. .

[0046] Specifically, the perception cycle constraint is that the perception cycle is within The internal control is used to control the frequency at which the drone generates status update data. When the perception cycle is short, the drone can acquire status information more frequently, but this will increase the pressure on the data queue and energy consumption. When the perception cycle is long, it can reduce resource consumption, but it may cause the edge server status update to lag.

[0047] The data queue stability constraint ensures that the data queue satisfies stability. The UAV generates state update data according to the perception cycle and stores it in the local data buffer. By setting the data queue stability constraint, the data generated by the UAV can be uploaded in a timely manner, avoiding the accumulation of a large amount of state update data, which would lead to an increase in the age of error information.

[0048] The energy queue stability constraint ensures that the energy queue meets the stability requirements. During the operation of the UAV, there is energy consumption for flight, hovering, sensing, and communication. By constructing an energy queue, the energy consumption of the UAV and the energy allocation process of the edge server are constrained, so that the UAV can operate stably for a long time.

[0049] The upload resource quantity constraint stipulates that the number of drones uploading in each time slot cannot exceed the uplink resource quantity N. Since the edge server has a limited number of uplink resource blocks, the number of drones simultaneously uploading status update data in each time slot is also limited. Furthermore, the energy allocated by the edge server to the drones cannot exceed the currently available energy.

[0050] In some embodiments, a data queue is used to characterize the amount of state update data generated by the drone but not successfully uploaded to the edge server; an energy queue is used to characterize the cumulative state between energy supply and energy consumption during drone operation. By updating the data queue and the energy queue, the resource allocation process can simultaneously meet the state update requirements and the long-term operational requirements of the drone. Specifically, the drone... Data queue Updated based on perception and transmission scheduling behavior, this queue is used to constrain long-term data backlogs and prevent excessive stale data from causing erroneous information to age prematurely. (Drone) Energy queue By maintaining energy queues based on energy allocation and total energy consumption updates, long-term energy shortages or excessive energy waste by drones can be avoided.

[0051] Regarding step S4 of this plan: Since the resource allocation optimization model involves random state changes, wireless channel changes, UAV movement changes, data queue changes, and energy queue changes, it is a long-term dynamic optimization problem with high complexity for direct solution. Therefore, this scheme adopts the Lyapunov optimization method to transform the resource allocation optimization model.

[0052] In step S4, a Lyapunov function consisting of a data queue and an energy queue is constructed, and the error information age is introduced into the Lyapunov drift plus penalty term. By minimizing the upper bound of the drift plus penalty term, the optimization objective of the resource allocation optimization model is transformed into a short-term resource allocation problem within each time slot.

[0053] To make the technical solution more complete, clear, and easier for those skilled in the art to understand and implement, the relevant derivation process of Lyapunov optimization has been added; Firstly, this scheme constructs a system state vector based on the data queue and energy queue. ,based on Establish a quadratic Lyapunov function ,based on Calculate Lyapunov drift ,in It expresses expectation.

[0054] This scheme then incorporates the age of the error message as a penalty term into the Lyapunov drift, forming a drift plus penalty objective function. Therefore, its upper bound can be derived as follows:

[0055]

[0056] Where B is a constant term.

[0057] Therefore, this scheme minimizes The upper bound can determine the resource allocation actions within the current time slot. Specifically, the optimization objective after optimization using the Lyapunov optimization method is expressed as:

[0058] The energy allocation for edge servers is as follows: The perception cycle of drones UAV transmission scheduling and the flight angle of the drone , Indicates the first One drone, It is a collection of drones. Indicates the first Each time slot Represents a set of time slots. Time slot drones at the time Error message: age For time slots drones at the time Data queue, For time slots drones at the time energy queue, drones In the time slot Sensing and transmission scheduling decisions, drones In the time slot Energy allocated and consumed V These are control parameters.

[0059] As can be seen, the optimized objective includes an AoII penalty term, a data queue benefit term, and an energy constraint term, thereby reducing AoII while maintaining the stability of the data queue and energy queue.

[0060] Regarding step S5 of this plan: To address the inconsistency between the rate of change in UAV perception cycles and the rate of change in communication resources, this solution constructs a frame-level perception control model with dual time scales and a time-slot-level resource adjustment model.

[0061] This scheme divides multiple consecutive time slots into a frame-level scheduling cycle, which is used to adjust the UAV perception cycle. In each time slot, resource allocation parameters such as edge server energy allocation, UAV transmission scheduling, and UAV flight angle are updated.

[0062] like Figure 2 As shown, the continuousK Each time slot constitutes a frame-level scheduling cycle, and the time frame index is denoted as . , The perception cycle of drones Updated at the beginning of each frame-level scheduling period and remains unchanged during that period, while energy allocation... Transmission scheduling and flight angle In each short time slot t The updated collaborative scheduling framework serves the following purposes: the sensing cycle changes slowly, and maintaining intra-frame stability can reduce queue fluctuations; energy allocation, transmission scheduling, and flight angle are affected by real-time channel, location, power, and queue, and need to be adjusted rapidly according to time slots.

[0063] To achieve collaborative optimization of different resource variables, the resource allocation optimization model is further modeled as a two-timescale distributed partially observable Markov decision process. The UAV's perception period is used as a frame-level decision variable, while edge server energy allocation, UAV transmission scheduling, and UAV flight angle are used as time-slot-level decision variables. Within the frame-level timescale, the UAV's perception period is determined based on a global state including remaining edge server energy, remaining UAV battery power, data queue status, energy queue status, error message age, UAV location, and channel status. Within the time-slot-level timescale, a resource adjustment strategy is determined based on the real-time system state, building upon the frame-level perception period. During the execution phase, each UAV independently executes decisions based on its local observations. During the training phase, global state information is used to optimize the policy network, thereby improving the collaborative resource allocation capability among multiple UAVs.

[0064] For frame-level scheduling cycles, the long-scale reward function is the cumulative value of short-scale rewards within a single time frame:

[0065] The short-timescale slot-level reward function for a slot can be expressed as:

[0066] in, Represents time frame Long-term frame-level rewards. Indicates time slot Short-timescale slot-level rewards Indicates the first One drone, It is a collection of drones. Indicates the first Each time slot Represents a set of time slots. Time slot drones at the time Error message: age For time slots drones at the time Data queue, For time slots drones at the time energy queue, drones In the time slot Sensing and transmission scheduling decisions, drones In the time slot Energy allocated and consumed These are control parameters.

[0067] Regarding the frame-level awareness control model: In this embodiment of the scheme, the frame-level perception control model is a multi-agent deep Q-network (MADQN). At the beginning of each frame-level scheduling cycle, each UAV inputs its own local observation state into the MADQN, and the MADQN outputs the perception cycle of the corresponding UAV. During the training process, the network parameters of the MADQN can be optimized using global state information.

[0068] That is, the Multi-Agent Deep Q-Network (MADQN) includes independent Q-networks and target networks corresponding to multiple drones; during the training phase, the global state information of the multi-drone auxiliary edge network is used to update the Q-network parameters; during the execution phase, each drone outputs its corresponding perception cycle through the Q-network based on its own local observation information.

[0069] like Figure 3 As shown, the first aspect concerns the frame-level perception periodicity strategy for long-term scales. At long-term scales, the perception period is a discrete decision variable; therefore, this scheme employs a multi-agent deep Q-network algorithm for perception periodicity decision-making. Specifically, each UAV is configured with an independent multi-agent deep Q-network and a corresponding target network. During training, global state is used to enhance Q-value estimation. During execution, each UAV selects its perception periodic action based solely on its local observations, thus meeting the requirements of distributed execution across multiple UAVs.

[0070] At the beginning of each frame-level scheduling period, the UAV outputs the sensing period for that frame-level scheduling period based on the current local observations. And remain unchanged within this frame-level scheduling period. The network parameters are updated by minimizing the error between the Q value and the target Q value. The target Q value and the loss function can be expressed as:

[0071]

[0072] in Indicates drone In time frame Local observations Indicates drone In time frame Local observation and This indicates the corresponding sensory cycle action. and These represent the Q network parameters and the target network parameters, respectively. Indicates the discount factor. It is a drone The corresponding Q network, It is a long-scale time frame The reward function, This represents the expected value.

[0073] The time-slot-level resource adjustment model is a multi-agent double critic proximal policy optimization (MADCPPO) algorithm. The MADCPPO algorithm further includes a distributed Actor network, a distributed Critic network, and a centralized Critic network. The distributed Actor network outputs resource adjustment actions based on the local state of the UAV, the distributed Critic network evaluates the actions of individual UAVs, and the centralized Critic network evaluates the overall collaborative behavior using the states of all UAVs and edge servers.

[0074] In each time slot, energy allocation, transmission scheduling, and flight angle decisions need to be processed simultaneously. Since each UAV can only obtain its own local observation information, a multi-agent dual-Critic proximal strategy optimization algorithm is adopted to improve the collaborative decision-making capability of multiple UAVs for short-term resource decisions. This algorithm includes distributed Actors, distributed Critics, and centralized Critics: Distributed Actor: Used to generate resource adjustment actions based on local observation information of UAVs. This network maps the local observations of each UAV in each time slot to the probability distribution of each action. Through interaction with the environment, the network can learn and discover the optimal strategy.

[0075] Distributed Critic: Used to evaluate corresponding actions based on local observation information of individual UAVs. This network evaluates the actions and observation data of the corresponding UAVs and provides feedback to the actuator network to guide policy updates, thereby optimizing the behavior of each UAV.

[0076] Centralized Critic: Used to evaluate overall collaborative behavior based on the joint state information of all drones and edge servers. It assesses the joint actions and observation data of all drones and edge servers and provides global feedback to all participants to promote collaborative behavior.

[0077] Furthermore, the time-slot-level resource adjustment model is trained and updated using a multi-agent dual-Critic proximal strategy optimization algorithm. This algorithm obtains a fused advantage function by fusing the centralized advantage function output by the centralized Critic network and the distributed advantage function output by the distributed Critic network. The strategy network parameters are updated based on the fused advantage function, so that the resource adjustment strategy simultaneously meets the global coordination requirements of multiple UAVs and the local optimization requirements of UAVs.

[0078] Specifically, a corresponding distributed Actor network is configured for each UAV. Each distributed Actor network outputs resource adjustment actions based on the local observation information of its corresponding UAV. Simultaneously, a centralized Critic network and a distributed Critic network are constructed to evaluate the resource adjustment actions using global state information and UAV local observation information, respectively, and update the parameters of the distributed Actor network according to the fusion advantage function. During training, the loss functions of the policy network, centralized Critic network, and distributed Critic network are minimized respectively, and the corresponding network parameters are updated accordingly. The policy network loss function is the same as the loss function of the distributed Actor network, the centralized Critic network loss function is the same as the loss function of the centralized Critic network, and the distributed Critic network loss function is the same as the loss function of the distributed Critic network.

[0079] To limit the magnitude of change before and after the policy update, the multi-agent dual-Critic proximal policy optimization algorithm uses a pruning loss function of proximal policy optimization for updating, assuming the drone... The policy network loss function is expressed as:

[0080] in, This represents the ratio of the current policy probability to the old policy probability. The threshold for pruning the strategy. For entropy regularization, Here is the entropy regularization coefficient. For the expected value, For the fusion of advantage functions, min() means minimizing.

[0081] This scheme limits the magnitude of a single policy update by pruning the probability ratio, maintains the policy's action exploration capability through entropy regularization, and integrates advantage functions to ensure that policy updates are guided by both global cooperative effects and local action effects.

[0082] Regarding the fusion dominance function, this invention calculates the centralized dominance function and the distributed dominance function using both centralized and distributed Criticism, and then performs a weighted fusion to obtain the fusion dominance function:

[0083]

[0084]

[0085] in, and The time difference errors are for centralized and distributed Critic, respectively. For time slots The reward It is a time slot The global state, It is a drone In the time slot The state of observation and The state values ​​and parameters for centralized and distributed Critic are respectively. The weights used to control these Critics; Discount factor; It is a weighting parameter in GAE, used to balance the trade-off between bias and variance.

[0086] For centralized Critic, global information is used to construct the target value. ,in For the target parameters, For having parameters The value of a centralized Critic network is determined, and the mean squared error (MSE) loss function is used to learn global action values. The specific loss function of the centralized Critic network is as follows:

[0087] in, For the centralized Critic network loss function, For the target value, For having parameters The value of centralized Critic networks.

[0088] For distributed Critic, the target value It is constructed based on local observation results. For the target parameters, Indicates that it has parameters as The value of a distributed Critic network is determined by the MSE loss function, which is used to learn the value of local actions. The specific loss function for a distributed Critic network is as follows:

[0089] in Indicates that it has parameters as The value of distributed Critic networks For the target value, This is the loss function for the distributed Critic network.

[0090] This scheme minimizes the policy network loss function, the centralized Critic network loss function, and the distributed Critic network loss function based on samples collected during short-term interaction, and updates the corresponding network parameters. Through the above dual-Critic structure, short-term decision-making can achieve collaborative optimization by utilizing global information, and optimize individual behavior based on local observations.

[0091] Regarding step S6 of this plan: In some embodiments, the output results include the UAV perception cycle output by the frame-level perception control model, and the edge server energy allocation, UAV transmission scheduling, and UAV flight angle output by the time-slot-level resource adjustment model.

[0092] In some embodiments, the reward feedback includes changes in error information age, changes in data queue, and changes in energy queue, and updates the network parameters of the frame-level perceptual control model and the network parameters of the time-slot-level resource adjustment model based on the reward feedback.

[0093] During training: frame-level states, perceptual cycle actions, and corresponding rewards are stored in the first experience replay pool to update the frame-level perceptual control model. The time slot-level state, resource adjustment actions, and corresponding rewards are stored in the second experience replay pool to update the Actor network and Critic network in the time slot-level resource adjustment model.

[0094] After multiple rounds of training, a collaborative resource allocation strategy model for multi-UAV assisted edge networks was obtained.

[0095] Specifically, at the beginning of each training round, the positions of each UAV, data queues, energy queues, error information age, available energy of the edge server, and channel status are initialized. At the beginning of each frame-level scheduling cycle, the perception cycle action is output and remains unchanged within that time frame. Within each short time scale slot contained in that time frame, energy allocation, transmission scheduling, and flight angle actions are output. After the actions are executed, the system updates the status and calculates the reward for the current time slot based on the short time scale reward function.

[0096] For each time slot's short-term timescale, the current time slot state, action, reward, and next time slot state are stored in the second experience replay pool D2. A small batch of samples is extracted from D2 to update the network parameters of the distributed Actor, distributed Critic, and centralized Critic, respectively. For the long-term timescale of the frame-level scheduling cycle, after the end of a frame-level scheduling cycle, all short-term timescale rewards within that cycle are accumulated to obtain the long-term timescale reward. The state, action, reward, and next time frame state of that frame-level scheduling cycle are stored in the first experience replay pool D1. Subsequently, a small batch of samples is extracted from D1 to update the Q-network parameters. After multiple rounds of training, a multi-UAV cooperative resource allocation strategy is formed. This multi-UAV cooperative resource allocation strategy determines the UAV perception cycle on the long-term timescale and determines edge server energy allocation, UAV transmission scheduling, and UAV flight angles on the short-term timescale.

[0097] Example 2 This embodiment verifies the resource allocation effect of the invention's method. The simulation scenario is set to a 500 m × 500 m service area, deploying 8 drones and 1 edge server. The drone flight altitude is set to 100 m, the flight speed to 5 m / s, and the minimum safe distance to 20 m. The invention's method, TRAO (Two-timescale Resource Allocation Optimization), is compared with MADQN-PPO (multi-agent deep Q-network—proximal policy optimization), MADQN-DDPG (multi-agent deep Q-network—deep deterministic policy gradient), MAPPO (multi-agent proximal policy optimization), Greedy, and Random, verifying the effectiveness of the invention.

[0098] Figure 4 The results comparing the average reward and average AoII (mean age of misinformation) during training between TRAO and different comparative algorithms are presented. Figure 4 In the diagram, (a) represents the comparison results of the average reward of different comparison algorithms, and (b) represents the comparison results of the average AoII of different comparison algorithms. Throughout the training process, TRAO consistently outperformed other algorithms, demonstrating strong convergence in terms of average reward, while other algorithms exhibited significantly lower reward values ​​or larger fluctuations. Regarding average AoII, compared to existing learning-based multi-agent resource allocation algorithms and heuristic algorithms, TRAO significantly reduced the average AoII, with a minimum reduction of 48.1%. This performance advantage mainly stems from our frame-level and slot-level dual-timescale optimization framework, which separates slot-level resource allocation from time-frame perception decisions, thereby promoting efficient collaboration among multiple agents. Therefore, these results validate the effectiveness of this invention in terms of overall resource allocation performance and AoII reduction.

[0099] Figure 5 The results comparing the average reward and average AoII of TRAO under different weight ratios in centralized and distributed Critic networks are presented. Figure 5 In the diagram, (a) represents the average reward result under different weight ratios of centralized and distributed Critic networks, and (b) represents the average AoII comparison result under different weight ratios of centralized and distributed Critic networks. Figure 5 It is known that using only a distributed Critic network, action evaluation is mainly based on local observations, which is insufficient to fully reflect the global collaborative relationship among multiple UAVs; while using only a centralized Critic network can utilize global state information, it is inadequate in characterizing the local action differences of individual UAVs. This invention achieves the lowest average AoII by weighted fusion of centralized and distributed advantage functions, allowing policy updates to be guided by both global collaborative information and local action value. Simulation results show that when the weight ratio of the centralized Critic network to the distributed Critic network is 0.7:0.3, the system achieves the lowest average AoII. Compared with a fully distributed Critic network, the average AoII of TRAO is reduced by 61.6%; compared with a fully centralized Critic network, the average AoII is reduced by 55.3%. These results demonstrate that the dual-Critic structure can balance global collaboration and local action optimization, improving the accuracy of time-slot-level resource adjustment and the efficiency of multi-UAV collaborative scheduling.

[0100] Example 3 This embodiment also provides an electronic device, see reference. Figure 6 It includes a memory 402 and a processor 401, the memory 402 storing a computer program and the processor 401 being configured to run the computer program to perform the steps in any of the embodiments of the method for constructing a resource allocation model for minimizing error information age for multi-UAV assisted edge networks.

[0101] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0102] The memory 402 may include a large-capacity memory 402 for data or instructions. The memory 402 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 401.

[0103] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the above embodiments of the method for constructing a resource allocation model for minimizing the age of error information for multi-UAV assisted edge networks.

[0104] Optionally, the electronic device may further include a transmission device 403 and an input / output device 404, wherein the transmission device 403 is connected to the processor 401 and the input / output device 404 is connected to the processor 401.

[0105] The transmission device 403 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 403 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0106] Input / output device 404 is used to input or output information. In this embodiment, the input information may be a drone-assisted edge network, etc., and the output information may be a multi-drone collaborative resource allocation strategy, etc.

[0107] Optionally, in this embodiment, the processor 401 can be configured to perform the following steps via a computer program: S1. Deploy multiple drones and edge servers within the target service area to build a multi-drone-assisted edge network; S2. Calculate the age of the error information of the corresponding drone based on the current real state of the monitoring area and the storage state of the edge server; S3. Construct a resource allocation optimization model. The resource allocation optimization model takes the minimum long-term average error information age of all UAVs as the optimization objective, and takes edge server energy allocation, UAV perception cycle, UAV transmission scheduling and UAV flight angle as optimization variables. Set UAV safe distance constraints, flight angle constraints, perception cycle constraints, data queue stability constraints, energy queue stability constraints, upload resource quantity constraints and energy allocation constraints. S4. Based on the Lyapunov optimization method, the resource allocation optimization model is transformed, and the optimization objective is transformed into a short-term resource allocation problem within each time slot; S5. Divide multiple consecutive time slots into a frame-level scheduling cycle. At the beginning of each frame-level scheduling cycle, update the UAV perception cycle and keep the UAV perception cycle unchanged within the corresponding frame-level scheduling cycle. In each time slot, update the edge server energy allocation, UAV transmission scheduling, and UAV flight angle according to the real-time status. A frame-level perception control model and a time-slot-level resource adjustment model are constructed. The frame-level perception control model outputs the UAV perception cycle through a multi-agent deep Q-network algorithm within the frame-level scheduling cycle. The time-slot-level resource adjustment model outputs the edge server energy allocation, UAV transmission scheduling, and UAV flight angle through a multi-agent dual-Critic proximal strategy optimization algorithm within each time slot. S6. Perform resource allocation based on the output results of the time slot-level resource adjustment model and the frame-level perception control model, and update the model parameters of the frame-level perception decision network and the time slot-level resource adjustment network based on the reward feedback generated during the execution process, so as to obtain a multi-UAV collaborative resource allocation strategy.

[0108] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0109] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0110] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer-executable components configured to perform embodiments when the program is run. One or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted that any block in the logical flow of the figures may represent a program step, or interconnected logical circuitry, blocks and functions, or a combination of program steps and logical circuitry, blocks and functions. The software may be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0111] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for constructing an error information age minimization resource allocation model for multi-UAV assisted edge networks, characterized in that, Includes the following steps: S1. Deploy multiple drones and edge servers within the target service area to build a multi-drone-assisted edge network; S2. Calculate the age of the error information of the corresponding drone based on the current real state of the monitoring area and the storage state of the edge server; S3. Construct a resource allocation optimization model, in which the optimization objective is to minimize the long-term average age of all UAVs, and the optimization variables are edge server energy allocation, UAV perception cycle, UAV transmission scheduling, and UAV flight angle. S4. Based on the Lyapunov optimization method, the resource allocation optimization model is transformed, and the optimization objective is transformed into a short-term resource allocation problem within each time slot; S5. Divide multiple consecutive time slots into a frame-level scheduling cycle. At the beginning of each frame-level scheduling cycle, update the UAV perception cycle and keep the UAV perception cycle unchanged within the corresponding frame-level scheduling cycle. In each time slot, update the edge server energy allocation, UAV transmission scheduling, and UAV flight angle according to the real-time status. A frame-level perception control model and a time-slot-level resource adjustment model are constructed. The frame-level perception control model outputs the UAV perception cycle through a multi-agent deep Q-network algorithm within the frame-level scheduling cycle. The time-slot-level resource adjustment model outputs the edge server energy allocation, UAV transmission scheduling, and UAV flight angle through a multi-agent dual-Critic proximal strategy optimization algorithm within each time slot. S6. Perform resource allocation based on the output results of the time slot-level resource adjustment model and the frame-level perception control model, and update the model parameters of the frame-level perception decision network and the time slot-level resource adjustment network based on the reward feedback generated during the execution process, so as to obtain a multi-UAV collaborative resource allocation strategy.

2. The method of claim 1, wherein, The drone operates in discrete time slots. Within each time slot, it performs state perception operations according to the corresponding perception cycle. After completing the perception, it generates a fixed length of state update data, stores the perceived state update data in a local cache, and sends the state update data to the edge server in an appropriate time slot. The edge server receives the data uploaded by the drone, processes and evaluates the state update data, and allocates energy to the drone according to the state.

3. The method of claim 1, wherein, Determine the storage status of the edge server based on the status update data received by the edge server; The penalty for state mismatch is determined based on the difference between the storage status of the edge server and the current real status of the monitoring area; The time penalty term is determined based on the duration of inconsistency between the storage state of the edge server and the current real state of the monitoring area; the state mismatch penalty term is combined with the time penalty term to obtain the age of the error information corresponding to the drone.

4. The method of claim 1, wherein, The resource allocation optimization model is configured with constraints on UAV safe distance, flight angle, perception cycle, data queue stability, energy queue stability, uploaded resource quantity, and energy allocation.

5. The method of claim 1, wherein, We construct a Lyapunov function consisting of a data queue and an energy queue, and introduce the error information age into the Lyapunov drift plus penalty term. By minimizing the upper bound of the drift plus penalty term, we transform the optimization objective of the resource allocation optimization model into a short-term resource allocation problem within each time slot.

6. The method of claim 1, wherein, The optimization objective after optimization using the Lyapunov optimization method is expressed as: The energy allocation for edge servers is as follows: The perception cycle of drones UAV transmission scheduling and the flight angle of the drone , Indicates the first One drone, It is a collection of drones. Indicates the first Each time slot Represents a set of time slots. Time slot drones at the time Error message: age For time slots drones at the time Data queue, For time slots drones at the time energy queue, drones In the time slot Sensing and transmission scheduling decisions, drones In the time slot Energy allocated and consumed These are control parameters.

7. The method for constructing a resource allocation model for minimizing the age of error information in multi-UAV assisted edge networks according to claim 1, characterized in that, The multi-agent deep Q-network includes independent Q-networks and a target network corresponding to multiple drones. During the training phase, the global state information of the multi-drone auxiliary edge network is used to update the Q-network parameters. During the execution phase, each drone outputs its perception cycle through the Q-network based on its own local observation information.

8. The method for constructing a resource allocation model for minimizing the age of error information in multi-UAV assisted edge networks according to claim 1, characterized in that, The multi-agent dual-Critic proximal policy optimization algorithm further includes a distributed Actor network, a distributed Critic network, and a centralized Critic network. The distributed Actor network adjusts actions based on the local state of the UAV and outputs resources. The distributed Critic network evaluates the actions of individual UAVs. The centralized Critic network evaluates the overall cooperative behavior using the states of all UAVs and the edge server.

9. The method for constructing a resource allocation model for minimizing the age of error information in multi-UAV assisted edge networks according to claim 8, characterized in that, For each time slot, the current time slot state, action, reward, and next time slot state are stored in the second experience replay pool. A small batch of samples is drawn from the second experience replay pool to update the network parameters of the distributed Actor, distributed Critic, and centralized Critic, respectively. For a frame-level scheduling period, after the end of a frame-level scheduling period, all short-timescale rewards within that frame-level scheduling period are accumulated to obtain a long-timescale reward. The state, action, reward, and next time frame state of that frame-level scheduling period are stored in the first experience replay pool. Subsequently, a small batch of samples is drawn from the first experience replay pool to update the Q network parameters. After multiple rounds of training, a multi-UAV cooperative resource allocation strategy is formed.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to execute the method for constructing a resource allocation model for minimizing error information age for multi-UAV assisted edge networks as described in any one of claims 1 to 9.

Citation Information

Patent Citations

  • Cloud-side multi-unmanned aerial vehicle collaborative resource optimization method assisted by large language model

    CN121099339A

  • Edge computing unloading and resource allocation method and system for space-ground integrated network

    CN122496846A