Low-altitude Internet of Things task optimization method based on auction and diffusion learning

Abstract

The invention discloses a low-altitude internet-of-things task optimization method based on auction and diffusion learning, and the method comprises the steps: constructing an air-ground cooperation edge calculation frame integrating an unmanned plane, an air base station and a ground base station, and facing the low-altitude application scenes with the uncertainty of task arrival, the heterogeneous calculation resources, the time-varying communication conditions and the like; on a large time scale, an auction algorithm based on a VCG mechanism is designed, and excitation compatible distribution of the unmanned aerial vehicle to a task area is realized; on a small time scale, a heterogeneous agent near-end strategy optimization algorithm (D-HAPPO) fused with a latent variable diffusion model is provided, and the strategy diversity and the environmental adaptability are improved by generating modeling. According to the invention, by introducing the condition generation process, the dynamic collaborative optimization of the task unloading decision and the route planning is realized, and the task completion rate and the energy consumption efficiency of the unmanned system in a complex scene are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of task offloading and trajectory optimization in low-altitude intelligent collaborative networks, and in particular to a low-altitude intelligent network task optimization method based on auction and diffusion learning. Background Art

[0002] With the development of integrated air-ground intelligent networks, drones (UAVs) are playing an increasingly critical role in tasks such as intelligent inspection, emergency rescue, and low-altitude logistics. To meet the demands of multi-tasking applications in dynamic environments, UAV systems must not only collect and process data but also possess strong autonomous decision-making and collaborative capabilities. However, limited by platform computing power and flight time, UAVs face high energy consumption and resource constraints when handling high-intensity and diverse computing tasks, making it difficult to ensure the timeliness and success rate of tasks. In recent years, the introduction of edge computing has provided new computing power support for UAV systems. By offloading computing tasks to ground base stations, aerial base stations, or other nearby UAVs, the computational pressure on individual nodes can be alleviated. However, resource status varies significantly between nodes, and communication channel quality fluctuates frequently. Therefore, efficient task offloading decisions and node coordination are key challenges in system optimization. Traditional centralized or static allocation strategies often struggle to adapt to spatiotemporal resource fluctuations in dynamic environments and cannot effectively coordinate resource competition and collaborative demands among multiple UAVs, leading to problems such as uneven system load and high task abandonment rates. Furthermore, drones must balance multiple factors, including mission coverage, communication quality, and energy consumption, along their trajectory. This makes the problem of jointly optimizing offloading decisions and trajectory control highly complex and time-varying. Therefore, there is an urgent need to design a collaborative scheduling mechanism for unmanned system task optimization that can dynamically allocate and offload drone tasks in a distributed environment, rationally plan their trajectories, and ultimately improve the overall system's mission completion rate and energy efficiency. Summary of the Invention

[0003] The present invention proposes a UAV task optimization method based on an auction mechanism and diffusion learning, which is used to solve the problems of uneven distribution of multiple tasks, limited computing resources, and inefficient path planning in low-altitude intelligent networks. This method decouples UAV trajectory optimization from task offloading decisions through hierarchical modeling, and performs optimization separately at different time scales. On a large time scale, a VCG auction mechanism is adopted to achieve energy-aware dynamic matching of UAVs and task areas; on a small time scale, a multi-agent reinforcement learning algorithm that integrates a latent variable diffusion model is used to make task offloading and execution mode decisions, thereby improving the mission success rate and system energy efficiency. This method has strong adaptability and scalability, and is particularly suitable for the intelligent collaborative task execution of low-altitude UAV clusters in complex dynamic environments.

[0004] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0005] A low-altitude intelligent network task optimization method based on auction and diffusion learning, the method comprising the following steps:

[0006] S1, builds an integrated air-ground collaborative network consisting of multiple drones, ground base stations, and aerial base stations to support task processing, edge collaboration, and task offloading in dynamic mission environments;

[0007] S2: Establish a multi-link communication model covering drones, ground base stations, and aerial base stations, and perform structured modeling of the task's data size, computing requirements, and deadline. A two-layer time slot partitioning strategy is used. On a large time scale, the drone's trajectory is modeled across multiple time slots, binding the drone to the task area. On a small time scale, task offloading decisions are dynamically modeled in each time slot, dynamically determining the offloading destination. With the optimization objectives of maximizing the system's energy efficiency and the weighted sum of the proportion of tasks completed on time in the integrated air-ground collaborative network, a joint optimization problem model is constructed to jointly optimize the connection relationship z between the drone and the task area, as well as the task offloading decision x.

[0008] S3, on a large time scale, calculates the expected utility value of each drone in different mission areas by combining the estimated total energy consumption and mission completion capability of the drones at intervals of multiple time slots. Based on the Vickrey-Clarke-Groves auction mechanism, the connection relationship z in the joint optimization problem model is solved in combination with the expected utility value to complete the mission area allocation of the drones.

[0009] S4, designs a diffusion model-enhanced heterogeneous agent proximal policy optimization algorithm. At a small time scale, it uses latent variable diffusion and conditional denoising mechanisms to solve the task offloading decision x for each time slot, and generates the task offloading and drone movement strategy for each time slot.

[0010] Step S1 further comprises:

[0011] Construct an integrated air-ground collaborative network, which includes an air base station, multiple aerial drones, and several ground base stations, where the drone set is represented as The ground base station set is represented as For aerial base stations The position of UAV u at time t is represented by l u (t)=(x u (t),y u (t),z u (t)), the position of the ground base station b is represented by l b (t)=(x b (t),yb (t),z b (t)), the coordinates of the aerial base station are expressed as l h (t)=(x h (t),y h (t),z h (t)); where x u (t), y u (t) and z u (t) represents the horizontal, vertical and height coordinates of the UAV u, respectively, and x b (t), y b (t) and z b (t) represents the horizontal, vertical and height coordinates of the ground base station b, respectively, and x h (t), y h (t) and z h (t) represents the coordinates of the aerial base station in the horizontal, vertical and height directions respectively;

[0012] Divide the system's running time into discrete time slots The length of each time slot is τ; tasks are generated in the specified task area and triggered when the UAV flies to the corresponding task area. The UAV processes the task locally or offloads the task to a nearby ground base station or aerial base station for collaborative processing.

[0013] Furthermore, in step S2, a multi-link communication model covering the UAV, ground base stations, and aerial base stations is established, and a structured modeling process of the task data size, computing requirements, and deadline is performed. The specific process includes the following steps:

[0014] The task area is modeled as a circular area on a two-dimensional plane. With position l m (t)=(x m ,y m ) as the center, and is defined as follows:

[0015]

[0016] Where r m For the mission area radius;

[0017] Each task area initializes a task queue at time slot t=0 It contains N m Tasks:

[0018]

[0019] Each task a m,n Use triples express, where d m,n is the data size, c m,n is the amount of computation required per bit, The number of deadline slots for the task.

[0020] Furthermore, in step S2, a two-layer time slot division strategy is adopted to perform UAV mission area binding at a large time scale and dynamically decide the unloading destination at a small time scale. The process includes:

[0021] A time scale separation strategy is adopted to decompose the original problem into two interrelated sub-problems. On a large time scale, the UAV's motion trajectory is modeled at intervals of multiple time slots, including the mission area definition, the UAV's movement pattern, and its flight energy consumption model. On a small time scale, the task offloading decision is dynamically modeled in each time slot, including channel conditions, queue status, and computing resource availability. The total energy consumption of the UAV and the total energy consumption of the system are calculated separately, and an energy efficiency index model is constructed to represent the amount of data successfully processed per unit energy consumption. With the optimization objectives of maximizing the system's energy efficiency and the weighted sum of the proportion of tasks completed on time in the integrated air-ground collaborative network, a joint optimization problem model is constructed to jointly optimize the connection relationship z between the UAV and the mission area and the task offloading decision x.

[0022] Furthermore, in step S2, the process of modeling the trajectory of the UAV at a large time scale and multiple time slots includes the following steps:

[0023] The motion trajectory of the UAV is modeled as a two-dimensional horizontal movement above the mission area. The flight energy consumption of the UAV u in time slot t consists of two parts: hovering power consumption and mobile power consumption The expression for hover power consumption is:

[0024]

[0025] Among them, M u is the mass of the drone, g is the acceleration due to gravity, ρ u is the propeller radius, p u is the number of propellers, is the air density; mobile power consumption The expression is:

[0026]

[0027] where v u (t) is the moving speed of the drone, is the maximum flight speed, is the power consumption at maximum speed, is the static power consumption when hovering;

[0028] Let δ u (t) represents the horizontal displacement distance of UAV y in time slot t, δ u (t)=‖l u (t+1)-l u (t)‖, the total energy consumption of flight is:

[0029]

[0030] Introduce binary variable z u,m (t), indicates whether UAV u is associated with the mission area at time slot t

[0031]

[0032] Furthermore, in step S2, at a small time scale, the process of dynamically making a task offloading decision at each time slot includes the following steps:

[0033] S21, introduction of accessibility indicator variables To determine whether the UAV u physically reaches the mission area at time slot t It is defined as follows:

[0034]

[0035] S22, the network topology is considered quasi-static in each time slot and is updated according to the latest parameters at the beginning of each time slot. The path loss between UAV u and the aerial base station is:

[0036]

[0037] Among them, u,h (t) is the elevation angle, f c is the carrier frequency, C is the speed of light; the available transmission rate from drone u to the aerial base station is:

[0038]

[0039] in, is the allocated bandwidth, P u (t) is the transmission power, N G is the noise power;

[0040] The path loss between the UAV and other nodes is modeled as:

[0041]

[0042] in The corresponding transmission rates are:

[0043]

[0044] S24, using binary variables Represents task a m,n Whether time slot t is processed locally by drone y:

[0045]

[0046] The completion time of the task processed locally is calculated as:

[0047]

[0048] in, Schedule the time for the task to start, is the waiting time, defined as:

[0049]

[0050] in represents the local task queue of drone u at time slot t, f u is the CPU frequency of the drone; the task queue is updated using the following formula:

[0051]

[0052] The local processing energy consumption is calculated as:

[0053]

[0054] in, is the effective switching capacitance coefficient of the UAV, and τ is the calculation period;

[0055] Defining variables Represents task a m,n Whether to unload to node α, where

[0056]

[0057] The calculated task completion time after unloading is:

[0058]

[0059] The waiting time is:

[0060]

[0061] The overall completion time is expressed as:

[0062]

[0063] After the task is unloaded, the task queue is updated using the following formula:

[0064]

[0065] The unloading energy consumption is calculated as:

[0066]

[0067] In the formula, let κ α represents the energy consumption coefficient of node α performing CPU / GPU calculation, where Represent other UAVs, ground base stations or aerial base stations respectively; each task can only be processed or offloaded locally once, with the following constraints:

[0068]

[0069] S25, define the task success set Used to indicate that in the task area The set of tasks that successfully completed before the deadline:

[0070]

[0071] Furthermore, in step S2, the process of constructing the joint optimization problem model includes the following steps:

[0072] The total energy consumption of UAV u in time slot t is calculated using the following formula:

[0073]

[0074]

[0075] In the formula Represents task a m,n The processing energy consumption, Indicates the total energy consumption of flight;

[0076] The total energy consumption of the system is calculated using the following formula:

[0077]

[0078] Define the overall energy efficiency η of the system, which is used to represent the amount of data successfully processed per unit energy consumption:

[0079]

[0080] in For all mission areas, For the mission area The set of tasks that are successfully completed before the intrinsic deadline;

[0081] Constructing a joint optimization problem model:

[0082]

[0083]

[0084] in, Indicates the task area The number of tasks completed on time in time slot t, N m For the mission area The total number of tasks in ; parameter β is the adjustment factor, which is used to balance the weight between task reliability and system energy efficiency.

[0085] Step S3 further comprises:

[0086] S31 calculates the horizontal flight distance between the UAV and the center of the mission area, and estimates the flight time and corresponding flight energy consumption based on the flight speed. For each task in the target area, evaluates its local execution energy consumption, compares the local execution energy consumption with the communication energy consumption and remote computing energy consumption required to offload the task to other nodes, and selects the solution with the lowest total energy consumption as the estimated UAV energy consumption. The UAV's total energy consumption includes the total overhead of flight and computing / offloading.

[0087] Calculate the number of successful tasks and estimate the task completion rate in the region; if a task is completed before the deadline, it is considered a successful task;

[0088] S32, combining the estimated total energy consumption of the drones and the mission completion rate to calculate the expected utility value of each drone in different mission areas. Based on the Vickrey-Clarke-Groves auction mechanism, the connection relationship z in the joint optimization problem model is solved in combination with the expected utility value. Specifically, the following sub-steps are included:

[0089] In step S321, each UAV u calculates its expected utility value for each mission area based on the estimated UAV energy consumption and mission success rate:

[0090]

[0091] in represents the estimated total energy consumption of the drone, represents the estimated task completion rate; parameters γ1 and γ2 control the weight balance between energy consumption and task success rate; the corresponding bid value is:

[0092] b u (m) = βU u (m)

[0093] Where β is the numerical scaling factor;

[0094] In step S322, after collecting the bids from all drones in step S321, the auctioneer assigns each drone to its optimal task area based on its expected utility value for the task area and the highest bid value. The auctioneer also updates the distribution of task areas and the overall energy efficiency of the system. After the assignment is completed, the VCG mechanism is used to calculate the payment value of each drone as its marginal contribution from participating in the task assignment. The specific formula is as follows:

[0095]

[0096] in represents the total system utility with all UAVs participating, represents the utility after removing the drone u;

[0097] In step S323, each UAV plans a minimum flight energy consumption path according to the assigned mission, and after going to the designated mission area to perform the mission, it enters step S4 to start the task offloading and scheduling process under a small time scale.

[0098] Step S4 further comprises:

[0099] S41, the UAV task offloading problem at a small time scale is modeled as a Markov game problem, where the UAVs are modeled as N intelligent agents collaborating in a shared environment, and the task offloading process is formalized as a five-tuple The Markov game defined by γ are the state, observation space, action space, reward function and reward parameter of the agent respectively;

[0100] Specifically, the state s of each agent n n (t) contains its own position l u (t), the location of the ground base station and the aerial base station l b (t) and l h (t), mission area information And the task queue and Specifically:

[0101]

[0102] Each agent can only observe local and partially shared information, and its observation space is:

[0103]

[0104] Action Space Indicates the optional offloading targets of the task currently assigned to drone n, including local processing or offloading to other nodes

[0105]

[0106] Reward Function Used to guide each drone to make a trade-off between energy consumption, timeliness, and mission completion:

[0107]

[0108] Among them E n is the total energy consumption of task processing or offloading, S n Indicates whether the task is successfully completed; parameter α is used to penalize energy consumption, and parameter γ is used to reward task success;

[0109] S43, introduce the latent variable of standard normal distribution As a learnable action generation prior; in the forward diffusion process, the latent variable is scheduled according to the variance within T steps Gradually add noise to form the noise latent variable z t :

[0110]

[0111] Among them, ∈ is the diffusion noise term, which is used to represent the randomness introduced in each diffusion process;

[0112] Training loss Used to optimize the denoising network D θ , so that it can be used in a given noise latent variable z t , observation code φ(o n ) and time step t, predict the diffusion noise ∈ added in the forward process as accurately as possible; the specific form is as follows:

[0113]

[0114] In the inference phase, the trained denoiser is used to extract random noise samples z t Reverse sampling to obtain clear latent variables

[0115]

[0116] The latent variables sampled in reverse are transformed into Decoded as final action a n ,

[0117] S44, embed the potential diffusion model into the sequence update structure, for each agent i m , the loss after clipping is:

[0118]

[0119] in It's an old strategy. is the recursively calculated composite advantage term:

[0120]

[0121] Where, Indicates the current strategy parameters of the mth agent The policy function under state s indicates taking action probability; represents the strategy parameters of the mth agent after the k+1th round update; is the action taken by the mth agent in the current state s; s is the current state of the environment, shared by all agents; Represents the old policy parameters of the mth agent in the kth iteration The strategy function under ; Clip is the clipping function, which is used to limit the update range of the strategy ratio; is the policy parameter θ of the first m-1 agents after update k+1 The probability of the joint strategy under ; is the strategy parameter θ of the first m-1 agents before updating k The probability of the joint strategy under ; The advantage function estimate of the joint action a in state s reflects the degree to which the action is better than the average strategy.

[0122] Compared with the prior art, the present invention has the following beneficial effects:

[0123] First, the low-altitude intelligent network task optimization method based on auction and diffusion learning of the present invention effectively improves the task allocation efficiency and energy efficiency of multiple unmanned systems in the low-altitude intelligent network.

[0124] Second, the low-altitude intelligent network task optimization method based on auction and diffusion learning of the present invention realizes energy-aware UAV-task area allocation by introducing the VCG auction mechanism on a large time scale, and constructs a distributed strategy network in combination with the diffusion modeling reinforcement learning algorithm on a small time scale, thereby realizing adaptive task offloading and path control between heterogeneous nodes, significantly improving the task success rate and system responsiveness.

[0125] Third, compared to existing centralized or static offloading methods, this method leverages generative strategies to enhance model expressiveness and action diversity, reducing task latency and energy consumption fluctuations while improving robustness in complex environments. Simulation results demonstrate that this method significantly outperforms baseline algorithms under dynamic network conditions, achieving significant improvements in key metrics such as task completion rate, average energy consumption, and algorithm convergence speed, demonstrating its potential for engineering promotion and deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] Figure 1 This is a schematic diagram of the system structure of the multi-layer unmanned system task collaborative offloading and trajectory planning under the low-altitude intelligent network of the present invention;

[0127] Figure 2 Schematic diagram of the collaborative optimization process of multi-time-scale task offloading and trajectory planning in the present invention;

[0128] Figure 3 Schematic diagram of the multi-time-scale unmanned system task optimization framework driven by auction and diffusion learning in the present invention;

[0129] Figure 4 This is a graph showing the average reward change trend of the D-HAPPO algorithm of the present invention under different learning rate settings;

[0130] Figure 5 This is a comparison chart of the latency performance of the algorithm in the present invention for heterogeneous unmanned system tasks;

[0131] Figure 6 This is a comparison chart of the energy consumption efficiency of the algorithm in the present invention under different numbers of drones. DETAILED DESCRIPTION

[0132] The embodiments of the present invention are described in further detail below with reference to the accompanying drawings.

[0133] The present invention discloses a low-altitude intelligent network task optimization method based on auction and diffusion learning, which comprises the following steps:

[0134] S1, builds a collaborative computing network framework consisting of multiple drones, ground base stations and aerial base stations to support task processing, edge collaboration and efficient offloading in dynamic task environments.

[0135] S2 establishes a multi-link communication model covering drones, ground base stations, and aerial base stations, and performs structured modeling of the task's data size, computing requirements, and deadline. A two-layer time slot partitioning strategy is adopted to bind drone task areas on a large time scale and dynamically decide on the unloading destination on a small time scale.

[0136] S3, based on the Vickrey-Clarke-Groves (VCG) auction mechanism, calculates the utility value of each drone in different areas and allocates task areas based on energy consumption estimation and task completion capability.

[0137] S4, designs a diffusion model-enhanced heterogeneous agent proximal policy optimization algorithm (D-HAPPO), which uses latent variable diffusion and conditional denoising mechanisms to generate unloading and movement strategies to achieve efficient collaboration of heterogeneous agents.

[0138] In step S1, the integrated air-ground collaborative network (IAGCN) consists of an aerial base station, multiple aerial drones, and several ground base stations. The IAGCN supports drones in performing complex tasks such as power inspections, agricultural monitoring, and emergency rescue within a designated mission area. As high-altitude nodes, aerial base stations provide wide coverage and powerful computing capabilities; ground base stations serve as stable computing and communication resource nodes on the ground and have high processing capabilities. Tasks are generated within designated mission areas and triggered when drones fly into these areas. Due to the limited computing power and energy of drones themselves, tasks usually need to be offloaded to nearby ground base stations or aerial base stations for collaborative processing. Single-hop task forwarding is also supported between drones, enabling more flexible resource sharing and dynamic scheduling.

[0139] In this embodiment, the drone set is represented as The ground base station set is represented as For aerial base stations The system's running time is divided into discrete time slots The length of each time slot is τ. The position of UAV u at time t is represented by l u (t)=(x u (t),y u (t),z u (t)), where x u (t),

[0140] y u (t) and z u (t) represents the horizontal, vertical and height coordinates respectively. Similarly, the position of the ground base station b is represented by l b (t)=(x b (t),y b (t),z b (t)), the coordinates of the aerial base station are expressed as l h (t)=(x h (t),y h (t),z h (t)).

[0141] Figure 1 This is a schematic diagram of the system structure of the multi-layer unmanned system task collaborative offloading and trajectory planning in the medium and low altitude intelligent network of the present invention. Figure 1As shown, the system comprises ground base stations, aerial base stations, drone clusters, edge servers, and multiple mission areas, forming a distributed, multi-layered intelligent collaborative system. The system is deployed in typical mission scenarios such as urban emergency response, agricultural inspections, and disaster monitoring. It achieves data exchange and computing power sharing by building integrated ground-to-air communication links. During mission execution, each mission area continuously generates heterogeneous computing tasks, such as high-resolution image processing, target recognition, and dynamic trajectory tracking. Ground base stations serve as stable resource nodes, providing highly reliable data relay and computing services. Aerial base stations, deployed on high-altitude platforms, boast wide coverage and strong computing power. They are primarily responsible for receiving relayed tasks from drones and coordinating airspace resources. Multiple distributed drones autonomously patrol the mission area, dynamically deciding whether to process tasks locally, offload to nearby drones, or forward them to ground or aerial base stations based on network status and node resource availability.

[0142] In step S2, to address the complexity of jointly optimizing UAV trajectory planning and task offloading decisions, this step adopts a time-scale separation strategy. This time-scale separation strategy decomposes the original problem into two interrelated sub-problems, enabling more efficient handling of large-scale spatial coordination and real-time response offloading decisions. At large time scales, due to the physical limitations of UAV speed and acceleration, their trajectory planning is usually carried out within a long time interval or planning cycle. At small time scales, task offloading decisions are made dynamically at each time slot and adjusted based on instantaneous observation information (such as channel conditions, queue status, and computing resource availability).

[0143] Modeling drone trajectories over a large time scale aims to minimize flight energy consumption while ensuring effective coverage of the mission area. To this end, it is necessary to define the mission area, the drone's movement pattern, and its long-term energy consumption model. This includes:

[0144] In the proposed system, the task area is modeled as a circular area on a two-dimensional plane. Each task area With position l m (t)=(x m ,y m ) is the center and its radius is r m The definition is as follows:

[0145]

[0146] Each task area initializes a task queue at time slot t=0 It contains N m Tasks:

[0147]

[0148] Each task a m,n Use triples Indicates that d m,n is the data size, c m,n is the amount of computation required per bit (expressed in CPU cycles), The number of deadline slots for the task:

[0149]

[0150] To improve energy efficiency, we assume that the UAV operates at a fixed cruising altitude H to avoid frequent lifting and lowering operations. Therefore, we model the UAV's motion as a two-dimensional horizontal movement above the mission area. The flight energy consumption of UAV u in time slot t consists of two parts: hovering power consumption and and mobile power consumption The expression for hovering power consumption is:

[0151]

[0152] Among them, M u is the mass of the drone, g is the acceleration due to gravity, ρ u is the propeller radius, p u is the number of propellers, is the air density. u (t) When moving, additional mobile power consumption will be generated:

[0153]

[0154] in is the maximum flight speed, is the power consumption at maximum speed, is the static power consumption when hovering (ie v u (t) = 0). Let δ u (t) represents the horizontal displacement of UAV u in time slot t, and δ u (t)=‖l u (t+1)-l u (t)‖. The total energy consumption of flight is:

[0155]

[0156] In order to describe the relationship between the UAV and the mission area, a binary variable z is introduced. u,m (t), indicates whether UAV u is associated with the mission area at time slot t

[0157]

[0158] Let κ αrepresents the energy consumption coefficient of node α performing CPU / GPU calculation, where That is, it represents other drones, ground base stations or air base stations. To determine whether drone u physically reaches the mission area at time slot t Introducing accessibility indicator variables It is defined as follows:

[0159]

[0160] Communication between the unmanned vehicle, the ground base station, and the aerial base station is an integral part of the task offloading process. The communication model determines the data transmission rate during task offloading. At small time scales, the network topology is considered quasi-static within each time slot, meaning that the channel state information remains approximately constant within the current time slot and is updated at the beginning of each time slot based on the latest parameters. The path loss between the drone u and the aerial base station is:

[0161]

[0162] Among them, u,h (t) is the elevation angle, f c is the carrier frequency, and C is the speed of light. Therefore, the available transmission rate from drone u to the aerial base station is:

[0163]

[0164] in, is the allocated bandwidth, P u (t) is the transmission power, N G is the noise power. The path loss between the UAV and other nodes (ground base stations or other UAVs) is modeled as:

[0165]

[0166] in The corresponding transmission rates are:

[0167]

[0168] Task a m,n Whether the time slot t is processed locally by UAV u is represented by a binary variable:

[0169]

[0170] The completion time of the local processing of this task is:

[0171]

[0172] in, Schedule the time for the task to start, is the waiting time, defined as:

[0173]

[0174] in represents the local task queue of drone u at time slot t, f u is the CPU frequency of the drone (unit: cycles / second). The task queue is updated as follows

[0175]

[0176] The local processing energy consumption is:

[0177]

[0178] in, is the effective switching capacitance coefficient of the UAV, and τ is the calculation period. To uniformly represent the unloading behavior, define the variable in Represents task a m,n Whether to uninstall to node α:

[0179]

[0180] The task completion time after uninstallation is:

[0181]

[0182] The waiting time is:

[0183]

[0184] The overall completion time can be expressed as:

[0185]

[0186] After the task is unloaded, the queue is updated as follows:

[0187]

[0188] Unloading energy consumption is:

[0189]

[0190] Each task can only be processed locally or offloaded once, subject to the following constraints:

[0191]

[0192] Defining a task success set In the mission area The set of tasks that successfully completed before the deadline:

[0193]

[0194] On this basis, a joint optimization problem is constructed. The total energy consumption of UAV u in time slot t is composed of the following three parts: flight energy consumption, computing energy consumption and communication energy consumption; task a m,n The processing energy consumption is:

[0195]

[0196] The total energy consumption of UAV u is:

[0197]

[0198] The total energy consumption of the system is:

[0199]

[0200] Define the energy efficiency index η, which represents the amount of data successfully processed per unit energy consumption: For all mission areas, It is the set of tasks that are successfully completed before the deadline in the area.

[0201]

[0202] To maximize the system's energy efficiency and the proportion of tasks completed on time in an integrated air-ground collaborative network, we jointly optimize the connectivity relationship z between UAVs and mission areas and the task offloading decision x, subject to multiple constraints.

[0203]

[0204] Where η represents the overall energy efficiency of the system, which is defined as the total amount of successfully processed task data (unit: bit) divided by the total energy consumption of the system. Indicates the task area The number of tasks completed on time in time slot t, N m is the total number of tasks within the region. Therefore, the sum reflects the average completion rate of tasks across time and space. The parameter β is a tuning factor used to balance task reliability with system energy efficiency. The constraints are as follows: Ensure that a UAV can only establish a task offloading connection after physically arriving at a task region; Ensure that both local computing and task offloading are permitted only when the UAV is within the corresponding task region.

[0205] In the task optimization framework proposed in this paper, the overall system energy efficiency η is defined as the total amount of data successfully processed per unit energy consumption, combined with the proportion of tasks completed on time in each task area to form a joint optimization objective function It is used to simultaneously measure the system's energy efficiency and mission completion reliability. A joint optimization problem model serves as the core objective function, defining the UAV-task area binding relationship (represented by variable z) and the task offloading decision (represented by variable x) as the core optimization objects. The optimization objective is the weighted sum of system energy efficiency and on-time task completion rate. In subsequent steps, an auction algorithm based on the VCG mechanism is introduced at large timescales (including multiple time slots). By calculating the expected utility value based on energy efficiency and mission success rate, the auction mechanism is used to solve for variable z, determining the optimal connection between each UAV's different mission areas within each large timescale cycle, thus achieving UAV task area binding at large timescales. Furthermore, a policy optimization method integrating a latent variable diffusion model is designed at small timescales (each time slot) to guide UAVs to dynamically select local processing or offloading targets in each time slot. The task offloading decision x is further solved, dynamically determining the task processing and offloading paths, influencing the formation of successful task sets and the change in system energy efficiency. This constitutes a comprehensive approach from problem modeling, task area allocation, to offloading path optimization.

[0206] Figure 2 This is a schematic diagram of the collaborative optimization process of multi-time scale task offloading and trajectory planning in the present invention. Figure 2 As shown, different types of tasks are generated concurrently from multiple areas and aggregated to the drone cluster through the task perception module, forming a queue of pending tasks. On a large timescale, the system uses variable z to establish relationships between drones and task areas and plan flight paths. On a small timescale, variable x is used to dynamically determine task offloading targets. Task processing strategies and path control are collaboratively optimized across two timescales, achieving a balance between mission success rate and energy efficiency.

[0207] In step S3, we propose an algorithm to evaluate the success rate and expected energy consumption of the UAV's mission within a specific mission area. By comparing the energy consumption required for local computation with that required for offloading to other nodes, the UAV can dynamically select the most energy-efficient execution mode while minimizing the mission's deadline.

[0208] First, the horizontal flight distance between the UAV and the center of the mission area is calculated, and the flight time and corresponding energy consumption are estimated based on the flight speed. For each task in the target area, the algorithm evaluates its local execution energy consumption and compares it with the communication energy consumption and remote computing energy consumption required to offload the task to other nodes (including other UAVs, ground base stations, and aerial base stations). The solution with the lowest total energy consumption is selected. If the task is completed before the deadline, it is considered a successful task. The number of successful tasks is used to estimate the task completion rate within the area, and the UAV's total energy consumption includes the total overhead of flight and computing / offloading.

[0209] To efficiently assign drones to mission areas within an integrated air-ground collaborative network, we propose an auction mechanism that balances energy efficiency and mission success rate. The algorithm consists of three main phases: utility evaluation and bidding, mission area allocation, and VCG pricing and path planning. Each drone u calculates its expected utility for each mission area based on its estimated energy consumption and mission success rate:

[0210]

[0211] in represents the estimated energy consumption, Represents the predicted task completion rate. Parameters γ1 and γ2 control the weight balance between energy consumption and task success rate. The corresponding bid value is:

[0212] b u (m) = βU u (m) (32)

[0213] Where β is a numerical scaling factor. After collecting all bids, the auctioneer assigns drones to mission areas in order of the highest bid value. Each drone is assigned to its optimal mission area, and the area assignment set and system utility are updated simultaneously.

[0214] After the task allocation is completed, the VCG mechanism is used to calculate the payment value (i.e. marginal utility contribution) of each drone:

[0215]

[0216] in represents the total system utility with all UAVs participating, represents the utility after removing drone u. While this payment value does not directly participate in the subsequent scheduling or path optimization process, it can be used to ensure the incentive compatibility of the auction results, ensuring that the optimal strategy for each drone is to truthfully report its estimated utility rather than manipulate bids. This mechanism encourages drones to submit bids truthfully, improving the incentive compatibility of allocations and ensuring the fairness and efficiency of task area allocation. Once the task is assigned, each drone plans a minimal flight energy route to the designated task area to execute the task and initiate the next phase of the offloading and scheduling process.

[0217] In step S4, based on the above hierarchical optimization framework, we further focus on the problem of drone task offloading at a small time scale. To characterize the decentralized and dynamic decision-making process, this problem is modeled as a Markov game, and a generative multi-agent reinforcement learning algorithm is proposed. This algorithm embeds the diffusion probability model into the multi-agent strategy network to improve the diversity of actions and environmental adaptability. In this setting, drones are modeled as N agents collaborating in a shared environment, and the task offloading process is formalized as a five-tuple The Markov game defined.

[0218] The state s of each agent n n (t) contains its own position l n (t), the location of the ground base station and the aerial base station l b (t),l h (t), mission area information And the task queue and Specifically:

[0219]

[0220] Each agent can only observe local and partially shared information. Its observation space includes:

[0221]

[0222] Action Space Indicates the optional offloading targets of the task currently assigned to UAV n, including local processing or offloading to other candidate nodes

[0223]

[0224] Reward Function Used to guide each drone to make a trade-off between energy consumption, timeliness, and mission completion:

[0225]

[0226] Among them E n is the total energy consumption of task processing or offloading, S n Indicates whether the task is successfully completed. The parameter α is used to penalize energy consumption, and γ is used to reward task success.

[0227] S43, we introduce a standard normally distributed latent variable It serves as a learnable action generation prior. During the forward diffusion process, the latent variable is scheduled according to the variance within T steps. Noise is gradually added to form a noise latent variable:

[0228]

[0229] The reverse process is performed by the denoising network D θ Modeling, the goal is to encode φ(o n ) and time step t, from z t Recover ∈ from . Its training goal is:

[0230]

[0231] In the inference phase, the trained denoiser is used to extract random noise samples z t Reverse sampling to obtain clear latent variables

[0232]

[0233] The latent variable is then decoded into the final action a through a deterministic mapping ψ n ,Right now This conditional generation process introduces a richer action space for the agent and enhances its adaptability in dynamic environments.

[0234] To ensure the collaborative convergence among multiple agents, this method embeds the potential diffusion model into the sequence update structure. m , the loss after clipping is:

[0235]

[0236] in It's an old strategy. is the recursively calculated composite advantage term:

[0237]

[0238] Where, Indicates the current strategy parameters of the mth agent The policy function under state s indicates taking action probability; represents the strategy parameters of the mth agent after the k+1th round update; is the action taken by the mth agent in the current state s; s is the current state of the environment, shared by all agents; Represents the old policy parameters of the mth agent in the kth iteration The strategy function under ; Clip is the clipping function, which is used to limit the update range of the strategy ratio to enhance the training stability, usually is the policy parameter θ of the first m-1 agents after update k+1 The probability of the joint strategy under . is the strategy parameter θ of the first m-1 agents before updating k The probability of the joint strategy under . The advantage function estimate of the joint action a in state s reflects the degree to which the action is better than the average strategy.

[0239] In step S45, each drone plans a minimal flight energy path to the designated mission area based on the mission area allocation results from step S322. During mission execution, it implements task offloading and dynamic scheduling according to the heterogeneous agent offloading optimization mechanism established in steps S41 to S44. This process combines local state observation, latent variable diffusion sampling, and policy update mechanisms to achieve collaborative offloading and policy convergence among multiple agents. After the current mission cycle ends, the system re-perceives the environmental state and mission generation status. All drones return to step S3 to initiate a new round of mission auctions and region allocation, forming a multi-timescale closed-loop optimization structure of "auction-execution-feedback-reauction."

[0240] This structure ensures that all agents share consistent update information while allowing them to generate their own stochastic policy representations through latent variable diffusion. This algorithm provides a unified framework that combines generative modeling capabilities with multi-agent learning mechanisms, offering a scalable solution for collaborative task scheduling of drones in edge intelligence systems.

[0241] Figure 3 Schematic diagram of the multi-time-scale unmanned system task optimization framework driven by auction and diffusion learning in the present invention. Figure 3 As shown in the figure, the overall system consists of a large-scale task allocation module and a small-scale offloading policy optimization module. The upper part illustrates the large-scale task allocation process based on the VCG mechanism. The drone generates bids based on mission observation information. The system allocates task areas and initializes trajectories based on the bids and utility calculation results. The lower part illustrates the small-scale task offloading decision process. This stage uses a diffusion model to construct an action sampling mechanism, generating policy actions through forward Gaussian perturbation and backward denoising. A comment network is used to evaluate the state-action value function, and a generative adversarial structure is introduced to enhance its representation capabilities, improving valuation accuracy and training stability in complex state spaces.

[0242] Figure 4 This is a graph showing the average reward change trend of the D-HAPPO algorithm under different learning rate settings. Figure 4 As shown in the figure, in a multi-agent task collaboration scenario, the algorithm's training performance under different learning rate parameters varies significantly. With learning rates of 0.001 and 5e-4, the algorithm achieves rapid convergence within the first 1 million steps and maintains a high average reward level in subsequent training stages, demonstrating good stability and policy effectiveness. In contrast, a learning rate that is too small results in slow convergence in the early stages of training, while a learning rate that is too large can easily cause fluctuations in the later stages of training, affecting system performance.

[0243] Figure 5 This is a comparison chart of the delay performance of the algorithm in the present invention for heterogeneous unmanned system tasks. Figure 5 The figure shows the comparative performance of various algorithms in terms of average task execution latency, with varying numbers of UAVs participating in collaborative tasks. As the number of UAVs increases from 2 to 10, the proposed method consistently maintains the lowest average latency, demonstrating good scalability and scheduling efficiency as the multi-agent scale increases. In contrast, traditional algorithms exhibit generally higher task latency, and some algorithms experience limited reduction as the number of UAVs increases, indicating their limited resource coordination capabilities in high-density environments.

[0244] Figure 6 This is a comparison chart of the energy efficiency of the algorithm in the present invention under different numbers of drones. Figure 6 As shown in the figure, as the number of drones in the system increases from 2 to 10, the method of the present invention always shows the highest energy efficiency among all algorithms, significantly outperforming the traditional benchmark algorithm. Especially in high-density intelligent agent scenarios (such as when the number of drones is 8 or 10), the strategy proposed by the present invention can better coordinate computational offloading and path planning tasks, thereby effectively reducing the energy consumption required per unit task. In contrast, the GM-SP algorithm has the lowest energy efficiency under all settings, indicating that it has large redundancy in resource scheduling and path control. HAPPO and LB-RBO are also gradually lagging behind in energy efficiency, and the growth rate is slow, making it difficult to adapt to the challenges brought by the expansion of system scale.

[0245] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0246] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A low-altitude intelligent network task optimization method based on auction and diffusion learning, characterized in that: The method comprises the following steps: S1, builds an integrated air-ground collaborative network consisting of multiple drones, ground base stations, and aerial base stations to support task processing, edge collaboration, and task offloading in dynamic mission environments; S2: Establish a multi-link communication model covering drones, ground base stations, and aerial base stations, and perform structured modeling of the task's data size, computing requirements, and deadline. A two-layer time slot partitioning strategy is used. On a large time scale, the drone's trajectory is modeled across multiple time slots, binding the drone to the task area. On a small time scale, task offloading decisions are dynamically modeled in each time slot, dynamically determining the offloading destination. With the optimization objectives of maximizing the system's energy efficiency and the weighted sum of the proportion of tasks completed on time in the integrated air-ground collaborative network, a joint optimization problem model is constructed to jointly optimize the connection relationship z between the drone and the task area, as well as the task offloading decision x. S3, on a large time scale, calculates the expected utility value of each drone in different mission areas by combining the estimated total energy consumption and mission completion capability of the drones at intervals of multiple time slots. Based on the Vickrey-Clarke-Groves auction mechanism, the connection relationship z in the joint optimization problem model is solved in combination with the expected utility value to complete the mission area allocation of the drones. S4, designs a diffusion model-enhanced heterogeneous agent proximal policy optimization algorithm. At a small time scale, it uses latent variable diffusion and conditional denoising mechanisms to solve the task offloading decision x for each time slot, and generates the task offloading and drone movement strategy for each time slot.

2. The low-altitude intelligent network task optimization method based on auction and diffusion learning according to claim 1 is characterized in that: Step S1 further comprises: Construct an integrated air-ground collaborative network, which includes an air base station, multiple air drones, and several ground base stations, where the drone set is represented as The set of ground base stations is represented as For aerial base stations The position of UAV u at time t is represented by l u (t)=(x u (t),y u (t),z u (t)), the position of the ground base station b is represented by l b (t)=(x b (t),y b (t),z b (t)), the coordinates of the aerial base station are expressed as l h (t)=(x h (t),y h (t),z h (t)); where x u (t), y u (t) and z u (t) represents the horizontal, vertical and height coordinates of the UAV u, respectively, and x b (t), y b (t) and z b (t) represents the horizontal, vertical and height coordinates of the ground base station b, respectively, and x h (t), y h (t) and z h (t) represents the coordinates of the aerial base station in the horizontal, vertical and height directions respectively; Divide the system's running time into discrete time slots The length of each time slot is τ; tasks are generated in the specified task area and triggered when the UAV flies to the corresponding task area. The UAV processes the task locally or offloads the task to a nearby ground base station or aerial base station for collaborative processing.

3. The low-altitude intelligent network task optimization method based on auction and diffusion learning according to claim 1 is characterized in that: In step S2, a multi-link communication model covering the UAV, ground base stations, and aerial base stations is established, and a structured modeling process is performed for the task data size, computing requirements, and deadline. The specific process includes the following steps: The task area is modeled as a circular area on a two-dimensional plane. With position l m (t)=(x m ,y m ) as the center, and is defined as follows: Where r m For the mission area radius; Each task area initializes a task queue at time slot t=0 It contains N m Tasks: Each task a m,n Use triples express, where d m,n is the data size, c m,n is the amount of computation required per bit, The number of deadline slots for the task.

4. The low-altitude intelligent network task optimization method based on auction and diffusion learning according to claim 1 is characterized in that: In step S2, a two-layer time slot division strategy is adopted to perform UAV mission area binding at a large time scale and dynamically decide the unloading destination at a small time scale. The process includes: A time scale separation strategy is adopted to decompose the original problem into two interrelated sub-problems. On a large time scale, the UAV's motion trajectory is modeled at intervals of multiple time slots, including the mission area definition, the UAV's movement pattern, and its flight energy consumption model. On a small time scale, the task offloading decision is dynamically modeled in each time slot, including channel conditions, queue status, and computing resource availability. The total energy consumption of the UAV and the total energy consumption of the system are calculated separately, and an energy efficiency index model is constructed to represent the amount of data successfully processed per unit energy consumption. With the optimization objectives of maximizing the system's energy efficiency and the weighted sum of the proportion of tasks completed on time in the integrated air-ground collaborative network, a joint optimization problem model is constructed to jointly optimize the connection relationship z between the UAV and the mission area and the task offloading decision x.

5. The low-altitude intelligent network task optimization method based on auction and diffusion learning according to claim 4 is characterized in that: In step S2, the process of modeling the trajectory of the UAV at a large time scale and multiple time slots includes the following steps: The motion trajectory of the UAV is modeled as a two-dimensional horizontal movement above the mission area. The flight energy consumption of the UAV u in time slot t consists of two parts: hovering power consumption and mobile power consumption The expression for hover power consumption is: Among them, M u is the mass of the drone, g is the acceleration due to gravity, ρ u is the propeller radius, p u is the number of propellers, is the air density; mobile power consumption The expression is: where v u (t) is the moving speed of the drone, is the maximum flight speed, is the power consumption at maximum speed, is the static power consumption when hovering; Let δ u (t) represents the horizontal displacement distance of UAV u in time slot t, δ u (t)=‖l u (t+1)-l u (t)‖, the total energy consumption of flight is: Introduce binary variable z u,m (t), indicates whether UAV u is associated with the mission area at time slot t 6. The low-altitude intelligent network task optimization method based on auction and diffusion learning according to claim 4 is characterized in that: In step S2, at a small time scale, the process of dynamically making task offloading decisions at each time slot includes the following steps: S21, introduction of accessibility indicator variables To determine whether the UAV u physically reaches the mission area at time slot t It is defined as follows: S22, the network topology is considered quasi-static in each time slot and is updated according to the latest parameters at the beginning of each time slot. The path loss between UAV u and the aerial base station is: Among them, u,h (t) is the elevation angle, f c is the carrier frequency, C is the speed of light; the available transmission rate from drone u to the aerial base station is: in, is the allocated bandwidth, P u (t) is the transmission power, N G is the noise power; The path loss between the UAV and other nodes is modeled as: in The corresponding transmission rates are: S24, using binary variables Represents task a m,n Whether time slot t is processed locally by drone u: The completion time of the task processed locally is calculated as: in, Schedule the time for the task to start, is the waiting time, defined as: in represents the local task queue of drone u at time slot t, f u is the CPU frequency of the drone; the task queue is updated using the following formula: The local processing energy consumption is calculated as: in, is the effective switching capacitance coefficient of the UAV, and t is the calculation period; Defining variables Represents task a m,n Whether to unload to node α, where The calculated task completion time after unloading is: The waiting time is: The overall completion time is expressed as: After the task is unloaded, the task queue is updated using the following formula: The unloading energy consumption is calculated as: In the formula, let κ α represents the energy consumption coefficient of node α performing CPU / GPU calculation, where Represent other UAVs, ground base stations or aerial base stations respectively; each task can only be processed or offloaded locally once, with the following constraints: S25, define the task success set Used to indicate that in the task area The set of tasks that successfully completed before the deadline:

7. The low-altitude intelligent network task optimization method based on auction and diffusion learning according to claim 4 is characterized in that: In step S2, the process of constructing the joint optimization problem model includes the following steps: The total energy consumption of UAV u in time slot t is calculated using the following formula: In the formula Represents task a m,n The processing energy consumption, Indicates the total energy consumption of flight; The total energy consumption of the system is calculated using the following formula: Define the overall energy efficiency η of the system, which is used to represent the amount of data successfully processed per unit energy consumption: in For all mission areas, For the mission area The set of tasks that are successfully completed before the intrinsic deadline; Constructing a joint optimization problem model: in, Indicates the task area The number of tasks completed on time in time slot t, N m For the mission area The total number of tasks in ; parameter β is the adjustment factor, which is used to balance the weight between task reliability and system energy efficiency.

8. The low-altitude intelligent network task optimization method based on auction and diffusion learning according to claim 1 is characterized in that: Step S3 further comprises: S31 calculates the horizontal flight distance between the UAV and the center of the mission area, and estimates the flight time and corresponding flight energy consumption based on the flight speed. For each task in the target area, evaluates its local execution energy consumption, compares the local execution energy consumption with the communication energy consumption and remote computing energy consumption required to offload the task to other nodes, and selects the solution with the lowest total energy consumption as the estimated UAV energy consumption. The UAV's total energy consumption includes the total overhead of flight and computing / offloading. Calculate the number of successful tasks and estimate the task completion rate in the region; if a task is completed before the deadline, it is considered a successful task; S32, combining the estimated total energy consumption of the drones and the mission completion rate to calculate the expected utility value of each drone in different mission areas. Based on the Vickrey-Clarke-Groves auction mechanism, the connection relationship z in the joint optimization problem model is solved in combination with the expected utility value. Specifically, the following sub-steps are included: In step S321, each UAV u calculates its expected utility value for each mission area based on the estimated UAV energy consumption and mission success rate: in represents the estimated total energy consumption of the drone, represents the estimated task completion rate; parameters γ1 and γ2 control the weight balance between energy consumption and task success rate; the corresponding bid value is: b u (m)=βU u (m) Where β is the numerical scaling factor; In step S322, after collecting the bids from all drones in step S321, the auctioneer assigns each drone to its optimal task area based on its expected utility value for the task area and the highest bid value. The auctioneer also updates the distribution of task areas and the overall energy efficiency of the system. After the assignment is completed, the VCG mechanism is used to calculate the payment value of each drone as its marginal contribution from participating in the task assignment. The specific formula is as follows: in represents the total system utility with all UAVs participating, represents the utility after removing the drone u; In step S323, each UAV plans a minimum flight energy consumption path according to the assigned mission, and after going to the designated mission area to perform the mission, it enters step S4 to start the task offloading and scheduling process under a small time scale.

9. The low-altitude intelligent network task optimization method based on auction and diffusion learning according to claim 1 is characterized in that: Step S4 further comprises: S41, the UAV task offloading problem at a small time scale is modeled as a Markov game problem, where the UAVs are modeled as N intelligent agents collaborating in a shared environment, and the task offloading process is formalized as a five-tuple The Markov game defined by γ are the state, observation space, action space, reward function and reward parameter of the agent respectively; Specifically, the state s of each agent n n (t) contains its own position l u (t), the location of the ground base station and the aerial base station l b (t) and l h (t), mission area information And the task queue and Specifically: Each agent can only observe local and partially shared information, and its observation space is: Action Space Indicates the optional offloading targets of the task currently assigned to drone n, including local processing or offloading to other nodes Reward Function Used to guide each drone to make a trade-off between energy consumption, timeliness, and mission completion: Among them E n is the total energy consumption of task processing or offloading, S n Indicates whether the task is successfully completed; parameter α is used to penalize energy consumption, and parameter γ is used to reward task success; S43, introduce the latent variable of standard normal distribution As a learnable action generation prior; in the forward diffusion process, the latent variable is scheduled according to the variance within T steps Gradually add noise to form the noise latent variable z t : Among them, ∈ is the diffusion noise term, which is used to represent the randomness introduced in each diffusion process; Training loss Used to optimize the denoising network D θ , so that it can be used under a given noise latent variable z t , observation code φ(o n ) and time step t, predict the diffusion noise ∈ added in the forward process as accurately as possible; the specific form is as follows: In the inference phase, the trained denoiser is used to extract random noise samples z t Reverse sampling to obtain clear latent variables The latent variables sampled in reverse are transformed into Decoded as final action a n , S44, embed the potential diffusion model into the sequence update structure, for each agent i m , the loss after clipping is: in It's an old strategy. is the composite advantage term computed recursively: Where, Indicates the current strategy parameters of the mth agent The policy function under state s indicates taking action probability; represents the strategy parameters of the mth agent after the update in the k+1th round; is the action taken by the mth agent in the current state s; s is the current state of the environment, shared by all agents; Represents the old policy parameters of the mth agent in the kth iteration The strategy function under ; Clip is the clipping function used to limit the update range of the strategy ratio; is the policy parameter θ of the first m-1 agents after update k+1 The probability of the joint strategy under ; is the strategy parameter θ of the first m-1 agents before updating k The probability of the joint strategy under ; The advantage function estimate of the joint action a in state s reflects the degree to which the action is better than the average strategy.

Images