Crowd sensing network resource management optimization method and system based on unmanned aerial vehicle assistance

Abstract

The invention discloses a crowd sensing network resource management optimization method and system based on unmanned aerial vehicle assistance, and relates to the technical field of wireless communication, and the method comprises the steps: obtaining user state data and unmanned aerial vehicle state data; wherein the user state data comprises a user position, maximum energy consumption, a sensing rate and an unmanned aerial vehicle position; the unmanned aerial vehicle state data comprises an unmanned aerial vehicle position, maximum energy consumption, a user position and other unmanned aerial vehicle positions; and inputting the user state data and the unmanned aerial vehicle state data into a pre-established joint optimization framework model, converting a joint optimization problem into a partially observable Markov decision process, solving based on a multi-agent deep reinforcement learning algorithm of heterogeneous agent near-end strategy optimization, and outputting to obtain a joint optimization result. And based on a joint optimization result, maximization of the sensing data processing amount in crowd sensing network resource management is realized.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, specifically to a method and system for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicles (UAVs). Background Technology

[0002] With the rapid development of smart city technologies and multi-access edge computing technologies, intelligent applications such as virtual reality and autonomous driving have been effectively promoted. Mobile crowdsourcing sensing, as an emerging paradigm, collects sensing data through mobile user collaboration, providing data support and a service foundation for various application scenarios. Compared to traditional sensor networks, mobile crowdsourcing sensing, relying on mobile user-driven data collection, not only reduces infrastructure deployment costs but also offers greater sensing coverage and flexibility. However, in traditional crowdsourcing sensing systems, ground-based sensing devices often struggle to effectively perform sensing tasks in complex or extreme environments or disaster-damaged areas. The development of unmanned aerial vehicle (UAV) technology offers a new opportunity to address these issues. With their high mobility and flexible deployment capabilities, UAVs can significantly improve the coverage and efficiency of crowdsourcing sensing services. Furthermore, UAVs equipped with high-precision sensors and onboard computing units can achieve more efficient data collection over a wider area in dynamic and complex environments.

[0003] Despite significant progress in UAV-assisted swarm intelligence networks in recent years, several key challenges remain. First, existing joint sensing-communication-computing frameworks neglect the computational coupling between users and UAVs, limiting data processing efficiency and failing to fully leverage the potential of collaborative computing resources. Second, in heterogeneous environments, the optimal time slot allocation between the sensing, communication, and computing phases dynamically adjusts with changes in network state and resource capabilities, necessitating an efficient and adaptive time slot scheduling and resource allocation strategy. Furthermore, the dynamic mobility of users further increases system uncertainty, making the network optimization problem strongly non-convex and high-dimensional, difficult to solve effectively with traditional optimization methods, resulting in low data management capabilities and resource utilization efficiency. Summary of the Invention

[0004] To address the shortcomings mentioned in the background section, the present invention aims to provide a method and system for optimizing resource management of a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance.

[0005] Firstly, the objective of this invention can be achieved through the following technical solution: a method for optimizing resource management of a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance, the method comprising the following steps: Acquire user status data and drone status data; wherein, the user status data includes user location, maximum energy consumption, sensing rate, and drone location; the drone status data includes its own drone location, maximum energy consumption, user location, and the locations of other drones; User status data and UAV status data are input into a pre-established joint optimization framework model. The joint optimization problem is transformed into a partially observable Markov decision process. The solution is obtained by a multi-agent deep reinforcement learning algorithm based on heterogeneous agent proximal policy optimization. The joint optimization result is output. Based on the joint optimization result, the amount of perception data processed in the resource management of the swarm intelligence perception network is maximized. The joint optimization results include optimal time slot allocation, resource allocation, and UAV 3D trajectory planning schemes; the pre-established joint optimization framework model includes a time slot partitioning model for time slot coordination and a system resource joint optimization problem model with the goal of maximizing the amount of processed sensing data.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the partially observable Markov decision process includes two types: user agents and drone agents; wherein the composition of the three elements of user agents and drone agents is different, and the three elements include state, action and reward; The reward function consists of two parts: the objective function term and the penalty term. Each agent selects the corresponding reward term according to the constraints and designs the penalty term function and penalty factor in a unified manner to achieve effective expression and optimization of the constraints.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the multi-agent deep reinforcement learning algorithm based on heterogeneous agent proximal policy optimization is improved by fusing convolutional neural networks and Kolmogorov-Arnold networks, wherein the convolutional neural network is used for feature extraction and the Kolmogorov-Arnold network is used to describe the nonlinear relationship between states and actions.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the joint optimization problem is defined as P1, as follows: Among them, constraints C1–C3 are used to limit the three-dimensional flight space range of the UAV. and These represent the side lengths of the rectangular region, and These represent the minimum and maximum flight altitudes of the UAV, respectively; C4–C5 are used to limit the allocation of communication bandwidth, where Indicates drone Distributed to users who establish communication connections bandwidth, This represents the total bandwidth available to the drone; C6–C8 collectively constrain user computing resources. And the computing resource allocation relationship between the user and the drone, where constraint C8 specifies the computing resources allocated to the user by the drone. It cannot exceed the total computing resources of the drone. C9–C10 constrain the communication connection variables between the user and the drone. Constraints C11 and C12 respectively limit the amount of data that can be processed within each time slot. Constraint C11 guarantees the amount of data processed by the user, including the amount of data processed locally. and the amount of data transmitted to the drone No more than the amount of data it perceives. Constraint C12 guarantees the amount of data processed by the drone. The flight speed of the UAV shall not exceed the amount of data it receives; constraints C13–C15 respectively constrain the flight speed of the UAV. acceleration and the minimum spacing between drones ;in, and These represent the maximum values ​​of the drone's speed and acceleration, respectively. and They represent drones and Exclusion The positions of other drones; C16 is used to constrain the time partitioning variable. , The range of values, while C17–C18 limits the energy consumption of both users and drones to not exceed their maximum energy limit, respectively using and express.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the three elements of the user intelligent agent and the drone intelligent agent are as follows: User intelligent agent: State: The state of the user's intelligent agent includes its own performance indicators and external environmental information. The performance indicators include user location, maximum energy consumption, and sensing rate, while also considering the location of the drone. The state is represented as: Actions: After observing the environment, the user agent's actions include deciding on time slot allocation, computational resource allocation, and the association strategy with the drone. The action can be represented as: in, It is a binary variable; Reward: The reward function includes both the expected target term and the penalty term for not meeting the constraints. The target term consists of two parts: the amount of data processed locally by the user and the amount of data transmitted by the associated drone for computation. Therefore, the user... The reward function can be expressed as follows: To standardize the expression of the penalty term, the penalty function is defined as follows: in, Represented as ; Indicates user The penalty for violating the maximum energy consumption constraint is denoted as: ,in For relevant penalty factors, This indicates a penalty when the amount of data processed exceeds the total amount of perceived data. Its corresponding penalty factor, Right now A penalty will be imposed when the amount of data transmitted by a user is less than the processing capacity of its associated drone. Its definition is ; Unmanned aerial vehicle (UAV) intelligent agents: State: The state of a drone agent includes its own position and maximum energy consumption constraints, as well as the user's position and the positions of other drones. Therefore, the drone's state... The state is defined as follows: Action: In each time slot, the drone agent needs to allocate resources to its associated users and control its own flight acceleration to adjust its trajectory. The action is defined as follows: Reward: For drones The reward function considers both the amount of computational data completed by the associated user and the amount of data to be computed received, and also includes a penalty term for constraint violations. (Drone) The reward function is defined as follows: in, Indicates drone Penalties for violating energy consumption constraints If the corresponding penalty factor is... In addition, definition Indicates drone The penalty for violating the data volume constraint is in the following form: ,in This represents the penalty factor associated with violations of data processing constraints, introduced by the penalty term. Ensure the drone flies safely within the designated operating area. Fly out of the preset horizontal range At that time, its penalty items ;in, Indicates drone Horizontal position, This indicates the penalties imposed on drones that violate the minimum safe distance, among which This indicates the penalty coefficient for collision violations.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the solution of the partially observable Markov decision process is simplified into a subproblem related to the user's computational frequency, defined as P2, as follows: By launching The optimal solution is expressed as .

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the convolutional neural network is used to extract key features from the input observation sequence, and the model complexity is reduced through a convolutional kernel parameter sharing mechanism; The Kolmogorov-Arnold network is used to generate agent actions. By introducing a learnable spline function to construct a residual activation function, and by optimizing the spline coefficients, the agent can generate reasonable actions based on different observation information.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: an improvement process for the penalty factor, comprising the following two categories: The first type of penalty factor is a penalty factor that is directly related to the optimization objective, and its constraints contain the same expression as the objective function; the second type of penalty factor is a penalty factor that has an indirect impact on the objective function. Although its constraints do not directly involve the objective function, they affect the overall optimization result by limiting the amount of data processing at each stage.

[0013] Secondly, in order to achieve the above objectives, this invention discloses a resource management and optimization system for a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance, comprising: The data acquisition module is used to acquire user status data and drone status data; wherein, the user status data includes user location, maximum energy consumption, sensing rate, and drone location; the drone status data includes its own drone location, maximum energy consumption, user location, and the locations of other drones; The management optimization module is used to input user status data and UAV status data into a pre-established joint optimization framework model, transform the joint optimization problem into a partially observable Markov decision process, solve it based on a multi-agent deep reinforcement learning algorithm for heterogeneous agent proximal policy optimization, output the joint optimization result, and maximize the amount of perception data processing within the resource management of the swarm intelligence perception network based on the joint optimization result. The joint optimization results include optimal time slot allocation, resource allocation, and UAV 3D trajectory planning schemes; the pre-established joint optimization framework model includes a time slot partitioning model for time slot coordination and a system resource joint optimization problem model with the goal of maximizing the amount of processed sensing data.

[0014] In another aspect of the present invention, in order to achieve the above-mentioned objective, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor. When the processor loads and executes the computer program, it employs the above-described method for optimizing resource management of a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance.

[0015] The beneficial effects of this invention are: This invention jointly utilizes the computing resources of users and drones, and performs joint optimization of the time slot allocation of the three stages of perception, communication and computing under heterogeneous constraints. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the model structure of the present invention; Figure 3 This is a schematic diagram of the time slot division scheme of the present invention; Figure 4 This is a schematic diagram of the algorithm framework of this invention; Figure 5 This is a schematic diagram illustrating the convergence of the user agent reward function under different comparison algorithms according to the present invention; Figure 6 This is a schematic diagram illustrating the convergence of the reward function of the UAV agent under different comparison algorithms according to the present invention; Figure 7 This is a schematic diagram illustrating the amount of perceived data processed by the present invention under different numbers of users; Figure 8 This is a schematic diagram showing the amount of sensing data processed by the present invention at different average sensing rates; Figure 9 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1: like Figure 1 As shown, a method for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicles (UAVs) includes the following steps: Specifically, in this embodiment of the invention, the target area is comprised of an internet-based sensing platform. A drone and The system consists of several users. The area publishes task requests through an internet-based sensing platform and selects suitable users based on task requirements and real-time user status to improve task completion efficiency. Then, drones are dispatched to the selected user locations to collect sensing data. When the drone communicates with the user, the following steps are performed: S101: Acquire user status data and drone status data; wherein, the user status data includes user location, maximum energy consumption, sensing rate, and drone location; the drone status data includes its own drone location, maximum energy consumption, user location, and the locations of other drones; S102: Input user status data and UAV status data into a pre-established joint optimization framework model, transform the joint optimization problem into a partially observable Markov decision process, solve it using a multi-agent deep reinforcement learning algorithm based on heterogeneous agent proximal policy optimization, output the joint optimization result, and maximize the amount of sensing data processed within the resource management of the swarm intelligence sensing network based on the joint optimization result. The joint optimization results include optimal time slot allocation, resource allocation, and UAV 3D trajectory planning schemes; the pre-established joint optimization framework model includes a time slot partitioning model for time slot coordination and a system resource joint optimization problem model with the goal of maximizing the amount of processed sensing data.

[0019] Partially observable Markov decision processes include two types: user agents and drone agents. The three elements of user agents and drone agents are different, and these three elements include state, action, and reward. The reward function consists of two parts: the objective function term and the penalty term. Each agent selects the corresponding reward term according to the constraints and designs the penalty term function and penalty factor in a unified manner to achieve effective expression and optimization of the constraints.

[0020] The multi-agent deep reinforcement learning algorithm based on heterogeneous agent proximal policy optimization is improved by fusing convolutional neural networks and Kolmogorov-Arnold networks. The convolutional neural network is used for feature extraction, and the Kolmogorov-Arnold network is used to describe the nonlinear relationship between state and action. The joint optimization problem is defined as P1, as follows: Among them, constraints C1–C3 are used to limit the three-dimensional flight space range of the UAV. and These represent the side lengths of the rectangular region. and These represent the minimum and maximum flight altitudes of the UAV, respectively; C4–C5 are used to limit the allocation of communication bandwidth, where Indicates drone Distributed to users who establish communication connections bandwidth, This represents the total bandwidth available to the drone; C6–C8 collectively constrain user computing resources. And the computing resource allocation relationship between the user and the drone, where constraint C8 specifies the computing resources allocated to the user by the drone. It cannot exceed the total computing resources of the drone. C9–C10 constrain the communication connection variables between the user and the drone. This ensures that each user can communicate with at most one drone in any given time slot. Constraints C11 and C12 limit the amount of data that can be processed in each time slot. Specifically, constraint C11 guarantees the amount of data processed by the user, including data processed locally. and the amount of data transmitted to the drone No more than the amount of data it perceives. Constraint C12 guarantees the amount of data processed by the drone. The flight speed of the UAV shall not exceed the amount of data it receives; constraints C13–C15 respectively constrain the flight speed of the UAV. acceleration and the minimum spacing between drones .in, and These represent the maximum values ​​of the drone's speed and acceleration, respectively. and They represent drones and Exclusion The positions of other drones; C16 is used to constrain the time partitioning variable. , The range of values, while C17–C18 limits the energy consumption of both users and drones to not exceed their maximum energy limit, respectively using and express; The three essential elements of user intelligent agents and drone intelligent agents are as follows: User intelligent agent: State: The state of the user's intelligent agent includes its own performance metrics and external environmental information. The performance metrics include user location, maximum energy consumption, and sensing rate, while also considering the location of the drone. Therefore, the user... The state is represented as: Action: After observing the environment, the user will determine the time slot allocation, computational resource allocation, and the association strategy with the drone. The action can be represented as: in, Since the variable is binary, to simplify the solution process, it is relaxed to a continuous variable with a value range between 0 and 1. This relaxation treatment is helpful for the design and implementation of subsequent algorithms.

[0021] Reward: To design an effective decision-making strategy, the reward function includes both the expected objective term and the penalty term for not meeting the constraints. The objective term consists of two parts: the amount of data processed locally by the user and the amount of data transmitted to its associated drone for computation. Therefore, the user's reward function can be expressed as follows: To standardize the expression of the penalty term, the penalty function is defined as follows: in, Represented as Based on this definition, Indicates user The penalty for violating the maximum energy consumption constraint is represented as follows: ,in This is a relevant penalty factor. Furthermore, This indicates a penalty when the amount of data processed exceeds the total amount of perceived data. This corresponds to a penalty factor. Furthermore, a penalty will be imposed when the amount of data transmitted by a user is less than the processing capacity of its associated drone. , It is the defined penalty factor.

[0022] Unmanned Aerial Vehicle (UAV) State: The state of a drone agent includes its own position and maximum energy consumption constraints, as well as the user's position and the positions of other drones. Therefore, the drone... The state is defined as follows: Actions: Each drone needs to allocate resources appropriately to its associated users; furthermore, its movement is determined by the control of the drone's acceleration. Therefore, drones... The action is defined as follows: Reward: For drones Its reward function considers both the amount of computational data completed by the associated user and the amount of data to be computed received. Furthermore, it includes penalty terms for violations of the constraint.

[0023] Therefore, drones The reward function is defined as follows: in, Indicates drone Penalties for violating energy consumption constraints If the corresponding penalty factor is... Furthermore, the definition Indicates drone Penalties for violating computational data volume constraints, among which This represents a penalty factor associated with violations of data processing constraints. Penalty terms are introduced to ensure the safe flight of drones within designated operating areas. When drones Fly out of the preset horizontal range At that time, its penalty items .in, Indicates drone The horizontal position. Furthermore. This indicates the penalties imposed on drones that violate the minimum safe distance, among which This indicates the penalty coefficient for collision violations.

[0024] Analysis of the penalty terms for the agents revealed that both user agents and drone agents contain multiple penalty factors, some of which directly influence the objective function and play a crucial role in the convergence performance of the training process. Based on this, penalty factors are categorized into two types, and the design of penalty factors directly affecting the objective function is improved using the interior-point method to enhance the model's convergence and overall optimization performance.

[0025] The process of improving the penalty factor includes the following two categories: The first type of penalty factor is directly related to the optimization objective, and its constraints contain the same expression as the objective function. The second type of penalty factor has an indirect impact on the objective function; although its constraints do not directly involve the objective function, they affect the overall optimization result by limiting the amount of data processing at each stage. This method focuses on designing the first type of penalty factor, including... and The value of the penalty factor directly affects the data processing volume in each time slot. When the penalty factor is too large, it leads to a decrease in the reward value, thereby reducing the learning efficiency of the agent; when the value is too small, the constraint effect is insufficient, and the rationality of the action strategy cannot be guaranteed. To determine a reasonable value for the penalty factor, this invention proposes a design method combining logarithmic scaling and rounding operations to construct an exponentially growing penalty coefficient, thereby improving the stability and optimization performance of the model. To illustrate this method, this invention uses the penalty factor... For example: First, calculate The logarithm of the result is calculated, and the result is rounded to obtain an integer value. The rounded value is denoted as . .in, This is a very small value, used to prevent illegal mathematical operations. This represents the floor function. Subsequently, the same method is used to... Perform the calculation, and define the result as follows: .at last, The following inequality can be used to obtain Among them, the first type of penalty factor and The value of was determined through experimental analysis, and the values ​​of the remaining penalty factors were set to 0.1.

[0026] To reduce the training burden, the user's computation frequency can be directly calculated when other variables are output through the network at each step. This invention simplifies the original problem P1 into a linear programming problem concerning the user's computation frequency, defined as P2, as follows: because Then this objective function is about It is a linear programming function. Furthermore, this invention analyzes constraints C11 and C17. First, analyzing constraint C11, it can be transformed into the following inequality: Similarly, analyzing constraint C17, regarding The inequality can be expressed as follows: Therefore, question P2 is about For linear programming problems, this invention can directly introduce their optimal solutions. For ease of representation, let... and Let C11 and C17 represent the right-hand terms of inequalities C11 and C17, respectively. Since the objective function is about... It is a monotonically increasing function, therefore the optimal value is... It can be represented as: In addition, due to Indicates user The CPU operating frequency, therefore, must be a non-negative value in practice.

[0027] Therefore, the optimal solution can be further expressed as: By observing the state definitions of users and drones, each agent needs to perceive not only its own state but also the states of other agents. As the number of agents increases, the observation space dimension expands significantly, thus increasing the input complexity of the policy network. Traditional MLP-based policy networks, being fully connected, lack the ability to extract observed features, and their parameter count increases linearly with the input dimension, easily leading to overfitting and reduced policy learning performance. Therefore, this invention designs a policy network for the HAPPO algorithm, integrating a hybrid policy network structure of one-dimensional CNN and KAN, forming a new algorithm, CKAN-HAPPO. Specifically, this network structure consists of an input layer, a one-dimensional convolutional layer, a pooling layer, and a KAN layer. The fully connected output layer of the original CNN is replaced by a KAN. The convolutional layer has a kernel size of 3 and a stride of 1; the convolutional layer uses a one-dimensional convolution with a kernel size of 3 and a stride of 1; the KAN uses a third-order spline function, and the number of internal node intervals is set to 5. In this architecture, the CNN module extracts key features from the input observation sequence, namely the states of the user and drone agents, and effectively reduces model complexity through a kernel parameter sharing mechanism. The KAN module replaces the traditional MLP as the output layer, generating the actions of the user and drone agents, i.e., the model's optimization variables. The KAN module effectively models complex nonlinear state-action dependencies while maintaining fewer model parameters. Its core employs a residual activation function, which is composed of learnable spline functions, defined as... , means as follows: in, Indicates trainable weights. The basis functions are defined by B-splines and their trainable coefficients. By optimizing these correlation coefficients, KAN can generate appropriate actions based on diverse observations from different agents. This hybrid structure can more fully characterize the complex dependencies between states and actions, thereby enhancing the agent's policy representation capabilities and improving overall performance and generalization ability in structurally similar time-slot tasks.

[0028] The proposed CKAN-HAPPO algorithm is used to train a multi-agent policy model offline to obtain a joint decision-making strategy between user agents and UAV agents. The model employs a cascaded structure of a CNN feature extraction module and a KAN action output module: the input layer receives the observation sequences from each agent; a one-dimensional convolutional layer (with 3 kernels and a stride of 1) extracts features from the observation sequences, with the composition of the observation sequences differing for different types of agents. For user agents, the observation sequence includes state information such as user position, maximum energy consumption, sensing rate, and UAV position. For UAV agents, the observation sequence includes state information such as the position of the current UAV, the positions of other UAVs, maximum energy consumption, and user position. The convolutional output is then dimensionality-reduced by a pooling layer and input into the KAN layer as the output layer. The KAN uses a third-order spline function with an internal node spacing of 5 to generate action vectors corresponding to the optimization variables of this application; similarly, the output action variables differ for different agents. For user agents, the output includes time slot partitioning variables, user computing resource allocation, and user-UAV association. For the UAV agent, the output includes UAV motion control (acceleration) and bandwidth and computing resource allocation to associated users. During training, this invention sets the maximum number of training rounds to 6000. In each time slot of each round, each agent obtains an action based on the current observation through forward inference via the aforementioned network. The environment executes the action and returns the next time slot observation and reward. The reward consists of the utility term corresponding to the objective function and the constraint violation penalty term, guiding the policy to gradually approach the system optimization objective while satisfying constraints such as communication, computing, and energy consumption. The algorithm adopts an actor-critic framework with centralized training and distributed execution. The actor and critic networks update parameters using gradient ascent and gradient descent, respectively, with learning rates set to 0.0002 and 0.0001. The training metrics include the stability of the cumulative reward and the moving average of the objective function value, the constraint violation rate, and the network loss. When the above training metrics meet the convergence criterion or the maximum number of training rounds is reached, the training is considered converged and the iteration is terminated. After training, the policy network and value network parameters of the user and the drone agent are stored for rapid inference output in subsequent online testing and real-time decision-making stages.

[0029] To further illustrate the algorithm's optimization objective and parameter update mechanism, the loss functions for the policy network and evaluation network are defined below. First, the loss function for the policy network is expressed as follows: Secondly, the loss function for evaluating the network is defined as follows: Furthermore, the CKAN-HAPPO algorithm training process includes the following steps: Initialize training parameters, including the maximum number of training epochs. Length of a single training session And the number of updates in each training round. ; Initialize the policy network parameters for each agent. Evaluate network parameters and establish an experience buffer. ; For each round of training , For each time slot , Acquire intelligent agents from the environment Observations , Obtain the agent's actions based on action distribution. , The optimal computation frequency for the user can be directly derived and calculated. , Calculate reward value And then transition to the next state. Will Store in the experience buffer; A small batch of data is randomly sampled from each experience buffer, and the order of the agents is randomly shuffled.

[0030] In each update round , Update parameters based on the loss function of the policy network. , Calculate the importance sampling ratio and adjust the objective function accordingly. .

[0031] Update parameters based on the loss function of the evaluation network. ; Repeat the above process until the training is complete.

[0032] Specifically, the present invention will be further illustrated below through embodiments: The resource management optimization method for swarm intelligence sensing networks based on multi-UAV assistance proposed in this invention, such as... Figure 2 As shown, the system consists of A drone and The system consists of several users. It employs an internet-based sensing platform to issue task requests and selects the most suitable user based on task requirements and the user's real-time status, thereby improving task completion efficiency. All users in the system are assumed to participate voluntarily. After a task is issued, the drone is dispatched to the selected user's location to collect their sensing data. The network allows users to preprocess the sensing data and offload some data to the drone for execution. This mechanism helps alleviate the pressure on the communication network and the drone's computing resources. User-side computation and drone-side computation occur in parallel. In this system, the invention considers a duration of... The task cycle is divided into There are 1 time slot, and the length of each time slot is 1. The workflow within each time slot is shown in Figure 3: First, the UAV receives a sensing request and is dispatched to collect sensing data; second, the UAV receives a sensing task request and flies to the target area to collect sensing data; third, the user participating in the sensing task of this time slot is scheduled to perform data acquisition; fourth, the user divides the collected data into two parts: one part is preprocessed locally, and the other part is uploaded to the UAV in its raw form for calculation. Since the data aggregation latency on the UAV side is relatively small, this invention ignores the impact of this latency on system performance. For users... Each time slot is further divided into three sub-time slots. This invention defines... and Variables are assigned to time slots to strictly constrain the execution time of each process. Specifically, Indicates user The perceived data latency time, For users Perform local calculations. Furthermore, Indicates user The data transmission delay time, and Then the data calculation is assigned to the drone.

[0033] The various model mechanisms involved in this invention are as follows: Moving Model User Mobility Model: Since user movement has a strong temporal dependence, their future position and speed are highly correlated with their past and current states. Therefore, this invention employs a Gaussian-Markov model, a stochastic process with memory properties, to effectively capture this time-dependent characteristic and simplify the modeling and implementation process. In the time slot The Cartesian coordinates are Its moving speed and direction angle are respectively and Then the formula for updating the user's location is: in, The time step is defined as [value]. This model is able to realistically depict the user's movement characteristics and their random changes while maintaining computational simplicity.

[0034] Unmanned Aerial Vehicle (UAV) Mobility Model: Setting Time Slots drones inside The three-dimensional Cartesian coordinates are Its velocity and acceleration are defined as follows: and Therefore, the motion of a drone can be described by the following formula: communication model This invention uses the Rician channel model to describe the wireless channel between the user and the drone.

[0035] Set time slot Internal users and drones The channel coefficient is in The large-scale channel gain is represented, taking into account the effects of path loss and shadowing fading. The small-scale fading part is represented as follows: in, For the line-of-sight component, This represents random scattering components. This model can effectively describe channel characteristics under different environments. To avoid interference, the system employs orthogonal frequency division multiple access (OFDMA) technology. Assume the user... The transmission power is Then it is in the time slot To drones The transmission rate is: Data perception and transmission model Set user The perception rate is Its value depends on the type of perceived data. The perception rate varies significantly depending on the data type. Therefore, the user... In the time slot The amount of internally generated sensory data can be expressed as According to the communication model, the user In the time slot The amount of sensing data uploaded to the drone is .

[0036] Computational model Set user The computing resources are The number of CPU cycles required to process 1 bit of data for The amount of data that the user computes locally within a time slot is then... At the same time, drones were deployed. The number of CPU cycles required per bit of data is and define Assigning drones to users The computing resources of the user Upload to drone The amount of data processed is .

[0037] Energy consumption model User energy consumption model: The total energy consumption of a user mainly includes three parts: perceived energy consumption, transmission energy consumption, and transmission energy consumption. And local calculation of energy consumption. First, let's assume the user... The sensing energy consumption per bit of data Then the energy consumption during the sensing phase can be expressed as: Secondly, when users When transmitting data to a drone, the energy consumption for transmission is expressed as follows: For data that has not been uploaded, users will process it locally, and the local computing energy consumption can be expressed as follows: Ultimately, the user The total energy consumption can be expressed as .

[0038] The UAV energy consumption model mainly consists of two parts: computational energy consumption and flight energy consumption. Specifically, the UAV's flight energy consumption is expressed as follows: ,in The drone's flight power is shown below: These parameters are related to the UAV's structure. Furthermore, the computational energy consumption of the UAV during mission execution can be expressed as... In summary, the total energy consumption of a drone consists of flight energy consumption and computing energy consumption, which can be expressed as: .

[0039] Based on the above model The total amount of data processed by users and drones is represented as follows: To verify the performance of the algorithm proposed in this invention, this invention considers comparing it with the following three algorithms: CNN-HAPPO: In this scheme, the policy network adopts a CNN structure, and its convolutional layer design is the same as the algorithm proposed in this invention, but it does not include the KAN module. This scheme serves as a benchmark for ablation experiments to evaluate the contribution of the KAN module to the algorithm's performance.

[0040] MLP-HAPPO: In this scheme, the policy network adopts an MLP structure, i.e., the original HAPPO algorithm. This scheme serves as a baseline for evaluating the role of CNN structures in policy learning.

[0041] MLP-MAPPO: Unlike HAPPO, MAPPO does not employ a sequential update strategy; its policy network is also based on an MLP structure. See Table 1 below: Table 1 Figure 5 and Figure 6 The convergence performance curves for user rewards and drone rewards are shown separately. It can be seen that as the number of training rounds increases, all reward values ​​show a continuous upward trend and eventually stabilize, indicating that the algorithm proposed in this invention has good convergence during training, thus verifying the effectiveness of the constructed MADRL algorithm. Among the various algorithms compared, the algorithm of this invention achieved the highest reward value. This performance advantage mainly stems from the hybrid structure of CNN and KAN used in the policy network of this invention. Compared with traditional MLP-based policy networks, CNN has good feature extraction capabilities and can significantly reduce the number of network parameters; at the same time, KAN enhances the model's ability to fit complex state-action relationships by introducing learnable spline functions, thereby improving exploration efficiency in high-dimensional continuous action spaces. This fusion architecture shows significant advantages in both training performance and generalization ability, enabling the algorithm to adapt more efficiently to structurally similar task scenarios. In contrast, the MAPPO algorithm performs the worst because it does not adopt the sequential update strategy of the HAPPO algorithm. This strategy can guarantee monotonic performance improvement during multi-agent joint updates. Therefore, MAPPO is inferior to HAPPO-based algorithms in terms of convergence speed and stability.

[0042] Figure 7The figure illustrates the change in the amount of perception data the system can process with the number of users, assuming a fixed number of 5 drones. As shown in the figure, the amount of perception data processed by the system continuously increases with the number of users. This is because more users provide more abundant local computing resources, thereby improving data preprocessing capabilities. When the number of users increases, the overall computing power of the user end is enhanced, enabling more perception data to be preprocessed locally, effectively reducing the communication pressure on the drones and improving the data collection efficiency of the drones. Notably, the algorithm proposed in this invention exhibits a more significant performance improvement with an increased number of users. This is mainly due to its CNN and KAN fusion policy network structure, which has stronger feature representation and nonlinear fitting capabilities, maintaining efficient policy learning performance in large-scale scenarios. These results further validate the effectiveness and scalability of the proposed network architecture in complex multi-user environments.

[0043] Figure 8 The figure illustrates the relationship between the total amount of data processed and the average sensing rate under different total bandwidth scenarios. As the figure shows, with the increase of the total system bandwidth, each UAV can obtain a larger bandwidth allocation, thereby improving the overall data transmission capability. Therefore, the total amount of data processed by the system increases with the increase of the average sensing rate. When the average sensing rate further increases, the data collection gap under different bandwidth scenarios gradually widens. This is because under lower sensing rate conditions, the amount of data generated in the sensing phase is limited by the sensing rate, and the impact of bandwidth on system performance is relatively small; while under higher sensing rate conditions, greater bandwidth can significantly improve data transmission capacity, thereby effectively enhancing data collection capabilities and improving the overall system performance.

[0044] Example 2: To achieve the above objective, such as Figure 9 As shown, based on Embodiment 1, this invention discloses a drone-assisted swarm intelligence sensing network resource management and optimization system, comprising: The data acquisition module 11 is used to acquire user status data and drone status data; wherein, the user status data includes user location, maximum energy consumption, sensing rate, and drone location; the drone status data includes its own drone location, maximum energy consumption, user location, and the locations of other drones; The management optimization module 12 is used to input user status data and UAV status data into a pre-established joint optimization framework model, transform the joint optimization problem into a partially observable Markov decision process, solve it based on a multi-agent deep reinforcement learning algorithm for heterogeneous agent proximal policy optimization, output the joint optimization result, and maximize the amount of perception data processing in the resource management of the swarm intelligence perception network based on the joint optimization result. The joint optimization results include optimal time slot allocation, resource allocation, and UAV 3D trajectory planning schemes; the pre-established joint optimization framework model includes a time slot partitioning model for time slot coordination and a system resource joint optimization problem model with the goal of maximizing the amount of processed sensing data.

[0045] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0046] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0047] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0048] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

Claims

1. A method for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicles (UAVs), characterized in that, The method includes the following steps: Acquire user status data and drone status data; wherein, the user status data includes user location, maximum energy consumption, sensing rate, and drone location; the drone status data includes its own drone location, maximum energy consumption, user location, and the locations of other drones; User status data and UAV status data are input into a pre-established joint optimization framework model. The joint optimization problem is transformed into a partially observable Markov decision process. The solution is obtained by a multi-agent deep reinforcement learning algorithm based on heterogeneous agent proximal policy optimization. The joint optimization result is output. Based on the joint optimization result, the amount of perception data processed in the resource management of the swarm intelligence perception network is maximized. The joint optimization results include optimal time slot allocation, resource allocation, and UAV 3D trajectory planning schemes; the pre-established joint optimization framework model includes a time slot partitioning model for time slot coordination and a system resource joint optimization problem model with the goal of maximizing the amount of processed sensing data.

2. The method for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance according to claim 1, characterized in that, The partially observable Markov decision processes include two types: user agents and drone agents. The three elements of user agents and drone agents are different, and the three elements include state, action, and reward. The reward function consists of two parts: the objective function term and the penalty term. Each agent selects the corresponding reward term according to the constraints and designs the penalty term function and penalty factor in a unified manner to achieve effective expression and optimization of the constraints.

3. The method for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance according to claim 1, characterized in that, The multi-agent deep reinforcement learning algorithm based on heterogeneous agent proximal policy optimization is improved by fusing convolutional neural networks and Kolmogorov-Arnold networks. The convolutional neural network is used for feature extraction, and the Kolmogorov-Arnold network is used to describe the nonlinear relationship between state and action.

4. The method for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance according to claim 1, characterized in that, The joint optimization problem is defined as P1, as follows: Among them, constraints C1–C3 are used to limit the three-dimensional flight space range of the UAV. and These represent the side lengths of the rectangular region, and These represent the minimum and maximum flight altitudes of the UAV, respectively; C4–C5 are used to limit the allocation of communication bandwidth, where Indicates drone Distributed to users who establish communication connections bandwidth, This represents the total bandwidth available to the drone; C6–C8 collectively constrain user computing resources. And the computing resource allocation relationship between the user and the drone, where constraint C8 specifies the computing resources allocated to the user by the drone. It cannot exceed the total computing resources of the drone. C9–C10 constrain the communication connection variables between the user and the drone. Constraints C11 and C12 respectively limit the amount of data that can be processed within each time slot. Constraint C11 guarantees the amount of data processed by the user, including the amount of data processed locally. and the amount of data transmitted to the drone No more than the amount of data it perceives. Constraint C12 guarantees the amount of data processed by the drone. The flight speed of the UAV shall not exceed the amount of data it receives; constraints C13–C15 respectively constrain the flight speed of the UAV. acceleration and the minimum spacing between drones ;in, and These represent the maximum values ​​of the drone's speed and acceleration, respectively. and They represent drones and Exclusion The positions of other drones; C16 is used to constrain the time partitioning variable. , The range of values, while C17–C18 limits the energy consumption of both users and drones to not exceed their maximum energy limit, respectively using and express.

5. The method for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance according to claim 2, characterized in that, The three essential elements of the user intelligent agent and the drone intelligent agent are as follows: User intelligent agent: State: The state of the user's intelligent agent includes its own performance indicators and external environmental information. The performance indicators include user location, maximum energy consumption, and sensing rate, while also considering the location of the drone. The state is represented as: Actions: After observing the environment, the user agent's actions include deciding on time slot allocation, computational resource allocation, and the association strategy with the drone. The action can be represented as: in, It is a binary variable; Reward: The reward function includes both the expected target term and the penalty term for not meeting the constraints. The target term consists of two parts: the amount of data processed locally by the user and the amount of data transmitted by the associated drone for computation. Therefore, the user... The reward function can be expressed as follows: To standardize the expression of the penalty term, the penalty function is defined as follows: in, Represented as ; Indicates user The penalty for violating the maximum energy consumption constraint is denoted as: ,in For relevant penalty factors, This indicates a penalty when the amount of data processed exceeds the total amount of perceived data. Its corresponding penalty factor, Right now A penalty will be imposed when the amount of data transmitted by a user is less than the processing capacity of its associated drone. Its definition ; Unmanned aerial vehicle (UAV) intelligent agents: State: The state of a drone agent includes its own position and maximum energy consumption constraints, as well as the user's position and the positions of other drones. Therefore, the drone's state... The state is defined as follows: Action: In each time slot, the drone agent needs to allocate resources to its associated users and control its own flight acceleration to adjust its trajectory. The action is defined as follows: Reward: For drones The reward function considers both the amount of computational data completed by the associated user and the amount of data to be computed received, and also includes a penalty term for constraint violations. (Drone) The reward function is defined as follows: in, Indicates drone Penalties for violating energy consumption constraints If the corresponding penalty factor is... In addition, definition Indicates drone The penalty for violating the data volume constraint is in the following form: ,in This represents the penalty factor associated with violations of data processing constraints, introduced by the penalty term. Ensure the drone flies safely within the designated operating area. Fly out of the preset horizontal range At that time, its penalty items ;in, Indicates drone Horizontal position, This indicates the penalties imposed on drones that violate the minimum safe distance, among which This indicates the penalty coefficient for collision violations.

6. The method for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance according to claim 5, characterized in that, The solution to the partially observable Markov decision process is simplified into a subproblem related to the user's computation frequency, defined as P2, as follows: By launching The optimal solution is expressed as .

7. The method for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance according to claim 3, characterized in that, The convolutional neural network is used to extract key features from the input observation sequence and reduces model complexity through a convolutional kernel parameter sharing mechanism; The Kolmogorov-Arnold network is used to generate agent actions. By introducing a learnable spline function to construct a residual activation function, and by optimizing the spline coefficients, the agent can generate reasonable actions based on different observation information.

8. The method for optimizing resource management in a swarm intelligence sensing network based on unmanned aerial vehicle (UAV) assistance according to claim 6, characterized in that, The improvement process of the penalty factor includes the following two categories: The first type of penalty factor is a penalty factor that is directly related to the optimization objective, and its constraints contain the same expression as the objective function; the second type of penalty factor is a penalty factor that has an indirect impact on the objective function. Although its constraints do not directly involve the objective function, they affect the overall optimization result by limiting the amount of data processing at each stage.

9. A resource management and optimization system for a swarm intelligence sensing network based on unmanned aerial vehicles (UAVs), employing the resource management and optimization method for a swarm intelligence sensing network based on unmanned aerial vehicles (UAVs) as described in any one of claims 1 to 8, characterized in that... include: The data acquisition module is used to acquire user status data and drone status data; wherein, the user status data includes user location, maximum energy consumption, sensing rate, and drone location; the drone status data includes its own drone location, maximum energy consumption, user location, and the locations of other drones; The management optimization module is used to input user status data and UAV status data into a pre-established joint optimization framework model, transform the joint optimization problem into a partially observable Markov decision process, solve it based on a multi-agent deep reinforcement learning algorithm for heterogeneous agent proximal policy optimization, output the joint optimization result, and maximize the amount of perception data processing within the resource management of the swarm intelligence perception network based on the joint optimization result. The joint optimization results include optimal time slot allocation, resource allocation, and UAV 3D trajectory planning schemes; the pre-established joint optimization framework model includes a time slot partitioning model for time slot coordination and a system resource joint optimization problem model with the goal of maximizing the amount of processed sensing data.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on the processor. When the processor loads and executes the computer program, it employs the resource management optimization method for swarm intelligence sensing network based on any one of claims 1 to 8.

Images