Cooperative crowd sensing system based on unmanned aerial vehicle group and resource optimization method

Abstract

The invention relates to the technical field of mobile crowd sensing, in particular to a cooperative crowd sensing system based on an unmanned aerial vehicle group and a resource optimization method, and adopts a working mode supporting'partial calculation unloading 'in the system. In the mode, after the unmanned aerial vehicle collects the original data, a part of data can be processed locally by utilizing the computing power of the unmanned aerial vehicle, and only the processing result and the remaining original data are transmitted back to the server. Through the mode, the system can dynamically balance the local calculation overhead of the unmanned aerial vehicle and the communication overhead of data return according to the real-time resource condition, so that the system can better adapt to a complex and changeable application environment, and the real-time state of the system can be used as input in the method, so that the real-time performance of the system is improved. A complete joint action strategy comprising the four dimensions can be directly output, step-by-step or heuristic search is not needed, after training convergence is carried out, millisecond-level real-time decision making can be carried out according to an instantly changing environment state, and a high-quality scheduling scheme is rapidly generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile crowdsensing, and in particular to a cooperative crowdsensing system based on a UAV group and a resource optimization method. BACKGROUND

[0002] In recent years, with the rapid development of 5.5G communication technology and embedded sensor technology, smart cities have entered the era of Internet of Everything. In the construction of smart cities, mobile crowdsensing (MCS) has become one of the key technologies and is widely used in traffic monitoring, environmental monitoring, noise pollution monitoring and other fields.

[0003] As shown in Figure 1 A typical MCS system usually consists of three main parts, namely a task publisher, a sensing platform and a task performer. For a certain number of sensing tasks, the task publisher first generates a sensing task, uploads relevant information (such as task coordinates, required sensing data volume, sensing capability, etc.) to the sensing platform, the sensing platform integrates the above information and distributes it to the task performer, the task performer completes the sensing task distributed by the platform, uploads the collected data to the platform, and then the platform processes the data and feeds back the processing result to the task publisher.

[0004] At present, most of the literature on MCS is studying the use of wearable devices (such as smart phones, smart watches, etc.) to perform sensing tasks. However, such devices face many challenges when performing MCS tasks: first, due to the relatively poor communication capabilities of wearable devices, they may be outside the coverage range of the server (sensing platform) in remote areas or disaster scenarios, making it difficult to collect sensing data; second, since wearable devices are not deployed directly by the sensing platform, an effective incentive mechanism needs to be designed to encourage user participation, but it is often difficult to find a suitable incentive mechanism; in addition, the computing power and data storage capacity of wearable devices are limited, which may cause performance bottlenecks when processing large-scale data; finally, the types of sensors on wearable devices are limited, and the quality of sensing data is usually poor.

[0005] To overcome the limitations of wearable devices in mobile crowd sensing tasks, deploying unmanned aerial vehicles (UAVs) to perform MCS tasks has become a better choice. First, UAVs have the advantages of high mobility and strong communication capabilities, which can quickly deploy to the task area to perform sensing tasks, and can effectively communicate with the server even in remote areas or disaster scenarios. Second, since the UAV is directly commanded by the server, it is more convenient to set up its reward and punishment mechanism, which helps to achieve better training results and thus higher system benefits and efficiency. In addition, compared with wearable devices, UAVs have stronger computing power and data storage capacity, which can assist in collecting and processing large-scale sensing data, thereby improving system efficiency. Finally, UAVs can be integrated with professional high-precision sensors, which can achieve a more comprehensive sensing range and better sensing capability than wearable devices for monitoring specific areas.

[0006] Although UAVs have many advantages in mobile crowd sensing scenarios, they still face challenges of limited multi-dimensional resources (such as communication resources and computing resources) in remote areas or disaster scenarios. Specifically, due to limited communication resources (transmission bandwidth), if the UAV directly uploads raw sensing data to the MCS server, data transmission may fail. Therefore, in this case, to reduce the consumption of communication resources and the burden of network traffic, the UAV's own computing power can be used to locally process some of the sensing data. In addition, in these special scenarios, the UAV's battery endurance, task planning, and collaborative scheduling also face severe challenges, which need to be further researched and optimized in coordination with the characteristics of MCS scenarios. SUMMARY

[0007] The purpose of the present application is to provide a collaborative crowd sensing system and resource optimization method based on a UAV swarm to solve the above problems.

[0008] The technical solution adopted by the present application is as follows: a collaborative crowd sensing system based on a UAV swarm, comprising a data center, a central base station, a sensing platform, and a plurality of same-model UAVs;

[0009] The data center is used to generate sensing task requirements and expects to collect data from a specific location PoI in the area;

[0010] The central base station and the sensing platform have the same coordinates, and the coordinates are (x0, y0, H0). The central base station serves as the communication hub to ensure stable wireless connections between the data center, the sensing platform, and the plurality of same-model UAVs;

[0011] The sensing platform is used to receive and process the sensing data uploaded by the plurality of same-model UAVs, and feed back the results to the data center;

[0012] A plurality of the same type of unmanned aerial vehicles are used to receive the task position issued by the perception platform, fly to a specific position PoI, and hover above it to collect perception data.

[0013] Further, a plurality of the same type of unmanned aerial vehicles collect perception data, a part of which is calculated locally and uploaded to the perception platform as local calculation results, and another part of which is directly uploaded to the perception platform as raw data without calculation processing.

[0014] The perception platform processes the received raw data and merges it with the received local calculation results as the perception result. After collecting all the perception data of the specific positions PoI, the perception platform integrates the perception results and returns all the perception results to the data center.

[0015] The unmanned aerial vehicle swarm-based collaborative crowd-sensing perception system in the present application also provides a perception system resource optimization method, comprising the following steps:

[0016] S1. System and network initialization, in the data center, construct and initialize the neural network model required by the algorithm, including the online policy network μ, the target policy network μ', the online evaluation network Q and the target evaluation network Q';

[0017] S2. Construct and perceive system state, at the beginning of each decision-making round t, the data center obtains and integrates the current state s(t) of the perception system;

[0018] S3. Generate and issue joint action, input the current state s(t) into the online policy network μ to generate a joint action a(t). The generated joint action a(t) contains all scheduling instructions for the plurality of the same type of unmanned aerial vehicles in the unmanned aerial vehicle swarm-based collaborative crowd-sensing perception system;

[0019] S4. Perform task and environment interaction, each unmanned aerial vehicle in the perception system performs the task according to the joint action a(t) issued by the data center. After the task is performed, the system environment transfers from the current state s(t) to the next round state s(t+1);

[0020] S5. Calculate the immediate reward and store the experience, the data center calculates the immediate reward R(t) according to the execution result of the current round task, and stores the experience quadruple (s(t), a(t), R(t), s(t+1)) generated in this interaction into the experience replay pool Z;

[0021] S6. Network model parameter update, when the number of samples in the experience replay pool Z reaches the preset training start threshold and the network update frequency condition is met, a training batch size B a of transition samples is randomly sampled from Z to update the online evaluation network Q and the policy network μ.

[0022] S7. Iterative and convergence judgment, repeat the steps of S2 to S6 until the algorithm reaches the preset maximum training round or the instantaneous reward R(t) curve of the perception system converges;

[0023] S8. Save the final parameters of the trained online policy network μ

[0024] Further, in S1, the following sub-steps are included:

[0025] Constructing online policy network μ and target policy network μ', and randomly initializing their network parameters And Ensure that at the beginning

[0026] Constructing online evaluation network Q and target evaluation network Q', and randomly initializing their network parameters And Ensure that at the beginning

[0027] Initialize the experience replay pool Z;

[0028] Initialize the training hyperparameters, including the learning rate α of Actor and Critic μ And α Q , reward discount factor γ, target network soft update rate τ, and training batch size B a .

[0029] Further, in S2, the current state s(t) of the perception system is defined as:

[0030] s(t) = {(S PoI (t), S UAV (t))};

[0031] Wherein, S PoI (t) is the coordinate vector of all specific positions PoI to be perceived at the current round, and S UAV (t) is the hovering position vector of all drones at the end of the last round.

[0032] Further, in S3, the joint action taken by the i drones is a i (t), and is defined as:

[0033]

[0034] Wherein, is the number of specific positions PoI selected by the i drones, is the perception time set by the i drones at execution round t, is the proportion of local computation of i-th UAV at t-th round, is the bandwidth allocated to i-th UAV at t-th round.

[0035] Further, in S5, the immediate reward R(t) is defined as:

[0036]

[0037] Further, in S6, when the number of samples in the experience replay pool Z reaches a threshold B a , the system uniformly samples a small batch of state transition samples from the experience replay pool has:

[0038]

[0039] Further, in S6, the online critic network parameters are updated including the following sub-steps:

[0040] The temporal difference (TD) target y is calculated by the target network j :

[0041]

[0042] where γ∈(0,1) is the discount factor, is the output value of the target critic network, the parameters of which are updated with a delay; is the target policy network used to generate the joint action a′ j for the next state;

[0043] The online critic network parameters are updated by minimizing the mean squared error loss function j between the online critic network prediction value and the TD target y

[0044]

[0045] where, is the output value of the online critic network, and α Q is the critic network learning rate;

[0046] The online policy network parameters are updated including the following sub-steps:

[0047] The policy gradient for updating the online policy network is calculated by sampling the policy gradient theorem

[0048] ​

[0049] wherein, is the gradient response of the evaluation network to action a, is the influence value of the policy network to the policy;

[0050] updating the online policy network parameters in the direction of policy gradient ascent to maximize the long-term return, we have:

[0051]

[0052] wherein α μ is the policy network learning rate, and α μ <α Q .

[0053] Further, in the S6, the updated online network parameters are smoothly synchronized to the corresponding target network at a small update rate τ (τ << 1) to maintain the stability of training, and we have:

[0054]

[0055] The beneficial effects of the present application include at least one of the following:

[0056] 1. A collaborative swarm intelligence perception system is provided, which includes a data center, a central base station, a perception platform, and a plurality of same-model UAVs. The system adopts a working mode supporting a "partial computation offloading". In this mode, after collecting raw data, the UAVs can first process a part of the data locally using their own computing power, and only return the processing results and the remaining raw data to the server. In this way, the system can dynamically balance the local computing overhead of the UAVs and the communication overhead of data return according to the real-time resource situation, thereby better adapting to complex and variable application environments.

[0057] 2. Meanwhile, aiming at the problem of long-term energy efficiency maximization, the collaborative swarm intelligence perception system based on UAVs provides a perception system resource optimization method, which jointly considers and cooperatively optimizes four key variable decision dimensions of task allocation and path planning, perception resource allocation, computing resource allocation, and communication resource allocation, and integrates the energy consumption and time overhead of the four stages of flight, perception, computation, and communication, and imposes multi-dimensional constraints such as task completion time, data volume requirement, and total bandwidth, thereby providing accurate mathematical basis for subsequent intelligent decision algorithms.

[0058] 3. An intelligent decision-making method named AC-TRO, i.e. Actor-Critic, is adopted for the dynamic optimization problem of complex multi-variable, continuous-discrete mixed action space. It can directly output a complete joint action policy containing the above four dimensions as input, without step-by-step or heuristic search. At the same time, it learns through the collaborative work of the "actor" network (formulating policy) and the "critic" network (evaluating value). The critic evaluates the long-term value of the current policy, and the actor adjusts the policy according to its feedback. The ultimate goal is to maximize the long-term cumulative reward (i.e. long-term average perceived energy efficiency). In addition, after training convergence, it can make real-time decisions in milliseconds according to the changing environment state, quickly generating high-quality scheduling schemes. Compared with traditional heuristic algorithms that require a large number of iteration times, it is more suitable for dynamic task scenarios. Finally, when the number of UAVs increases and the problem complexity grows exponentially, the performance degradation of this method is much smaller than that of traditional heuristic algorithms, showing superior scalability and robustness. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a schematic diagram of a typical MCS system;

[0060] Figure 2 is a schematic diagram of a UAV swarm-based collaborative swarm intelligence perception system;

[0061] Figure 3 is a workflow diagram of UAV swarm intelligence perception;

[0062] Figure 4 is a convergence analysis diagram of the algorithm of the present embodiment;

[0063] Figure 5 is a comparison of system utility of the AC-TRO algorithm and other baseline algorithms under different numbers of UAVs;

[0064] Figure 6 is a comparison of the completion rate of perception tasks of the AC-TRO algorithm and other baseline algorithms under different numbers of UAVs;

[0065] Figure 7 is a comparison of the average energy consumption of the AC-TRO algorithm and other baseline algorithms under different numbers of UAVs. DETAILED DESCRIPTION

[0066] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described in the drawings can be arranged and designed in various different configurations.

[0067] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0068] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0069] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0070] As shown in Figure 2 A perception system based on cooperative crowd intelligence of UAV swarm, comprising a data center, a center base station, a perception platform and a plurality of same-model UAVs;

[0071] The data center is used to generate a perception task demand and expects to collect data from a specific location PoI in a region;

[0072] The center base station and the perception platform have the same coordinates, and the coordinates are (x0, y0, H0). The center base station serves as a communication hub and is used to ensure stable wireless connection between the data center, the perception platform and the plurality of same-model UAVs;

[0073] The perception platform is used to receive and process the perception data uploaded by the plurality of same-model UAVs, and feed back the results to the data center;

[0074] The plurality of same-model UAVs are used to receive the task location issued by the perception platform, fly to the specific location PoI and hover above it to collect perception data.

[0075] At the same time, the plurality of same-model UAVs upload a part of the collected perception data as local calculation results to the perception platform after local calculation, and upload another part of the collected perception data as raw data directly to the perception platform without calculation and processing;

[0076] The perception platform processes the received raw data and merges the received local computing results as the perception result. After collecting all the perception data of the specific positions PoI, the perception platform integrates the perception results and returns all the perception results to the data center.

[0077] The purpose of such design is to provide a collaborative swarm intelligence perception system containing a data center, a central base station, a perception platform, and a plurality of unmanned aerial vehicles of the same model. In the system, a working mode supporting "partial computation offloading" is adopted. In this mode, after collecting raw data, the unmanned aerial vehicle can first process a part of the data locally using its own computing power, and only return the processing result and the remaining raw data to the server. In this way, the system can dynamically balance the local computing overhead of the unmanned aerial vehicle and the communication overhead of data return according to the real-time resource situation, so as to better adapt to the complex and changeable application environment.

[0078] In specific scene use, the scene is composed of a data center (Data Center), a central base station (BS), an MCS server (perception platform), and N unmanned aerial vehicles (UAV) of the same model. Let denote the set of UAVs. The data center acts as a task publisher, responsible for generating perception task requirements, and expects to collect data from specific positions (PoI, Point of Interest) in the region. The BS and the MCS server are co-located at coordinates (x0, y0, H0). The BS acts as a communication hub, ensuring stable wireless connection between the data center, the MCS server, and the UAVs; the MCS server undertakes the functions of core task scheduling and data processing, responsible for receiving and processing the perception data uploaded by the UAVs, and feeding back the results to the data center.

[0079] At the beginning of each task execution round, the data center generates N different PoIs, each representing a key position that needs to collect data, let denote the set of POIs. Then, the data center sends all PoI information to the MCS server as the target for all UAVs to collect data in this round.

[0080] The whole system contains mechanical model, perception model, computing model and communication model. In the mechanical model, at the beginning of each task execution round t, the UAV i hovers at the position (x i (t-1), y i (t-1), h0), where h0 is the working flight height of the UAV. Then, the UAV i flies to the perception task position assigned in this round, i.e. the coordinates of the target interest point (PoI) above and hovers at this point to perform the perception task.

[0081] To describe the assignment relationship between UAV i and PoI h, a binary variable p k,i (t) = {0, 1} is defined to represent whether UAV i chooses to fly to PoI k and perform the perception task. Wherein, p k,i (t) = 1 represents that UAV i chooses to perform the perception task at PoI k in the tth task execution round, and p k,i (t) = 0 represents that UAV i does not perform the perception task at PoI k in the tth task execution round.

[0082] Therefore, the flight distance d i (t) of UAV i in the task execution round t can be represented as:

[0083]

[0084] Wherein, d i,k (t) represents the Euclidean distance of UAV i flying from the last task position to PoI k in the round t, and the calculation formula is:

[0085]

[0086] Therefore, the flight time t of UAV in this process can be represented as:

[0087]

[0088] Wherein, d i,k (t) represents the flight distance of UAV i in the task execution round t, and v0 represents the flight speed of UAV.

[0089] The flight power of UAV i can be represented as:

[0090]

[0091] In the formula, γ is a constant related to the attribute of UAV itself; v is the flight power constant, which indicates that the flight power of UAV is proportional to a certain power of the flight speed. Therefore, the flight energy consumption E of UAV i in the flight time t can be represented as:

[0092]

[0093] In addition, the hovering energy consumption E of UAV i can be represented as:

[0094]

[0095] Wherein, is the hovering power of UAV i, is the hovering duration of UAV i.

[0096] Since the total time T0of each task round t is fixed, and the flight time of UAVs and the hovering time jointly constitute the total time of a task round, i.e.,

[0097]

[0098] In the perception model, after completing the flight task in the tth task execution round, UAV i hovers directly above the target point of interest (PoI) to start the perception task. Let is the perception time of UAV i in round t, i.e., the duration of UAV i performing perception data collection at the current PoI.

[0099] UAV i collects environmental information at a fixed rate o i in round t. i (t) can be expressed as:

[0100]

[0101] where o i is the number of bits perceived by UAV i per second, which is related to the sensor performance of UAV and the characteristics of the perceived area.

[0102] In the perception process, the perception energy consumption of UAV is proportional to the amount of data it collects. Let be the energy consumption of UAV i to perceive 1 bit of data, then the total perception energy consumption of UAV i in task round t can be expressed as:

[0103]

[0104] In the calculation model, generally speaking, the calculation and communication processes of the crowd-sensing system need to rely on complete perception data. If the calculation and communication processes are started before the completion of the perception task, it may lead to incomplete or invalid data transmission. Therefore, UAVs start local calculation and upload operations on the original data they collect after completing the perception task. It is assumed that the processing of perception data by UAVs can be offloaded at any ratio, i.e., the process of processing perception data has sufficient flexibility. In view of the limitation of the communication link bandwidth of UAVs and the demand for improving transmission efficiency, each UAV needs to first calculate part of the perception data locally, and transmit the remaining original data together with the local calculation result to the server.

[0105] Let c iThe number of CPU cycles required to process 1 bit of sensing data, μ i (t) represents the proportion of UAV i that computes sensing data locally in round t (0≤μ). i (t)≤1). Therefore, the time for UAV i to perform local computation in round t is... It can be represented as:

[0106]

[0107] Among them, z i (t) represents the amount of sensory data collected by UAV i within round t, and f i It is the CPU frequency of UAV i, which measures the computing power of UAV i.

[0108] Energy consumption of UAV i during round t for local calculations It can be represented as:

[0109]

[0110] Here, κ is the effective switched capacitance, and its value depends on the chip architecture.

[0111] like Figure 3 As shown, in the system's communication model, since the UAV flies at a high altitude and its operating environment is usually unobstructed, the air-to-air (ATA) channel between the UAV and the MCS server can be modeled as a line-of-sight (LoS) channel. This means that the path loss between the BS and the UAV is mainly determined by the free-space path loss. Assume that in mission round t, the distance between UAV i and the MCS server is d. i,0 If (t), then within round t, the channel gain β from UAV i to the MCS server within that round is... i (t) can be expressed as:

[0112] β i (t)=β0(d i,0 (t)) -μ #(12)

[0113] Where β0 is the channel power gain per unit distance, and μ is the path loss exponent of the LoS channel.

[0114] To avoid mutual interference between UAVs, Orthogonal Frequency Division Multiplexing (OFDM) technology is used between UAVs. In order to improve the bandwidth utilization, the bandwidth allocated to each UAV is regarded as an optimization variable, which is dynamically allocated to each UAV by the MCS server at the beginning of each round. In addition, when dynamically allocating bandwidth, the following constraints need to be followed:

[0115]

[0116] where B i (t) represents the bandwidth allocated to UAV i in time slot t, and B is the total bandwidth available to the system. This constraint ensures that the sum of the bandwidth allocated to all UAVs does not exceed the total available bandwidth, thereby effectively managing network resources and optimizing overall system performance.

[0117] According to Shannon's theorem, the communication rate r i (t) between UAV i and the MCS server can be represented as:

[0118]

[0119] where, is the transmit power of UAV i, is the channel noise power density, and μ is the path loss factor.

[0120] Assuming that in round t, the proportion of raw data directly uploaded by UAV i to the original perception data volume is 1-μ i (t), then the transmission time of its uploaded raw data is:

[0121]

[0122] where z i (t) is the perception data volume collected by UAV i in round t. Without loss of generality, since the data volume of data processing results is much smaller than that of raw data, the time delay and energy consumption of UAV uploading data processing results can be reasonably ignored, so this embodiment only discusses the communication model of uploading raw data.

[0123] Therefore, the transmission energy consumption of UAV i uploading perception data is:

[0124]

[0125] Figure 3 From the time perspective of UAV i, the working process of the system is clearly illustrated. Therefore, the hovering time of UAV i in round t can be represented as:

[0126]

[0127] wherein, is the time for UAV i to perform the perception task, is the time for local computation, is the time for data transmission. The scheduling and resource allocation of the system need to be reasonably planned according to these time parameters to ensure that the UAV completes the perception task within a limited time.

[0128] In mathematical calculations, for a general UAV swarm perception system, the energy consumption and task execution delay are usually as low as possible under the premise of guaranteeing key performance indicators such as task coverage and perception data collection amount; or in the scene where the communication bandwidth and flight path are limited, the overall perception task completion quality of the system is improved. For the scenario studied in this paper, improving the collection amount of perception data usually accompanies higher energy and resource consumption, which is in conflict with the endurance capability of the UAV and the limitation of computing and communication resources, therefore, the joint optimization of the two needs to be considered.

[0129] This embodiment will establish an optimization problem of improving the collection amount of perception data and reducing energy consumption under the premise of meeting resource limitations and task constraints from the perspective of the overall system. For UAV i, the perception energy efficiency function R i (t) that measures its perception and data processing capability in task execution round t can be defined as:

[0130]

[0131] wherein, as described in equation (7), z i (t) represents the data amount collected by UAV i in completing all perception tasks in execution round t; E i (t) reflects the energy consumption of UAV i in executing the perception task in this round, which can be represented as:

[0132]

[0133] Considering all UAVs in the system, the total utility R(t) of the system at task execution round t is:

[0134]

[0135] In summary, for the UAV swarm perception system studied in this embodiment, the long-term cumulative utility brought by long-term sequential decision-making is considered, the long-term average perception energy efficiency of the system is maximized, and therefore the optimization objective of the final model can be represented as:

[0136]

[0137] Formula (22) specifies the minimum requirement of the amount of data that the UAV needs to collect when performing the perception task, that is, the perception data collected by any UAV in one round should not be less than z min Formula (23) constrains each UAV i to select a PoI k to perform the perception task, to ensure the task allocation of the UAV and the overall coordination of the system, in which the binary variable p i,k (t)∈{0,1} indicates whether the UAV i performs the perception task on the PoI k; formula (24) indicates that a PoI can only be selected by one UAV, ensuring that each POI is only allocated once in the task execution round t, which, in combination with formula (23), ensures that each UAV can only focus on one specific task at the same time, thereby improving the clarity of task allocation and the specificity of execution; formula (25) specifies the upper limit of the bandwidth allocation; and formula (26) constrains the UAV i to complete all perception, calculation and transmission tasks within the hovering time.

[0138] In this embodiment, based on the cooperative swarm intelligence perception system of the UAV group, a cooperative swarm intelligence perception system resource optimization method based on the UAV group is also provided, which includes the following steps:

[0139] S1. System and network initialization, in the data center, construct and initialize the neural network model required by the algorithm, including the online policy network μ, the target policy network μ', the online evaluation network Q and the target evaluation network Q';

[0140] S2. Construct and perceive the system state, at the beginning of each decision round t, the data center obtains and integrates the current state s(t) of the perception system;

[0141] S3. Generate and issue joint action, input the current state s(t) to the online policy network μ to generate a joint action a(t), the generated joint action a(t) contains all scheduling instructions for multiple UAVs of the same type in the cooperative swarm intelligence perception system based on the UAV group;

[0142] S4. Perform task and environment interaction, each UAV in the perception system performs the task according to the joint action a(t) issued by the data center, and after the task is performed, the system environment is transferred from the current state s(t) to the state s(t+1) of the next round;

[0143] S5. Calculate the immediate reward and store the experience, the data center calculates the immediate reward R(t) according to the execution result of the task in this round, and stores the experience quadruple (s(t), a(t), R(t), s(t+1)) generated in this interaction into the experience replay pool Z;

[0144] S6. Network model parameter update, when the number of samples in the experience replay pool Z reaches the preset training start threshold, and the network update frequency condition is met, a training batch size B of transition samples is randomly sampled from Z a Update the online evaluation network Q and the online policy network μ.

[0145] S7. Iteration and convergence judgment, repeat the steps of S2 to S6 until the algorithm reaches the preset maximum training round or the instantaneous reward R(t) curve of the perception system converges.

[0146] S8. Save the final parameters of the online policy network μ after training convergence

[0147] The purpose of such design is that, for the problem of long-term energy efficiency maximization, the cooperative swarm intelligence perception system based on UAV swarm provides a perception system resource optimization method, which jointly considers and cooperatively optimizes four key variable decision dimensions of task allocation and path planning, perception resource allocation, computing resource allocation, and communication resource allocation, and integrates the energy consumption and time overhead of the four stages of flight, perception, computing, and communication, and imposes multi-dimensional constraints such as task completion time, data volume requirement, and total bandwidth, providing accurate mathematical basis for subsequent intelligent decision algorithms. For the dynamic optimization problem of complex multivariate, continuous-discrete mixed action space, an intelligent decision algorithm named AC-TRO (Actor-Critic) is adopted, which includes policy network Actor part and evaluation network Critic part, which can take the real-time state of the system as input and directly output a complete joint action strategy containing the above four dimensions, without step-by-step or heuristic search. At the same time, learning is carried out through the cooperative work of the "actor" network (formulating strategy) and the "critic" network (evaluating value). The critic evaluates the long-term value of the current policy, and the actor adjusts the policy according to its feedback, with the ultimate goal of maximizing the long-term cumulative reward (i.e. long-term average perception energy efficiency). In addition, after training convergence, real-time decision-making within milliseconds can be made according to the rapidly changing environmental state, quickly generating high-quality scheduling schemes. Compared with traditional heuristic algorithms that require a large number of iteration times, this method is more suitable for dynamic task scenarios. Finally, when the number of UAVs increases and the problem complexity grows exponentially, the performance degradation of this method is much smaller than that of traditional heuristic algorithms, showing superior scalability and robustness.

[0148] ​In a specific implementation, the above-mentioned UAV swarm cognitive sensing system is modeled using an MDP model. Considering that the multi-UAV cooperation of the system in a dynamic environment is a sequential decision-making process, and there is a long-term trade-off between energy constraints and task benefits, the MDP framework can effectively depict the state transition process of the above dynamic system, the reward mechanism can be quantified as the influence of immediate decision-making on long-term goals, and the discount factor can balance the trade-off between short-term benefits and system sustainability, which makes it particularly suitable for UAV swarm cognitive sensing scenarios that require online decision-making.

[0149] Specifically, a Markov decision process is usually composed of five parts, which can be represented by a five-tuple {S, A, P, R, γ}. Among them, S represents a finite state space, A represents a finite action set, P represents a state transition probability, describes the environmental dynamics, R is the immediate reward, quantifies the benefit of single-step decision-making, and γ∈(0, 1) is the discount factor, which represents the importance of future rewards compared to immediate rewards for long-term returns.

[0150] For the state space:

[0151] Let the state space be S, and the system state s(t) at the t-th round can be specifically defined as:

[0152] s(t)={(S PoI (t),S UAV (t))}#(27)

[0153] Where S PoI (t) is a vector composed of all sensing task PoI coordinates at the t-th round, which can be represented as:

[0154]

[0155] S UAV (t) is a vector of the hovering positions of all UAVs at the start of the t-th round, that is, the hovering positions of the UAVs performing sensing tasks in the last round, which can be represented as:

[0156]

[0157] For the finite action set:

[0158] Let the action space be A={a1(t),…,a i (t),…} N Where the action a i (t) taken by UAV i at the t-th round can be defined as:

[0159]

[0160] is the number of the selected PoI by the i-th UAV, specifically, if the i-th UAV chooses to go to the k-th PoI to perform the perception task at the t-th round, then it satisfies:

[0161]

[0162] Similarly, is the perception time set by the i-th UAV when performing the round t, which can be expressed as:

[0163]

[0164] is the proportion of the i-th UAV's local calculation at the t-th round, which is expressed as:

[0165]

[0166] is the bandwidth allocated to the i-th UAV at the t-th round, which is expressed as:

[0167]

[0168] In order to better cope with the "exploration-exploitation" dilemma widely existing in reinforcement learning, the action selection is based on the ε-greedy strategy, that is, randomly selecting an action with a probability of ε, and selecting an action a(t) generated by the Actor network based on the current state with a probability of (1-ε).

[0169] For the state transition probability

[0170] The probability of the system transitioning from state s(t) to state s'(t) after taking action a(t) is expressed as

[0171]

[0172] In this model, since the transition of the state not only depends on the action selected by the system, but also is affected by the PoI location of the newly released perception task by the MCS server, this transition probability is unknown, which is called model-free reinforcement learning.

[0173] For the reward function

[0174] The total utility R(t) of the system at the task execution round t has been defined in equation (19), therefore in the current model, the core goal of optimization is to maximize this utility function under the condition of meeting resource limitations and task constraints, so as to improve the perception data collection and reduce the system energy consumption. Based on this goal, the immediate reward function R(s(t), a(t)) obtained by the system when performing action a(t) at state s(t) is set, which is specifically defined as follows:

[0175]

[0176] The reward function assigns a reward or penalty by judging whether the system complies with all resource and task constraints after performing an action. When all constraints are met, the system will obtain a corresponding positive utility R(t), i.e. the reward after successful completion of the task; if any constraint is violated, the system will be subject to a greater penalty -P, where P is a significant negative constant to punish actions that do not meet the constraints. The design of this reward function ensures the target direction of model optimization, enabling the system to maximize the reward while avoiding the violation of constraints when performing each action.

[0177] For the method provided in the embodiment, compared with the discretization limitation of the traditional DQN method in the continuous action space, the Actor-Critic (AC) framework can realize direct optimization of the continuous action space by decoupling the policy function and the value function, and significantly improve the scalability of multi-variable collaborative decision-making. In order to improve the stability of the training process, a double network architecture is adopted in the embodiment, specifically, the Actor network is composed of an online policy network and a target policy network, wherein the online policy network directly outputs control instructions as the policy maker of the agent; the Critic network includes an online evaluation network and a target evaluation network.

[0178] In each update round of the Actor-Critic framework, the system stores the four-tuple (s j ,a j ,R i ,s′ j ) of MDP state transition in the experience replay pool . When the sample size in the experience replay pool reaches a threshold B a , the system uniformly samples a small batch of state transition samples

[0179]

[0180] As a "policy evaluator", the training target of the Critic network is to accurately predict the long-term value of the action, and to minimize the temporal difference error. The system will first update the parameters of the Critic network, and for each sampled sample The system first calculates the temporal difference target y j

[0181]

[0182] Wherein, γ∈(0,1) is a discount factor, is the output value of the target Critic network, and the parameters of the target Critic network will be updated with a delay; is the target Actor network to generate the action a' for the next state j . Then, the error of the Critic network is defined as the mean squared error between the online network's prediction and the target value, and the online Critic network parameters are updated by the backpropagation strategy, i.e.,

[0183]

[0184] where is the output value of the online Critic network, and a Q is the Critic learning rate. To avoid the sudden change of the value estimate, the algorithm gradually synchronizes the target network parameters by a soft update strategy , which ensures the smooth transition of the value estimate, i.e.

[0185]

[0186] After the Critic network parameters are updated, the Actor network, as the "policy executor", will optimize its policy according to the value gradient information provided by the Critic. Specifically, the action value output of the Critic network based on the current policy will indicate how the Actor adjusts the policy parameters to obtain a higher return. The Actor network first calculates and determines the growth direction of the action value (gradient direction) to improve the policy performance:

[0187]

[0188] The gradient can be decomposed into the Critic's gradient response to the action a and the Actor's influence value on the policy . Then, the online Actor network parameters are updated in the direction of gradient ascent to maximize the long-term return, i.e.

[0189]

[0190] where a Q is the Actor learning rate, usually a μ < a Q . Similarly to the Critic, the Actor also updates its parameters by a soft update strategy.

[0191]

[0192] ​The Actor-Critic based collaborative optimization framework successfully solves the continuous action space optimization problem in UAV swarm collaborative scheduling by decoupling the iterative mechanism of policy evaluation and policy improvement. It directly generates high-dimensional continuous control instructions in an end-to-end manner, ensuring algorithm stability while significantly improving decision-making efficiency in complex dynamic environments.

[0193] During the entire step implementation process, in the initialization phase, the algorithm first constructs the Actor network for selecting actions and the Critic network for evaluating action values, and initializes the parameters of the two networks and the experience replay pool. At the same time, the system also randomly deploys all UAVs in the task execution area to wait for task release.

[0194] After entering the task execution phase, the algorithm starts to execute by rounds. In each round, the system first updates the state information according to the current PoI and UAV information. Then, the MCS server assigns the perception task according to the current state information of the UAV. The flight path, perception time, and local calculation proportion of each UAV determine its action mode in this round. Specifically, the system selects an action for each UAV, including the PoI the UAV flies to, the perception time at the task point, the local calculation proportion, and the allocated transmission bandwidth value. After the action is selected, the UAV executes the flight, data collection, calculation, and communication processes according to the selected action and obtains a reward value, which reflects the contribution of the current action to the overall system utility.

[0195] After the UAV executes the action, the system stores the current state, action, reward, and next state into the experience replay pool. When the the batch size threshold B a is reached, a small batch of data is randomly extracted for training. This mechanism breaks the time correlation of data and improves the learning efficiency. The Critic network, as the "policy evaluator", aims to accurately predict the long-term value of the action. First, the Critic uses the target network to generate stable value estimates, and then iteratively adjusts the Critic parameters by minimizing the mean square error between the predicted value and the target value. The Actor network, as the "policy executor", optimizes its strategy based on the value feedback provided by the Critic. The Critic network outputs the action value as the gradient direction of the current strategy, indicating how the Actor adjusts the strategy parameters to obtain higher returns. Then, the Actor network updates the network parameters along the gradient direction, making the strategy tend to select high-value actions recognized by the Critic. This process forms a closed loop with the Critic update, realizing the progressive improvement of the strategy.

[0196] To avoid the abrupt change of the target value estimation, a soft update strategy is adopted between the target network and the online network: the parameters of the online network are gradually synchronized to the target network in a small proportion, ensuring the smooth transition of the value evaluation. This mechanism significantly improves the training stability. At the same time, in order to balance exploration and utilization, Gaussian noise is injected when the actor generates actions, and the noise intensity is gradually attenuated during the training process in order to achieve the purpose of focusing on extensive exploration of the environment at the beginning and relying on the optimized strategy to efficiently execute tasks at the end.

[0197] In the present model, since the selection of the state is ergodic, the algorithm converges after multiple task execution rounds, at which time each state corresponds to a unique optimal policy with the maximum return, which can maximize the data collection of the UAV within a limited time while saving energy as much as possible. Thereafter, each action of the agent represents the optimal crowd sensing and task scheduling method in the current state.

[0198] In order to verify the effect of the system and method provided in the present embodiment, a simulation experiment is set up, including the training environment and the hyperparameters of the neural network; subsequently, the convergence performance of the algorithm is analyzed in detail through experimental simulation, and the proposed algorithm is compared with several baseline algorithms in different environments to prove the effectiveness of the proposed algorithm in various experimental scenarios.

[0199] For model training, the crowd sensing and task scheduling algorithm for UAV is implemented using PyTorch 1.11.0, and the NVIDIA GTX 3090 graphics card is used to train the deep neural network of the agent (i.e., the Actor network and the Critic network). Specifically, the input layer of the Actor network and the Critic network receives an input of state dimension size, and after passing through a hidden layer with a hidden width of 256, the Actor network is connected to the output layer for generating actions, and the Critic network is connected to the output layer for evaluating the state value.

[0200] During training, the Adam optimizer is used to optimize the parameters of the neural network with a learning rate of 0.001, and the neural network is updated regularly. The training batch size and the replay buffer size are set to 64 and 10,000, respectively. The discount factor γ of future rewards is 0.99, and the soft update rate τ of the target network is set to 0.05.

[0201] Table 1 shows the remaining neural network parameter settings during training.

[0202]

[0203] In the simulation, several UAVs are set to perform the perception task, where the execution location of the perception task is randomly and uniformly generated and distributed in the target area by the task publisher, and the target area is a two-dimensional square with a size of 200unit*200unit, and each UAV will be responsible for perceiving the PoI distributed in the target area. The flight speed of the UAV is set to [10, 20] units / s, the flight height is fixed at 10 units, the flight power is 40W, and the hovering power is 30W. In terms of data perception, the perception rate of each UAV is 1.5Mbps, and the energy consumption of the perception task is [10 -7 ,10 -6 ]J / bit; in terms of computing tasks, it is assumed that the CPU operating frequency of the UAV is [2, 3]GHz, and the UAV needs about 100-1000 CPU cycles to process each bit of data. In addition, the time slot length of each task execution round is fixed at 10 seconds; in terms of data transmission, the transmission power of the UAV is set to 0.1-0.2W, the total bandwidth of the available channel of all UAVs is 10MHz, the channel noise power density is-100dBm / Hz, and the path loss factor is 4.

[0204] Table 2 summarizes all the parameters used in the MCS system:

[0205]

[0206] To evaluate the convergence performance of the algorithm, the average system utility value of the agent in the training process is analyzed in this embodiment. In this set of experiments, the perception task execution environment of five UAVs is set, Figure 4 The performance of the algorithm in the training process is shown.

[0207] As shown in Figure 4 , in the early stage of training (about 250 training steps), the reward curve shows a rapid growth trend. This indicates that the agent has begun to learn an effective strategy, and the parameters of the strategy network are gradually adjusted through gradient optimization in the training process, improving the effect of the agent interacting with the environment. In the middle stage of training, although the reward curve shows a certain degree of fluctuation, the overall trend is still rising. This fluctuation may be due to the uncertainty introduced by the Actor-Critic algorithm in updating the policy network and value network using the multi-step temporal difference (TD) method. At this stage, the model needs to balance between exploring new strategies and optimizing the current strategy. In the later stage of training, the reward curve gradually tends to be stable, showing good convergence, indicating that the model has learned an optimal strategy. At this time, the agent can more stably obtain a higher reward from the environment, indicating that the learned strategy has good adaptability and robustness in different states.

[0208] To evaluate the performance of the proposed algorithm, the following four baseline algorithms are designed for comparison:

[0209] Random algorithm: In each task execution round, the random algorithm will randomly generate actions for each UAV within the set action value range, including the selected PoI number, local offloading ratio, and perception time. The random algorithm is simple and fast, and can generate actions instantly based on any state. However, it cannot effectively consider the current environmental state and task requirements, so this algorithm also serves as a baseline lower bound for performance comparison.

[0210] Upper bound strategy: The optimal action strategy will traverse all possible action combinations in a certain state and calculate the corresponding utility value. Then, based on the greedy strategy, the action combination with the highest utility value is selected and executed. This strategy can theoretically guarantee that the task scheduling utility and scheduling success rate of the UAV reach the global optimum in the current state, providing a theoretical upper bound for algorithm performance. However, due to the exponential growth of the number of action space combinations, the optimal strategy has very high computational complexity, and its calculation time is thousands of times longer than the time slot length of the task execution round, so it cannot be used in practice.

[0211] Genetic algorithm (GA): Genetic algorithm optimizes action selection by simulating the process of natural selection. In the specific implementation, the parameter configuration of fixed population size N = 20, mutation probability p u = 0.1, and maximum iteration number R max is dynamically adjusted according to the number of UAVs participating in the task to balance search efficiency and calculation time. In terms of action formulation, genetic algorithm needs to encode the UAV's action and randomly generate multiple different action genes. Then, these genes are subjected to a certain number of iterations of crossover, mutation, selection, and other operations. Finally, in a certain time period, the UAV's action is selected as the action corresponding to the gene with the highest fitness. Specifically, the fitness function of the gene is set as the reward function, and the individual with the highest fitness in the last generation is set as the action value of the UAV in this round.

[0212] Simulated annealing algorithm (SA): Simulated annealing algorithm borrows the principle of gradually cooling to reach the lowest energy state in the physical annealing process of metals to gradually optimize action selection. The algorithm searches for the neighborhood solution of the current solution and accepts suboptimal solutions with a certain probability, gradually converging to the global optimum. For the continuous variable characteristics of the UAV action space, in the specific implementation, the initial temperature T0 = 100 and the termination temperature T min = 10 -2The cooling rate is dynamically adjusted according to the number of UAVs to balance the calculation time. Compared with the genetic algorithm, the simulated annealing is more suitable for scenarios containing continuous variables in the action space, and the current solution obtained after the temperature is reduced to the threshold or the number of iterations reaches the upper limit is selected as the action of the UAV in this round.

[0213] In this embodiment, the AC-TRO (Actor-Critic for Task-based Resource Optimization) algorithm will be compared with three baseline algorithms from the perspective of multiple performance indicators. It is worth mentioning that in the design of the comparative experiment, in view of the difference between the heuristic algorithm and the paradigm of reinforcement learning-the former needs to restart the iterative search at any state (without the ability of strategy reasoning), and the latter has the ability of instant state-action mapping, based on the time consumption equivalence principle, the super parameter of the heuristic algorithm is adjusted: by constraining the population size, the annealing temperature decay rate and other key super parameters, ensure that the single step decision calculation delay and the reasoning time delay of the strategy network of the reinforcement learning keep the order of magnitude alignment (±15% error threshold), so as to eliminate the interference of the difference of the calculation overhead on the performance evaluation of the system. Specifically, simulation is carried out in three groups of experimental environments, and there are 3, 5 and 8 UAVs in the perception scene respectively. Since the strategy after the algorithm converges represents the approximation state of the theoretically optimal performance, in order to evaluate the long-term performance of the algorithm in the stable stage, Figure 5 The average system utility of AC-TRO and baseline algorithms in each round after convergence in each test scene is compared. When the number of UAVs is only 3, the average system utility of the AC-TRO algorithm is not much different from that of other baseline algorithms, however, with the increase of the number of UAVs, the performance of the AC-TRO algorithm is greatly superior to that of other algorithms, which can be attributed to the following points:

[0214] (1) With the increasing number of UAVs, the state space of the problem expands exponentially. Genetic algorithms and simulated annealing algorithms rely on random search and iterative optimization. In a larger state space, the difficulty of searching all possible solutions comprehensively increases significantly. This leads to a rapid increase in computational complexity, and the algorithm needs more time and computational resources to find the optimal solution or an approximate optimal solution.(2) In a smaller state space, local optimal solutions are usually close to the global optimal solution, so heuristic algorithms can gradually approach the global optimal solution through iteration. However, as the state space increases, the number of local optimal solutions increases, and some local optimal solutions may be far from the global optimal solution. Heuristic algorithms are more likely to be misled by local optimal solutions in this case, making it difficult to find the true global optimal solution.(3) Heuristic algorithms are highly sensitive to the selection of initial values. In genetic algorithms, the quality of the initial population directly affects the search efficiency. If the initial population is of low quality, the algorithm may fall into a local optimal solution in the early stages. In simulated annealing algorithms, the setting of the initial solution affects the neighborhood structure and the generation of new solutions, which in turn determines the algorithm's ability to escape local optimal solutions. As the state space expands, the difficulty of selecting a high-quality initial solution increases significantly, which makes the algorithm more susceptible to the quality of the initial solution, causing it to deviate from the global optimal solution during iteration. In addition, the larger the state space, the more complex the problem characteristics and constraints. If the initial solution does not fully reflect these characteristics, it will further increase the risk of the algorithm falling into a local optimal solution, leading to performance degradation.

[0215] In view of the limitations of heuristic algorithms in large state spaces, the AC-TRO algorithm exhibits significant advantages. First, the algorithm can learn an effective strategy directly from a complex high-dimensional state space in an end-to-end manner through the collaborative work of the Actor network and the Critic network. Unlike heuristic algorithms that rely on random search and fixed rules, AC-TRO dynamically adjusts the strategy in each iteration, effectively avoiding falling into a local optimal solution through real-time feedback from the Critic. At the same time, since the algorithm uses reinforcement learning-based training, it can use the more efficient experience replay mechanism in larger state spaces, avoiding excessive sensitivity to initial solutions. More importantly, AC-TRO has strong generalization ability and adaptability. Even in the case of rapid expansion of the state space, it can still capture the global characteristics of the problem through deep networks, thereby optimizing the global strategy. In addition, AC-TRO gradually approaches the optimal strategy during training, avoiding the problem of excessive dependence on high-quality initial solutions in large state spaces, significantly improving the computational efficiency and performance of the algorithm in complex scenarios.

[0216] Meanwhile, in addition to the average system utility indicator, the task orchestration success rate is proposed to measure the effectiveness of the perception task execution. This indicator reflects the degree of satisfaction of the UAVs to the perception task requirements during the task execution process. Specifically, when the PoI allocation between multiple UAVs does not conflict, and the sum of the flight time and the perception task execution time of each UAV is less than the duration of a single task execution round, the perception task orchestration is considered successful. If the task orchestration fails, the data center will automatically trigger a multi-round calculation process to try to generate a feasible solution by dynamically adjusting the POI parameters. This mechanism is designed for extreme POI setting scenarios. After the first round of failure, the system will retain the current task context and start a constraint relaxation strategy, such as relaxing the time threshold or increasing the redundancy of collaborative UAVs, and then perform iterative parameter optimization calculation. If the adjustment still cannot meet the orchestration conditions after multiple rounds, it is determined that there is no global feasible solution, and the system will directly terminate the task orchestration of the current round, clear the execution queue, and trigger a new round of task assignment process.

[0217] As shown in Figure 6 , the task orchestration success rate of the algorithm is evaluated and compared, and the results roughly show similar trends as the average system utility indicator in Figure 5 . However, in the case of growing state space, the heuristic algorithm shows more significant performance decline in task orchestration success rate, with a larger decline in average system utility. This reflects a significant feature of the heuristic algorithm in optimization strategy: utility priority. Specifically, the heuristic algorithm prefers to select actions with higher individual task utility, which leads to a significant decline in task success rate. This phenomenon reveals the limitations of the heuristic algorithm in complex state space: when the state space expands, due to the need to handle more possible action combinations, the heuristic algorithm may focus on improving the contribution of a few high-utility tasks in the optimization process, neglecting the overall task orchestration optimization. The heuristic algorithm sacrifices the comprehensiveness and balance of task orchestration in pursuit of local optimal single-task utility.

[0218] In contrast, the AC-TRO algorithm exhibits good adaptability to the increase of the state space in this scenario, which indicates that the AC-TRO algorithm does not achieve higher system utility by reducing the number of completed perception tasks. Through the combination of the policy network and the value network, the AC-TRO algorithm can effectively balance the relationship between task utility and task scheduling success rate. In the case of state space growth, the AC-TRO algorithm can still evaluate the overall value of different actions from a global perspective, avoiding the neglect of the overall success rate of tasks by only focusing on a few high-utility tasks. Specifically, the AC-TRO algorithm uses the policy network to efficiently sample complex action spaces that are continuous or discrete, and at the same time uses the state value provided by the value network as the optimization direction. This mechanism enables the algorithm to more accurately evaluate the long-term value of actions in complex state spaces, significantly reducing the negative impact of state space expansion on performance. In contrast, heuristic algorithms are more susceptible to the influence of complex state spaces due to the lack of dynamic adjustment mechanisms and global value evaluation capabilities, especially in scenarios with task allocation conflicts and limited time resources.

[0219] Figure 7 The average energy consumption of the AC-TRO algorithm and other comparative algorithms in executing intelligent perception tasks in various scenarios is shown. It can be seen that the energy consumption of the AC-TRO algorithm proposed in this embodiment in completing perception tasks is lower than that of other comparative algorithms, indicating that the algorithm is more efficient in energy utilization. Through reasonable resource optimization and action selection, AC-TRO can effectively reduce the average energy consumption, thereby demonstrating better energy efficiency in the execution of perception tasks.

[0220] In summary, the resource optimization method provided in this embodiment is superior to other benchmark algorithms in multiple performance indicators, and can balance task utility and success rate to maintain high overall performance in complex task scenarios, demonstrating good scalability and stability.

[0221] Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for part of the technical features, and any modification, equivalent substitution, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A collaborative swarm intelligence sensing system based on unmanned aerial vehicle (UAV) swarms, characterized in that, This includes a data center, a central base station, a sensing platform, and multiple drones of the same model; The data center is used to generate sensing task requirements and expects to collect data from specific Points of Interest (PoIs) in the region; The central base station and the sensing platform have the same coordinates (x0, y0, H0). The central base station serves as the communication hub to ensure a stable wireless connection between the data center, the sensing platform, and multiple UAVs of the same model. The sensing platform is used to receive and process sensing data uploaded by multiple drones of the same model, and feed the results back to the data center; Multiple drones of the same model are used to receive the task location issued by the sensing platform, fly to a specific location (PoI), hover above it, and collect sensing data.

2. The sensing system based on collaborative swarm intelligence of unmanned aerial vehicle (UAV) swarms according to claim 1, characterized in that, The perception data collected by multiple drones of the same model are processed locally, and then uploaded to the perception platform as the local calculation result. The other part is uploaded directly to the perception platform as raw data without any calculation processing. The sensing platform processes the received raw data and merges it with the received local calculation results to form the sensing results. After collecting sensing data from all specific location Points of Interest (PoIs), the sensing platform integrates the sensing results and returns all sensing results to the data center.

3. A resource optimization method for a sensing system based on collaborative collective intelligence of unmanned aerial vehicle (UAV) swarms, applicable to the sensing system based on collaborative collective intelligence of UAV swarms as described in claim 1 or 2, characterized in that, Includes the following steps: S1. System and network initialization: In the data center, construct and initialize the neural network model required by the algorithm, including the online policy network μ, the target policy network μ′, the online evaluation network Q, and the target evaluation network Q'; S2. Construct and perceive the system state. At the beginning of each decision round t, the data center acquires and integrates the perception of the current system state s(t); S3. Generate and issue a joint action. Input the current state s(t) into the online policy network μ to generate a joint action a(t). The generated joint action a(t) contains all the scheduling instructions for multiple UAVs of the same model in the collaborative swarm intelligence perception system based on UAV swarms. S4. Execute tasks and interact with the environment. Each UAV in the perception system executes tasks according to the joint action a(t) issued by the data center. After the task is executed, the system environment transitions from the current state s(t) to the state s(t+1) of the next round. S5. Calculate the immediate reward and store the experience. The data center calculates the immediate reward R(t) based on the execution result of the task in this round, and stores the experience quadruple (s(t), a(t), R(t), s(t+1)) generated in this interaction into the experience replay pool Z. S6. Network model parameter update: When the number of samples in the experience replay pool Z reaches the preset training start threshold and the network update frequency condition is met, a training batch of size B is randomly sampled from Z. a Transfer sample Update the online evaluation network Q and the linear policy network μ; S7. Iteration and convergence judgment: Repeat steps S2 to S6 until the algorithm reaches the preset maximum number of training rounds or the instantaneous reward R(t) curve of the perception system converges. S8. Save the final parameters of the online policy network μ after training convergence.

4. The resource optimization method for a sensing system based on collaborative swarm intelligence of unmanned aerial vehicle (UAV) swarms according to claim 3, characterized in that, S1 includes the following sub-steps: Construct the online policy network μ and the target policy network μ′, and randomly initialize their network parameters. and Ensure initial Construct the online evaluation network Q and the target evaluation network Q′, and randomly initialize their network parameters. and Ensure initial Initialize the experience replay pool Z; Initialize the training hyperparameters, including the learning rate α for both Actor and Critic. μ and α Q The reward discount factor γ, the target network soft update rate τ, and the training batch size B a .

5. A resource optimization method for a sensing system based on collaborative swarm intelligence of unmanned aerial vehicle (UAV) swarms according to claim 4, characterized in that, In S2, the current state s(t) of the sensing system is defined as: s(t)={(S PoI (t),S UAV (t))}; Among them, S PoI (t) is the coordinate vector of all PoIs to be sensed in the current round, S UAV (t) is the hovering position vector of all drones at the end of the previous round.

6. The resource optimization method for a sensing system based on collaborative swarm intelligence of unmanned aerial vehicle (UAV) swarms according to claim 5, characterized in that, In S3, the joint action taken by the i drones is a i (t), and defined as: in, It is the PoI number selected by i drones at a specific location. It is the sensing time set by drone i during round t. It represents the proportion of local computation for i drones in round t. It is the bandwidth allocated to i drones in round t.

7. A resource optimization method for a sensing system based on collaborative swarm intelligence of unmanned aerial vehicle (UAV) swarms according to claim 6, characterized in that, In S5, the instant reward R(t) is defined as:

8. A resource optimization method for a sensing system based on collaborative swarm intelligence of unmanned aerial vehicle (UAV) swarms according to claim 7, characterized in that, In step S6, when the sample size in the experience replay pool Z reaches the threshold B... a Then, the system will uniformly sample small batches of state transition samples from the experience replay pool. have:

9. A resource optimization method for a sensing system based on collaborative swarm intelligence of unmanned aerial vehicle (UAV) swarms according to claim 7, characterized in that, In step S6, the online evaluation network parameters are updated. Includes the following sub-steps: The temporal difference (TD) target y is computed through the target network. j : Where γ∈(0,1) is the discount factor. It is the output value of the target evaluation network, and its parameters are... Updates will be delayed; It is a target policy network used to generate the joint action a′ for the next state. j ; By minimizing the online evaluation network prediction value and the TD target y j Mean squared error loss function between To update the parameters of the online evaluation network in, It is the output value of the online evaluation network, α. Q It is used to evaluate the network learning rate; Update online policy network parameters Includes the following sub-steps: The policy gradient used to update the online policy network is calculated using the sampling policy gradient theorem. in, It evaluates the gradient response of the network to action a. It is the influence value of the policy network on the policy; Update the online policy network parameters along the policy gradient ascent direction To maximize long-term returns, we have: Where α μ Let α be the learning rate of the policy network, and let α be the learning rate of the network. μ <α Q .

10. A resource optimization method for a sensing system based on collaborative swarm intelligence of unmanned aerial vehicle (UAV) swarms according to claim 9, characterized in that, In step S6, the updated online network parameters are smoothly synchronized to the corresponding target network at a small update rate τ (τ << 1) to maintain training stability, as follows:

Images