An evolutionary game theory based method and system for cooperative search of distributed unmanned aerial vehicle swarm

Abstract

The application provides a kind of based on the cooperative search method and system of distributed unmanned aerial vehicle cluster of evolution game theory, the method comprises: first, the expected return of each unmanned aerial vehicle search strategy is initially predicted by Monte Carlo method, the relationship between strategy and expected return is initially determined, then enter the strategy optimization stage based on evolution game, that is, with the search process, each unmanned aerial vehicle constantly updates expected return according to certain rules, the strategy evaluation value of the current strategy of each unmanned aerial vehicle in subnet is calculated, and each unmanned aerial vehicle optimizes and updates strategy based on evolution game mechanism according to strategy evaluation value, wherein the principle of strategy optimization and update is that excellent strategy will be accepted by more unmanned aerial vehicles, and poor strategy will be gradually eliminated. Through this dynamic evolution process, the unmanned aerial vehicle cluster can adaptively select the optimal search strategy in the connected subnet, so as to realize efficient cooperation of the cluster under distributed architecture, and significantly improve the efficiency of cooperative search.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) swarm technology, and in particular to a cooperative search method and system for distributed UAV swarms based on evolutionary game theory. Background Technology

[0002] Research on collaborative search missions using drone swarms is becoming increasingly important, as it is crucial for improving efficiency and success rates, especially for complex tasks. Compared to single drones, multiple drones, through interaction and understanding of environmental information, can significantly improve mission efficiency, mission scenario coverage, and robustness by rationally allocating tasks, making collaborative decisions, and adaptively adjusting. Swarms containing multiple drones have high practical application value for the collaborative search of multiple dynamic targets within a region, and swarm collaborative search is fundamental research for many practical applications.

[0003] For drone swarms, distributed architecture is a commonly used approach, delegating decision-making authority to each drone. Compared to centralized architecture, drones in a distributed architecture can make fully autonomous decisions based on their own sensor information and communication with other drones. Furthermore, distributed control architecture offers a wider range of applications and greater development potential (especially in large-scale swarms or highly dynamic environments). However, ensuring efficient and effective information exchange and understanding between drones in a distributed swarm, and guaranteeing the global nature of the distributed swarm system, remains a key challenge.

[0004] Furthermore, regarding the problem of collaborative search in a distributed architecture-based UAV swarm, existing technologies are mostly based on deterministic collaborative strategies, i.e., using preset rules or algorithms. While these methods are simple to implement, for fully autonomous decision-making individuals, fixed strategies often fail to fully leverage their autonomous decision-making advantages. Moreover, fixed strategies not only make UAVs difficult to adapt to various complex dynamic environments, but also involve cumbersome parameter optimization issues.

[0005] Therefore, there is an urgent need for a novel collaborative search method for UAV swarms under a distributed architecture, which can fully leverage the advantages of fully autonomous decision-making by UAVs under a distributed architecture, thereby dynamically optimizing strategies, while also ensuring that the group decisions made by the distributed swarm system have a certain degree of globality. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a cooperative search method and system for distributed drone swarms based on evolutionary game theory, which can dynamically optimize the search strategy when the drone swarm performs cooperative search tasks, thereby giving full play to the advantages of the drones' fully autonomous decision-making, and the strategy optimization can also enable the drones to adapt to complex dynamic environments more quickly.

[0007] One aspect of the present invention provides a cooperative search method for distributed drone swarms based on evolutionary game theory. For each drone within the drone swarm, the method includes:

[0008] During the strategy optimization phase, at the planning step, the following steps are performed:

[0009] Drone networking: Based on the maximum communication distance between drones, the current drone communicates with other drones in the cluster to form a connected subnet; the current drone applies a strategy in the strategy space, the connected subnet contains multiple drones applying different strategies, and the current drone stores the correspondence between each strategy and expected reward in the strategy space, as well as a search information graph;

[0010] UAV information fusion and optimal decision path determination: The current UAV receives the search information map stored by other UAVs in its connected subnet, and updates its own stored search information map using the received search information map; based on the updated search information map and the strategy applied by the current UAV, it selects an optimal decision path in the rolling planning time domain from the task search area, and calculates the expected execution benefit of the optimal decision path;

[0011] The correspondence between various drone strategies and expected benefits is updated: the expected benefits corresponding to the current drone application strategy are updated based on the expected execution benefits, and the current drone receives the strategies and updated expected benefits of other drone applications in its connected subnet, thereby updating the stored correspondence according to the set expected benefit update rules.

[0012] Drone strategy evaluation value: Based on the expected execution revenue and the expected update revenue corresponding to the current drone application strategy, the relative evaluation value of the strategy applied by the current drone is calculated. Then, based on the relative evaluation value at the current planning step and the relative evaluation value at a specific planning step, the weighted relative evaluation value of the strategy applied by the current drone is calculated. Here, the specific planning step refers to the planning step of the current drone using the strategy within the time period from the historical planning step with a distance of a set sliding window length from the current planning step.

[0013] The drone updates its current policy: The drone determines the maximum weighted relative evaluation value within its connected subnet. Based on the weighted relative evaluation value of the policy applied by the drone and the maximum weighted relative evaluation value, the drone calculates the policy learning probability corresponding to the current drone. Based on the policy learning probability, the drone updates its current policy to the policy corresponding to the maximum weighted relative evaluation value.

[0014] During the strategy optimization phase, the following steps are performed within the execution time domain:

[0015] Drones execute tasks and update search information: In the execution time domain, the current drone executes the search task according to the optimal decision path, captures the target and outputs the target position when the target search conditions are met, and updates its stored search information map based on the detection information during the search task execution process; wherein, the execution time domain and the planning time domain are the planning step time when the optimal decision path is selected and one or more consecutive planning step time times thereafter, and the planning time domain is greater than or equal to the execution time domain; after the steps in the execution time domain are completed, if the search stopping condition is not met, it jumps to the next planning step time and repeats all the steps in the above planning step time and execution time domain, and finally completes the collaborative search of the drone swarm.

[0016] In some embodiments of the present invention, the expected return update rules set for each strategy in the correspondence relationship include:

[0017] If the expected return corresponding to the strategy in the correspondence is not 0, then the updated expected return corresponding to the strategy is the original expected return in the correspondence.

[0018] If the expected return corresponding to the strategy in the correspondence is 0, and the strategy is not the strategy currently applied by any drone in the connected subnet to which the current drone belongs, then the updated expected return corresponding to the strategy is the default value.

[0019] If the expected return corresponding to the strategy in the correspondence is 0, and the strategy is the strategy currently applied by other drones in the connected subnet to which the current drone belongs, then the updated expected return corresponding to the strategy is the average of the updated expected returns obtained by other drones in the connected subnet to which the drone belongs that use the strategy.

[0020] In some embodiments of the present invention, the weighted relative evaluation value of the strategy applied by the UAV at the current planning step time is calculated based on the relative evaluation value at the current planning step time and the relative evaluation value at a specific planning step time, including:

[0021] Given a specific planning step, calculate the average of the relative evaluation values ​​at that specific planning step; by weighted summing the relative evaluation value and the average value at the current planning step, obtain the weighted relative evaluation value of the strategy applied by the current UAV.

[0022] In the absence of a specific planning step, the relative evaluation value of the strategy currently applied by the UAV is used as the weighted relative evaluation value of the strategy currently applied by the UAV.

[0023] In some embodiments of the present invention, the correspondence between each strategy and expected return in the strategy space stored by the UAV is obtained through Monte Carlo prediction operation;

[0024] For each drone within a drone swarm, the Monte Carlo prediction operation includes the following steps: at the planning step, performing drone networking and drone information fusion and determining the optimal decision path; in the execution time domain, performing drone tasks and updating search information; and calculating the expected revenue for each drone.

[0025] The calculation of expected revenue for the UAV involves the following steps: after the prediction stopping condition is met, the average of the actual revenue obtained by the current UAV applying the strategy to perform the search task is calculated at all planning steps of the Monte Carlo prediction operation, and this average is used as the expected revenue corresponding to the current UAV applying the strategy. This leads to the correspondence between the various strategies stored by the current UAV and their expected revenues.

[0026] In some embodiments of the present invention, the search information map stored by the UAV is determined based on the target presence probability, environmental uncertainty, and pheromone information detected by the UAV when performing a search task; and a two-dimensional Gaussian distribution is used to initialize and model the target presence probability in the search information map stored by the UAV.

[0027] The search stopping condition is when the set number of search planning steps is reached at a given time, or when the number of targets captured by the drone swarm reaches a given number; the prediction stopping condition is when the set number of prediction planning steps is reached at a given time; and

[0028] The policy learning probability is calculated using the Fermi function.

[0029] In some embodiments of the present invention, an optimal decision path within the rolling planning time domain is selected from the task search area based on an updated search information graph and the current UAV application strategy, and the expected execution benefit of the optimal decision path is calculated, including:

[0030] Within the planning time domain, a traversal algorithm is used to calculate all feasible paths for the current UAV within the task search area. Based on the updated search information graph, the path revenue of each feasible path is calculated. Based on the calculated path revenue, a set number of paths are selected from all feasible paths as the UAV's pre-decision paths. The path revenue of a feasible path is determined based on the search value revenue and coordination revenue of the feasible path.

[0031] Based on the current drone application strategy and the path revenue of the pre-decision path, the expected execution revenue of the pre-decision path is calculated, and the optimal decision path is selected from the pre-decision paths based on the calculated expected execution revenue.

[0032] In some embodiments of the present invention, the expected execution benefits of the pre-decision path are determined based on the current UAV application strategy, the search value benefits of the pre-decision path, and the coordination benefits of the pre-decision path.

[0033] The search value gain of the pre-decision path is the sum of the pheromone concentration differences in all task sub-regions traversed by the pre-decision path; the coordination gain of the pre-decision path is the sum of the target existence probability and environmental uncertainty in all task sub-regions traversed by the pre-decision path; and the pheromone concentration difference is the difference between the attracting pheromone concentration and the repulsive pheromone concentration.

[0034] In some embodiments of the present invention, the current UAV receives search information maps stored by other UAVs within its connected subnet, and updates its own stored search information map using the received search information maps, including:

[0035] For the target existence probability map and environmental uncertainty map in the search information map, the UAV receives the target existence probability map and environmental uncertainty map stored by other UAVs in its connected subnet;

[0036] For each task sub-region within the task search area, if the sub-region is within the current UAV's search range, the target existence probability map and environmental uncertainty map stored by the current UAV are used as the target existence probability map and environmental uncertainty map in the search information map updated by the current UAV, respectively.

[0037] For each task sub-region within the task search area, if the sub-region is outside the search range of the UAV, calculate the average target existence probability of the sub-region in the target existence probability map stored by other UAVs in the connected subnet to which the current UAV belongs, calculate the square root of the product of the environmental uncertainties of the sub-region in the environmental uncertainty map stored by other UAVs in the connected subnet to which the current UAV belongs, and use the calculation results as the target existence probability map and environmental uncertainty map in the updated search information map of the current UAV, respectively.

[0038] Another aspect of the present invention provides a cooperative search system for a distributed unmanned aerial vehicle (UAV) swarm based on evolutionary game theory, including a processor, a memory, and a computer program / instructions stored in the memory. The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method described in any of the above embodiments.

[0039] Another aspect of the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method described in any of the above embodiments.

[0040] This invention proposes a cooperative search method and system for distributed UAV swarms. Based on evolutionary game theory, it enables adaptive learning and optimization of strategies during the cooperative search process. Through this dynamic evolutionary process, UAVs within the swarm can adaptively select the optimal search strategy, thereby achieving efficient swarm collaboration in a distributed architecture and significantly improving the efficiency of cooperative search.

[0041] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0042] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0044] Figure 1 This is a schematic diagram of a drone swarm collaboratively searching for moving targets in one embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram illustrating the use of a rasterization method to divide the task search area in one embodiment of the present invention.

[0046] Figure 3 This is a schematic diagram of a drone flight path in one embodiment of the present invention.

[0047] Figure 4 This is a schematic diagram of the search range of an airborne sensor for a drone in one embodiment of the present invention.

[0048] Figure 5 This is a flowchart illustrating a collaborative search method for a distributed unmanned aerial vehicle (UAV) swarm based on evolutionary game theory in one embodiment of the present invention.

[0049] Figure 6 This is a schematic diagram of the initial distribution of search information in one embodiment of the present invention.

[0050] Figure 7 This is a schematic diagram of the task area initialization partition in one embodiment of the present invention.

[0051] Figure 8 This is a flowchart illustrating a collaborative search method for a distributed unmanned aerial vehicle (UAV) swarm based on evolutionary game theory, according to another embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0053] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0054] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0055] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0056] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0057] In a distributed architecture, each drone in a drone swarm can make completely autonomous decisions. However, current technologies mostly rely on fixed cooperative strategies to execute cooperative search tasks, which often fails to fully leverage the advantages of autonomous decision-making. Moreover, fixed cooperative strategies not only make it difficult for drones to adapt to various complex and dynamic environments, but also present cumbersome parameter optimization problems.

[0058] Evolutionary Stable Strategy (ESS) is a core concept in evolutionary game theory. Specifically, it refers to the fact that, since the players are boundedly rational, they cannot find the optimal strategy and the optimal equilibrium point at the beginning of the game. Therefore, the players need to continuously learn, gradually correct their strategic errors, and constantly imitate and improve their own and other players' most advantageous strategies in the past. After a period of imitation and error correction, all players will tend to a certain stable strategy.

[0059] Based on this, this application proposes a distributed cooperative search method for UAV swarms based on evolutionary game theory. This method dynamically learns and optimizes strategies during the cooperative search process, fully leveraging the autonomous decision-making advantages of UAVs. Specifically, as the search progresses, each UAV in the swarm updates the expected payoff of its current strategy and evaluates the current strategies of all UAVs in the connected subnet based on the updated expected payoff. Each UAV optimizes and updates its strategy using an evolutionary game mechanism based on its strategy evaluation value, thereby ensuring that high-performing strategies are adopted by more UAVs. Furthermore, in the method proposed in this application, UAVs can make completely autonomous decisions. That is, during the game phase, UAVs can continuously update their search strategies based on their own sensor information and communication information with other UAVs, but during the decision-making phase, they make their own decisions without considering the decisions of other UAVs in the swarm.

[0060] The current collaborative search process of the drone swarm is as follows: Figure 1 As shown, within a specific task search area (also called the task area), there are m moving targets to be detected. The prior information regarding the number, location, and direction of movement of these targets is unknown. A swarm of n UAVs conducts a cooperative search for these moving targets within the task search area without any prior information. The objective of this cooperative search task is, considering the performance of airborne sensors and errors caused by environmental factors, for each UAV to efficiently cooperate in the search, capture as many targets as possible within the set task time domain, minimize the number of false positives, and ensure its own safety.

[0061] To better describe environmental information, simplify the solution space for search decisions, and improve the efficiency of solving UAV swarm collaborative search problems, a gridding method can be introduced. That is, during UAV collaborative search, a gridding method can be used to divide the task search area. By dividing the task search area into multiple task sub-regions, each UAV searches one or more sub-regions, thereby making the information detected by the UAVs more refined. For example, ... Figure 2 As shown, the task area can be divided into N x ×N y Each sub-region is composed of several equally sized grid cells (also called task sub-regions), and each grid cell can serve as an independent information carrier. For ease of subsequent representation, each sub-region can be numbered based on its position within the task search area. For example, the position of each sub-region can be represented as (in this application, (x,y) represents the task sub-region in column x and row y), and each sub-region can be denoted as G. xy . Figure 2 The task sub-region division shown is only an example. This application requires that each task sub-region does not overlap and has the same area, and does not impose specific limitations on the division method and shape of the task sub-region.

[0062] Furthermore, compared to the task area, UAVs conducting swarm cooperative search within a larger task area can be considered as translational point masses on a two-dimensional plane, ignoring the influence of altitude changes. Assuming each UAV in the swarm is located in a certain task sub-region at each planning step, the UAV can perform target search within the task sub-region of its search range, acquiring information about the task area through onboard sensors, and the UAV's flight path is constrained by boundary and velocity constraints. The two-dimensional motion model of UAV i can be described as follows:

[0063]

[0064] Among them, (x i (t), y i (t) represents the sub-region where the projection of UAV i onto the mission area is located at planning step time t, and d i (t) represents the heading angle of UAV i at planning step time t. For example, as... Figure 3 As shown, when the drone type is a rotary-wing drone, it can be assumed that the drone has nine possible directions of motion, corresponding to nine different heading angles di(t)={45°·v|v=0,±1,±2,±3,±4}.

[0065] Furthermore, the search range (also known as the detection range) of a drone is affected by various parameters, including the drone's altitude, the physical parameters of the sensors themselves, and uncertainties in the detection environment. For example... Figure 4 As shown, it is assumed that the detection range of the UAV's onboard sensor does not change with the UAV's altitude, and the UAVs in the cluster of this application can be homogeneous UAVs. Therefore, the detection range of each UAV within the mission area is fixed. Moreover, the search range of UAV i in this application can be defined as: the grid cell where UAV i is located, and the grid cells that share an edge with the grid cell where UAV i is located belong to the search range of UAV i within the mission area.

[0066] The drone swarm performs cooperative search at each planning step. The process of performing cooperative search is the same at each planning step, and the steps of cooperative search performed by each drone in the swarm are the same. Therefore, in the following text, the cooperative search method proposed in this application takes planning step t as an example and describes the cooperative search method proposed in this application as a single drone in the swarm as the execution subject.

[0067] The collaborative search method proposed in this application mainly performs policy optimization operations during the policy optimization phase, such as... Figure 5 As shown, the method may include steps S110 to S160 in the strategy optimization stage.

[0068] Step S110: UAV Networking. Based on the maximum communication distance between UAVs, the current UAV establishes communication connections with other UAVs in the cluster to form a connected subnet. Since the positions of the UAVs in this application are dynamically changing, step S110 needs to be executed at each planning step to ensure the effectiveness of UAV communication, that is, the connected subnet is continuously updated during the cooperative search task.

[0069] Specifically, if multiple drones within a cluster can communicate directly or indirectly with each other (e.g., drone i and drone j can communicate directly or indirectly through other drones), then these drones can form a connected subnet. A drone cluster can form at least one connected subnet, and no two connected subnets will contain the same drone.

[0070] As an example, existing solutions typically form connected subnets based on the maximum communication distance between UAVs. During cooperative search tasks, the UAV swarm can form a network based on a communication distance threshold at each planning step, and use a directed graph G = (U, D) to represent the communication connections between UAVs, where U represents the set of all UAVs in the swarm, and D represents the set of Euclidean distances between each UAV within the task search area (or simply the task area). If the communication connections between UAVs in the swarm are stored in matrix form, the adjacency matrix A of the UAV swarm can be defined as:

[0071]

[0072] Among them, a ij This represents the communication connection status between drone i and drone j, n represents the number of drones in the drone set U, and d represents the communication connection status between drone i and drone j. ij C represents the Euclidean distance between drone i and drone j. r This represents the maximum communication distance between drones. As can be seen from equation (2), when the distance between two drones is less than or equal to the threshold C... r When they are in a certain state, it is assumed that a communication link can be established between them, i.e., a ij =1 (at this time, drone i and drone j are in a communication connection state), when the distance between the two drones is greater than the threshold C r When this happens, it is assumed that a communication link cannot be established between them, i.e., a ij =0. Additionally, by default, drone i can be set to not be able to establish a communication link with itself, i.e., a ii =0.

[0073] Furthermore, based on the adjacency matrix A mentioned above, the communication topology matrix C of the entire UAV swarm (also known as the reachability matrix of the directed graph G = (U, D)) can be calculated. The formula for calculating the communication topology matrix C is as follows:

[0074] For example, suppose the total number of drones in the cluster is n = 4, and the adjacency matrix at planning step t is:

[0075]

[0076] It can be calculated that:

[0077]

[0078] Element c in the communication topology matrix C ij Indicates whether there exists at least one communicative reachable path from UAV i to UAV j. If c ij If the expression is not equal to 0, it indicates that there exists a communication reachable path from UAV i to UAV j. Therefore, the communication topology matrix C calculated above can be further simplified to a binary matrix C′:

[0079]

[0080] The results of C′ above indicate that in this example, the four drones in the drone swarm are in a fully connected state at the planning step t, that is, there is a direct or indirect communication path between any two drones in the swarm. At this time, the drone swarm contains only one connected subnet, and the drones in this subnet are the swarm itself.

[0081] As shown in the above example, the specific process of forming a connected subnet in a cluster is as follows: The adjacency matrix A of multiple drones is determined based on the maximum communication distance between them, and a communication topology matrix C is constructed based on adjacency matrix A; if each element c in the communication topology matrix C... ij All are c ij If the value is not equal to 0, then these multiple drones can form a connected subnet, enabling communication between drones located within the same connected subnet. A drone swarm can form one or more connected subnets. For example, at planning step t, the set of connected subnets N(t) formed by the drone swarm can be represented as:

[0082] N(t) = {(UVA1,UVA2,...,UVA...} K1 (UVA1, UVA2, ..., UVA) K2 ),...,(UVA1,UVA2,...,UVA Kw )};...(6)

[0083] Among them, UVA Kw This represents the Kw-th UAV within a connected subnet at the current planning step time t.

[0084] Furthermore, if at planning step t, a drone is unable to communicate directly or indirectly with other drones in the drone swarm, then that drone will still perform the cooperative search task at planning step t, but will not perform steps S140 and S150.

[0085] In some embodiments of the present invention, during the execution of a cooperative search task, the UAV collects information within the task search area, fuses its own stored historical information, detected information, and information from other UAVs obtained through communication via a connected subnet, and optimizes strategies and makes decisions based on the fused information. In the cooperative search method proposed in this application, step S120 includes: step S121, the UAV fuses information to obtain an updated search information map: the current UAV receives the search information maps stored by other UAVs within its connected subnet, and updates its own currently stored search information map based on the received search information map.

[0086] In some embodiments of the present invention, the search information stored by each UAV within the cluster can be presented in the form of a search information map. Since the search information maps stored by each UAV may differ, the search information maps obtained from other UAVs through communication via connected subnets may also differ, and the search areas and collected environmental information of each UAV may differ. Therefore, at the same planning step, the updated search information maps of each UAV within the cluster may be different. Furthermore, the types of search information stored by UAVs typically include various types, such as target presence probability, and can be designed according to the collaborative search task. To specifically illustrate the subsequent information fusion process, this application uses three types of search information—target presence probability, environmental uncertainty, and pheromone concentration—as examples. Therefore, the search information map may include a Target Probability Map (TPM), an Environmental Uncertainty Map (EUM), and a Digital Pheromone Map (DPM). That is, the search information map stored by the UAVs is determined based on the target presence probability, environmental uncertainty, and pheromone information detected by the UAVs during the search task.

[0087] The search information map is a collection of historical information about the search area stored by UAVs. It uses individual sub-regions as information carriers, recording information about each sub-region within the task area based on the UAVs' detection results and communication information. This reflects the UAV swarm's cognitive state of the current task area during collaborative search missions. The search information map is continuously updated during the UAV swarm's detection process. Updated maps further deepen the swarm's understanding of the task area, guiding the UAV swarm to plan search paths and make decisions based on the updated search information map—a positive feedback process.

[0088] More specifically, at the planning step time t, the drone swarm is based on the maximum communication distance C. r After forming a connected subnet, drones within the same subnet need to obtain search information maps from other drone members in addition to their own stored search information maps. That is, in order to enhance the drones' global perception of the mission search area, deepen their understanding of the environment, and guide the drone swarm to make better decisions, the drones within the swarm need to perform information fusion.

[0089] As an example, the target existence probability information mainly includes the probability of the target existing within the task area (in this application, the target existence probability is designed to be [0,1], where a target existence probability of 0 indicates that the UAV believes that there is definitely no target in the sub-area, and a target existence probability of 1 indicates that the UAV believes that there is definitely a target in the sub-area). The environmental uncertainty information mainly includes environmental uncertainty (used to reflect the degree of information mastery of the UAV cluster over the task area). The pheromone information is used to characterize the concentration of attracting information and the concentration of repulsive pheromones within the task area to guide the movement direction of the UAV.

[0090] In some embodiments of the present invention, each UAV within the cluster selectively accepts a fusion strategy during information fusion, that is, it only accepts and fuses environmental information outside its own detection range, while retaining local information within its detection range. This fusion strategy is designed to fully utilize global information while preserving the individual UAV's own detection information, and to perform focused information classification processing. Therefore, in this application, the process by which a UAV receives a search information map stored by other UAVs within its connected subnet and updates its own stored search information map includes:

[0091] For the target existence probability map and environmental uncertainty map in the search information map, the UAV receives the target existence probability map and environmental uncertainty map stored by other UAVs in its connected subnet;

[0092] For each task sub-region within the task search area, if the sub-region is within the current UAV's search range, the target existence probability map and environmental uncertainty map stored by the current UAV are used as the target existence probability map and environmental uncertainty map in the search information map updated by the current UAV, respectively. If the sub-region is outside the UAV's search range, the average target existence probability of the sub-region is calculated from the target existence probability maps stored by other UAVs in the connected subnet to which the current UAV belongs. The square root of the product of the environmental uncertainties of the sub-region in the environmental uncertainty maps stored by other UAVs in the connected subnet to which the current UAV belongs is calculated. The square root of the product of the average target existence probability and the environmental uncertainty is used as the target existence probability map and environmental uncertainty map in the search information map updated by the current UAV, respectively.

[0093] Specifically, the fusion mechanism of the target existence probability map and the environmental uncertainty map can be expressed by the following formula:

[0094]

[0095] Where, p i (x, y, t) represents the subregion G in the target existence probability map stored by UAV i at planning step time t. xy The probability that the target exists, p i (x, y, t) co This indicates that the subregion G is being referred to. xy The probability of the target existing when the drone i updates at time t during the planning step is G. i (t) represents the set of sub-regions included in the search range of UAV i at planning step t. This represents the total number of drones within the connected subnet to which drone i belongs.

[0096] Where, η i (x, y, t) represents the sub-region G in the environmental uncertainty map stored by UAV i at planning step t. xy Environmental uncertainty, η i (x, y, t) co This indicates that the subregion G is being referred to. xy The environmental uncertainty obtained by updating the drone i at time t during the planning step.

[0097] According to equations (7) and (8), if the subregion G xy Located within the search range of UAV i (i.e., G) xy in G i If (t)), then the target existence probability map and environmental uncertainty map stored by the UAV are retained respectively, and information fusion is not required; otherwise, if the sub-region G xy If the target is located outside the detection range of UAV i, it receives information from other UAVs within the connected subnet, and these information are fused to obtain the target presence probability and environmental uncertainty of the sub-region corresponding to UAV i. This fusion mechanism ensures the integrity of information within the detection range, allowing UAVs to extend their global perception capabilities while maintaining the integrity of information in key areas, thereby achieving efficient decision-making.

[0098] Furthermore, since the pheromone concentration in each sub-region within the mission area is fixed and independent (for example, each sub-region within the mission area may store different concentrations of pheromone), this application does not fuse the pheromone information stored by each UAV. Moreover, although the pheromone information stored by each UAV is not fused, the pheromone concentration gradually dissipates over time within each decision cycle (in this application, executing steps S110 to S160 once is considered one decision cycle), as shown in equations (11) to (15) below.

[0099] As can be seen from the above, after executing step S121, the search information map stored by the UAV is updated to a fused target existence probability map, a fused environmental uncertainty map, and a pheromone information map showing the concentration change over time.

[0100] As an example, a search information graph can be defined as:

[0101] ① A target existence probability map is used to represent the probability of a target existing within each task sub-region. TPM can be defined as:

[0102] P i (t)={p i (x,y,t)|x≤N x y≤N y};…….……...……..…….…..…………(9)

[0103] Among them, P i (t) represents the probability of the existence of the target in the mission area stored by UAV i at planning step t, p i (x, y, t) represents the sub-region G stored by UAV i at planning step t. xy The probability of the target existing at point {x≤N} x y≤N y} represents subregion G xy Located within the mission area. i (x, y, t) = 0 indicates that at planning step time t, the UAV i believes that G... xy It is impossible for a target to exist at p. i (x, y, t) = 1 indicates that at planning step time t, the UAV i considers G to be... xy There must be a goal there.

[0104] ② Environmental uncertainty maps are used to reflect the degree of information a UAV has about each task sub-region, and are usually measured by information entropy. EUM can be defined as:

[0105] U i (t)={η i (x, y, t)|x≤Nx ，y≤Ny};…….…….....….…………..…………(10)

[0106] Among them, U i (t) represents the degree of information that UAV i has about the mission area at time t during the planning step, and η i (x, y, t) represents the value of UAV i on subregion G at planning step t. xy Information mastery level, η i (x, y, t) = 1 indicates that at planning step t, the drone i pairs the subregion G. xy The information within is completely unknown, η i (x,y,t)=0 indicates that at planning step t, the drone i pairs the subregion G. xy We have complete control over the information within.

[0107] ③ A digital pheromone map is used to represent the pheromone concentration within the task area at each planning step. Each UAV maintains its own digital pheromone map. Since different UAVs may not make the same judgment about the same location, the pheromone maps stored by each UAV may be different.

[0108] Pheromones, also known as pheromones, are a key medium for information transmission between organisms in nature. Secreted by individuals, they are dispersed through media such as air and water, and perceived by other organisms of the same species through their sense of smell, triggering changes in behavior, emotions, psychological or physiological mechanisms. Pheromones are widely used in the cooperative behavior of social organisms such as ants and bees. For example, ants release pheromones to mark their paths, guiding their companions to food sources or avoiding dangerous areas. In collaborative search tasks involving drone swarms, drawing inspiration from the communication and interaction mechanisms between organisms in nature, the concept of pheromones can be introduced to mark the search area and target location, simulating and optimizing the swarm's cooperative behavior. Specifically, two types of pheromones—attractive and repulsive—can be designed to guide the drone swarm to better complete the collaborative search task, thereby improving the algorithm's performance and robustness. When a target is detected in a sub-region, considering the uncertainty of airborne sensors, multiple confirmations are needed to confirm the target's presence; therefore, pheromones can also be called revisit pheromones.

[0109] For drone i, DPM can be defined as:

[0110]

[0111] Where s(t) represents the pheromone information within the task region at planning step t, s a (x,y,t) represents the subregion G at planning step t. xy The concentration of attracting pheromones within the body, s r(x, y, t) represents the subregion G at planning step t. xy The concentration of repulsive pheromones within the body.

[0112] (1) Regarding attracting pheromones: Attracting pheromones guide UAVs to previously detected key areas through gravity, allowing them to revisit sub-regions within the area to confirm the target's presence. To simplify the model, the process of attracting pheromones is described as follows: Release – When certain conditions are met (e.g., the sub-region has not been visited beyond the revisit time threshold; the detection result indicates the presence of a target within the sub-region), the sub-region releases pheromones; Evaporation – The pheromones within the sub-region decrease at a certain rate during each decision cycle. If the UAV swarm detects sub-region G… xy When memory is in the target, sub-region G xy It will activate the pheromone switch, releasing new pheromones to guide the drone to revisit the potential target area as soon as possible.

[0113] At time t, subregion G xy The concentration of attractive pheromones within the body can be defined as:

[0114]

[0115] Where ρ represents the pheromone volatilization rate, s a (x, y, t-1) represents the subregion G at planning step time t-1. xy The concentration of attracting pheromones within the body, Δs a This indicates the unit that attracts pheromones. The pheromone attraction switching coefficient, which can be adjusted according to the probability of the detected target's presence, is defined as follows:

[0116]

[0117] Where q(x, y, t-1) represents the number of times UAV i detects subregion G at planning step time t-1. xy Is there a target inside? This indicates that the pheromone attraction switch is turned on. This indicates that the pheromone attraction switch is off.

[0118] Furthermore, in this application, the planning step time can be discontinuous. The planning step time refers to the moment when the UAV performs the cooperative search task. Therefore, the time interval between adjacent planning step times is one decision cycle. For simplicity, this application uses planning step time t-1 to represent the previous planning step time of planning step time t.

[0119] (2) Regarding the repulsive pheromone, at planning step time t, subregion G xy The probability of the internal target existing is greater than the set target existence probability threshold p.t (p t When the probability of target presence in the sub-region is set to 0 (based on empirical values), it is considered that the UAV has already acquired the target in that sub-region. In this case, the probability of target presence in the sub-region needs to be updated to 0, and the UAV should not revisit the sub-region for a short period. To improve the globality of the search and reduce invalid searches, a repulsive pheromone is introduced. Its function is described as follows: Release – When certain conditions are met (e.g., the UAV releases the repulsive pheromone after executing a capture command on the corresponding sub-region), the sub-region releases the pheromone; Evaporation – The pheromone within the sub-region decreases by a certain proportion in each decision cycle. When sub-region G... xy When the probability of a target existing within a certain region exceeds a set threshold, the target is considered to have been captured by the drone, meaning that the target has been captured within a short period of time in sub-region G. xy No new targets will appear within the area. The repulsion pheromone switch is activated, and the sub-region releases new repulsion pheromones, driving the drone to other potential areas for searching.

[0120] At time t, subregion G xy The repulsive pheromone inside is s r (x,y,t) can be defined as follows:

[0121]

[0122] Among them, s r (x, y, t-1) represents the subregion G at planning step time t-1. xy The concentration of repulsive pheromones within, Δs r This indicates a repulsive pheromone increment unit. The pheromone repulsion switching coefficient, which can be adjusted according to the probability of the detected target, is defined as follows:

[0123]

[0124] in, This indicates that the pheromone repulsion switch is on. This indicates that the pheromone repulsion switch is off, p i (x,y,t-1) represents the sub-region G stored by UAV i at planning step time t-1. xy The probability of the target existing.

[0125] In some embodiments of the present invention, when the UAV cluster does not search the mission area, it is necessary to initialize the search information map of each UAV. In addition, since the environmental uncertainty and pheromone information can be updated according to the target existence probability (the environmental uncertainty is updated according to equation (31), and the pheromone information is updated according to equations (12) to (15), the initial setting process of the search information is described below using the target existence probability as an example. The environmental uncertainty and pheromone information can be initialized according to the initial target existence probability.

[0126] Specifically, under local communication conditions, limited by communication distance, UAV swarms typically conduct effective collaborative searches through connected subnets in the initial stage. If a uniform distribution is used to initialize the target presence probability in the task area, the UAV swarm may get trapped in a local optimum in the early stages of the search, resulting in a waste of search resources. Therefore, in the absence of prior information, this application, based on the assumption of rationality, designs a two-dimensional Gaussian distribution to initialize the target presence probability in the task area (i.e., the initial target presence probability map in the search information map stored by the UAVs in this application presents a two-dimensional Gaussian distribution). This initialization method can provide the UAV swarm with a key search area where targets may exist in the initial stage by reasonably setting the center point and variance of the Gaussian distribution (for example, by setting the center point and variance through rapid networking and interaction between the UAVs and environmental information).

[0127] As an example, the initial modeling process for the probability of the existence of a target is as follows:

[0128] At the initial planning step (assuming t=0), each subregion G within the task area... xy The probability density function of the probability that the target exists can be expressed as:

[0129]

[0130] Among them, [x tar y tar ] T This represents the peak position of a two-dimensional Gaussian distribution, used to reflect the initial position where the target is most likely to exist within the mission area. For example, the peak position can be defined as the center position of the mission area. σ 2 σ represents the variance of a two-dimensional Gaussian distribution, and its magnitude determines the degree of dispersion of the target distribution. 2 A larger value indicates higher uncertainty in the target position, σ 2 A smaller value indicates lower uncertainty in the target location, which can be calculated using the following formula:

[0131] σ 2 =(H / k) 2;……..………..……..……..……..……..……..……..……..…………(17)

[0132] Where H is the length or width of the task area, and k is the scaling factor.

[0133] Considering that this application only needs to characterize the relative probability of a target's existence within the mission area, the relative value of the probability density can be directly used as the representation of the target's existence probability. By preserving the relative magnitude of the probability densities (e.g., normalizing the maximum value to 1), the probability distribution of the target at different locations within the mission area can be effectively reflected. Specifically, areas with higher probability density indicate a greater probability of the target's existence, while areas with lower probability density indicate a lower probability of the target's existence. Initialize the target existence probability (see...). Figure 6 (a) in the text and uncertainty of the initial environment (see Figure 6 The distribution diagram of (b) in the figure is as follows: Figure 6 As shown.

[0134] The initial pheromone concentration of each sub-region within the design task area of ​​this application is 0.

[0135] Furthermore, to maximize the dispersion of the UAV swarm in the initial stage and thus improve the information collection efficiency of the mission area, the mission search area can be divided into multiple regions (each region contains one or more sub-regions), and the target existence probability and environmental uncertainty of each sub-region can be independently initialized. For example... Figure 7 As shown, the task search area can be divided into four regions on average.

[0136] In some embodiments of the present invention, each drone in the drone swarm applies a policy from the policy space at each planning step. During the policy optimization phase, the policy applied by each drone may change at different planning steps.

[0137] As an example, to implement the cooperative search function of UAVs, this application sets up a policy space Θ. The policy space Θ contains multiple policies, and the policy adopted by the UAVs in the cluster at each planning step can be obtained from the policy space (i.e., regardless of whether the policy is before or after optimization, the policy applied by the UAV is a policy from the policy space). The policy space can be represented as:

[0138] Θ = {θ1, θ2, ..., θ} b ,...,θ B};….……...……...……...……...……..………(18)

[0139] Where, θ bLet B represent the b-th strategy, and let B represent the number of strategies in the strategy space (for subsequent optimization strategies, B≥2). The strategies in the strategy space can provide a way to calculate the benefits when the UAV performs the search task. For example, in step S120, the expected execution benefits can be calculated using the strategies currently applied by the UAV.

[0140] In some embodiments of the present invention, after obtaining the updated search information through step S121, step S122 is also required to enable the UAV to determine the optimal decision path, including: based on the updated search information map and the current UAV application strategy, selecting an optimal decision path within the rolling planning time domain (hereinafter, the rolling planning time domain can also be simply referred to as the planning time domain) from the task search area, and calculating the expected execution benefit of the optimal decision path. Step S122 specifically includes:

[0141] Individual pre-decision: Within the planning time domain, all feasible paths of the current UAV are selected from the task search area using a traversal algorithm. The path revenue of each feasible path is calculated based on the updated search information graph. Based on the calculated path revenue, a set number of paths are selected from the feasible paths as the pre-decision paths of the current UAV.

[0142] Group optimal decision-making: Based on the current UAV application strategy and the path revenue of the pre-decision paths, the expected execution revenue of the pre-decision paths is calculated, and the optimal decision path is selected from the pre-decision paths based on the calculated expected execution revenue. Here, a feasible path refers to the path that the UAVs can travel when performing cooperative search tasks, and the optimal decision path is the path among the pre-decision paths that maximizes the expected execution revenue of the UAV swarm.

[0143] As an example, the planning time domain refers to the time domain consisting of the current planning step (i.e., the planning step at which the optimal decision path is selected) and one or more consecutive planning steps thereafter. That is, it is a rolling planning time domain that starts at the current planning step and ends at a planning step selected from all subsequent planning steps. Furthermore, in this application, a pruning strategy can be used to select a pre-decision path from all feasible paths based on path benefits (for example, all feasible paths can be sorted in descending order of path benefits, and pre-decision paths can be selected from all feasible paths in descending order of path benefits).

[0144] More specifically, the path revenue of a feasible path can be determined based on the search value revenue and coordination revenue of the feasible path. Since the optimization process of each UAV in a distributed architecture aims to maximize the revenue gained by a single UAV performing a search task and improve decision-making efficiency through fully autonomous decision-making, this application can model the calculation method of path revenue for a flight path, and the constructed model can be used to represent the search planning process of each UAV. The path revenue calculation model can be expressed by the following formula:

[0145]

[0146] Among them, [t s ,t s +T s [T] represents the planning time domain for the drone swarm to perform the search task. s To plan the length of the time domain, R i (t) is the decision variable, representing the time step t when the drone i is in the planning step (at which time t∈[t]). s ,t s +T s The feasible path of J) Si (t,R i (t) represents the decision of UAV i at planning step t. i The single-machine path revenue under (t), J Vi (t,R i (t)) and J Ci (t,R i (t) represent the state of the UAV i in decision R at time t of the planning step. i Search value gains and coordination gains under (t).

[0147] As an example, the search value revenue J Vi (t,R i (t) is the mathematical representation of the most valuable search area at present. It intuitively reflects the search priority of the UAV swarm within the task area at planning step time t, and can be determined based on the pheromone concentration in the pheromone map of UAV i. Specifically, due to the existence of sensor detection probability and false alarm probability, UAVs need to revisit multiple times during the search process to confirm the existence of the target and capture it. When a UAV detects a target in a sub-region, the sub-region will release attractive pheromones to guide the UAV to revisit the area, indicating that the area has high search value; conversely, if the target has been hit, the potential gains from continuing to search the area are significantly reduced. At this time, the sub-region will release repulsive pheromones, driving the UAV to search for other potential areas. Therefore, the search value gain can be defined as:

[0148]

[0149] Where (x, y) ∈ G i (R i (t) represents the path R i (t) contains subregion G xy If drone i's path Ri is partially or entirely... ( The projection of t) lies in subregion G. xy If it is internal, then path R is considered to be... i (t) contains subregion G xy In this application, if the UAV can detect sub-region G... xy Based on the information within, it is assumed that the drone's path passes through sub-region G. xy This can also be considered as the path of the drone containing sub-region G. xy .

[0150] Moreover, the coordination benefit J Ci (t,R i The design of (t) aims to reasonably balance local search and global search. On the one hand, it effectively avoids the problem of prematurely falling into local optima due to relying solely on the probability of target existence. On the other hand, it also limits the over-search of low-probability regions. Therefore, the coordinated benefit is designed by combining the probability and uncertainty of target existence in the TPM and EUM of UAV i, which can be defined as follows:

[0151]

[0152] According to equations (20) and (21), the coordination benefit of a feasible path (or pre-decision path) is the sum of the probability of target existence and environmental uncertainty (information from the target existence probability map and environmental uncertainty map updated in step S121) in all task sub-regions traversed by the feasible path; the search value benefit of a feasible path (or pre-decision path) is the sum of the pheromone concentration differences (difference between attracting pheromone concentration and repelling pheromone concentration) in all task sub-regions traversed by the feasible path.

[0153] Furthermore, the expected execution benefit of the pre-decision path can be determined based on the current UAV application strategy, the search value benefit of the pre-decision path, and the coordination benefit of the pre-decision path. Similar to the coordination benefit and search value benefit of available tracks, the coordination benefit of the pre-decision path is the sum of the target existence probability and environmental uncertainty in all task sub-regions traversed by the pre-decision path; the search value benefit of the pre-decision path is the sum of the pheromone concentration differences in all task sub-regions traversed by the pre-decision path; where the pheromone concentration difference is the difference between the attracting pheromone concentration and the repelling pheromone concentration. That is, the path benefit of the pre-decision path can also be calculated using equations (20) and (21).

[0154] Since the strategy used in drone applications is a calculation strategy to provide revenue for drones, the formula for calculating the expected execution revenue of the pre-decision path can be expressed as:

[0155]

[0156] Where, θ b，i (t) represents the strategy θ applied by UAV i at planning step t. b Alternatively, it can be used Represents θ b，i (t), For the planning step t, the drone i follows the policy θ b，i (t) executes the trajectory R i (t)(R at this time) i (t) can be represented as the expected execution benefit of the pre-decision path.

[0157] In some embodiments of this invention, the present application designs an update to the current UAV application strategy based on evolutionary game theory during the strategy optimization phase. However, this process requires the use of the correspondence between each strategy and expected payoff in the strategy space determined during the prediction phase (the present application performs Monte Carlo prediction operations during the prediction phase, so the prediction phase can also be called the Monte Carlo prediction phase) (hereinafter simply referred to as the strategy-expected payoff relationship or correspondence). The correspondence between the strategy and expected payoff is used to indicate the expected payoff obtained by adopting each strategy in the strategy space before the current planning step.

[0158] More specifically, the strategy-expected payoff relationship stored by the UAVs in this application can be set manually based on empirical values, or it can be determined by performing Monte Carlo prediction operations through prediction experiments. Since the accuracy of the strategy-expected payoff relationship affects the subsequent collaborative search steps of the UAVs, this application specifically limits the steps for determining the strategy-expected payoff relationship through Monte Carlo prediction operations in the prediction phase: For each UAV in the UAV cluster, at the planning step, the strategy-expected payoff relationship can be obtained by executing steps S210 to S250.

[0159] As an example, before executing steps S210 to S250 in the prediction phase, some parameters need to be initialized, including: the maximum communication distance C between drones. r Detection probability P D False alarm probability P F Probability correction value Pheromones increment unit Δs a With Δs r The parameters include: pheromone evaporation rate ρ, set quantity (i.e., the number of pre-decision paths retained in the feasible paths), trust factor λ, decay factor β, set sliding window length Z, and set target existence probability threshold p.t Planning time domain [t] s , t s +T s ], execution time domain [t s , t s +T e ], the initial search information graph, and the set task duration [t0, T] for the prediction phase. 预测 ].

[0160] When performing Monte Carlo prediction operations, the policies applied by each drone are fixed (i.e., no policy learning and optimization are performed during the prediction phase). Each drone can be randomly assigned a policy from a policy space, and since the drones make completely autonomous decisions in a distributed architecture, the policy-expected payoff relationship stored by each drone can be different.

[0161] In some embodiments of the present invention, the Monte Carlo prediction operation process for each drone in a drone swarm is as follows:

[0162] At each planning step in the forecasting phase, perform the following steps:

[0163] Step S210, UAV networking: Based on the maximum communication distance between UAVs, the current UAV establishes communication connections with other UAVs in the cluster to form a connected subnet.

[0164] Step S220, UAV information fusion and determination of optimal decision path: The current UAV receives the search information map stored by other UAVs in its connected subnet, and updates its own stored search information map using the received search information map; based on the updated search information map and the strategy applied by the current UAV, it selects an optimal decision path in the planning time domain from the task search area, and calculates the expected execution benefit of the optimal decision path (the optimal decision path is selected based on the principle of maximizing benefits).

[0165] Within the execution time domain, perform the following steps:

[0166] Step S230: The UAV executes the task and updates the search information: The UAV executes the search task according to the optimal decision path selected in step S220, captures the target and outputs the target position when the target search conditions are met; and updates its currently stored search information map according to the information detected during the execution of the search task (see equations (29) and (30) below); wherein, the condition for issuing the capture command can be q i (x, y, t) ≥ p t (At planning time t, UAV i believes that the probability q of a target in a certain grid exists based on the detected information.) i (x, y, t) is greater than the set probability threshold p of the existence of the target.t ).

[0167] The planning time domain in step S220 refers to the current planning step time in the prediction phase, which means the current planning step and one or more consecutive planning step times thereafter; the execution time domain in step S230 also refers to the current planning step time in the prediction phase, which means the current planning step and one or more consecutive planning step times thereafter; and the execution time domain length is less than or equal to the planning time domain length.

[0168] During the prediction phase, after completing the steps within the execution time domain, if the prediction stopping condition is not met, the process jumps to the next planning step and repeats the above steps S210 to S230; wherein, the prediction stopping condition can be that the current prediction planning step has reached the set number of prediction planning steps.

[0169] Step S240: Calculate the expected return for the UAV: ​​After the prediction stopping condition is met, calculate the average of the actual returns obtained by the UAV in performing the search task using the current strategy at all planning steps during the prediction phase, and use this average as the expected return corresponding to the current UAV strategy. This yields the correspondence between the various strategies stored by the UAV and their expected returns. That is, the average of the actual returns of the UAV at all planning steps in the Monte Carlo prediction operation is used to obtain the preset strategy θ. b The expected return (i.e., the fixed strategy assigned to the drone).

[0170] In addition to calculating the expected return after the predicted stopping condition is met, other existing techniques can also be used to calculate the expected return in this application, such as the incremental averaging method. However, this invention is not limited to these methods.

[0171] Specifically, during the strategy evaluation process, due to the inherent differences in returns among different strategies, the magnitude of the return cannot directly reflect the superiority or inferiority of the strategy. This application uses relative returns to represent the degree of superiority or inferiority of the strategy. In the prediction phase, the expected return of the strategy currently applied by UAV i can be estimated using the following formula:

[0172]

[0173] Where, [t0, T 预测 The time domain for setting the task to perform Monte Carlo prediction operations. This indicates that at time t, during the prediction planning step, the drone i follows policy θ. b The actual benefits under, For drone i in strategy θ b The expected return, ultimately determined through Monte Carlo operations, is the initial policy-expected return relationship θ during the policy optimization phase. b，i and The correspondence between them.

[0174] Furthermore, in the Monte Carlo prediction phase, since each drone is only randomly assigned a strategy, although each drone stores the correspondence between all strategies and expected payoffs in the strategy space, each drone only uses formula (23) to calculate the expected payoff of the assigned strategy, and the expected payoffs of other strategies can be set to 0 by default. Moreover, the step of the Monte Carlo operation designed in this application is to estimate the expected payoffs of multiple strategies in the strategy space. Although no strategy learning is performed in the prediction phase, if all drones in the drone cluster adopt the same strategy, it is impossible to learn and optimize the strategy in the subsequent strategy optimization phase. Therefore, although the strategy applied by each drone is randomly assigned in the prediction phase, it is required that there are at least two strategies in the drone cluster so that the cluster can carry out the strategy evolution game process.

[0175] As an example, the prediction phase precedes the strategy optimization phase. If the task time domain for the drone swarm to perform the collaborative search task in the strategy optimization phase includes many planning steps, the strategy-expected benefit relationship can be determined first based on the actual situation of the task area during the actual search task execution, and then further strategy optimization can be performed. That is, in the overall task time domain [0, T], r [0, T] l Perform Monte Carlo prediction operations for collaborative search in [T] l T r Perform cooperative search by implementing strategy optimization operations based on evolutionary game theory.

[0176] For distributed systems, the prerequisite for game theory is the ability to communicate. Therefore, the participants in the game are limited to drones that form the same connected subnet at each planning step. That is, in the method proposed in this application, the players are drones in the same connected subnet at each game, so each connected subnet can also be regarded as a game subnet. During the game, the information shared by each drone is real and accurate. From the perspective of each game subnet, all the drones in the subnet are in a cooperative relationship. The cooperative search method proposed in this application is based on evolutionary game theory. Since the game and decision-making are carried out at the same planning step, when each drone makes a decision using the current application strategy in the subsequent step S160, it cannot receive the strategies of other drones in time. The strategy optimization stage of this application is designed to consider the strategies of other drones and update its own current application strategy during the game process (mainly including steps S130 to S150), and only execute the strategy updated at the previous planning step during the decision-making process (including step S160).

[0177] Furthermore, when performing strategy optimization operations through steps S130 to S150 at the current planning step, drones within the same game subnet can engage in game play based on the expected payoff obtained at the previous planning step, thereby achieving strategy optimization. To enable subsequent strategy learning and optimization, each connected subnet should contain multiple drones with different current application strategies (for example, a connected subnet can contain drones with different application strategies as well as drones with the same application strategy).

[0178] Step S130: Update the correspondence between the drone's strategy and expected revenue: Update the expected revenue corresponding to the current drone application's strategy based on the expected execution revenue, and the current drone receives the strategies and updated expected revenues of other drone applications within its connected subnet, thereby updating the correspondence according to the set expected revenue update rules.

[0179] In a distributed architecture, all drones within a drone swarm store policy-expected payoff relationships. They can maintain the expected payoff for each policy in the policy space through their own game theory and decision-making processes. That is, for each policy in the policy space, each drone maintains the expected payoff for all policies in the policy space. For drone i, the currently applied policy θ... b The expected return at time t of the planning step can be expressed as: or

[0180] More specifically, the strategy θ applied to drone i at planning step time t. b，i or The expected return of the current application strategy for drone i can be updated based on the expected execution return, using the following formula:

[0181]

[0182] in, This indicates that at the deadline planning step t-1, the drone i follows policy θ. b The expected return (which can be determined from the correspondence before the update) This indicates that at planning step t, the drone i follows policy θ. b The expected execution benefit is calculated based on the expected execution benefit. Specifically, the expected benefit corresponding to the current drone application strategy is updated based on the expected execution benefit. This includes: for the strategy applied by the drone at the current planning step, the updated expected benefit is obtained by averaging the expected benefit and the expected execution benefit corresponding to the strategy in the corresponding relationship.

[0183] As an example, at planning step t, if the current policy applied by drone i is θ b Then, based on equation (24), the strategy for the application of drone i can be calculated as θ.b The corresponding expected return, for other strategies (e.g., strategy θ) a The expected performance benefit of drone i under these strategies can be assumed to be 0. That is, this application does not use equation (24) to update the expected performance benefit corresponding to other strategies that are not currently adopted, but uses the following equation (25) to update it.

[0184] In some embodiments of the present invention, during the strategy game, each drone updates the expected payoff corresponding to its applied strategy at the current planning step, and also updates the expected payoffs corresponding to other strategies in the corresponding relationship. That is, each drone updates the stored strategy-expected payoff relationship at each planning step. For each strategy in the corresponding relationship, the set expected payoff update rules (i.e., the rules for updating the strategy-expected payoff relationship) include:

[0185] If the expected return corresponding to the strategy in the correspondence is not 0, then the updated expected return corresponding to the strategy is the original expected return in the correspondence. If the expected return corresponding to the strategy in the correspondence is 0, and the strategy is not the strategy currently applied by any drone in the connected subnet to which the current drone belongs, then the updated expected return corresponding to the strategy defaults to 0. If the expected return corresponding to the strategy in the correspondence is 0, and the strategy is the strategy currently applied by any other drone in the connected subnet to which the current drone belongs, then the updated expected return corresponding to the strategy is the average of the updated expected returns obtained by other drones in the connected subnet to which the drone belongs using the strategy.

[0186] Specifically, for the strategy θ applied by drone i b，i Other strategies (such as strategy θ in the mapping relationship stored by drone i) a，i At planning step t, UAV i can update its stored policy-expected payoff relationship based on the updated expected payoffs of other UAVs in the connected subnet. Let policy θ be... a，i For example, at planning step time t, the strategy θ in the correspondence stored by UAV i a，i The updated formula for expected return is as follows:

[0187]

[0188] Among them, stg j (t)=θ a Let θ represent the strategy applied by UAV j at planning step time t. a , This indicates that drone j and drone i are located in the same connected subnet (and the number of drones in that subnet is...). Let I represent the connected subnet where drone i is located, and the applied strategy is θ. aThe number of drones.

[0189] Step S140, Calculate the evaluation value of the drone strategy: Based on the expected execution revenue and the expected update revenue corresponding to the current drone strategy, calculate the relative evaluation value of the current drone strategy. Then, based on the relative evaluation value at the current planning step and the relative evaluation value at a specific planning step, calculate the weighted relative evaluation value of the drone strategy.

[0190] In the process of strategy evaluation, since there are inherent differences in the magnitude of benefits calculated by different strategies, it is not reasonable to directly compare the magnitude of benefits. Therefore, this application uses relative expected benefits. However, in order to remove the influence of inherent differences, this application further uses strategy evaluation values ​​to evaluate the magnitude of benefits of the strategies applied by the UAV.

[0191] Specifically, assuming that at planning step t, the drone i is based on the application's policy θ b，i Making decisions, relative evaluation of strategies The following formula can be used for calculation:

[0192]

[0193] Furthermore, when strategy θ b When the expected return is low, a high return in a single instance will lead to a high evaluation value; when the strategy θ b When the expected return is high, a single low return can penalize potentially high-quality strategies. Directly using the single-return evaluation value as the criterion for judging the quality of a strategy introduces significant random noise. To reduce noise while preserving the dynamic characteristics of the strategy game, a secondary calculation of the evaluation value is performed based on a sliding window weighting method. At planning step t, the UAVs within the connected subnet calculate the currently applied strategy θ using the following formula. b,i Weighted relative evaluation value:

[0194]

[0195] in, This indicates the strategy θ applied by UAV i at planning step t. b The weighted relative evaluation value, β (β∈(0,1)) represents the decay factor, Z represents the set sliding window length (greater than the number of steps at a specific planning step), and F represents the set strategy applied by UAV i within the window as θ. b The total number of steps at a specific planning time, stg(T) = θ b Let θ represent the current application strategy of drone i at planning step t. b .

[0196] In some embodiments of the present invention, according to equation (27), the weighted relative evaluation value of the current application strategy of the UAV is calculated based on the relative evaluation value at the current planning step time and the relative evaluation value at a specific planning step time. This includes: when a specific planning step time exists, calculating the average value of the relative evaluation value at the specific planning step time; obtaining the weighted relative evaluation value of the strategy applied by the UAV at the current time by weighted summing the relative evaluation value at the current planning step time and the calculated average value; when no specific planning step time exists, using the relative evaluation value of the strategy applied by the UAV at the current time as the weighted relative evaluation value of the strategy applied by the UAV at the current time. Herein, the specific planning step time refers to the planning step time in the selected time domain (starting from the planning step time with a sliding window length set from the current planning step time, and the selected historical time domain before the current planning step time) in which the UAV adopts the application strategy.

[0197] Step S150, UAV updates current policy: The current UAV determines the maximum value of the weighted relative evaluation value within its connected subnet. Based on the maximum value of the weighted relative evaluation value and the weighted relative evaluation value of the policy applied by the current UAV, the policy learning probability corresponding to the current UAV is calculated. Based on the policy learning probability, the policy applied by the current UAV is updated to the policy corresponding to the maximum value of the weighted relative evaluation value.

[0198] Traditional game theory requires all players to be perfectly rational and operates under conditions of complete information. Therefore, if traditional game theory is used, all drones, when estimating the optimal strategy for the next planning step, will tend to choose the strategy that maximizes their own gain. Evolutionary game theory, however, assumes all players are "boundedly rational individuals." This means that all drones, when estimating the optimal strategy for the next planning step, will tend to choose the strategy that maximizes their own gain. Therefore, players are not necessarily able to choose the strategy that maximizes their own gain; instead, evolutionary stable strategies lead all players to a certain stable strategy. Further, this can be understood as follows: at each iteration, the drones within the game subnet will compare their own strategy with the strategies that maximize the gain of all members within the subnet, and with a certain probability, learn the strategy of the player with the highest gain. For example, in this application, the strategy learning process is a strategy replacement, that is, replacing the drone's current strategy with the strategy corresponding to the maximum weighted relative evaluation value.

[0199] As an example, the policy learning probability can be calculated using the Fermi function. Assume the relative policy evaluation value of drone i within the current connected subnet is... Among all members in the connected subnet to which drone i belongs, drone j has the largest relative policy evaluation value, denoted as: The probability that UAV i learns the policy corresponding to UAV j at the current planning step can be calculated using the following formula:

[0200]

[0201] Here, λ is the trust factor, representing the drone's acceptance of external strategies. When λ→0, it means that the strategy will be updated completely randomly regardless of the payoff, that is, the drone will randomly choose whether to learn the strategy of the member with the highest payoff in the current game. Conversely, if λ→∞, it represents a deterministic update rule, that is, when the maximum payoff in the subnet is higher than the payoff brought by its current strategy, the drone will definitely adopt the same strategy in the next moment.

[0202] The process of calculating the policy learning probability using the Fermi function mentioned above is only an example, and other methods can also be used. This invention is not limited to this.

[0203] After executing steps S110 to S150 at the current planning step time, step S160 needs to be executed again within the execution time domain including the current planning step time.

[0204] Step S160: The UAV performs the task and updates the search information: The UAV performs the search task according to the optimal decision path, captures the target and outputs the target position when the target search conditions are met, and updates its stored search information map according to the information detected during the search task execution.

[0205] As an example, the execution time domain and the planning time domain are relative terms. The execution time domain refers to the current planning step time and one or more subsequent consecutive planning step times; and the length of the execution time domain is less than or equal to the length of the planning time domain. In this application, both the planning time domain and the execution time domain include the current planning step time.

[0206] The target search condition in step S160 of this application can be q. i (x, y, t) ≥ p t That is, at time t, the drone i is in the planning step for subregion G xy The probability q of the detected target exists i (x, y, t) is greater than or equal to the set probability threshold p of the existence of the target. t The target search conditions in steps S230 and S160 above are merely examples, and this application does not specifically limit the target search conditions.

[0207] Furthermore, for a given sub-region, if the sub-region is outside the detection range of all drones within the connected subnet, the drone detection information can be set to 0 by default.

[0208] During the collaborative search mission performed by the UAV swarm, the UAV can update its stored search information map based on the information detected by its onboard sensors at planning step t. Specifically, at planning step t, the UAV updates its search information map updated in step S120 based on the information detected in step S160. Since environmental uncertainty and pheromone information can be updated based on the target presence probability, the following discussion uses the probability q of UAV i detecting the target presence in step S160. i Taking (x, y, t) as an example, the search information update process in step S160 is described (the search information can also be updated directly based on the probability of the target's existence and environmental uncertainty detected by the UAV).

[0209] Specifically, to more accurately reflect the dynamic update process of search information, taking the probability of target existence as an example, we consider the detection probability P of the sensor. D With the false alarm probability P F This allows for dynamic information updates based on the Bayesian criterion. Assume that for sub-region G... xy The probability of the target detected by UAV i at time t during the planning step is q. i Given (x, y, t), at planning step t, the prior updated target existence probability can be updated based on the posterior target existence probability, resulting in the target existence probability map stored by UAV i at planning step t. The specific formula is as follows:

[0210]

[0211] Where, q i (x, y, t) represents the sub-region G detected by UAV i at time t during the planning step. xy Does the target (q) exist inside? i (x, y, t) takes the value 0 or 1, q i (x, y, t-1) = 1 indicates that UAV i has detected subregion G. xy With a goal, q i (x, y, t-1) = 0 indicates that UAV i has detected subregion G. xy There is no target.

[0212] Furthermore, at planning step t, if sub-region G xy The probability p of the existence of the internal target i (x, y, t) is greater than the probability threshold of the target existence. Then it is considered that subregion G xy A target exists within the memory, and a capture command is issued. After capturing the target, in order to more accurately reflect the actual situation of the task area, it is necessary to analyze the sub-region G. xy The probability of the target within is adjusted, and the adjustment mechanism is as follows:

[0213]

[0214] in, This represents a probability correction value, which is an empirical value.

[0215] As an example, the detection probability P in equation (29) D With the false alarm probability P F The probability P can be customized or determined based on the specific engineering application. Specifically, during UAV detection, the UAV updates its stored search information map in real time based on the detection information obtained from its onboard sensors, thereby making further decisions. However, due to factors such as sensor performance, environmental obstruction, and resource constraints, there may be instances of missed targets or false target detections. Therefore, the detection probability P needs to be set. D And the false alarm probability P F These represent the probability that the sensor detects the target or makes a mistake within the search range (or detection range), respectively, and are defined as follows:

[0216] Detection probability P D =P(Detection result: Target present | Actual target present), P D ∈[0,1];

[0217] False alarm probability P F =P(Detection result: Target present | Actual target absent), P F ∈[0,1].

[0218] Furthermore, for sub-region G xy The environmental uncertainty of UAV i at planning step time t can be updated by the information entropy of the target existence probability updated by equation (31), as follows:

[0219] η i (x,y,t)=-p i (x,y,t)log2p i (x,y,t)-(1-p i (x,y,t))log2(1-p i (x,y,t));….(31)

[0220] The pheromone information is updated based on the target existence probability updated by equation (30), and is updated according to equations (12) to (15).

[0221] After the steps in the execution time domain are completed (i.e., after step S160 is completed), if the search stopping condition is not met, the process jumps to the next planning step and repeats the above steps S110 to S160; when the search stopping condition is met, the collaborative search of the UAV swarm is stopped.

[0222] As an example, the search stopping condition could be reaching a set number of search planning steps at the current planning step time, the number of targets captured by the drone swarm reaching a set number of targets, or the proportion of drones using a certain strategy within the drone swarm reaching a set percentage. Both the target search conditions and the search stopping conditions in this application can be set according to requirements, and this invention is not limited thereto.

[0223] As an example, similar to the prediction phase, before executing steps S110 to S160 in the strategy optimization phase, this application needs to initialize and set some parameters, including: the maximum communication distance C between drones. r Detection probability P D False alarm probability P F Probability correction value Pheromones increment unit Δs a With Δs r The parameters include: pheromone evaporation rate ρ, set quantity, trust factor λ, decay factor β, set sliding window length Z, and set target existence probability threshold (the first target existence probability threshold is set during the strategy optimization phase). Planning time domain [t] s , t s +T s ], execution time domain [t s , t s +T e [The initial search information graph, the strategy-expected return relationship determined in the prediction phase, and the task time domain set in the strategy optimization phase [T]] l T r ].

[0224] As shown in Table 1 and Figure 8 As shown, the collaborative search method proposed in this application uses an evolutionary game approach to randomly assign multiple pre-designed collaborative strategies to cluster members, and achieves strategy optimization through the evolutionary game process.

[0225] Table 1. Pseudocode of the distributed cooperative search method for UAV swarms based on evolutionary game theory.

[0226]

[0227]

[0228] The UAV swarm cooperative search method based on evolutionary game theory in a distributed architecture proposed in this application has the following significant advantages:

[0229] ① This application organically integrates evolutionary game theory and distributed cooperative search by UAV swarms. During the search process, strategy evaluation and game theory continuously optimize and eliminate inferior search strategies, while each UAV makes completely autonomous decisions, significantly improving the efficiency of cooperative search. The proposed method divides the entire cooperative search process into a Monte Carlo prediction stage and an evolutionary game-based strategy optimization stage. In these two stages, each UAV in the swarm executes cooperative search tasks through completely autonomous decision-making, maintaining its own search information graph and strategy-expected payoff relationship. A path payoff calculation model and pruning strategies are integrated to determine the optimal search trajectory for each UAV. In the Monte Carlo prediction stage, each UAV in the swarm is assigned an initial strategy and performs cooperative search according to the initial strategy, thereby estimating the expected payoff of the initial strategy to quantify its search efficiency. After initially establishing the strategy-expected payoff relationship, the strategy optimization phase begins. As the search progresses, the expected payoff for each drone's current strategy is continuously updated according to certain rules. Based on these expected payoffs, the current strategies of each drone within the subnet are evaluated. Each drone optimizes and updates its strategy based on its evaluation value using an evolutionary game mechanism. The principle of strategy optimization and updating is that superior strategies will be adopted by more drones, while poorly performing strategies will be gradually eliminated. Through this dynamic evolutionary process, the drone swarm can adaptively select the optimal search strategy within the subnet, thereby achieving efficient cluster collaboration in a distributed architecture and significantly improving the efficiency and effectiveness of collaborative search.

[0230] This method provides a feasible solution to the problem of collaborative search for moving targets by UAV swarms under a distributed architecture in complex scenarios, while avoiding the tedious parameter tuning process, and has important practical application value.

[0231] ② Based on the concept of evolutionary game theory, this application designs a new strategy evaluation mechanism and expected payoff update rule, thereby constructing a novel UAV swarm evolutionary game mechanism under a distributed architecture. Specifically, a connected subnet is set up based on communication distance. Within the connected subnet, UAVs perform information fusion. Based on the optimal decision path and the current corresponding strategy of the UAV, the search strategy is evaluated and played within the connected subnet, thereby achieving autonomous evolutionary optimization of the UAV search strategy, ultimately causing each UAV to converge to its own optimal strategy.

[0232] ③ This application uses a Gaussian distribution to initialize the search information map. In the absence of prior information, based on reasonable assumptions, a two-dimensional Gaussian distribution is used to initialize the target existence probability and environmental uncertainty in the task area. This initialization method can initially provide the UAV with a key search area where targets may exist by reasonably setting the center point and variance of the Gaussian distribution. Furthermore, to maximize the dispersion of the UAV swarm and improve the efficiency of environmental information collection in the initial stage, this application can also divide the task area into multiple regions and independently initialize the search information for each region.

[0233] Corresponding to the above method, the present invention also provides a cooperative search system for a distributed unmanned aerial vehicle (UAV) swarm. The system includes a computer device, which includes a processor and a memory. The memory stores computer programs / instructions, and the processor executes the computer programs / instructions stored in the memory. When the computer programs / instructions are executed by the processor, the system implements the steps of the method described above.

[0234] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0235] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0236] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0237] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0238] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cooperative search method for distributed unmanned aerial vehicle (UAV) swarms based on evolutionary game theory, characterized in that, For each drone within a drone swarm, the method includes: During the strategy optimization phase, at the planning step, the following steps are performed: Drone networking: Based on the maximum communication distance between drones, the current drone communicates with other drones in the cluster to form a connected subnet; the current drone applies a strategy in the strategy space, the connected subnet contains multiple drones applying different strategies, and the current drone stores the correspondence between each strategy and expected reward in the strategy space, as well as a search information graph; The UAV integrates information and determines the optimal decision path: The current UAV receives the search information map stored by other UAVs in its connected subnet, and updates its own stored search information map using the received search information map; based on the updated search information map and the strategy applied by the current UAV, it selects an optimal decision path in the rolling planning time domain from the task search area, and calculates the expected execution benefit of the optimal decision path; The correspondence between various strategies and expected benefits of drone updates is as follows: Based on the expected execution benefits, the expected benefits corresponding to the current drone application strategy are updated, and the current drone receives the strategies and updated expected benefits of other drone applications within its connected subnet, thereby updating the stored correspondence according to the set expected benefit update rules. Drone strategy evaluation value: Based on the expected execution revenue and the expected update revenue corresponding to the current drone application strategy, calculate the relative evaluation value of the strategy applied by the current drone. Then, based on the relative evaluation value at the current planning step and the relative evaluation value at a specific planning step, calculate the weighted relative evaluation value of the strategy applied by the current drone. The specific planning step refers to the planning step of the current drone using the strategy within the time period from the historical planning step to the current planning step, which is a distance of a set sliding window from the current planning step. The drone updates its current policy: The drone determines the maximum value of the weighted relative evaluation value within its connected subnet. Based on the weighted relative evaluation value of the policy applied by the drone and the maximum value of the weighted relative evaluation value, the drone calculates the policy learning probability corresponding to the current drone. Based on the policy learning probability, the drone updates its current policy to the policy corresponding to the maximum value of the weighted relative evaluation value. During the strategy optimization phase, the following steps are performed within the execution time domain: The UAV performs tasks and updates search information: In the execution time domain, the current UAV performs a search task according to the optimal decision path, captures the target and outputs the target position when the target search conditions are met, and updates its stored search information map according to the detection information during the execution of the search task; wherein, the execution time domain and the planning time domain are both the planning step time of selecting the optimal decision path and one or more consecutive planning step times thereafter, and the planning time domain is greater than or equal to the execution time domain; After the steps in the execution time domain are completed, if the search stopping condition is not met, the process jumps to the next planning step and repeats all the steps in the above planning step and execution time domain, finally completing the collaborative search of the UAV swarm.

2. The method according to claim 1, characterized in that, For each strategy in the aforementioned correspondence, the set expected return update rules include: If the expected return corresponding to the strategy in the correspondence is not 0, then the updated expected return corresponding to the strategy is the original expected return in the correspondence. If the expected return corresponding to the strategy in the correspondence is 0, and the strategy is not the strategy currently applied by any drone in the connected subnet to which the current drone belongs, then the updated expected return corresponding to the strategy is the default value. If the expected return corresponding to the strategy in the correspondence is 0, and the strategy is the strategy currently applied by other drones in the connected subnet to which the current drone belongs, then the updated expected return corresponding to the strategy is the average of the updated expected returns obtained by other drones in the connected subnet to which the drone belongs that use the strategy.

3. The method according to claim 1, characterized in that, The step of calculating the weighted relative evaluation value of the strategy applied by the UAV at the current planning step time based on the relative evaluation value at the current planning step time and the relative evaluation value at a specific planning step time includes: Given a specific planning step, calculate the average of the relative evaluation values ​​at that specific planning step; by weighted summing the relative evaluation value at the current planning step and the average value, obtain the weighted relative evaluation value of the strategy applied by the current UAV. In the absence of a specific planning step, the relative evaluation value of the strategy currently applied by the UAV is used as the weighted relative evaluation value of the strategy currently applied by the UAV.

4. The method according to claim 1, characterized in that, The correspondence between each strategy and expected return in the strategy space stored by the drone is obtained through Monte Carlo prediction operations; For each drone in a drone swarm, the Monte Carlo prediction operation includes the following steps: at the planning step, performing drone networking and drone information fusion and determining the optimal decision path; in the execution time domain, performing drone tasks and updating search information; and calculating the expected revenue of the drones. The calculation of expected revenue by the UAV involves the following steps: after the prediction stopping condition is met, the average of the actual revenue obtained by the current UAV applying the strategy to perform the search task is calculated at all planning steps of the Monte Carlo prediction operation, and this average is used as the expected revenue corresponding to the current UAV applying the strategy, thereby obtaining the correspondence between the various strategies and expected revenues stored by the current UAV.

5. The method according to claim 4, characterized in that, The search information map stored by the UAV is determined based on the target existence probability, environmental uncertainty, and pheromone information detected by the UAV when performing the search mission; and a two-dimensional Gaussian distribution is used to initialize and model the target existence probability in the search information map stored by the UAV. The search stopping condition is when the planned step time reaches the set number of search planning steps or the number of targets captured by the drone swarm reaches the set number of targets; the prediction stopping condition is when the planned step time reaches the set number of prediction planning steps; and The learning probability of the strategy is calculated using the Fermi function.

6. The method according to claim 5, characterized in that, The step of selecting an optimal decision path within the rolling planning time domain from the task search area based on the updated search information graph and the current UAV application strategy, and calculating the expected execution benefit of the optimal decision path, includes: Within the planning time domain, a traversal algorithm is used to calculate all feasible paths of the current UAV from the task search area. Based on the updated search information graph, the path revenue of each feasible path is calculated, and a set number of paths are selected from all feasible paths as the UAV's pre-decision paths based on the calculated path revenue. The path revenue of a feasible path is determined based on the search value revenue and coordination revenue of the feasible path. Based on the current UAV application strategy and the path revenue of the pre-decision path, the expected execution revenue of the pre-decision path is calculated, and the optimal decision path is selected from the pre-decision paths based on the calculated expected execution revenue.

7. The method according to claim 6, characterized in that, The expected execution benefits of the pre-decision path are determined based on the current UAV application strategy, the search value benefits of the pre-decision path, and the coordination benefits of the pre-decision path. Wherein, the search value gain of the pre-decision path is the sum of the pheromone concentration differences in all task sub-regions traversed by the pre-decision path; the coordination gain of the pre-decision path is the sum of the target existence probability and environmental uncertainty in all task sub-regions traversed by the pre-decision path; wherein, the pheromone concentration difference is the difference between the attracting pheromone concentration and the repelling pheromone concentration.

8. The method according to claim 5, characterized in that, The current UAV receives search information maps stored by other UAVs within its connected subnet, and updates its own stored search information map using the received search information maps, including: For the target existence probability map and environmental uncertainty map in the search information map, the UAV receives the target existence probability map and environmental uncertainty map stored by other UAVs in its connected subnet; For each task sub-region within the task search area, if the sub-region is within the current UAV's search range, the target existence probability map and environmental uncertainty map stored by the current UAV are used as the target existence probability map and environmental uncertainty map in the search information map updated by the current UAV, respectively. For each task sub-region within the task search area, if the sub-region is outside the search range of the UAV, calculate the average target existence probability of the sub-region in the target existence probability map stored by other UAVs in the connected subnet to which the current UAV belongs, calculate the square root of the product of the environmental uncertainties of the sub-region in the environmental uncertainty map stored by other UAVs in the connected subnet to which the current UAV belongs, and use the calculation results as the target existence probability map and environmental uncertainty map in the updated search information map of the current UAV, respectively.

9. A cooperative search system for a distributed unmanned aerial vehicle (UAV) swarm based on evolutionary game theory, comprising a processor, a memory, and a computer program / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 8.

Images