Hierarchical intention-based multi-uav cooperative search and rescue intelligent decision method and device

Abstract

The present application relates to a layered intention-based multi-UAV cooperative search and rescue intelligent decision-making method and device. The method comprises: obtaining an environment simulation map; constructing a multi-UAV cooperative search and rescue model; extracting features of the search target and obstacles through the multi-UAV cooperative search and rescue model to obtain target position features and obstacle position features; performing information gain grid division based on the environment simulation map to determine the local information gain field of the agent UAV; calculating the local gain information of the agent UAV according to the local information gain field of the agent UAV and the information of the neighboring UAVs; extracting features of the local gain information to obtain local gain features; focusing attention on the target position features, obstacle position features, and local gain features to output multi-source aggregated features; processing the multi-source aggregated features through a multi-layer perception machine to output the action normal distribution of the UAV. The present application can improve the search and rescue efficiency of the UAV in the search and rescue scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle cooperation, in particular to a multi-unmanned aerial vehicle cooperative search and rescue intelligent decision-making method and device based on hierarchical intentions. BACKGROUND

[0002] The application of multi-agent reinforcement learning (MARL) in unmanned aerial vehicle (UAV) swarm cooperation tasks, such as tracking moving targets, searching unknown areas, and obstacle avoidance, has become a research hotspot in recent years. The core focuses on the cooperation between agents in the UAV swarm in scenarios such as tracking moving targets, searching unknown areas, and obstacle avoidance, as well as the optimization of division of labor, adaptation to partially observable environments, and decision-making in continuous action spaces. In recent years, through the combination of reinforcement learning frameworks and cooperation mechanisms, the technical solutions based on MARL have continuously improved the performance of UAV swarm cooperation, providing technical support for multi-task execution in complex dynamic environments.

[0003] Current mainstream technical solutions mainly fall into three categories: first, mechanisms based on graph neural networks (GNN) or graph attention (GAT), which treat UAVs as graph nodes, build edge relationships through communication ranges, and use attention to aggregate neighbor information to facilitate cooperative decision-making, often combined with frameworks such as MAPPO for path planning and information sharing; second, multi-agent reinforcement learning (MARL) solutions based on roles, which solve the problem of homogeneous division of labor through dynamic role assignment, generate role representations with the help of variational inference or attention, and integrate them into the CTDE framework to achieve task division; third, solutions based on exploration mechanisms, such as information gain field (IGF) or intrinsic motivation guidance, which quantify environmental uncertainty as rewards to shape exploration, and combine algorithms such as PPO to improve the exploration coverage and search efficiency of UAVs.

[0004] However, mechanisms based on graph neural networks (GNN) or graph attention (GAT) rely on stable communication links, and communication disruptions in dynamic or sparse environments can easily cause attention mechanisms to fail. Moreover, the computational overhead of graph structure increases quadratically with the number of UAVs, making it difficult to support large-scale clusters. Multi-agent reinforcement learning based on roles has the problem of role staticity, and role assignment is prone to overfitting local observations, making it less adaptable in dynamic tasks and leading to lag in division of labor adjustment. In the exploration mechanism, modules such as IGF have low integration with decision-making networks, and there are problems such as insufficient exploration efficiency and slow convergence in sparse reward environments. In addition, the decision-making robustness in partially observable environments is poor, and the scalability of heterogeneous UAV swarms is insufficient, further restricting the actual performance of UAV swarm cooperation tasks. SUMMARY

[0005] Therefore, it is necessary to provide a multi-unmanned aerial vehicle cooperative search and rescue intelligent decision method and device based on hierarchical intention to solve the problems of communication dependence leading to cooperation failure, insufficient role division adaptability, and low integration of exploration mechanism and decision.

[0006] A multi-unmanned aerial vehicle cooperative search and rescue intelligent decision method based on hierarchical intention, the method comprising:

[0007] An environment simulation map is obtained, the environment simulation map comprising unmanned aerial vehicles, search and rescue targets, and obstacles; wherein the unmanned aerial vehicles comprise agent unmanned aerial vehicles and neighborhood unmanned aerial vehicles;

[0008] A multi-unmanned aerial vehicle cooperative search and rescue model is constructed;

[0009] The search and rescue targets and the obstacles are subjected to feature extraction by the multi-unmanned aerial vehicle cooperative search and rescue model, to obtain target position features and obstacle position features;

[0010] The environment simulation map is subjected to information gain grid division, and based on the grid division, a local information gain field of the agent unmanned aerial vehicle is determined;

[0011] The local information gain field of the agent unmanned aerial vehicle is subjected to calculation by fusing information of the neighborhood unmanned aerial vehicles, to obtain local gain information of the agent unmanned aerial vehicle; the local gain information is subjected to feature extraction, to obtain local gain features;

[0012] The target position features, the obstacle position features, and the local gain features are subjected to attention focusing, to output multi-source aggregated features;

[0013] The multi-source aggregated features are subjected to processing by a multi-layer perception machine, to output action normal distribution of the unmanned aerial vehicle.

[0014] In another aspect, a multi-unmanned aerial vehicle cooperative search and rescue intelligent decision device based on hierarchical intention is also provided, comprising:

[0015] An environment simulation map acquisition module is configured to obtain an environment simulation map, the environment simulation map comprising unmanned aerial vehicles, search and rescue targets, and obstacles; wherein the unmanned aerial vehicles comprise agent unmanned aerial vehicles and neighborhood unmanned aerial vehicles;

[0016] A model construction module is configured to construct a multi-unmanned aerial vehicle cooperative search and rescue model;

[0017] A feature extraction module is configured to subject the search and rescue targets and the obstacles to feature extraction by the multi-unmanned aerial vehicle cooperative search and rescue model, to obtain target position features and obstacle position features;

[0018] The local information gain module is configured to perform information gain grid division based on the environment simulation map, determine a local information gain field of the agent UAV based on the grid division, calculate local gain information of the agent UAV according to the local information gain field of the agent UAV and information of neighbor UAVs, and perform feature extraction on the local gain information to obtain local gain features.

[0019] The feature aggregation module is configured to focus attention on the target position features, the obstacle position features and the local gain features, and output multi-source aggregated features.

[0020] The action output module is configured to process the multi-source aggregated features by using a multi-layer perception machine, and output an action normal distribution of the UAV.

[0021] Compared with the prior art, the multi-UAV cooperative search and rescue intelligent decision method and device based on hierarchical intentions provided by the application have the following effects:

[0022] 1. Without relying on external communication graphs or edge calculations, the attention degree is calculated only among the agent's own intention features (exploration / tracking / obstacle avoidance / movement), ensuring stable cooperation even in scenarios with high communication failure rates, and the calculation overhead is low, which is suitable for large-scale UAV cooperation; implicit cooperation is realized by combining the global role overview, avoiding independent actions of each agent, to overcome the problems of communication dependence and graph structure vulnerability, and to provide independent cooperation.

[0023] 2. By separating the intention to extract behavior features, a dynamic focus intention embedding is generated by attention aggregation, so that the role naturally emerges from end-to-end learning without explicit assignment or clustering of roles; when the target escapes or the obstacles change, the attention weight can be adjusted in real time, reducing overfitting of local observations, improving training stability and global cooperation effect, to overcome the static nature and poor adaptability of role division, and to realize dynamic implicit division of labor.

[0024] 3. The information gain field is integrated into the exploration intention to optimize the balance between exploration and tracking, reduce manual intervention, and improve exploration efficiency and coverage; for partial observability and heterogeneity, noise robustness is enhanced by intention extraction, scalability is improved by parameter sharing, and generalization ability is enhanced by combining information gain field and gradual reward, and the performance is more stable in the obstacle-dense environment. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the application, and those skilled in the art can obtain other related drawings from these drawings without creative labor.

[0026] Figure 1This is a flowchart illustrating the multi-UAV collaborative search and rescue intelligent decision-making method based on hierarchical intent provided in Example 1;

[0027] Figure 2 This is a framework diagram of the multi-UAV collaborative search and rescue model and evaluation network structure provided in Example 1;

[0028] Figure 3 This is a schematic diagram of the testing and verification environment provided in Example 1;

[0029] Figure 4 The method proposed in this invention is used in Example 1. Figure 3 A diagram illustrating the capture rate in the given scenario;

[0030] Figure 5 The method proposed in this invention is used in Example 1. Figure 3 Survival rate diagram in the scenario;

[0031] Figure 6 This is a structural block diagram of the multi-UAV collaborative search and rescue intelligent decision-making device based on hierarchical intent provided in Example 2.

[0032] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0034] It should be noted that in this invention, the use of terms such as "first," "second," etc., is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0035] It is understood that the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0036] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0037] Embodiment 1

[0038] As Figure 1 shown, the embodiment provides a layered intention-based multi-UAV cooperative search and rescue intelligent decision-making method, including the following steps:

[0039] Step 201, obtaining an environment simulation map, which includes UAVs, search and rescue targets and obstacles.

[0040] It can be understood that when performing cooperative search and rescue, each UAV is regarded as an agent UAV, and its adjacent UAVs are regarded as neighborhood UAVs.

[0041] Step 202, constructing a multi-UAV cooperative search and rescue model.

[0042] Step 203, extracting features of the search and rescue target and the obstacle through the multi-UAV cooperative search and rescue model to obtain target position features and obstacle position features.

[0043] It can be understood that by extracting features of the search and rescue target and the obstacle, the intention separation behavior feature is realized, which can accurately focus on the core elements in the search and rescue scene and filter irrelevant information in the environment. At the same time, such a way makes up for the limited field of view of a single UAV, reduces the misjudgment caused by insufficient local information, enhances the comprehensiveness and pertinence of information utilization, and reduces decision bias.

[0044] Step 204, performing information gain grid division based on the environment simulation map, and determining the local information gain field of the agent UAV based on the grid division.

[0045] Step 205, calculating the local gain information of the agent UAV according to the local information gain field of the agent UAV and the information of the neighborhood UAVs; extracting features of the local gain information to obtain local gain features.

[0046] The gain information is integrated into the multi-UAV cooperative search and rescue, which can make the UAVs move preferentially to high information value areas, optimize the balance of exploration and tracking, and improve the search and rescue efficiency.

[0047] Step 206, focusing attention on the target position features, obstacle position features and local gain features, and outputting multi-source aggregated features.

[0048] It can be understood that through attention focusing and dynamic focus intention embedding, the role naturally emerges from end-to-end learning without explicit assignment or clustering of roles. When facing target evasion or obstacle changes, the attention weight can be adjusted in real time to reduce overfitting of local observations, improve training stability and global collaboration effect, overcome the static nature and poor adaptability of role division, and realize dynamic implicit division of labor.

[0049] Step 207: Process the multi-source aggregated features using a multilayer perceptron to output the normal distribution of the UAV's actions.

[0050] In one embodiment, the multi-UAV collaborative search and rescue model includes a feature extraction module, an information gain module, a first multi-head attention module, and a first multi-layer perceptron. Specifically, the feature extraction module extracts features from the search and rescue target and obstacles to obtain target location features and obstacle location features.

[0051] The information gain grid is divided using the information gain module. Based on the grid division, the local information gain field of the agent drone is determined. Then, based on the local information gain field of the agent drone, the information of neighboring drones is fused to calculate the local gain information of the agent drone. Feature extraction is performed on the local gain information to obtain the local gain features.

[0052] The first multi-head attention module focuses attention on the target location features, obstacle location features, and local gain features, and outputs multi-source aggregated features.

[0053] The multi-source aggregated features are processed by the first multilayer perceptron to output the normal distribution of the drone's actions.

[0054] Specifically, such as Figure 2 As shown, the feature extraction module includes two parallel multilayer perceptron networks (MLPs), used for searching and rescuing targets. and obstacles Feature extraction is performed to obtain the target location features. Location characteristics of obstacles The processing expression is:

[0055] ;

[0056] ;

[0057] In the formula, This indicates a partial observation of the search and rescue target by the drone; This indicates a partial observation of an obstacle by a drone; This indicates that the output features are 64-dimensional.

[0058] The information gain module includes a CNN network and a multilayer perceptron network (MLP). Within the information gain module, the information gain grid is first divided based on the environmental simulation image, with a resolution of [resolution value missing]. Covering a world size of Then, the local information gain field of the agent drone is determined. , According to the local information gain field of the agent UAV 、 , the information of the neighboring UAVs is fused for calculation to obtain the local gain information of the agent UAV . The local gain information is sequentially input into a CNN network and a multi-layer perception network for processing to obtain local gain features , and the processing process is expressed as:

[0059] .

[0060] Then, the target position features , obstacle position features and local gain features are spliced and input into a first multi-head attention module for attention focusing, and then average calculation is performed to output multi-source aggregation features.

[0061] The processing process of the attention focusing through the first multi-head attention module is expressed as:

[0062] ;

[0063] In the formula, denotes the first multi-head attention module; denotes the multi-source aggregation features; denotes the number of heads of the multi-head attention.

[0064] Finally, the multi-source aggregation features are processed by a first multi-layer perception to output the action normal distribution of the UAV .

[0065] In one embodiment, the multi-UAV cooperative search and rescue model further includes a context state module; the context state module obtains and processes the state information of the agent UAV and the neighboring UAVs to obtain the context state information of the agent UAV.

[0066] After the context state information is fused with the multi-source aggregation features, the second multi-layer perception is used for processing to output the role representation of the agent UAV.

[0067] Specifically, the context state module adopts a multi-layer perception structure.

[0068] The state information of the agent UAV and the neighboring UAVs is input into the context state module for processing to obtain the context state information of the agent UAV , and the processing process is expressed as:

[0069] ​ ;

[0070] Then, the context state information is spliced with the multi-source aggregation features , and the processing expression is:

[0071] ;

[0072] Then, it is processed by a second multi-layer perception (MLP2) to output the role representation of the agent drone. The processing expression is:

[0073] ;

[0074] In the formula, represents the fusion feature; represents the role representation.

[0075] In one embodiment, in the information gain module, the information gain grid is divided based on the environment simulation map, and based on the grid division, the local information gain field of the agent drone is determined, including:

[0076] The information gain grid is divided based on the environment simulation map, and the environment uncertainty field and the search and rescue target uncertainty field of the agent drone are initialized.

[0077] The environment uncertainty field is dynamically updated to obtain the environment field of the agent drone; and the target field of the agent drone is generated according to the target position.

[0078] Specifically, the local information gain field of the agent drone includes the environment uncertainty field and the search and rescue target uncertainty field .

[0079] First, the environment uncertainty field of the agent drone is initialized as , and the search and rescue target uncertainty field is initialized as , wherein represents an initial value, which is also the maximum value of the environment uncertainty.

[0080] Then, the environment uncertainty field is dynamically updated, including a recovery process and a decay process, wherein the recovery process is represented as ;

[0081] The decay process is represented as .

[0082] Then, the target field of the Gaussian distribution is generated according to the target position, and the expression is:

[0083] ;

[0084] wherein, represents the recovery speed of the environment uncertainty; represents the decay speed of the environment uncertainty; represents the map coordinate; represents the observed target position; represents the variance of the Gaussian distribution field.

[0085] In one embodiment, in the information gain module, the local information gain field of the agent UAV is calculated according to the information of the neighboring UAVs, and the local gain information of the agent UAV is obtained, and the calculation expression is:

[0086] ;

[0087] wherein, represents the local gain information of the agent UAV; represents the environment field of the agent UAV; represents the target field of the agent UAV; represents the number of neighboring UAVs; represents the maximum value of the sum of the environment uncertainty field and the target uncertainty field at all grid points; represents the minimum value of the sum of the environment uncertainty field and the target uncertainty field at all grid points. In one embodiment, in the calculation of the local gain information of the agent UAV, the environment uncertainty of the UAV is further calculated according to the environment field and the target field, and the expression is: ;

[0088] Based on the change of the uncertainty before and after the update, the reward of the local information gain is determined, and the expression is:

[0089] ;

[0090] wherein,

[0091] ;

[0092] wherein, represents the reward of the local information gain; represents the reward coefficient; represents the environment uncertainty of the UAV before the update; represents the environment uncertainty of the UAV after the update, wherein, .

[0093] ​​​In the above embodiment, the information gain is integrated into the multi-UAV cooperative search and rescue model, which dynamically updates the environmental uncertainty field, simulates the interaction between the environment and the target through the dynamic change of the field, and optimizes the exploration-tracking balance through the shaping reward (uncertainty reduction + gradual proximity penalty) end-to-end, reduces manual dependence, and improves efficiency.

[0094] In one embodiment, the evaluation network is further included, and the multi-UAV cooperative search and rescue model is trained through the evaluation network; the evaluation network includes a third multi-layer perception machine, a second multi-head attention module, and a fourth multi-layer perception machine.

[0095] The global state is obtained based on the environment simulation map, and the role representation of the agent UAV is obtained.

[0096] The global state is input into the third multi-layer perception machine for processing to output a query vector; and the role representation and the query vector are input into the second multi-head attention module for processing to output a role query perception vector.

[0097] After the query vector and the role query perception vector are fused, the fourth multi-layer perception machine is input for processing to output an evaluation value.

[0098] The action normal distribution output by the multi-UAV cooperative search and rescue model is valued based on the evaluation value.

[0099] As shown in the formula (1), first, the global state S is input into the third multi-layer perception machine M3 for processing to output a query vector q, and the processing process is expressed as Figure 2 . As shown in the formula (2), the role representation set R is input into the second multi-head attention module A2 together with the query vector q for processing to output a role query perception vector r, and the processing process is expressed as . After the query vector q and the role query perception vector r are fused, the fourth multi-layer perception machine M4 is input for processing to output an evaluation value v, and the processing process is expressed as .

[0100] In one embodiment, in the evaluation network, the loss function is: In the formula (3), n represents the number of samples; and represents the sample .

[0101] In one embodiment, in the evaluation network, the loss function is:

[0102] In one embodiment, in the evaluation network, the loss function is:

[0103] ;

[0104] In the formula (3), n represents the number of samples; and represents the sample . represents the sample ​Dot product similarity between them; Indicates sample With sample Dot product similarity between them; This represents the set of positive samples after k-means clustering. This represents the set of negative samples after k-means clustering. Indicates the index of the agent being processed; , Indicates different positive sample indices.

[0105] In one embodiment, when training the multi-UAV collaborative search and rescue model using an evaluation network, a reward function is also set up to calculate the reward value of the UAV actions based on the reward function; the expression of the reward function is:

[0106]

[0107] in:

[0108] ;

[0109] ;

[0110] In the formula, Indicates the total reward; Indicates the basic reward; Represents the reward for local information gain; Indicates distance reward; Indicates a time penalty; Indicates the distance reward coefficient; Indicates the current step The nearest target distance; Indicates the next step The nearest target distance; This indicates the penalty at each time step.

[0111] To overcome communication dependencies and graph structure vulnerabilities, this invention shifts to internal self-attention in multi-UAV collaborative search and rescue models. Attention is calculated only between the agent's own intent features (exploration / tracking / obstacle avoidance), eliminating the need for external communication graphs or edge computation. This ensures that even with high communication failure rates, focus intent embeddings can be generated independently, promoting decision robustness. It also has low computational overhead, making it suitable for large-scale UAV operations. The evaluation network implicitly encourages cooperation through a global role overview (average focus intent), preventing degeneration into independent agents.

[0112] Meanwhile, to overcome the staticity and poor adaptability of role division, three behavior characteristics of the unmanned aerial vehicle, the search and rescue target and the obstacle are extracted by intention separation, and a dynamic focus intention embedding is generated by using multi-head self-attention aggregation, and the role naturally emerges from end-to-end learning without explicit assignment or clustering. This improves adaptability: when the target escapes or the obstacle changes, the attention weight adjusts in real time, which can reduce overfitting to local observations, train more stably, better global cooperation, drive division through rewards, avoid "clustering", and realize dynamic implicit division.

[0113] Further, to overcome the low exploration efficiency and sparse reward problem, the information gain field is fused into the exploration intention as an input, and the exploration-tracking balance is optimized end-to-end through the shaping reward (uncertainty reduction + gradual neighbor penalty). This reduces manual dependence and improves efficiency.

[0114] In addition, for partial observability and heterogeneity, the method uses intention extraction to ensure noise robustness. Parameter sharing combined with intention attention improves scalability. Generalization is enhanced through information gain and gradual reward; the overall performance is more stable in obstacle-dense environments.

[0115] In one embodiment, the method is verified. The number of unmanned aerial vehicles is 4, and the search and rescue target is 10. As shown in Figure 3 , the blue circle represents the observation range of the unmanned aerial vehicle, the dark red color represents the obstacle, and the light red color represents the search and rescue target.

[0116] As shown in Figure 4 and Figure 5 , by using the method provided by the present application, the capture rate can reach 0.9 and the survival rate can reach 0.98, which has obvious advantages.

[0117] It should be understood that although each step in the present embodiment Figure 1 is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the present embodiment may include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or sub-steps or stages of other steps.

[0118] Embodiment 2

[0119] Based on the multi-unmanned aerial vehicle cooperative search and rescue intelligent decision method based on hierarchical intention in embodiment 1, the embodiment discloses a multi-unmanned aerial vehicle cooperative search and rescue intelligent decision device based on hierarchical intention, as shown in the figure. Figure 6 The multi-unmanned aerial vehicle cooperative search and rescue intelligent decision device based on hierarchical intention comprises an environment simulation map acquisition module 401, a model construction module 402, a feature extraction module 403, a local information gain module 404, a feature aggregation module 405, and an action output module 406, wherein:

[0120] The environment simulation map acquisition module 401 is used to acquire an environment simulation map, wherein the environment simulation map comprises unmanned aerial vehicles, search and rescue targets, and obstacles; the unmanned aerial vehicles comprise agent unmanned aerial vehicles and neighborhood unmanned aerial vehicles.

[0121] The model construction module 402 is used to construct a multi-unmanned aerial vehicle cooperative search and rescue model.

[0122] The feature extraction module 403 is used to perform feature extraction on the search and rescue targets and the obstacles through the multi-unmanned aerial vehicle cooperative search and rescue model, to obtain target position features and obstacle position features.

[0123] The local information gain module 404 is used to perform information gain grid division based on the environment simulation map, to determine a local information gain field of the agent unmanned aerial vehicle based on the grid division; to calculate the local gain information of the agent unmanned aerial vehicle by fusing the information of the neighborhood unmanned aerial vehicles according to the local information gain field of the agent unmanned aerial vehicle; and to perform feature extraction on the local gain information to obtain local gain features.

[0124] The feature aggregation module 405 is used to focus attention on the target position features, the obstacle position features, and the local gain features, and to output multi-source aggregated features.

[0125] The action output module 406 is used to process the multi-source aggregated features through a multi-layer perception machine, and to output an action normal distribution of the unmanned aerial vehicle.

[0126] It can be understood that the multi-unmanned aerial vehicle cooperative search and rescue intelligent decision device based on hierarchical intention is arranged on the unmanned aerial vehicle, and is used to generate action instructions. The unmanned aerial vehicle is also provided with a data acquisition module, an execution control module, and a communication module. The data acquisition module is used to acquire unmanned aerial vehicle sensor data and information gain (IGF) input; the execution control module is used to control the flight of the unmanned aerial vehicle and realize cooperative search and rescue; and the communication module is used to support neighborhood information fusion.

[0127] In this embodiment, the specific working processes and working principles of the environment simulation map acquisition module 401, the model construction module 402, the feature extraction module 403, the local information gain module 404, and the feature aggregation module 405 are the same as those in the method in Embodiment 1, and thus the above modules will not be described again in this embodiment. The various unit modules can be implemented in whole or in part by software, hardware, and combinations thereof, and can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to the above various unit modules.

[0128] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments can be included. In each embodiment provided in the present application, any reference to a memory, storage, database, or other medium can include a non-volatile and / or volatile memory. The non-volatile memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory can include a random access memory (RAM) or an external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0129] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0130] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A layered intention-based multi-UAV cooperative search and rescue intelligent decision-making method, characterized in that, The method comprises: acquiring an environment simulation map, the environment simulation map comprising a UAV, a search and rescue target and an obstacle; constructing a multi-UAV cooperative search and rescue model; extracting features of the search and rescue target and the obstacle through the multi-UAV cooperative search and rescue model to obtain target position features and obstacle position features; performing information gain grid division based on the environment simulation map, and determining a local information gain field of an agent UAV based on the grid division; calculating local gain information of the agent UAV by fusing information of neighbor UAVs according to the local information gain field of the agent UAV; extracting features of the local gain information to obtain local gain features; focusing attention on the target position features, the obstacle position features and the local gain features to output multi-source aggregated features; processing the multi-source aggregated features through a multi-layer perception machine to output action normal distribution of the UAV; wherein the calculation expression of the local gain information of the agent UAV by fusing information of neighbor UAVs according to the local information gain field of the agent UAV is: ； the processing expression of the local gain features by extracting features of the local gain information is: ； In the formula, represents a map coordinate; represents local gain information of an agent UAV ; represents an environment field of a first tier UAV; represents a target field of a first tier UAV; represents a number of neighbor UAVs; represents a maximum value of a sum of an environment uncertainty field and a target uncertainty field at all grid points ; represents a minimum value of a sum of an environment uncertainty field and a target uncertainty field at all grid points .

2. The layered intent-based multi-UAV cooperative search and rescue intelligent decision method according to claim 1, characterized in that, the multi-UAV cooperative search and rescue model comprises a feature extraction module, an information gain module, a first multi-head attention module and a first multi-layer perception machine; wherein the feature extraction module is used to extract features of the search and rescue target and the obstacle to obtain target position features and obstacle position features; the information gain module is used to perform information gain grid division, and determine a local information gain field of an agent UAV based on the grid division; then, the local gain information of the agent UAV is calculated by fusing information of neighbor UAVs according to the local information gain field of the agent UAV; features of the local gain information are extracted to obtain local gain features; the first multi-head attention module is used to focus attention on the target position features, the obstacle position features and the local gain features to output multi-source aggregated features; the first multi-layer perception machine is used to process the multi-source aggregated features to output action normal distribution of the UAV.

3. The layered intent-based multi-UAV cooperative search and rescue intelligent decision method according to claim 2, characterized in that, The multi-UAV cooperative search and rescue model further comprises a context state module; the context state module acquires state information of the agent UAV and the neighbor UAVs and processes the state information to obtain context state information of the agent UAV; after the context state information is fused with the multi-source aggregated features, the second multi-layer perception machine is used to process the context state information and the multi-source aggregated features to output role representation of the agent UAV.

4. The layered intent-based multi-UAV cooperative search and rescue intelligent decision method according to claim 2, characterized in that, In the information gain module, the information gain grid division based on the environment simulation map is performed, and a local information gain field of an agent UAV is determined based on the grid division, comprising: performing information gain grid division based on the environment simulation map, and initializing an environment uncertainty field and a search and rescue target uncertainty field of the agent UAV; dynamically updating the environment uncertainty field to obtain an environment field of the agent UAV; and generating a target field of the agent UAV according to the target position.

5. The layered intent-based multi-UAV cooperative search and rescue intelligent decision method according to claim 4, characterized in that, In the calculation of the local gain information of the agent UAV, further comprising: calculating the UAV's uncertainty to the environment according to the environment field and the target field; Based on the change of uncertainty before and after updating, the reward of local information gain is determined, and the expression is: ； wherein represents a reward of local information gain; represents a reward coefficient; represents the uncertainty of the environment for the UAV before updating; represents the uncertainty of the environment for the UAV after updating. 6.The layered intention based multi-UAV cooperative search and rescue intelligent decision method according to any one of claims 1 to 5, characterized in that, Further comprising an evaluation network, the multi-UAV cooperative search and rescue model is trained through the evaluation network; The evaluation network comprises a third multi-layer perception machine, a second multi-head attention module and a fourth multi-layer perception machine; Based on the environment simulation map, the global state is obtained, and the role representation of the agent UAV is obtained; The global state is input into the third multi-layer perception machine for processing, and a query vector is output; The role representation and the query vector are input into the second multi-head attention module for processing, and a role query perception vector is output; After the query vector and the role query perception vector are fused, the fourth multi-layer perception machine is input for processing, and an evaluation value is output; Based on the evaluation value, the action normal distribution output by the multi-UAV cooperative search and rescue model is evaluated.

7. The layered intent-based multi-UAV cooperative search and rescue intelligent decision method according to claim 6, characterized in that, In the evaluation network, the loss function is: ； wherein represents the number of samples; represents the samples represents the dot product similarity between the samples represents the dot product similarity between the samples represents the dot product similarity between the samples represents the set of positive samples after k-means clustering; represents the set of negative samples after k-means clustering; represents the index of the agent being processed; , represents the different positive sample indices.​​ 8.The layered intention based multi-UAV cooperative search and rescue intelligent decision method of claim 6, wherein, When the multi-UAV cooperative search and rescue model is trained through the evaluation network, a reward function is further set, and the reward value of the UAV action is calculated according to the reward function; the expression of the reward function is: ； wherein represents the total reward; represents the base reward; represents the reward for local information gain; represents the distance reward; represents the time penalty.

9. A layered intention-based multi-UAV cooperative search and rescue intelligent decision device, characterized in that, The device comprises the multi-UAV cooperative search and rescue intelligent decision-making method based on layered intentions according to any one of claims 1 to 8. An environment simulation map acquisition module is configured to acquire an environment simulation map, wherein the environment simulation map includes a UAV, a search and rescue target, and an obstacle; the UAV includes an agent UAV and a neighborhood UAV; A model construction module is configured to construct a multi-UAV cooperative search and rescue model; A feature extraction module is configured to extract features of the search and rescue target and the obstacle through the multi-UAV cooperative search and rescue model to obtain target position features and obstacle position features; A local information gain module is configured to perform information gain grid division based on the environment simulation map, determine a local information gain field of the agent UAV based on the grid division, calculate the local information gain field of the agent UAV by fusing information of the neighborhood UAV, and extract features of the local information gain to obtain local gain features; A feature aggregation module is configured to focus attention on the target position features, the obstacle position features, and the local gain features to output multi-source aggregated features; An action output module is configured to process the multi-source aggregated features through a multi-layer perception machine to output an action normal distribution of the UAV.

Images