A consensus-based multi-unmanned aerial vehicle safety cooperation sensing method

Abstract

The present application relates to the technical field of unmanned aerial vehicle cooperative perception, and particularly relates to a multi-unmanned aerial vehicle safe cooperative perception method based on consensus. The method comprises the following steps: constructing a multi-unmanned aerial vehicle cooperative perception network, each intelligent agent collects three-dimensional point clouds and extracts intermediate feature maps, and broadcasts the intermediate feature maps to other intelligent agents; each intelligent agent obtains a benchmark detection result based on its own feature map, and detects after fusing its own feature map with the feature map of each other intelligent agent to obtain a comparative detection result; the difference information of each comparative result and the benchmark result is calculated, trusted intelligent agents are screened, and a local target set is constructed; each intelligent agent broadcasts the local target set and aggregates it into a global target set, excludes intelligent agents outside the global target set, fuses the feature maps of itself and the remaining trusted intelligent agents, and obtains a purified fusion feature map. Through feature map exchange, difference detection and consensus screening, the present application realizes safe and reliable cooperative perception when there are abnormal or malicious nodes.

Description

Technical Field

[0001] This invention relates to the field of collaborative perception technology for unmanned aerial vehicles (UAVs), and more particularly to a consensus-based method for safe collaborative perception among multiple UAVs. Background Technology

[0002] Currently, multi-UAV collaborative sensing systems have broad application prospects in environmental monitoring, target tracking, and search and rescue. However, in actual deployment, UAV nodes may generate abnormal or erroneous data due to sensor noise, communication interference, or malicious attacks, leading to degradation or even failure of collaborative sensing results. Existing methods typically employ the direct fusion of feature maps or raw data from all nodes, lacking an effective mechanism for identifying and filtering abnormal nodes, making it difficult to guarantee the reliability and security of sensing in the presence of malicious nodes.

[0003] Chinese Patent Publication No. CN117492460A discloses a method for interactive control of unmanned aerial vehicle (UAV) missions and a storage medium. It utilizes the SLAM algorithm to establish the UAV's position and map in an unknown environment in real time, and then employs DS theory to handle uncertainties, enabling information sharing and mission collaboration among multiple UAVs, thereby improving mission execution efficiency, safety, and intelligence. Through real-time mapping and localization using the SLAM algorithm, the UAV can accurately perceive the environment and optimize mission execution based on evidence merging according to DS theory. This allows the UAV to plan paths, avoid obstacles, and coordinate tasks more quickly and efficiently, thus improving overall mission execution efficiency. By building a map using the SLAM algorithm, UAVs can jointly explore and cover a larger area. Utilizing evidence merging and mission collaboration based on DS theory to optimize resource allocation achieves a wider mission coverage, suitable for complex tasks such as large-scale search and monitoring.

[0004] Therefore, the existing technology has the following problems: The lack of a security screening and exclusion mechanism for false / erroneous data generated by abnormal or malicious nodes makes it difficult to resist malicious attacks or perception biases in multi-drone collaborative perception scenarios. Summary of the Invention

[0005] To address this, the present invention provides a consensus-based multi-UAV security collaborative perception method to overcome the lack of security identification and exclusion mechanisms for false / erroneous data generated by abnormal or malicious nodes in existing technologies, which makes it difficult to resist malicious attacks or perception deviations in multi-UAV collaborative perception scenarios.

[0006] To achieve the above objectives, the present invention provides a consensus-based multi-UAV security cooperative perception method, comprising: Step S1: Construct a multi-UAV collaborative perception network composed of several intelligent agents. Each intelligent agent collects the original three-dimensional point cloud data of its environment and extracts features from the original three-dimensional point cloud data to obtain an intermediate feature map. Step S2: Each agent broadcasts its own intermediate feature map to other agents in the multi-UAV cooperative perception network and receives the intermediate feature maps broadcast by other agents. Step S3: Each agent performs target detection based on its own intermediate feature map to obtain a baseline detection result; and fuses its own intermediate feature map with the intermediate feature map of each other agent, and performs target detection on the fused feature map to obtain the corresponding comparative detection result. Step S4: Each agent calculates the difference information between each comparison detection result and the benchmark detection result, selects target agents from other agents based on the difference information, and constructs a local target set for the agent based on all selected target agents. Step S5: Each intelligent agent broadcasts its own local target set to the multi-UAV cooperative perception network, and aggregates all local target sets to obtain a global target set; Step S6: Each agent, based on the global target set, excludes the intermediate feature maps corresponding to the agents in the global target set, and fuses its own intermediate feature map with the intermediate feature maps of the remaining agents to obtain a purified fused feature map.

[0007] Further, in step S4, the difference information between each comparison detection result and the benchmark detection result is calculated, including: Step S41: Match the positional overlap of the comparison detection result with the benchmark detection result to identify redundant targets, missing targets, and detection attribute deviations; Step S42: Calculate the information gain index for each other agent based on the redundant targets, the missing targets, and the detection attribute deviation; Step S43: Verify the redundant targets and correct the information gain index based on the verification results; Step S44: Select other agents whose corrected information gain index is greater than the preset information gain index as target agents.

[0008] Further, in step S41, redundant targets, missing targets, and detection attribute biases are identified, including: Each target in the comparison detection result is paired with each target in the benchmark detection result, and the positional overlap between each pair of targets is calculated. Target pairs with a positional overlap greater than a preset positional overlap are determined to be a successful match, identified as the same target, and the detection attribute deviation of the target between the comparison detection result and the benchmark detection result is calculated. Mark all targets that did not match successfully in the comparison detection results as redundant targets; All targets that did not match successfully in the benchmark detection results are marked as missing targets.

[0009] Further, the detection attribute bias is calculated, including: Calculate the size deviation between the size of the same target in the comparative detection result and the size in the benchmark detection result, and use the size deviation as the detection attribute deviation.

[0010] Further, in step S42, the information gain metric for each other agent is calculated, including: Calculate the positive gain value based on the number of redundant targets and the detection quality; Calculate the positive gain value based on the positive change in the detected attribute deviation; Calculate the negative gain value based on the number of missing targets and the detection quality; The positive gain value and the negative gain value are combined to obtain the information gain index of each other agent.

[0011] Further, in step S43, the redundant targets are verified, including: Obtain the first observation direction vector of the redundant target in the coordinate system of its own agent, and the second observation direction vector in the coordinate system of other agents; Calculate the epipolar constraint error between the first observation direction vector and the second observation direction vector; If the epipolar constraint error is less than the preset epipolar error, then the redundant target is determined to have passed the verification, and the information gain index remains unchanged. If the epipolar constraint error is greater than or equal to the epipolar error threshold, the redundant target is determined to have failed verification, and the information gain index is corrected.

[0012] Further, the epipolar constraint error between the first observation direction vector and the second observation direction vector is calculated, including: Calculate the epipolar distance between the first observation direction vector and the second observation direction vector. The epipolar distance is the shortest distance in three-dimensional space between the ray corresponding to the first observation direction vector and the ray corresponding to the second observation direction vector. Use the epipolar distance as the epipolar constraint error.

[0013] Further, in step S5, the global target set is obtained, including: Each intelligent agent broadcasts its local target set to other intelligent agents in the multi-UAV cooperative perception network; Aggregate the local target sets of each agent and determine the number of times each agent is included in the local target set by other agents; Agents that have been included in the global target set with a number of times greater than or equal to a preset threshold are included.

[0014] Further, in step S6, the purified fusion feature map is obtained, including: The intermediate feature maps broadcast by each agent are filtered according to the global target set, and the intermediate feature maps corresponding to the agents in the global target set are excluded. The purified fused feature map is obtained by fusing its own intermediate feature map with the remaining intermediate feature maps after exclusion.

[0015] Further, in step S1, an intermediate feature map is obtained, including: The raw 3D point cloud data collected by each intelligent agent is voxelized to generate a bird's-eye view and a grid map. Feature extraction is performed on the bird's-eye view occupies the grid map to obtain an intermediate feature map.

[0016] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention defines multiple drones deployed in traffic scenarios and a central roadside unit as a set of intelligent agents. The roadside unit serves as the initial master node of the consensus protocol, providing stable computing and communication support. Each drone uses sensors to collect 3D point cloud data of the environment. Then, through voxelization processing, the original sparse point cloud is converted into a structured bird's-eye view occupies a grid map, and the height dimension is encoded as the channel dimension, eliminating the need for viewpoint transformation. Finally, a shared encoder within the cluster is used to perform spatial downsampling and feature compression on the bird's-eye view, generating a compact and information-rich intermediate feature map, which is then broadcast and received among the intelligent agents. This effectively solves the problems of the original point cloud data being sparse and irregular and difficult to process directly, and the original data transmission communication overhead being too high. It achieves efficient compression and sharing of perception data, enabling each intelligent agent to perceive environmental information beyond its own field of view.

[0017] Furthermore, this invention accurately identifies redundant targets, missing targets, and detection attribute deviations through position overlap matching, quantifying the differentiated impact of each collaborator on perception performance. Secondly, based on the number and detection quality of redundant targets, the positive change in detection attribute deviations, and the number and detection quality of missing targets, a weighted information gain index is calculated to achieve a quantitative assessment of the collaborator's net contribution. Further, epipolar constraints are used for multi-view geometric verification of redundant targets. The authenticity of targets is judged by calculating the epipolar distance between observation direction vectors. False targets that fail verification are penalized with gain correction based on the product of their detection confidence and epipolar constraint error, effectively identifying maliciously injected adversarial perturbations. Finally, a local target set is constructed by selecting trustworthy collaborators through gain thresholding. This enables each agent to independently complete anomaly detection and trust scoring of other nodes without global labels, significantly improving the security and robustness of the multi-UAV cooperative perception system in adversarial environments.

[0018] Furthermore, this invention, through a constructed consensus aggregation mechanism, broadcasts and aggregates the local target sets independently generated by each agent through majority voting, effectively solving the problem of unreliable local judgments by a single agent. Each agent broadcasts its own local target set to all other agents in the network. After collecting all local target sets, each agent counts the number of times it has been included in the local target sets by other agents, and includes agents with an inclusion count greater than or equal to a preset threshold into the global target set. This formalized consensus mechanism ensures that decisions are made only when there is a significant majority consensus within the cluster, effectively preventing a minority of attackers from colluding to frame innocent agents or protect malicious nodes.

[0019] Furthermore, this invention, through a constructed secure feature fusion mechanism guided by a consensus-driven global target set, achieves precise filtering and purification fusion of feature maps of malicious or untrusted agents. Based on the obtained global suspect set, it excludes intermediate feature maps corresponding to all agents consistently identified as attackers by the cluster, and only aggregates and fuses its own intermediate feature map with the feature maps of the remaining trusted collaborators to generate a purified fused feature map. Simultaneously, this method also incorporates a fault-safe fallback mechanism. When the consensus process fails to converge, the UAV automatically switches to an independent perception mode relying solely on its own feature map, ensuring basic operational safety. This effectively prevents malicious disturbances or contaminated data from entering the fusion stage, guaranteeing the reliability and integrity of the feature information flow. Attached Figure Description

[0020] Figure 1 This is a flowchart of the consensus-based multi-UAV security cooperative perception method in this embodiment; Figure 2 This is a flowchart illustrating the calculation of difference information in the consensus-based multi-UAV safety collaborative perception method of this embodiment. Detailed Implementation

[0021] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0022] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0023] Please see Figures 1-2 As shown, Figure 1 This is a flowchart of the consensus-based multi-UAV security cooperative perception method in this embodiment; Figure 2 This is a flowchart illustrating the calculation of difference information in the consensus-based multi-UAV safety collaborative perception method of this embodiment.

[0024] This embodiment provides a consensus-based multi-UAV security cooperative perception method, including: Step S1: Construct a multi-UAV collaborative perception network composed of several intelligent agents. Each intelligent agent collects the original three-dimensional point cloud data of its environment and extracts features from the original three-dimensional point cloud data to obtain an intermediate feature map. Step S2: Each agent broadcasts its own intermediate feature map to other agents in the multi-UAV cooperative perception network and receives the intermediate feature maps broadcast by other agents. Step S3: Each agent performs target detection based on its own intermediate feature map to obtain a baseline detection result; and fuses its own intermediate feature map with the intermediate feature map of each other agent, and performs target detection on the fused feature map to obtain the corresponding comparative detection result. Step S4: Each agent calculates the difference information between each comparison detection result and the benchmark detection result, selects target agents from other agents based on the difference information, and constructs a local target set for the agent based on all selected target agents. Step S5: Each intelligent agent broadcasts its own local target set to the multi-UAV cooperative perception network, and aggregates all local target sets to obtain a global target set; Step S6: Each agent, based on the global target set, excludes the intermediate feature maps corresponding to the agents in the global target set, and fuses its own intermediate feature map with the intermediate feature maps of the remaining agents to obtain a purified fused feature map.

[0025] In this embodiment of the invention, the intelligent agent includes one or more drones deployed in a traffic scenario and a central roadside unit (RSU). A multi-drone cooperative perception network is formed by N intelligent agents, and the set of intelligent agents is denoted as […]. This set contains one RSU (index: )and Mobile drones (indexed as) to ), where N is the total number of agents, Let i be the i-th drone. RSU, as the initial master node of the consensus protocol, provides stable computing resources and reliable communication connections; each drone is equipped with a LiDAR sensor to collect 3D point cloud data of the environment and perform collaborative perception tasks.

[0026] Specifically, in step S1, the intermediate feature map is obtained, including: The raw 3D point cloud data collected by each intelligent agent is voxelized to generate a bird's-eye view and a grid map. Feature extraction is performed on the bird's-eye view occupies the grid map to obtain an intermediate feature map.

[0027] In this embodiment of the invention, each intelligent agent Equipped with a LiDAR sensor for acquiring 3D point clouds of raw environmental data. Point cloud It is a set of points in three-dimensional Euclidean space, that is To facilitate processing using standard deep learning architectures, the raw point cloud is converted into a structured bird's-eye view (BEV) representation. The BEV graph is a discrete three-dimensional binary tensor. This indicates that the top view occupies the grid after the surrounding space has been quantized. Specifically, when the voxel unit... When a space contains LiDAR scan points, the value is 1, indicating that the space is occupied; a value of 0 indicates that the space is free. All such BEV maps are denoted as... .

[0028] In this embodiment of the invention, each intelligent agent (drone or RSU) It uses its onboard LiDAR sensor to collect raw 3D point cloud data. The unique aerial perspective of drones naturally aligns the point clouds they acquire with the Bird's-Eye View (BEV) coordinate system, thus completely eliminating the need for complex perspective transformations common in ground vehicle systems. However, the raw point cloud data is sparse and irregular, making it unsuitable for direct processing by standard convolutional neural networks. To address this issue, this invention employs voxelization, dividing the 3D space into a discrete voxel grid and projecting the point cloud into a 2D pseudo-image. In this 2D pseudo-image, the height dimension is encoded as a channel dimension, thereby generating a structured BEV occupancy grid map. The BEV diagram is consistent with the occupied mesh defined in the previous section: voxel elements with a value of 1. This indicates that the 3D space is occupied by LiDAR scan points, while a value of 0 indicates that the space is free.

[0029] Although converted to a structured BEV graph, the original data still contains a large amount of redundant information, which severely impacts communication efficiency. Therefore, each agent... Use an encoder shared within a cluster BEV diagram Transform into a compact and information-rich intermediate feature map:

[0030] Where K is the spatial downsampling rate, which compresses the amount of data by reducing the spatial resolution; This is the feature dimension, used to preserve key semantic information. The resulting feature map... That is, an intelligent agent Broadcast of collaborative messages. Subsequently, each agent receives a set of feature maps shared by other members within the cluster. This feature-sharing mechanism enables each agent to perceive environmental information beyond its own field of view, thereby forming a unified and broader collaborative perception system.

[0031] This invention defines a collection of intelligent agents by defining multiple drones deployed in a traffic scenario and a central roadside unit. The roadside unit serves as the initial master node of the consensus protocol, providing stable computing and communication support. Each drone uses sensors to collect 3D point cloud data of the environment. Then, through voxelization processing, the original sparse point cloud is converted into a structured bird's-eye view occupies a grid map, encoding the height dimension as the channel dimension, eliminating the need for viewpoint changes. Finally, a shared encoder within the cluster is used to spatially downsample and compress the bird's-eye view, generating a compact and information-rich intermediate feature map, which is then broadcast and received among the intelligent agents. This effectively solves the problems of the original point cloud data being sparse and irregular, making it difficult to process directly, and the excessive communication overhead of the original data transmission. It achieves efficient compression and sharing of perceived data, enabling each intelligent agent to perceive environmental information beyond its own field of view.

[0032] Specifically, in step S4, the difference information between each comparison detection result and the benchmark detection result is calculated, including: Step S41: Match the positional overlap of the comparison detection result with the benchmark detection result to identify redundant targets, missing targets, and detection attribute deviations; In this embodiment of the invention, for each other intelligent agent, the agent pairs its comparative detection result with the benchmark detection result one by one, and calculates the positional overlap between each pair of targets. Specifically: target pairs with a positional overlap greater than a preset threshold are determined to be successfully matched and regarded as the same target, and the detection attribute deviation of the target between the comparative detection result and the benchmark detection result is calculated, including the size deviation; all targets that are not successfully matched in the comparative detection result are marked as redundant targets, indicating that the comparative detection result has false detections relative to the benchmark detection result; all targets that are not successfully matched in the benchmark detection result are marked as missing targets, indicating that the comparative detection result has missed detections relative to the benchmark detection result.

[0033] Step S42: Calculate the information gain index for each other agent based on the redundant targets, the missing targets, and the detection attribute deviation; In this embodiment of the invention, based on the three types of difference information mentioned above, the agent calculates the information gain index for each other agent. This index comprehensively evaluates the net contribution of the feature map of the other agent to the perception performance: a positive gain component is calculated based on the number of redundant targets and the detection quality, reflecting the negative contribution of false detections; a positive gain component is calculated based on the positive change in the detection attribute deviation; a negative gain component is calculated based on the number of missing targets and the detection quality, reflecting the information loss caused by missed detections; and the above components are weighted and combined to obtain the information gain index for each other agent.

[0034] Step S43: Verify the redundant targets and correct the information gain index based on the verification results; In this embodiment of the invention, for each identified redundant target, the agent further performs epipolar constraint verification to determine whether the redundant target is introduced by perceptual noise or malicious perturbation. Specifically: the agent obtains the first observation direction vector of the redundant target in its own agent coordinate system and the second observation direction vector in other agent coordinate systems; the epipolar distance between the two direction vectors, i.e., the shortest distance between the two rays in three-dimensional space, is calculated as the epipolar constraint error. If the epipolar distance is less than a preset epipolar error threshold, the redundant target is determined to have passed geometric verification and is considered a real target, maintaining the original information gain index unchanged; if the epipolar distance is greater than or equal to the preset epipolar error threshold, the redundant target is determined to be a false target, and the information gain index is penalized and its gain value is reduced.

[0035] Step S44: Select other agents whose corrected information gain index is greater than the preset information gain index as target agents.

[0036] In this embodiment of the invention, the modified information gain index is compared with a preset information gain threshold: if the modified information gain index is greater than the preset information gain threshold, the corresponding other agents are identified as target agents and regarded as trusted collaborators; if the above condition is not met, they are identified as suspicious or malicious agents and are not included in the local target set. Finally, the agent constructs its local target set based on all the selected target agents. This local target set represents the set of trusted collaborators identified by the agent based on local perception and difference analysis.

[0037] Specifically, in step S41, redundant targets, missing targets, and detection attribute biases are identified, including: Each target in the comparison detection result is paired with each target in the benchmark detection result, and the positional overlap between each pair of targets is calculated. Target pairs with a positional overlap greater than a preset positional overlap are determined to be a successful match, identified as the same target, and the detection attribute deviation of the target between the comparison detection result and the benchmark detection result is calculated. Mark all targets that did not match successfully in the comparison detection results as redundant targets; All targets that did not match successfully in the benchmark detection results are marked as missing targets.

[0038] Specifically, calculating the detection attribute bias includes: Calculate the size deviation between the size of the same target in the comparative detection result and the size in the benchmark detection result, and use the size deviation as the detection attribute deviation.

[0039] In this embodiment of the invention, for each other intelligent agent, the agent pairs its comparison detection result with the benchmark detection result one by one. Specifically, it iterates through each detected target in the comparison detection result and calculates the positional overlap with each detected target in the benchmark detection result, i.e., the ratio of the intersection area to the union area of ​​the two bounding boxes. Target pairs with a positional overlap greater than a preset overlap threshold are determined to be successfully matched, and it is assumed that the pair of targets represents the same object in the real world. The detection attribute deviation of the same target between the comparison detection result and the benchmark detection result is calculated, specifically including calculating the size deviation of the target bounding box in the three dimensions of length, width, and height, and using the size deviation as the detection attribute deviation. For all targets in the comparison detection result that do not successfully match any target in the benchmark detection result, they are marked as redundant targets, indicating that the comparison detection result has false detections relative to the benchmark detection result. For all targets in the benchmark detection result that do not successfully match any target in the comparison detection result, they are marked as missing targets, indicating that the comparison detection result has missed detections relative to the benchmark detection result.

[0040] Specifically, in step S42, the information gain metric for each other agent is calculated, including: Calculate the positive gain value based on the number of redundant targets and the detection quality; Calculate the positive gain value based on the positive change in the detected attribute deviation; Calculate the negative gain value based on the number of missing targets and the detection quality; The positive gain value and the negative gain value are combined to obtain the information gain index of each other agent.

[0041] In this embodiment of the invention, for each other agent, the agent calculates its information gain index based on the identified redundant targets, missing targets, and detection attribute deviations. Specifically, firstly, a positive gain value is calculated based on the number of redundant targets and the detection quality. Redundant targets represent false detections in the comparison detection result relative to the benchmark detection result. The higher the number and confidence level of these false detections, the more false information is introduced into the feature map of the other agent, negatively impacting perception performance. Therefore, this positive gain value is essentially a negative contribution component. Secondly, a positive gain value is calculated based on the positive change in the detection attribute deviation. The detection attribute deviation includes the size deviation of the same target between the comparison detection result and the benchmark detection result. If the deviation is positive and exceeds a preset threshold, it indicates that the target size estimation is more accurate after fusing the feature map of the other agent, i.e., a positive gain is generated; if the deviation is negative, it indicates that the size estimation accuracy decreases after fusion, generating a negative contribution.

[0042] Next, a negative gain value is calculated based on the number of missing targets and the detection quality. Missing targets represent missed detections in the comparative detection results relative to the baseline detection results. The more missing targets there are and the higher the detection confidence of the missed targets, the more real targets are missed due to the feature map of this other agent, resulting in a significant negative contribution to perception performance. Finally, the calculated positive gain value (including the negative contribution from false detections) and the negative gain value are weighted together to obtain the information gain index for each other agent. This index comprehensively reflects the net contribution to the agent's perception performance after introducing the feature map of this other agent: a higher information gain index indicates that the feature map of this other agent is more helpful in improving perception accuracy; a lower information gain index, or even a negative value, indicates that the feature map of this other agent contaminates or interferes with perception performance and should be considered a suspicious agent.

[0043] Specifically, in step S43, the verification of the redundant targets includes: Obtain the first observation direction vector of the redundant target in the coordinate system of its own agent, and the second observation direction vector in the coordinate system of other agents; Calculate the epipolar constraint error between the first observation direction vector and the second observation direction vector; If the epipolar constraint error is less than the preset epipolar error, then the redundant target is determined to have passed the verification, and the information gain index remains unchanged. If the epipolar constraint error is greater than or equal to the epipolar error threshold, the redundant target is determined to have failed the verification, and the information gain index is corrected.

[0044] In this embodiment of the invention, for each marked redundant target, the agent further performs geometric verification to determine whether the redundant target is caused by an object existing in the real environment or by perceptual noise or false disturbances injected by a malicious agent. Specifically, firstly, the agent obtains the first observation direction vector of the redundant target in its own agent coordinate system and the second observation direction vector in other agent coordinate systems. The observation direction vector represents the spatial direction from the agent's position to the target. Secondly, the epipolar constraint error between the first and second observation direction vectors is calculated. Epipolar constraint is a fundamental principle in multi-view geometry, describing the geometric constraint relationship satisfied by the projection of the same three-dimensional point in space under different viewpoints. The larger the epipolar constraint error, the more inconsistent the observation directions of the two agents are towards the target, and the more likely the target is to be a false target.

[0045] Then, the calculated epipolar constraint error is compared with a preset epipolar error threshold. If the epipolar constraint error is less than the preset threshold, the redundant target is deemed to have passed geometric verification, indicating that it has consistent multi-view observations in 3D space and is a real object. In this case, the information gain index of the other agent remains unchanged. If the epipolar constraint error is greater than or equal to the preset threshold, the redundant target is deemed to have failed geometric verification, indicating that it lacks consistent multi-view observations and is likely a false target caused by perceptual noise or adversarial perturbations injected by a malicious agent. In this case, the information gain index of the other agent is corrected. Specifically, the penalty value corresponding to the redundant target is deducted from the information gain index. The penalty value is calculated based on the product of the detection confidence of the redundant target and the epipolar constraint error. The larger the epipolar constraint error and the higher the detection confidence, the larger the penalty value is deducted. At the same time, for multiple redundant targets that fail verification generated by the same other agent, the corresponding penalty values ​​are cumulatively deducted to obtain the corrected information gain index.

[0046] This invention accurately identifies redundant targets, missing targets, and detection attribute deviations through position overlap matching, quantifying the differentiated impact of each collaborator on perception performance. Secondly, based on the number and detection quality of redundant targets, the positive change in detection attribute deviations, and the number and detection quality of missing targets, a weighted information gain index is calculated to quantitatively assess the net contribution of collaborators. Furthermore, epipolar constraints are used for multi-view geometric verification of redundant targets. Target authenticity is determined by calculating the epipolar distance between observation direction vectors. False targets that fail verification are penalized with gain correction based on the product of their detection confidence and epipolar constraint error, effectively identifying maliciously injected adversarial perturbations. Finally, a local target set is constructed by selecting trustworthy collaborators through gain thresholding. This enables each agent to independently complete anomaly detection and trust scoring of other nodes without global labels, significantly improving the security and robustness of the multi-UAV cooperative perception system in adversarial environments.

[0047] Specifically, calculating the epipolar constraint error between the first observation direction vector and the second observation direction vector includes: Calculate the epipolar distance between the first observation direction vector and the second observation direction vector. The epipolar distance is the shortest distance in three-dimensional space between the ray corresponding to the first observation direction vector and the ray corresponding to the second observation direction vector. Use the epipolar distance as the epipolar constraint error.

[0048] In this embodiment of the invention, the shortest distance in three-dimensional space between the ray corresponding to the first observation direction vector and the ray corresponding to the second observation direction vector is calculated. This shortest distance is taken as the epipolar distance, and the epipolar distance is taken as the epipolar constraint error. Specifically, the first observation direction vector represents a spatial ray originating from the position of the agent itself and pointing towards the direction of the redundant target; the second observation direction vector represents a spatial ray originating from the position of another agent and pointing towards the same redundant target. Under ideal multi-view geometry, if the redundant target is a real object existing in three-dimensional space, then these two rays should intersect at the spatial point where the target is located, and the epipolar distance is zero. However, due to the influence of perception noise, positioning errors, or malicious perturbations, the two rays are usually difficult to intersect precisely.

[0049] Understandably, the shortest Euclidean distance between two rays is used to measure their deviation; this shortest distance is called the epipolar distance. The smaller the epipolar distance, the more consistent the observation directions of the two agents towards the redundant target, and the more likely the target is a real object. Conversely, the larger the epipolar distance, the greater the difference in observation directions between the two agents, and the more likely the target is a false target caused by perceptual noise or maliciously injected false perturbations.

[0050] Specifically, in step S5, the global target set is obtained, including: Each intelligent agent broadcasts its local target set to other intelligent agents in the multi-UAV cooperative perception network; Aggregate the local target sets of each agent and determine the number of times each agent is included in the local target set by other agents; Agents that have been included in the global target set with a number of times greater than or equal to a preset threshold are included.

[0051] In this embodiment of the invention, to filter out potential false local opinions and obtain a single, globally reliable decision, each agent participates in a consensus protocol. This protocol aggregates all individual suspect sets. This produces the final robust suspect set. :

[0052] Specifically, this framework implements the protocol through majority voting rules, formally expressed as:

[0053] For any given drone The total number of votes cast for blacklisting is calculated. (Summation term) That is The total number of votes received. Indicator function Represents a single intelligent agent Voting: If Will If a device is blacklisted, the value is 1; otherwise, it is 0. (Drone) Only when the total number of votes reaches or exceeds the required majority threshold Only then were they officially declared suspicious and added to the final blacklist. The threshold It serves a protective function, ensuring that decisions are made only when there is significant consensus, thereby preventing a small number of attackers from conspiring to frame innocent drones.

[0054] This invention, through a constructed consensus aggregation mechanism, broadcasts and aggregates the locally generated target sets independently by each agent through majority voting, effectively solving the problem of unreliable local judgments by a single agent. Each agent broadcasts its local target set to all other agents in the network. After collecting all local target sets, each agent counts the number of times it has been included in the local target sets by other agents, and includes agents with inclusion counts greater than or equal to a preset threshold into the global target set. This formalized consensus mechanism ensures that decisions are made only when there is a significant majority consensus within the cluster, effectively preventing a minority of attackers from colluding to frame innocent agents or protect malicious nodes.

[0055] Specifically, in step S6, the purified fusion feature map is obtained, including: The intermediate feature maps broadcast by each agent are filtered according to the global target set, and the intermediate feature maps corresponding to the agents in the global target set are excluded. The purified fused feature map is obtained by fusing its own intermediate feature map with the remaining intermediate feature maps after exclusion.

[0056] In this embodiment of the invention, after identifying a set of trustworthy collaborators through the execution of a security protocol, the ego drone... Only its own feature map With the message of successful verification To integrate, utilize aggregation Generate the purified fusion feature map:

[0057] in, It is a global suspect set (i.e., the set of nodes identified as attackers) driven by consensus. If the consensus process fails to converge, the ego drone will automatically fall back to fail-safe mode, relying solely on its own feature map. Independent perception is performed to ensure basic security. This trust-based filtering mechanism effectively purifies the information flow and guarantees the reliability of feature fusion.

[0058] Ultimately, the Ego drone The purified fusion feature map The signal is fed into the decoder (i.e., the detection head). The decoder outputs the final 3D object detection result, which is a set of 3D bounding boxes: Each bounding box It accurately describes the location, size, orientation, and category of the detected target. These detection results will serve as the basis for decision-making in downstream tasks such as path planning and cooperative obstacle avoidance.

[0059] This invention, through a constructed secure feature fusion mechanism guided by a consensus-driven global target set, achieves precise filtering and purification fusion of feature maps of malicious or untrusted agents. Based on the obtained global suspect set, it excludes intermediate feature maps corresponding to all agents consistently identified as attackers by the cluster, aggregating and fusing only its own intermediate feature map with the feature maps of the remaining trusted collaborators to generate a purified fused feature map. Simultaneously, the method incorporates a fault-safe fallback mechanism; when the consensus process fails to converge, the UAV automatically switches to an independent perception mode relying solely on its own feature map, ensuring basic operational safety. This effectively prevents malicious disturbances or contaminated data from entering the fusion stage, guaranteeing the reliability and integrity of the feature information flow.

[0060] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A consensus-based multi-UAV security cooperative perception method, characterized in that, include: Step S1: Construct a multi-UAV collaborative perception network composed of several intelligent agents. Each intelligent agent collects the original three-dimensional point cloud data of its environment and extracts features from the original three-dimensional point cloud data to obtain an intermediate feature map. Step S2: Each agent broadcasts its own intermediate feature map to other agents in the multi-UAV cooperative perception network and receives the intermediate feature maps broadcast by other agents. Step S3: Each agent performs target detection based on its own intermediate feature map to obtain a baseline detection result; and fuses its own intermediate feature map with the intermediate feature map of each other agent, and performs target detection on the fused feature map to obtain the corresponding comparative detection result. Step S4: Each agent calculates the difference information between each comparison detection result and the benchmark detection result, selects target agents from other agents based on the difference information, and constructs a local target set for the agent based on all selected target agents. Step S5: Each intelligent agent broadcasts its own local target set to the multi-UAV cooperative perception network, and aggregates all local target sets to obtain a global target set; Step S6: Each agent, based on the global target set, excludes the intermediate feature maps corresponding to the agents in the global target set, and fuses its own intermediate feature map with the intermediate feature maps of the remaining agents to obtain a purified fused feature map.

2. The consensus-based multi-UAV security cooperative perception method according to claim 1, characterized in that, In step S4, the difference information between each comparison detection result and the benchmark detection result is calculated, including: Step S41: Match the positional overlap of the comparison detection result with the benchmark detection result to identify redundant targets, missing targets, and detection attribute deviations; Step S42: Calculate the information gain index for each other agent based on the redundant targets, the missing targets, and the detection attribute deviation; Step S43: Verify the redundant targets and correct the information gain index based on the verification results; Step S44: Select other agents whose corrected information gain index is greater than the preset information gain index as target agents.

3. The consensus-based multi-UAV security cooperative perception method according to claim 2, characterized in that, In step S41, redundant targets, missing targets, and detection attribute biases are identified, including: Each target in the comparison detection result is paired with each target in the benchmark detection result, and the positional overlap between each pair of targets is calculated. Target pairs with a positional overlap greater than a preset positional overlap are determined to be a successful match, identified as the same target, and the detection attribute deviation of the target between the comparison detection result and the benchmark detection result is calculated. Mark all targets that did not match successfully in the comparison detection results as redundant targets; All targets that did not match successfully in the benchmark detection results are marked as missing targets.

4. The consensus-based multi-UAV security cooperative perception method according to claim 3, characterized in that, Calculate the detection attribute bias, including: Calculate the size deviation between the size of the same target in the comparative detection result and the size in the benchmark detection result, and use the size deviation as the detection attribute deviation.

5. The consensus-based multi-UAV security cooperative perception method according to claim 2, characterized in that, In step S42, the information gain metric for each other agent is calculated, including: Calculate the positive gain value based on the number of redundant targets and the detection quality; Calculate the positive gain value based on the positive change in the detected attribute deviation; Calculate the negative gain value based on the number of missing targets and the detection quality; The positive gain value and the negative gain value are combined to obtain the information gain index of each other agent.

6. The consensus-based multi-UAV security cooperative perception method according to claim 2, characterized in that, In step S43, the redundant targets are verified, including: Obtain the first observation direction vector of the redundant target in the coordinate system of its own agent, and the second observation direction vector in the coordinate system of other agents; Calculate the epipolar constraint error between the first observation direction vector and the second observation direction vector; If the epipolar constraint error is less than the preset epipolar error, then the redundant target is determined to have passed the verification, and the information gain index remains unchanged. If the epipolar constraint error is greater than or equal to the epipolar error threshold, the redundant target is determined to have failed verification, and the information gain index is corrected.

7. The consensus-based multi-UAV security cooperative perception method according to claim 6, characterized in that, Calculating the epipolar constraint error between the first observation direction vector and the second observation direction vector includes: Calculate the epipolar distance between the first observation direction vector and the second observation direction vector. The epipolar distance is the shortest distance in three-dimensional space between the ray corresponding to the first observation direction vector and the ray corresponding to the second observation direction vector. Use the epipolar distance as the epipolar constraint error.

8. The consensus-based multi-UAV security cooperative perception method according to claim 1, characterized in that, In step S5, the global target set is obtained, including: Each intelligent agent broadcasts its local target set to other intelligent agents in the multi-UAV cooperative perception network; Aggregate the local target sets of each agent and determine the number of times each agent is included in the local target set by other agents; Agents that have been included in the global target set with a number of times greater than or equal to a preset threshold are included.

9. The consensus-based multi-UAV security cooperative perception method according to claim 1, characterized in that, In step S6, the purified fusion feature map is obtained, including: The intermediate feature maps broadcast by each agent are filtered according to the global target set, and the intermediate feature maps corresponding to the agents in the global target set are excluded. The purified fused feature map is obtained by fusing its own intermediate feature map with the remaining intermediate feature maps after exclusion.

10. The consensus-based multi-UAV security cooperative perception method according to claim 1, characterized in that, In step S1, an intermediate feature map is obtained, including: The raw 3D point cloud data collected by each intelligent agent is voxelized to generate a bird's-eye view and a grid map. Feature extraction is performed on the bird's-eye view occupies the grid map to obtain an intermediate feature map.

Images