Unmanned aerial vehicle-mounted nuclear, chemical and biological composite detection system based on multi-source sensor fusion and intelligent identification method

Abstract

The invention discloses an unmanned aerial vehicle-mounted nuclear, chemical and biological composite detection system based on multi-source sensor fusion and an intelligent identification method. The system comprises an unmanned aerial vehicle flight platform, a multi-source sensor array, an airborne edge calculation unit, a cloud federated learning platform and a modular processing unit. Through a cross-modal association fusion algorithm and a lightweight space-time diagram neural network model, real-time and accurate identification and traceability prediction of nuclear, chemical and biological threats are realized at an airborne end, and marking, sampling or neutralization and other treatment actions can be intelligently started according to an identification result to form a complete detection-detection-identification-treatment closed-loop response system. And the automation and intelligence level of nuclear, chemical and biological emergency response is improved.

Description

Technical Field

[0001] This invention relates to the field of environmental safety and emergency detection technology, specifically to an unmanned aerial vehicle (UAV) nuclear, chemical, and biological composite detection system and intelligent identification method based on multi-source sensor fusion. Background Technology

[0002] The detection and identification of nuclear, chemical, and biological threats (hereinafter referred to as NCB) is a significant challenge in the fields of public safety and national defense security. Traditional manual detection methods are not only slow in response but also expose personnel to high-risk environments. With the maturity of UAV technology, using UAVs as mobile platforms for NCB detection has become an effective solution. However, existing UAV-borne detection systems and methods still have a series of technical shortcomings that need to be addressed in practical applications.

[0003] Existing technologies mainly follow three development paths, but all of them have obvious limitations:

[0004] First, in the single-platform multi-functional integration approach, existing systems mostly focus on adding multiple sensors to a single UAV platform. For example, the LSSNIFF 800 and other nuclear and chemical reconnaissance UAVs on the market integrate nuclear radiation and chemical agent detectors. However, such systems generally have two major drawbacks: First, the types of integrated sensors are limited, and they usually lack effective detection methods for biological warfare agents, making it impossible to achieve true nuclear, chemical, and biological composite detection. Second, and more importantly, the multiple sensors inside are often in an isolated state, only able to perform simple parallel data reporting or display, lacking a deep fusion mechanism at the information and decision levels. This makes it impossible for the system to understand the inherent logical relationship between different sensor data. For example, it cannot use nuclear radiation anomalies as clues to actively guide chemical or biological sensors to conduct collaborative verification, resulting in insufficient ability to identify composite and covert threats and a high false alarm rate.

[0005] Secondly, in the integrated nuclear, chemical, and biological monitoring approach, although the academic community has proposed a conceptual design for an integrated nuclear and chemical monitoring system that uses a unified information model to handle nuclear and chemical threats, this technical approach is still in the laboratory stage and faces challenges such as complex information recovery algorithms and unstable performance under complex measurement conditions. In particular, existing research focuses on the back-end information processing model, while seriously neglecting to establish an effective closed-loop linkage with the front-end UAV mobile platform and the back-end intelligent response action. Its system design usually stops at detection and identification, forming an information cliff, and is unable to automatically convert the identification results into precise control commands for UAV platforms or response units. This results in an excessively long response chain from threat detection to action, which cannot meet the real-time requirements of emergency response.

[0006] Third, in order to expand the detection range, researchers have explored multi-UAV collaborative operation modes on multi-UAV collaborative detection routes. However, existing multi-UAV systems mostly rely on preset formation flight or simple task allocation, lacking a core architecture that can intelligently integrate group detection data and perform dynamic task planning. Its shortcomings are: on the one hand, the system cannot construct a unified situation map from the data collected by multiple UAVs at different times and locations, making it difficult to accurately locate the threat source and accurately predict the spread trend; on the other hand, when some UAV nodes fail or the environment changes, the system lacks sufficient robustness to adaptively adjust, resulting in reduced collaborative efficiency.

[0007] Furthermore, from a broader technical perspective, existing unmanned aerial vehicle (UAV)-borne nuclear, chemical, and biological detection systems also share the following common problems:

[0008] The contradiction between intelligent recognition algorithms and airborne computing resources: complex intelligent recognition algorithms, such as large-scale deep learning models, usually require a large amount of computation and are difficult to implement in real-time inference on the airborne edge computing unit of the drone. If the data is sent back to the cloud for processing, it will introduce unacceptable communication delays and lose the meaning of emergency response.

[0009] The system's evolutionary capability is insufficient, and the detection experience and data of individual UAV systems cannot be effectively shared and learned from other units, resulting in slow updates to the knowledge base of the entire detection network and difficulty in responding to new or unknown nuclear, chemical and biological threats.

[0010] Therefore, there is an urgent need in this field for a new generation of unmanned aerial vehicle (UAV) systems and methods that can deeply integrate nuclear, chemical, and biological multi-source sensor information, possess airborne real-time intelligent identification and decision-making capabilities, and achieve an integrated closed loop of detection, inspection, identification, and processing. Summary of the Invention

[0011] To overcome the problems of the prior art, this invention discloses an unmanned aerial vehicle (UAV) nuclear, chemical, and biological composite detection system and intelligent identification method based on multi-source sensor fusion.

[0012] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0013] A UAV-borne nuclear, chemical, and biological composite detection system based on multi-source sensor fusion includes:

[0014] Unmanned aerial vehicle (UAV) flight platform;

[0015] A multi-source sensor array, integrated on a drone flight platform, is used to simultaneously collect raw data on nuclear, chemical, and biological threats;

[0016] The airborne edge computing unit communicates with the multi-source sensor array to process raw data in real time; the airborne edge computing unit has a built-in multi-source information fusion engine and intelligent recognition module;

[0017] Among them, the multi-source information fusion engine is configured to run cross-modal correlation fusion algorithms;

[0018] The intelligent identification module is configured to identify and classify single and combined nuclear, chemical, and biological threats based on the output of a multi-source information fusion engine and through a lightweight spatiotemporal graph neural network model.

[0019] Preferably, the multi-source sensor array includes a nuclear detection module for detecting gamma rays and neutrons, a chemical detection module employing a time-of-flight ion mobility spectrometer, and a biological detection module containing a laser-induced fluorescence radar and a particle collector.

[0020] Preferably, the cross-modal association fusion algorithm specifically includes:

[0021] Trigger-verifier rule base, predefined data association logic for nuclear detectors, chemical detectors and biological detectors;

[0022] The dynamic task orchestration engine automatically generates a verification task sequence based on the rule base when the data from any sensor exceeds a preset threshold and becomes a trigger. It also adjusts the working mode and sampling parameters of other related sensors in real time to perform collaborative verification.

[0023] Preferably, the execution results of the verification task sequence are weighted and fused with the confidence level of the initial trigger to finally output a comprehensive threat type, threat level and overall credibility assessment.

[0024] Preferably, the lightweight spatiotemporal graph neural network model processes data in the following manner:

[0025] The detection data of multiple waypoints or spatial locations during a single flight are constructed into a spatiotemporal graph structure. The nodes in the spatiotemporal graph structure contain sensor readings, and the edges are defined by geographical location relationships, time series, and environmental wind field data.

[0026] The model learns both the spatial diffusion patterns and temporal evolution trends of threats, and outputs estimates of the geographical location of threat sources, predictions of diffusion directions, and classification results of threat types.

[0027] Preferably, the system also includes a cloud-based federated learning platform that is communicatively connected to the airborne edge computing unit;

[0028] The cloud-based federated learning platform is configured as follows:

[0029] The model parameter updates, which have been desensitized, are aggregated from multiple drone systems to train and generate an enhanced global intelligent recognition model.

[0030] Through a knowledge distillation process, the knowledge of the global intelligent recognition model is compressed into a lightweight spatiotemporal graph neural network model, and then distributed to the intelligent recognition modules of each drone.

[0031] Preferably, the system also includes a modular processing unit that can be mounted on the UAV flight platform;

[0032] The modular processing unit communicates with the airborne edge computing unit and receives instructions from the intelligent identification module;

[0033] The modular disposal unit is one of the following: a marker ball dispenser, a disinfectant spraying system, or a sampling container delivery device.

[0034] Preferably, the system further includes a digital twin simulation module, which is used to simulate the effect of the treatment plan in the digital space after receiving the recognition result from the intelligent recognition module, and to select the optimal treatment plan instruction to the modular treatment unit for execution based on the simulation result.

[0035] Preferably, the system constructs a collaborative detection network formed by multiple drones through a self-organizing network;

[0036] One drone serves as the master node, responsible for integrating detection data from other drone nodes and coordinating the flight paths and sensing tasks of various drones within the collaborative detection network.

[0037] Preferably, a UAV-borne nuclear, chemical, and biological composite detection and intelligent identification method based on multi-source sensor fusion is applied to the above-mentioned system, and the method includes:

[0038] Nuclear, chemical, and biological environmental parameters are collected synchronously using a multi-source sensor array;

[0039] In the airborne edge computing unit, multi-source heterogeneous data are fused in real time through a cross-modal correlation fusion algorithm;

[0040] Based on the fused feature data, a lightweight spatiotemporal graph neural network model is used for inference to identify and output the type, location, and confidence level of nuclear, chemical, and biological threats.

[0041] Based on the identification results, disposal instructions are generated and executed, including controlling the drone to avoid pollution, issuing alarms, and locating the pollution source.

[0042] The beneficial effects of this invention are as follows:

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] To achieve true nuclear, chemical, and biological composite detection and deep information fusion, a multi-source sensor array integrating nuclear, chemical, and biological specialized sensors is used. By running a cross-modal correlation fusion algorithm, the system establishes intelligent logical connections between sensor data, such as a trigger-verifier mechanism. This transforms simple data superposition into information-level collaborative verification, improving the accuracy of identifying composite threats and effectively reducing the false alarm rate.

[0045] It possesses forward-looking capabilities for threat prediction and attribution. By using a lightweight spatiotemporal graph neural network model to process detection data, the system can learn the diffusion and evolution patterns of threats in the spatiotemporal dimension, outputting the location estimation of threat sources and the prediction of diffusion trends, realizing the leap from perceiving the present to predicting the future, and providing critical lead time for command and decision-making.

[0046] To build a continuously evolving intelligent system that collaborates with cloud, edge, and device, we introduce a cloud-based federated learning platform. While ensuring the privacy of each user's data, we aggregate collective wisdom to optimize the model. We also use knowledge distillation technology to sink the enhanced global model capabilities into a lightweight model, enabling the intelligent model deployed on the airborne edge computing unit to continuously evolve and become more intelligent with use.

[0047] By forming an integrated closed loop of detection, inspection, identification, and response, and by introducing modular response units that are linked with intelligent identification results, the system can directly convert identification instructions into response actions, such as marking, disinfection, and sampling, reducing response time from minutes to seconds. This solves the problem of information gaps that stop at alarms in existing technologies, and improves the overall efficiency of emergency response and personnel safety. Attached Figure Description

[0048] Figure 1 This is a flowchart of an unmanned aerial vehicle (UAV) nuclear, chemical, and biological composite detection system based on multi-source sensor fusion, provided in Embodiment 1 of the present invention.

[0049] Figure 2 The flowchart illustrates a method for intelligent identification of nuclear, chemical, and biological composite detection on unmanned aerial vehicles based on multi-source sensor fusion, as provided in Embodiment 1 of the present invention. Detailed Implementation

[0050] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, exemplary embodiments will be described in detail below, examples of which are illustrated in the accompanying drawings. In the following description relating to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of methods and systems consistent with some aspects of this application as detailed in the appended claims.

[0051] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0052] The following detailed description of the specific implementation methods, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided in detail.

[0053] Example 1

[0054] Please refer to Figure 1 This invention provides an unmanned aerial vehicle (UAV) nuclear, chemical, and biological composite detection system based on multi-source sensor fusion, comprising:

[0055] The drone flight platform uses an industry-grade hexacopter drone as its basic carrier. The platform provides stable flight capabilities, sufficient payload (greater than 10kg), and open secondary development interfaces, providing a reliable mobile base for subsequent sensor integration and intelligent decision-making.

[0056] The multi-source sensor array is rigidly connected to the center of gravity of the UAV flight platform via a rigid carbon fiber clamp. This array is not a simple stack of sensors, but rather collects raw data on nuclear, chemical and biological threats in parallel in a synchronous triggering manner.

[0057] For example, the LSSNIFF 800 nuclear and chemical reconnaissance UAV has already integrated nuclear detection and chemical detection modules on the same platform. Based on this, the present invention further introduces a biological warfare agent detection module to form a nuclear, chemical and biological composite reconnaissance capability.

[0058] Airborne edge computing unit: The airborne edge computing unit adopts NVIDIA Jetson AGX Orin or similar high-performance, low-power embedded AI computing modules, and is connected to the multi-source sensor array through a customized CAN bus and serial communication protocol, responsible for processing massive and heterogeneous raw data in real time.

[0059] The airborne edge computing unit has two core software modules built in: a multi-source information fusion engine and an intelligent recognition module. This allows computing tasks to be moved from the cloud to the drone, reducing the latency of data transmission and improving the system's autonomous response speed and reliability in complex electromagnetic environments.

[0060] Among them, the multi-source information fusion engine is configured to run a cross-modal correlation fusion algorithm. This algorithm goes beyond traditional data-level fusion, such as simply splicing data, and focuses on deep fusion at the information level and decision level. It can understand the physicochemical meaning behind different sensor data and establish the logical relationship between them.

[0061] The intelligent recognition module is configured to be based on the output of the multi-source information fusion engine, and the intelligent recognition module is configured to be based on the high-level feature information output by the fusion engine, and the final inference is performed through a lightweight spatiotemporal graph neural network model.

[0062] This model is designed specifically for limited airborne computing resources. Its parameters have been carefully optimized, enabling it to run on the Jetson platform in real-time or near real-time with a latency of less than 1 second. Ultimately, it achieves accurate identification and classification of single nuclear, chemical, and biological threats, as well as complex composite threat patterns such as nuclear-chemical and chemical-biological threats.

[0063] Specifically, by combining a highly integrated multi-source sensor array with powerful airborne edge computing capabilities, and running advanced cross-modal fusion algorithms and lightweight intelligent recognition models, a drone detection platform capable of autonomously responding to complex nuclear, chemical, and biological threats is constructed.

[0064] Furthermore, the multi-source sensor array includes a nuclear detection module for detecting gamma rays and neutrons, a chemical detection module employing a time-of-flight ion mobility spectrometer, and a biological detection module containing a laser-induced fluorescence radar and a particle collector.

[0065] The nuclear detection module for detecting gamma rays and neutrons is specifically implemented using a detection scheme based on europium-doped strontium iodide scintillator crystal coupled with a silicon photomultiplier array.

[0066] Gamma-ray detection uses europium-doped strontium iodide crystals as the core detection material. When irradiated by gamma rays, it will ionize or be excited, emitting fluorescence of a specific wavelength, which is visible light.

[0067] Europium-doped strontium iodide crystals have high light output, good energy resolution, and moderate density, making them suitable for gamma-ray energy spectroscopy measurements and nuclide identification.

[0068] Neutron detection involves wrapping a scintillator crystal with a layer of enriched neutrons. 6 Li fluoride materials as neutron converters;

[0069] When the neutron and 6 When Li undergoes a nuclear reaction, it produces alpha particles and tritium nuclei. These charged particles then excite the scintillator crystal to emit light, enabling indirect detection of neutron signals.

[0070] Signal conversion: The silicon photomultiplier tube array is mounted close to the scintillator crystal and is responsible for converting weak fluorescence signals into electrical signals. It adopts a structure of multiple independent avalanche photodiode micro-elements connected in parallel, which has the advantages of high gain, low operating voltage and insensitivity to magnetic fields, making it suitable for stable operation in the harsh environment of UAV vibration.

[0071] The chemical detection module employing time-of-flight ion mobility spectrometry is a miniaturized atmospheric pressure chemical ionization source and drift tube system;

[0072] Ambient air is drawn into the ionization region by a miniature pump, in a weak radiation source, such as... 63 Under the action of Ni or non-radioactive corona discharge needle, sample molecules are ionized to form parent ions;

[0073] Ions enter a drift tube filled with inert buffer gas under the action of a uniform electric field and begin to drift in a direction. Different chemical substances have different migration rates in the drift tube due to differences in their mass-to-charge ratio and collision cross section.

[0074] Specifically, ions with compact structures and smaller masses migrate faster, while ions with loose structures and larger masses migrate slower.

[0075] The ions eventually reach the Faraday cup detector, where the system accurately records their flight time. By comparing the measured flight time spectrum with the characteristic spectra of known chemical warfare agents in the built-in database, such as sarin, soman, and VEX, qualitative and quantitative analysis of chemical agents can be achieved. The detection limit of this technology for the above-mentioned agents can reach the level of milligrams per cubic meter to micrograms per cubic meter, with a response time of less than a few seconds.

[0076] The biological detection module of laser-induced fluorescence radar and particle collector combines two technical approaches: remote sensing and in-situ sampling.

[0077] Laser-induced fluorescence radar emits ultraviolet laser beams of specific wavelengths into the air, such as the third harmonic output of Nd:YAG lasers at 355nm.

[0078] When a laser beam irradiates bioaerosols in the air, such as bacterial spores and viral clusters, it excites their internal fluorescent groups, such as tryptophan, tyrosine, and nicotinamide adenine dinucleotide, to produce fluorescence.

[0079] The detector receives this fluorescence signal and forms a fluorescence spectrum. Different types of biological warfare agents will exhibit unique fluorescence fingerprints due to differences in their internal components and structures, which can be used for preliminary classification and warning.

[0080] The particle collector works in conjunction with the detector to actively collect suspended particles in a specific volume of air through inertial impaction or filter enrichment. The collected samples can be temporarily stored in a sealed container.

[0081] This provides a material basis for more accurate on-site analysis, such as using immunochromatographic test strips or bringing samples back to the laboratory for polymerase chain reaction identification, thereby improving the reliability of biothreat confirmation.

[0082] Specifically, by defining the advanced technical paths for each of the nuclear, chemical, and biological detection modules, the system integrates multi-source sensors and possesses high-quality, high-reliability raw data acquisition capabilities. The energy spectrum analysis capability of the nuclear detection module enables nuclide identification, the high sensitivity and rapid response of the chemical detection module ensure the detection of toxic agents, and the introduction of the biological detection module fills a key gap in existing UAV reconnaissance systems. The three are organically combined to form an indispensable hardware foundation and data source for the entire system to achieve accurate and intelligent identification.

[0083] Furthermore, the cross-modal association fusion algorithm specifically includes:

[0084] The trigger-verifier rule base is a collection of digital rules pre-installed in the system. It is a structured database or configuration file that defines the inherent physicochemical relationship logic between the data of nuclear detectors, chemical detectors, and biological detectors.

[0085] The rule base contains multiple production rules in the form of IF-THEN, such as IF, gamma ray dose rate > 1.0 μSv / h, trigger is nuclear detector; THEN, activate chemical detection module to perform high-frequency sampling verification of nerve agents such as sarin and VEX, and simultaneously increase the monitoring priority of radioactive drug resistant bacterial spores in biological detection module.

[0086] IF: Laser-induced fluorescence radar detects a specific fluorescence spectrum of suspected ricin aerosol, triggering a biological detector; THEN: Instructs the chemical detection module to switch to the high-sensitivity mode of time-of-flight ion mobility spectrometry, focusing on detecting toxic precursor compounds that may coexist with biotoxins.

[0087] These rules are based on the scientific principle that nuclear, chemical, and biological threatening substances may coexist or be causally related in the actual environment;

[0088] For example, the handling of certain nuclear materials may be accompanied by the volatilization of specific chemical solvents, and the release of certain biological warfare agents may be to cover up signs of a chemical attack;

[0089] The dynamic task orchestration engine is a real-time running software service that continuously monitors the data streams from all sensors. Once the trigger conditions are met, the engine will immediately intervene and take over the scheduling of the sensors.

[0090] The engine continuously scans real-time data from nuclear detection modules, such as detectors using scintillator + SiPM technology, chemical detection modules, such as detectors using time-of-flight ion mobility spectrometers, and biological detection modules, such as laser-induced fluorescence radar.

[0091] When any data value exceeds its preset threshold, such as when the chemical detector detects sarin at a concentration exceeding 0.2 mg / m³, the sensor is marked as a trigger.

[0092] After receiving the trigger signal, the engine immediately queries the trigger-verifier rule base. Based on the matched rules, the engine does not simply activate other sensors, but generates an optimal and ordered sequence of verification tasks.

[0093] For the kernel trigger mentioned above, the generated sequence might be:

[0094] Step 1: Adjust the chemical detector to focus on the specific toxic agent; Step 2: Increase the flow rate of the bioaerosol sampler; Step 3: Combine the data from the three sources to calculate the overall confidence level.

[0095] To perform the verification task, the engine sends control commands to other associated sensors, adjusting their operating modes and sampling parameters in real time; this includes:

[0096] Operating modes, such as switching the chemical detection module from the regular inspection mode to a fixed-point tracking mode for specific toxic agents;

[0097] Sampling parameters, such as increasing the inhalation pump flow rate of the biological particle collector to capture more air samples, or reducing the integration time of the spectral sensor to improve detection real-time performance and avoid loss of critical data;

[0098] Specifically, through the above mechanism, the system achieves multi-sensor collaborative verification. The collaborative verification process is no longer a simple superposition of the independent work of each sensor, but an organic whole behavior under the command of a unified intelligent hub. This reduces the probability of false alarms caused by environmental interference or device noise from a single sensor. For example, an increase in nuclear radiation readings may be due to environmental background fluctuations or medical radiation sources. However, if it is accompanied by synchronous anomalies of characteristic chemical agents and specific biomarkers, the system can determine it as a composite attack with a high degree of confidence, thereby providing a more reliable and accurate decision-making basis for subsequent responses.

[0099] Furthermore, the execution results of the verification task sequence are weighted and fused with the confidence level of the initial trigger to finally output a comprehensive threat type, threat level and overall credibility assessment;

[0100] After the dynamic task orchestration engine generates and executes the verification task sequence, each sensor participating in the verification will produce a result, which contains two parts of information:

[0101] A binary conclusion, namely whether the sensor confirmed the existence of a threat;

[0102] Local confidence, a value between 0 and 1, represents the degree of confidence the sensor has in its own conclusions. This confidence level is calculated internally by the sensor based on factors such as signal strength, signal-to-noise ratio, and comparison with environmental background values.

[0103] For example, when a nuclear detector is used as a trigger, a chemical detector is scheduled to verify a specific toxic agent and may return an execution result that a sarin signature spectrum has been detected with a local confidence level of 0.85.

[0104] The confidence level of the initial trigger refers to the initial reliability score assigned by the system to the sensor that first issues an alarm, i.e., the trigger when its data exceeds a preset threshold. The calculation of this confidence level takes into account the following factors:

[0105] The greater the signal exceeds the threshold, the higher the confidence level.

[0106] Sensor historical reliability, including the false alarm rate and missed alarm rate of the sensor in past missions;

[0107] Environmental interference factors, including the influence coefficients of current ambient temperature and humidity on sensor performance;

[0108] Weighted fusion is performed by a dedicated submodule within the multi-source information fusion engine. Its implementation employs a data fusion model based on weighted averages, and the mathematical expression of this process can be summarized as follows:

[0109]

[0110] Among them, the overall credibility represents the overall credibility assessment of the final output, which is a normalized value between 0 and 1;

[0111] W t The weight representing the initial trigger is predefined in the trigger-validator rule base and reflects the historical authority of such triggers;

[0112] C t Represents the confidence level of the initial trigger;

[0113] W vi The weight representing the execution result of the i-th verification task sequence is also defined by the rule base. The weight depends on the type of verification sensor and the importance of the specific verification task it performs.

[0114] C vi Represents the local confidence level returned by the i-th verification sensor;

[0115] After fusion computing, the system ultimately outputs a structured comprehensive judgment, which includes the following three dimensions:

[0116] Threat type: Based on the conclusions of all participating sensors, the most likely threat type is determined through voting or the principle of maximum membership, such as pure nuclear threat, chemical-biological composite threat, etc.

[0117] Threat levels are mapped to discrete levels such as low, medium, high, and extremely high based on the numerical range of comprehensive credibility, providing an intuitive basis for emergency response;

[0118] The overall credibility assessment, namely the specific numerical value of the comprehensive credibility obtained from the above calculation, provides a continuous and refined probabilistic reference for command and decision-making.

[0119] Specifically, through a rigorous weighted fusion mathematical model, the multi-source and heterogeneous information generated by collaborative verification is transformed into a single, clear, and quantifiable action guide, which effectively suppresses accidental false alarms from a single sensor and improves the reliability of the system.

[0120] Furthermore, the lightweight spatiotemporal graph neural network model processes data in the following ways:

[0121] Constructing a spatiotemporal graph structure from detection data of multiple waypoints or spatial locations during a single flight is the foundation for data processing in this model. This structure abstracts continuous detection tasks into a dynamic topological network, and its specific construction method is as follows:

[0122] A node is defined as a single stop measurement of a UAV at a single flight waypoint, or a snapshot scan of different spatial locations during a single flight.

[0123] Each node's attribute is a multidimensional vector containing all sensor readings at that time and location, such as gamma-ray dose rate, concentration of a specific chemical agent, and fluorescence intensity of bioaerosols.

[0124] The node feature vector is a 7-dimensional floating-point array containing gamma dose rate, neutron count, chemical agent concentration, biofluorescence intensity, wind speed, wind direction, and timestamp; the model output is a 3-dimensional tensor representing the latitude and longitude of the threat source, threat type number, and confidence level.

[0125] The edges between nodes are defined by three types of information to reflect the real physical environment on which threat propagation depends;

[0126] Geographical relationships are determined by calculating the Euclidean distance between nodes based on the latitude and longitude coordinates of the nodes obtained from GPS or BeiDou positioning modules. Nodes that are closer in distance usually have higher edge connection weights.

[0127] Time series data determines the chronological order of nodes on a timeline based on the timestamps of data collection, and is used to build relationships along the time dimension.

[0128] Environmental wind field data is crucial for accurately predicting the direction of threat spread. The model will incorporate real-time wind field data, including wind direction and speed.

[0129] For node A upwind and node B downwind, the model will construct a directed edge from A to B, with the weight determined by the wind speed and the frequency of the prevailing wind direction.

[0130] This modeling method, which takes into account the anisotropy of the geographical environment, has been proven to improve the accuracy of predicting the diffusion of pollutants such as PM2.5.

[0131] In this way, discrete detection data points are organized into a graph structure containing rich spatiotemporal physical relationships;

[0132] The model learns both the spatial diffusion patterns and temporal evolution trends of threats simultaneously, achieved through a hybrid spatiotemporal graph convolutional network. This network references the model architecture in remote sensing intelligent analysis of surface landscape pattern evolution and is deeply optimized for UAV onboard computing resources.

[0133] Spatial diffusion pattern learning uses Graph Convolutional Network (GCN) as a spatial feature extractor. GCN obtains the topological structure of the graph through its adjacency matrix, i.e. the edges between nodes, and allows each node to aggregate information of its neighboring nodes through a message passing mechanism.

[0134] If a node detects slight pollution and is located downwind and close to the pollution source node, then by aggregating information from the direction of the pollution source, the node can also obtain a high pollution risk characteristic. This process enables the model to accurately capture the spatial diffusion pattern of the threat.

[0135] In terms of temporal evolution trend learning, the model uses a gated recurrent unit (GRU) to process the feature sequence of each node arranged in chronological order. The GRU can remember historical states, thereby learning the evolution trend of threat concentration increasing, decreasing or fluctuating periodically over time.

[0136] To meet the real-time requirements of airborne edge computing units, the model undergoes multiple lightweight design features;

[0137] Adopting a similar approach to LGS-CNN, depthwise separable convolutions are used instead of standard image convolutions to reduce the number of parameters;

[0138] Knowledge distillation is performed on the trained model to compress the knowledge from the large model into a small model;

[0139] Fixed-point quantization technology is used to convert model weights from 32-bit floating-point numbers to 8-bit integers, further improving inference speed and facilitating deployment on embedded systems;

[0140] After learning the spatiotemporal features, the model outputs structured results in the following three dimensions:

[0141] The model estimates the geographic location of the threat source by analyzing the activation intensity of nodes on the spatial feature map. It can then inversely locate one or more nodes that contribute the most to the current contamination field and output the geographic locations of these nodes as the coordinates of the threat source. This is similar to the method of locating multiple radioactive sources in a large cargo yard.

[0142] The diffusion direction prediction model combines learned spatial patterns and real-time wind field data to predict the main diffusion path and impact range of the threat cloud in the future, such as in the next 5-10 minutes.

[0143] The model inputs the fused spatiotemporal features of each node into the classifier to output the classification results of the threat type, such as pure chemical threat or nuclear-biological complex threat, and attaches a confidence level.

[0144] Specifically, by constructing discrete detection data into a spatiotemporal graph structure with clear physical meaning, and using a lightweight deep learning model to simultaneously mine its spatiotemporal correlations, the system can ultimately achieve accurate location, source tracing, and prediction of threats.

[0145] Furthermore, the system also includes a cloud-based federated learning platform that communicates with the airborne edge computing units. The cloud-based federated learning platform is a software system deployed on a remote high-performance server cluster. It establishes a communication connection with the airborne edge computing units deployed on each UAV through an encrypted wireless communication network, such as 4G / 5G or satellite links. This design draws on the idea of ​​decentralized federated learning frameworks, which exchange information through point-to-point communication. However, in this system, the cloud-based federated learning platform serves as a logical coordination center for efficiently aggregating global knowledge.

[0146] The cloud-based federated learning platform is configured to aggregate anonymized model parameter updates from multiple drone systems to train and generate a reinforced global intelligent recognition model. The specific process is as follows:

[0147] Each drone participating in federated learning trains a lightweight spatiotemporal graph neural network model locally using private data collected by its multi-source sensor array. The resulting model parameters are updated, such as gradients or weight differences, rather than the original kernel, chemical, and biological detection data.

[0148] These parameters themselves do not contain sensitive environmental sample information, thus achieving data desensitization and effectively protecting the privacy of the task area;

[0149] Desensitization refers to uploading only the model gradient difference, while the original detection data remains stored locally and is not uploaded to the cloud.

[0150] During the aggregation process, the cloud platform receives parameter updates from multiple drone systems and uses a federated averaging algorithm or other advanced aggregation algorithms, such as FedProx, to perform a weighted average of these updates. This is then used to train and generate a reinforced global intelligent recognition model. The allocation of aggregation weights can be determined based on the amount of data from each drone or the quality of the model updates.

[0151] Through a knowledge distillation process, the knowledge of the global intelligent recognition model is compressed into a lightweight spatiotemporal graph neural network model, and then distributed to the intelligent recognition modules of each UAV. This is a key step in achieving lightweight airborne intelligence.

[0152] The knowledge distillation process consists of a teacher-student network framework. The aggregated global intelligent recognition model, which is more powerful but also larger in size, plays the role of the teacher, while the original lightweight spatiotemporal graph neural network model, which is designed for airborne environments, plays the role of the student.

[0153] Knowledge transfer and knowledge distillation involve having student models imitate the behavior of teacher models.

[0154] Specifically, a public dataset containing various nuclear, chemical, and biological threat patterns is used as input for both the teacher and student models. The student model learns to match the soft-label output of the teacher model. This involves a probability distribution that contains more information about inter-category relationships. The loss function typically uses KL divergence to minimize the difference between the two output distributions, as expressed in the formula:

[0155]

[0156] Among them, L KD This represents the loss value from knowledge distillation.

[0157] and These represent the original output scores of the teacher model and the student model for the i-th category, respectively.

[0158] T is a temperature parameter used to control the smoothness of the output probability distribution;

[0159] After knowledge distillation training, the student model, namely the lightweight spatiotemporal graph neural network model, successfully acquired the core knowledge of the teacher model, and its performance was improved. At the same time, it maintained its lightweight characteristics and met the requirements of airborne real-time reasoning. Finally, the enhanced lightweight model was distributed from the cloud platform to the intelligent recognition module of each drone, completing a global model evolution and deployment.

[0160] Specifically, by integrating collective intelligence through a federated learning mechanism to generate a more powerful global model, and then using knowledge distillation technology to inject the essence of the large model into a lightweight model, this process enables each drone to benefit from the collective experience of the entire fleet, continuously improving its ability to identify nuclear, chemical, and biological threats, while ensuring its independent, real-time, and reliable operation at the edge, thus resolving the contradiction between the limitations of individual drone intelligence and the bottleneck of onboard computing power.

[0161] Furthermore, the system also includes modular processing units that can be mounted on the UAV flight platform, and the specific implementation adopts standard mechanical and electrical interfaces to achieve rapid integration;

[0162] The drone flight platform uses an industry-grade hexacopter drone as its basic carrier. The bottom of the platform has reserved automated mounting points that meet the standards. The outer shell of the modular processing unit integrates matching mechanical latches and waterproof electrical connectors.

[0163] In practice, ground personnel can load or switch specific treatment units within seconds according to on-site task requirements, such as marking, disinfection or sampling.

[0164] This design draws on the modular design concept of drones, effectively overcoming the shortcomings of traditional drones, such as the difficulty in disassembling and assembling components and the inability to meet the needs of various payloads, and realizing plug-and-play functionality and flight platform.

[0165] The modular processing unit communicates with the airborne edge computing unit and receives instructions from the intelligent identification module to establish a reliable low-latency instruction and control link.

[0166] At the hardware level, the processing unit establishes a physical connection with the onboard edge computing unit inside the UAV body through the electrical interface of the mounting point. The communication protocol adopts CAN bus or high-speed serial communication. Both protocols have good anti-interference capabilities and are suitable for achieving reliable data transmission in complex electromagnetic environments.

[0167] After completing threat identification and location, the intelligent identification module will generate structured disposal instructions if it confirms the presence of VEX poison contamination at the (X,Y) coordinates.

[0168] The instruction is sent to the modular treatment unit in real time through the aforementioned communication link. The instruction includes at least the treatment mode and action parameters, such as the amount of disinfectant sprayed and the target location information.

[0169] The microcontroller embedded in the processing unit, such as the STM32 series, will parse and drive the corresponding actuators, such as pumps, motors, and solenoid valves, to complete the specified actions after receiving the instructions.

[0170] The modular disposal unit is one of the following: a marker ball dispenser, a disinfectant spraying system, or a sampling container delivery device. The following are specific implementation schemes for these three typical disposal units:

[0171] The marker ball dispenser is used to quickly and visually mark the boundaries of contaminated areas on the ground, warning people to avoid them;

[0172] The device includes a cylindrical ammunition magazine, a rotating wheel driven by a stepper motor, and a delivery tube. The ammunition magazine contains hundreds of highly visible fluorescent or red environmentally friendly biodegradable marker balls.

[0173] Upon receiving the delivery instruction, the microcontroller precisely controls the rotation angle of the stepper motor, causing the dial to release a fixed number of marker balls with each rotation. The marker balls fall through the delivery tube under gravity and land accurately on the boundary of the contaminated area.

[0174] Disinfectant spraying system is used for preliminary decontamination of identified nuclear, chemical, and biological contaminated areas;

[0175] The disinfectant spraying system includes a corrosion-resistant disinfectant storage tank, a miniature precision diaphragm pump, an aerosol spray head, and a liquid control solenoid valve.

[0176] The system draws on the aerosol spray disinfection method in the technical guidelines for disinfection of public places. When a spraying instruction is received, the micro pump is activated to pump the disinfectant, such as a 5g / L peracetic acid solution, from the storage tank and atomize it into tiny droplets with a diameter of less than 20μm, accounting for more than 90% of the total droplets, through the aerosol spray head.

[0177] This allows the disinfectant to better suspend in the air and cover object surfaces, combining the effects of spraying and fumigation to effectively neutralize chemical or biological warfare agents. The spraying rate can be infinitely adjusted according to instructions, typically at 100ml / m². 2 Up to 300ml / m 2 between;

[0178] Sampling container delivery device for safely collecting air, soil or liquid samples in high-risk areas for subsequent precise laboratory analysis;

[0179] The device includes a multi-compartment turret, mechanical grippers, and a buffer release mechanism. Each compartment is pre-installed with a sterile, sealed sampling container, such as a sampling bag or sample bottle.

[0180] Upon receiving the sampling instruction, the turret rotates to deliver the sampling container from the designated compartment to the delivery position. With the assistance of the buffer release mechanism, the mechanical gripper smoothly delivers the sampling container to the target location, preventing the container from breaking upon impact with the ground and ensuring the integrity of the sample.

[0181] Specifically, through standardized mounting with the UAV flight platform, reliable communication with the airborne edge computing unit, and dedicated devices for three typical tasks—identification, disinfection, and sampling—traditional reconnaissance UAVs are upgraded into aerial outposts with immediate autonomous response capabilities. This effectively solves the problem of interruptions in the existing technology system, reducing the response time from threat detection to threat handling from minutes to seconds, thereby improving the efficiency of handling nuclear, chemical, and biological emergencies and enhancing personnel safety.

[0182] Furthermore, the digital twin simulation module is a software system that runs on an airborne edge computing unit or a near-ground edge server connected via a low-latency link;

[0183] The digital twin simulation module is activated after receiving the recognition result from the intelligent recognition module. This recognition result is a structured data packet, which contains at least information such as threat type, threat level, geographical coordinates of pollution source, and pollution concentration distribution. This data will serve as the initial and boundary conditions for the digital twin simulation.

[0184] Simulating the effects of different treatment options in the digital space involves the following steps:

[0185] Based on the geographic information system data and environmental parameters transmitted back by the UAV in real time, such as temperature, humidity, wind speed and direction, as well as the pollution distribution map output by the intelligent identification module, the module constructs a three-dimensional virtual mapping layer in the digital space that is consistent with the real environment.

[0186] The module contains a pre-built disposal plan library, which includes various disposal plan templates for different threat types, such as chemical agents, biological warfare agents, and radioactive dust, such as constant volume spraying, ultra-low volume spraying, and boundary marking.

[0187] Meanwhile, the module integrates a lightweight physics engine that encapsulates core computational fluid dynamics algorithms, enabling it to simulate the diffusion and sedimentation of substances such as disinfectants and markers in real-world environments.

[0188] For each feasible disposal plan, the module performs rapid simulations in digital space. For example, regarding chemical agent pollution, the module will simulate the effects of Plan A (using 5% peracetic acid at a spraying intensity of 200 ml / m²) and Plan B (using 10% sodium hypochlorite at a spraying intensity of 150 ml / m²) in sequence. The simulation solves the mass transport equation, which can be simplified as follows:

[0189]

[0190] Where C represents the concentration field of the treated material or pollutant, t represents time, u represents the wind speed vector defined by the environmental wind field data, D represents the turbulent diffusion coefficient, and S represents the source term, which simulates the spray source of the treatment unit here.

[0191] By solving this equation, the module can predict the spatial distribution of disinfectants and their interaction with pollutants under different scenarios, and finally quantify the simulation effect indicators of each scenario, such as the expected neutralization rate, treatment coverage and estimated completion time.

[0192] Selecting the optimal handling plan based on simulation results and instructing modular handling units to execute it is a decision-making process based on multi-objective optimization.

[0193] The digital twin simulation module has a built-in decision logic unit that performs a weighted comprehensive score on the effectiveness indicators of all simulation schemes and task priorities, such as fastest handling and most thorough handling.

[0194] In scenarios where a safe passage needs to be opened quickly, the processing time has the highest weight, so the solution that can complete the area coverage the fastest will be selected as the optimal solution.

[0195] Once the optimal solution is selected, the module will immediately generate a detailed and executable sequence of control instructions, which will be sent directly to the modular treatment unit. If the optimal solution is disinfectant spraying, the instructions will precisely control the flow rate of the micro pump, the nozzle diameter of the spray head, and the flight path of the drone in the disinfectant spraying system to ensure that the actual treatment effect is infinitely close to the simulation effect in the digital space.

[0196] Specifically, by creating a high-fidelity virtual environment, multiple handling schemes are simulated, and the optimal strategy is selected autonomously based on quantitative indicators. This improves the accuracy, safety, and overall efficiency of modular handling unit actions, and solves the problem of uncertain handling effects of existing systems in complex scenarios.

[0197] Furthermore, the system constructs a collaborative detection network formed by multiple drones through a self-organizing network. By utilizing drone self-organizing network technology, multiple drones are dynamically grouped into a decentralized, self-healing communication network.

[0198] Each UAV flight platform participating in the swarm mission is equipped with a self-organizing network communication module that conforms to IEEE 802.11s or similar standards;

[0199] Once multiple drones take off, they will automatically detect nearby drone nodes and build a mesh network using dynamic routing protocols, such as optimized OLSR or AODV protocols.

[0200] This process is similar to the self-organizing network of drones studied in the research, whose topology can change dynamically as the drone moves;

[0201] This network structure does not rely on ground base stations or satellite communication, can work in areas without network coverage, and has strong resilience.

[0202] When some nodes fail or move, the network can automatically update routes to ensure smooth data links. Through optimized routing protocols, it can effectively reduce the total energy consumption of such networks and balance the node load, thus extending the battery life of the entire cluster.

[0203] One drone serves as the master node, responsible for integrating detection data from other drone nodes and coordinating the flight paths and sensing tasks of drones within the detection network, embodying a hybrid architecture that combines centralized and distributed approaches.

[0204] The master node is not pre-designated, but dynamically elected in the cluster based on factors such as the drone's remaining battery power, computing resources, communication link quality, and geographical location through a distributed negotiation algorithm. A similar architecture exists in intelligent drone cluster command and control systems, where the master node undertakes core functions such as intelligent perception, intelligent decision-making, and intelligent control.

[0205] All drones acting as child nodes transmit the raw data or pre-processed feature data collected by their multi-source sensor arrays to the master node in real time through the self-organizing network.

[0206] The airborne edge computing unit of the master node has stronger processing capabilities and uses a multi-source information fusion engine, such as cross-modal correlation fusion algorithm and intelligent recognition module, such as lightweight spatiotemporal graph neural network model, to perform spatiotemporal registration and deep fusion of all child node data.

[0207] This knowledge layer fusion method of swarm intelligence, similar to its application in the security of radioactive material transportation, can effectively improve the accuracy of global perception information and generate a unified and comprehensive regional nuclear, chemical, and biological threat situation map.

[0208] The master node coordinates the flight paths and sensing tasks of each UAV in the collaborative detection network. Based on the global threat situation and mission objectives, the master node dynamically plans the optimal flight path for each UAV in the sub-nodes, based on a distributed Markov coverage model or a similar multi-agent task allocation algorithm, to avoid detection blind spots and prevent collisions or repeated scanning between UAVs, thereby achieving collaborative coverage of the target area.

[0209] The master node will issue task instructions to specific child node drones through the self-organizing network based on the overall situation, and dynamically adjust their sensing tasks.

[0210] For example, instructing a drone to conduct a detailed investigation of a high-threat area, or instructing another drone to switch its sensor operating mode to verify a specific type of threat;

[0211] This dynamic task switching capability is also reflected in intelligent drone swarm command and control systems, where sub-drones can switch to different roles such as reconnaissance nodes and payload delivery units.

[0212] Specifically, multiple drones are connected into an intelligent organic whole through self-organizing network technology, and centralized management of data fusion and task coordination is achieved through the master node. This expands the physical detection range of the single-machine system, improves the accuracy and reliability of threat identification through collective intelligence, and endows the system with strong survivability and adaptability in complex and denial-of-control environments.

[0213] Further, please refer to Figure 2 A method for intelligent detection and identification of nuclear, chemical, and biological composites on unmanned aerial vehicles (UAVs) based on multi-source sensor fusion, applied to the aforementioned system, includes:

[0214] Synchronously acquiring nuclear, chemical, and biological environmental parameters through a multi-source sensor array is the first step in achieving composite detection. Specifically, in implementation:

[0215] A multi-source sensor array integrated into the UAV flight platform includes nuclear, chemical, and biological detection modules, which collect environmental data at the same time through hardware synchronous triggering signals.

[0216] The nuclear detection module collects gamma-ray dose rate and neutron count rate;

[0217] The chemical detection module acquires ion mobility spectrometry data;

[0218] The biodetection module collects laser-induced fluorescence spectra and particle counting information;

[0219] All data is tagged with a unified timestamp and spatial location label, originating from the drone's GPS / IMU, laying the foundation for subsequent spatiotemporal correlation analysis. This step solves the problem of misaligned time sequences and difficulty in correlation of data from different sensors in traditional detection.

[0220] In the airborne edge computing unit, multi-source heterogeneous data are fused in real time through a cross-modal correlation fusion algorithm;

[0221] A cross-modal correlation fusion algorithm running on an airborne edge computing unit processes the received multi-source heterogeneous data;

[0222] The algorithm first performs data standardization preprocessing to eliminate differences in the units and magnitudes of different sensors;

[0223] Semantic relationships between data are established through a trigger-verifier rule base. For example, when nuclear radiation readings are abnormal, the algorithm will automatically increase the priority of chemical agent detection, forming a verification task sequence.

[0224] The fusion process is completed in real time at the edge, avoiding delays in data backhaul and ensuring that the system completes the correlation analysis of multi-source information within seconds. This real-time fusion mechanism improves the system's response speed to sudden threats.

[0225] Based on the fused feature data, a lightweight spatiotemporal graph neural network model is used for inference to identify and output the type, location, and confidence level of nuclear, chemical, and biological threats, thus realizing the transformation from data to knowledge.

[0226] The lightweight spatiotemporal graph neural network model receives fused feature data as input;

[0227] The model constructs spatiotemporally discrete detection data into a graph structure, where nodes contain sensor readings and edges are defined by spatiotemporal relationships and the environmental wind field;

[0228] Through spatiotemporal graph convolution and gated recurrent units, the model simultaneously learns the spatial diffusion patterns and temporal evolution trends of threats;

[0229] Finally, the model outputs structured recognition results, including:

[0230] Threat types, such as single or combined threats like VX nerve agents, anthrax spores, or cesium-137 radioactive sources;

[0231] Location estimation, using the threat source coordinates determined through a backpropagation attention mechanism;

[0232] Confidence score, a reliability score based on the model's output probability, a value between 0 and 1;

[0233] This step involves transforming discrete detection readings into threat situation assessments with clear physical meaning;

[0234] Based on the identification results, response instructions are generated and executed. These instructions include controlling the drone to avoid pollution, issuing alarms, and locating the pollution source, thus completing a closed loop from perception to action.

[0235] Based on the threat type, level, and confidence level identified in the identification results, the system generates the optimal handling instructions using a pre-set decision rule base.

[0236] For low-level threats, the instruction may only include issuing an alarm and sending structured alarm information to the command center via data radio;

[0237] For high-level chemical or biological threats, the source of pollution may be located and drones may be controlled to circle at a safe distance to continuously monitor the spread of the pollution cloud;

[0238] In extreme cases, such as when a high concentration of radioactive material is detected, the drone will be immediately controlled to evade it and autonomously plan an escape route to avoid entering a more dangerous area.

[0239] When equipped with modular disposal units, disposal instructions may also include specific operations such as initiating the deployment of marker balls, spraying disinfectant, or collecting samples;

[0240] Specifically, by constructing a complete closed loop for responding to nuclear, chemical, and biological threats through four closely linked steps—synchronous data acquisition, real-time fusion, intelligent reasoning, and decision execution—this method achieves deep fusion and intelligent analysis of multi-source heterogeneous data, directly transforming the analysis results into actionable responses, thereby improving the efficiency and reliability of nuclear, chemical, and biological emergency response. Compared with existing technologies, this method has made progress in terms of the accuracy of threat identification, the timeliness of response, and the level of intelligence in handling.

[0241] Example 2

[0242] This embodiment describes an unmanned aerial vehicle (UAV) nuclear, chemical, and biological composite detection system based on multi-source sensor fusion, which includes a multi-source sensor array, an airborne edge computing unit, and a self-organizing network module, but does not carry modular disposal units such as marker ball dispensers or disinfectant spraying systems.

[0243] The self-organizing network communication module is equipped with each drone in the cluster. Each drone is equipped with a self-organizing network communication module that conforms to IEEE 802.11s or similar standards. The Mesh network is built through dynamic routing protocols, such as optimized OLSR or AODV protocols. This network structure does not rely on ground base stations and has strong resilience. When some nodes fail or move, the network can automatically update the routes to ensure the smooth flow of data links.

[0244] The airborne edge computing unit uses high-performance, low-power embedded AI computing modules such as the RK3588 to serve as the core of edge computing power for the drone swarm. This unit is responsible for running lightweight spatiotemporal graph neural network models and cross-modal correlation fusion algorithms.

[0245] Dynamic master node election: The master node is not pre-designated, but is dynamically elected in the cluster based on factors such as the drone's remaining battery power, computing resources, communication link quality, and geographical location, through a distributed negotiation algorithm.

[0246] The intelligent identification and processing method in this embodiment includes the following steps:

[0247] The swarm of drones receives unified synchronous data collection instructions through a self-organizing network over the target area. The multi-source sensor arrays of each drone collect environmental data synchronously according to a preset cycle. All data are packaged with a unified timestamp and their own spatial location information, laying the foundation for subsequent spatiotemporal correlation and fusion.

[0248] Data aggregation: All slave node drones transmit the collected raw data or pre-processed feature data to the dynamically elected master node in real time through an ad hoc network.

[0249] Fusion and identification: The airborne edge computing unit of the master node uses a multi-source information fusion engine to run a cross-modal correlation fusion algorithm and an intelligent identification module. It runs a lightweight spatiotemporal graph neural network model to perform spatiotemporal registration and deep fusion of all child node data, generate a unified regional nuclear, chemical and biological threat situation map, and complete the identification, location and classification of threats.

[0250] Once a threat is identified, the system generates handling instructions that do not involve physical handling units, but rather focus on information-level coordination and alerts. Specific instructions are as follows:

[0251] The master node distributes the threat source coordinates, type, confidence level, and estimated spread range to all slave nodes in the cluster through a self-organizing network;

[0252] The master node commands all slave node drones to form a ring formation to track and continuously monitor the identified pollution cloud; this collaborative strategy can effectively contain and monitor the target area.

[0253] The master node will send a complete threat situation map, early warning information and cluster status back to the command center via 5G or satellite link, providing real-time and intuitive on-site information for back-end decision-making.

[0254] Example 3

[0255] This embodiment specifically optimizes the intelligent recognition module within the airborne edge computing unit. This module is equipped with a lightweight spatiotemporal graph neural network model based on graph attention network (GAT), and improves the model's computational efficiency through channel pruning and 8-bit integer quantization techniques.

[0256] The hardware platform, the airborne edge computing unit adopts NVIDIA Jetson Orin series modules, which integrates an ARM-based CPU core and a GPU core equipped with Tensor Cores, supports INT8 quantization operations, and provides a hardware foundation for the efficient operation of the model;

[0257] In terms of software environment, deploying TensorRT or similar inference optimization SDKs on Jetson Orin can fully leverage the hardware acceleration potential by performing layer fusion, accuracy calibration, and kernel selection optimization on trained models.

[0258] This embodiment focuses on the specific implementation and optimization of a lightweight spatiotemporal graph neural network model, and its process is as follows:

[0259] The model structure is modified, and spatial convolution of the graph attention network is used as a substitute.

[0260] This scheme uses the Graph Attention Network (GAT) instead of the standard Graph Convolutional Network (GCN) as the spatial feature extractor. GAT calculates a dynamic, non-fixed attention weight for each node pair in the graph through an attention mechanism.

[0261] For node i and its neighbor node j, GAT first calculates the attention coefficient:

[0262]

[0263] Among them, h i and h j is the node feature, W is the shared linear transformation weight, a is a single-layer feedforward neural network, and || denotes vector concatenation;

[0264] The coefficients are then normalized using the softmax function to obtain the final attention weights α. ij This process can be represented as:

[0265]

[0266] Where, N i N is the set of neighbors of node i, and k is the loop variable used to iterate through the neighbor set. i For all nodes in the array, LeakyReLU is the activation function and exp is the exponential function;

[0267] Compared with the scheme in Example 1 that relies on predefined edge weights based on wind field data, GAT can dynamically learn the weights of relationships between nodes. In complex wind fields, such as environments without a dominant wind direction or with eddies, the model no longer mechanically relies on potentially inaccurate wind direction data. Instead, it adaptively captures more critical spatial dependencies through the learned attention weights, which may result in better performance.

[0268] Lightweighting measures include channel pruning and integer quantization;

[0269] First, an unpruned GAT model is trained on the server using complete data. After the training converges, a layer-by-layer channel pruning strategy is adopted.

[0270] For convolutional or linear layers, output channels that contribute less are identified and removed based on the L1 norm of their weight parameters. Typically, a pruning rate of 50% is set to reduce model parameters and computational cost.

[0271] This importance-based pruning method is intrinsically similar to the compression approach that preserves key information through gradient matching. The pruned model requires fine-tuning to restore performance.

[0272] The pruned and fine-tuned model is converted from FP32 format to INT8 precision. This process uses post-training quantization to calculate the distribution range of activation values ​​in each layer of the network using a representative calibration dataset, and determines the scaling factor from FP32 to INT8 accordingly.

[0273] Specifically, the model's environmental adaptability is enhanced. GAT's dynamic attention mechanism improves its adaptability to complex wind field environments, enabling the system to maintain robust threat spread prediction and source tracing performance even under variable weather conditions in the real world. By comprehensively utilizing pruning and quantization, the model is accelerated on airborne edge computing units, ensuring the real-time performance of nuclear, chemical, and biological threat identification and response.

[0274] Although alternative embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make further changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0275] The above specific embodiments further illustrate the purpose, technical solution and beneficial effects of this application. It should be understood that the above are only specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of this application should be included within the scope of protection of this invention.

Claims

1. A UAV-borne nuclear, chemical, and biological composite detection system based on multi-source sensor fusion, characterized in that, include: Unmanned aerial vehicle (UAV) flight platform; A multi-source sensor array, integrated on the UAV flight platform, is used to simultaneously collect raw data on nuclear, chemical, and biological threats; An airborne edge computing unit is communicatively connected to the multi-source sensor array and is used to process the raw data in real time; the airborne edge computing unit has a built-in multi-source information fusion engine and an intelligent recognition module. The multi-source information fusion engine is configured to run a cross-modal correlation fusion algorithm. The intelligent identification module is configured to identify and classify single and combined nuclear, chemical, and biological threats based on the output of the multi-source information fusion engine through a lightweight spatiotemporal graph neural network model.

2. The system according to claim 1, characterized in that, The multi-source sensor array includes a nuclear detection module for detecting gamma rays and neutrons, a chemical detection module employing a time-of-flight ion mobility spectrometer, and a biological detection module containing a laser-induced fluorescence radar and a particle collector.

3. The system according to claim 1, characterized in that, The cross-modal association fusion algorithm specifically includes: Trigger-verifier rule base, predefined data association logic for nuclear detectors, chemical detectors and biological detectors; The dynamic task orchestration engine automatically generates a verification task sequence based on the rule base when the data from any sensor exceeds a preset threshold and becomes a trigger. It also adjusts the working modes and sampling parameters of other related sensors in real time to perform collaborative verification.

4. The system according to claim 3, characterized in that, The execution results of the verification task sequence are weighted and fused with the confidence level of the initial trigger to finally output a comprehensive assessment of threat type, threat level, and overall credibility.

5. The system according to claim 1, characterized in that, The lightweight spatiotemporal graph neural network model processes data in the following ways: The detection data of multiple waypoints or spatial locations during a single flight are constructed into a spatiotemporal graph structure. The nodes in the spatiotemporal graph structure contain sensor readings, and the edges are defined by geographical location relationships, time series, and environmental wind field data. The model simultaneously learns the spatial diffusion patterns and temporal evolution trends of threats, and outputs estimates of the geographical location of the threat source, predictions of the diffusion direction, and classification results of the threat type.

6. The system according to claim 1, characterized in that, The system also includes a cloud-based federated learning platform that is communicatively connected to the airborne edge computing unit; The cloud-based federated learning platform is configured as follows: The model parameter updates, which have been desensitized, are aggregated from multiple drone systems to train and generate an enhanced global intelligent recognition model. Through a knowledge distillation process, the knowledge of the global intelligent recognition model is compressed into the lightweight spatiotemporal graph neural network model and then distributed to the intelligent recognition module of each UAV.

7. The system according to claim 1, characterized in that, The system also includes a modular processing unit that can be mounted on the UAV flight platform; The modular processing unit is communicatively connected to the airborne edge computing unit and receives instructions output by the intelligent identification module; The modular disposal unit is one of the following: a marker ball dispenser, a disinfectant spraying system, or a sampling container delivery device.

8. The system according to claim 7, characterized in that, The system also includes a digital twin simulation module, which, after receiving the recognition result from the intelligent recognition module, simulates the effect of the treatment plan in the digital space, and selects the optimal treatment plan based on the simulation result to instruct the modular treatment unit to execute it.

9. The system according to claim 1, characterized in that, The system constructs a collaborative detection network formed by multiple drones through a self-organizing network; One drone acts as the master node, responsible for integrating detection data from other drone nodes and coordinating the flight paths and sensing tasks of each drone within the collaborative detection network.

10. A method for intelligent identification and detection of nuclear, chemical, and biological composites on unmanned aerial vehicles based on multi-source sensor fusion, characterized in that, The method, applied to the system as described in any one of claims 1 to 9, comprises: Nuclear, chemical, and biological environmental parameters are collected synchronously through the multi-source sensor array. In the airborne edge computing unit, multi-source heterogeneous data are fused in real time using the cross-modal correlation fusion algorithm; Based on the fused feature data, the lightweight spatiotemporal graph neural network model is used for inference to identify and output the type, location, and confidence level of nuclear, chemical, and biological threats. Based on the identification results, disposal instructions are generated and executed, including controlling the drone to avoid pollution, issuing alarms, and locating the pollution source.

Images