Intelligent security and explosive ordnance disposal system based on internet of things

Abstract

This invention discloses an intelligent security and bomb disposal system based on the Internet of Things, relating to the field of public safety technology. The system includes a ubiquitous sensing layer, an edge computing layer, a cloud-based intelligent analysis platform, an unmanned execution layer, and a unified command terminal. The sensing layer collects multi-dimensional signals through various heterogeneous sensors. The edge computing layer performs local AI processing and feature fusion to generate structured warnings. The cloud platform performs multi-modal AI deep fusion analysis on the warnings, quantifies the threat level, and automatically generates a response plan containing the optimal robot, path, and atomic operation sequence based on the real-time situation of a digital twin through an intelligent decision engine. After authorization by the command terminal, the plan is automatically executed by inspection or bomb disposal robots for reconnaissance and response. This achieves a closed-loop process from intelligent sensing and analysis decision-making to unmanned execution, improving the response speed, accuracy, and safety of security and bomb disposal, and enabling continuous system learning through federated learning and post-evaluation.

Description

Technical Field

[0001] This invention relates to the field of public safety and emergency response technology, and more specifically, to an intelligent security and bomb disposal system based on the Internet of Things. Background Technology

[0002] In densely populated public places such as large transportation hubs, cultural and sports venues, and city squares, preventing and handling explosive threats is of paramount importance in security work. Traditional security models mainly rely on fixed video surveillance and manual patrols, which face the following bottlenecks: Limited perception capabilities; video surveillance struggles to identify non-visual threats, such as chemical explosives and radioactive materials, and has limited detection capabilities for obscured or disguised items, resulting in blind spots; Delayed response and decision-making; anomaly detection relies on manual monitoring, leading to long chains of warning confirmation, information reporting, and decision-making, which cannot meet emergency response requirements; Extremely high handling risks; bomb disposal operations require close-range operation by specialized personnel, posing a direct threat to personal safety, causing significant psychological stress, and being limited by individual experience; Poor system coordination; various security subsystems, such as video, access control, and alarm monitoring equipment, typically operate independently, resulting in fragmented information and making it difficult to form an integrated closed loop from early warning, analysis, decision-making, to handling; Low level of intelligence; lack of deep analysis and fusion capabilities for massive amounts of multi-source data, hindering accurate threat assessment and scientific contingency plan generation. In recent years, the Internet of Things (IoT), edge computing, AI, and robotics technologies have developed rapidly. However, existing technologies mostly simply combine these technologies, and have not yet formed a systematic solution that can seamlessly connect front-end multi-dimensional perception, near-end real-time intelligence, cloud-based deep decision-making, and terminal unmanned execution, and achieve continuous evolution through the bidirectional flow of data and models. Therefore, there is an urgent need for an integrated intelligent security and bomb disposal system that can achieve all-time early warning, intelligent decision-making, and unmanned handling. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide an intelligent security and explosive ordnance disposal system based on the Internet of Things. This system, by constructing a cloud-edge-device collaborative architecture, achieves full-process automation and intelligence from multi-dimensional perception, intelligent analysis, dynamic decision-making to unmanned execution, thereby improving the ability to detect, accurately assess, respond quickly, and safely handle explosive threats.

[0004] The above-mentioned technical objective of this invention is achieved through the following technical solution: an intelligent security and explosive ordnance disposal system based on the Internet of Things, comprising: The ubiquitous sensing layer consists of several heterogeneous IoT sensing nodes deployed within the protected area, used to collect visible light video, non-visible material components, penetrating imaging signals, and environmental disturbance signals within the protected area. The edge computing layer, consisting of edge computing gateways distributed in each logical partition of the protected area, is connected to the ubiquitous sensing layer and is used to receive and process the raw data of the sensing nodes. The cloud-based intelligent analysis platform communicates with the edge computing layer to receive and aggregate structured warning events and associated raw data samples from different edge computing gateways. It then calls a multimodal AI fusion analysis model to perform decision-level fusion analysis on the aggregated multi-dimensional data across the entire domain. This enables the identification of suspicious targets, judgment of behavioral patterns, and quantitative assessment of threat levels. Based on the threat level assessment results and the real-time situation of the built-in digital twin, the platform automatically generates a contingency plan that includes the optimal execution unit, dynamic path planning, and atomic operation sequences through an intelligent decision engine. The unmanned execution layer includes at least one security patrol robot and at least one bomb disposal robot, which are respectively connected to the cloud-based intelligent analysis platform. They are used to receive precise control instructions transformed from the disposal plan and execute the entire process from on-site reconnaissance and target confirmation to final disposal. At the same time, they transmit their own status, operation data and environmental perception data back in real time. The unified command terminal interacts with the cloud-based intelligent analysis platform to visualize the overall digital twin situation, the decision-making basis and execution details of the emergency response plan, and provides a human-machine interface for commanders to review, modify, authorize and issue emergency interventions for the plan. The edge computing layer and the cloud-based intelligent analysis platform form a collaborative processing architecture. The edge computing layer performs low-latency, high-concurrency initial perception and filtering, while the cloud-based intelligent analysis platform performs high-complexity, global in-depth analysis and decision-making. The cloud-based intelligent analysis platform and the edge computing layer are linked through bidirectional data streams of structured warning events and model parameters.

[0005] Optionally, the heterogeneous IoT sensor nodes in the ubiquitous sensing layer specifically include: a high-definition smart camera for acquiring video streams and having a built-in moving target detection and tracking algorithm; a trace gas sensor for real-time adsorption and analysis of air samples to identify the volatile organic compound components characteristic of explosives; a millimeter-wave radar for transmitting and receiving millimeter-wave signals to generate radar images reflecting the outline, material, and internal structure of objects; and an array of acoustic and vibration sensors for acquiring abnormal acoustic signals and structural vibration signals in the environment and locating disturbance sources through direction-of-arrival estimation. All sensor nodes synchronize their clocks through a unified network time protocol to ensure that the timestamps of the original data are consistent.

[0006] Optionally, the processing of the edge computing layer specifically includes: performing real-time human pose estimation, package-related event detection, and abnormal running and gathering behavior recognition on the video stream from the high-definition smart camera using a loaded lightweight convolutional neural network model; performing spatiotemporal alignment of radar image data from millimeter-wave radar with video frames from the high-definition smart camera, and enhancing the detection rate of suspicious items under occlusion or disguise through a feature-level fusion algorithm; and triggering the edge warning generation module when the local inference result or fusion analysis result exceeds a preset confidence threshold, packaging the time, location, sensor type, confidence level, and key evidence snapshots to form the structured warning event.

[0007] Optionally, the multimodal AI fusion analysis model in the cloud-based intelligent analysis platform adopts a Transformer-based multi-head attention mechanism fusion network. The specific steps for performing decision-level fusion analysis are as follows: video features, radar point cloud features, gas concentration time series features, and acoustic vibration spectrum features reported from different edge gateways that are related to the same spatiotemporal event are mapped to a high-dimensional embedding space; the relevance weights of different modal features in determining the current threat are dynamically calculated through the multi-head attention mechanism; the weighted fusion global feature vector is input into the downstream multi-task learning network, and target classification, behavior classification, and comprehensive threat level score are output in parallel, thereby fitting and generating a threat level assessment result.

[0008] Optionally, the logic for the intelligent decision-making engine to generate the response plan includes: Contingency plan triggering: When the comprehensive threat level score exceeds the first threshold, the contingency plan generation process is automatically triggered; Resource scheduling: Query the status, location and capabilities of all available execution units in the digital twin, and calculate and select the optimal security patrol robot to perform the first reconnaissance with multiple objectives such as shortest path, least time and best concealment; Dynamic programming: Based on the real-time 3D map and dynamic obstacle information in the digital twin, plan a collision-free optimal path for the selected robot from its current position to the target position; Task Serialization: If the threat level rises to the second threshold after reconnaissance and verification, an atomic operation sequence is generated for the bomb disposal robot. The sequence includes branch instructions for autonomous navigation to precise coordinates, robotic arm deployment, activation of X-ray fluoroscopic scanning, and selection of robotic arm grabbing and transferring to the explosion-proof container or activation of water cannon for in-situ destruction based on the scan results.

[0009] Optionally, the bomb disposal robot includes: a modular load interface that allows the robot's end effector to quickly switch between different functional modules according to instructions; The functional modules include at least: a high-resolution binocular vision camera for close-range 3D modeling and visual servo control; a miniature X-ray backscatter imager for non-destructive internal imaging of suspicious items; and a high-pressure liquid water jet cannon for launching precise water bullets for cutting or detonation. The robot's controller is embedded with a local path planning and arm-hand coordination control algorithm based on reinforcement learning training, enabling it to semi-autonomously complete the atomic operation sequence.

[0010] Optionally, a federated learning framework and model update mechanism are also provided: the cloud-based intelligent analysis platform acts as the central server for federated learning, periodically distributing initialized global AI models to each edge computing gateway; each edge computing gateway uses locally generated de-identified data to train the model, and uploads the trained model parameters incrementally with encryption; the cloud-based intelligent analysis platform aggregates all uploaded parameters, updates the global model, and redistributes the optimized model to each edge computing gateway in the edge computing layer.

[0011] Optionally, it also includes a post-disposal effect evaluation and knowledge base construction module, which is used to automatically extract the entire chain of data from initial perception to final disposal after a complete disposal task closure, and generate structured cases; based on preset key performance indicators, automatically evaluate the system's response time, recognition accuracy, and disposal success rate; and use the structured cases as enhanced samples, label them and store them in a dedicated knowledge base for subsequent supervised incremental training of the multimodal AI fusion analysis model.

[0012] In summary, the present invention has the following beneficial effects: 1. At the perception and early warning level, this invention achieves an improvement from single-vision to full-dimensional perspective, solving the problems of perception blind spots and missed detections. Traditional systems rely on video surveillance, which cannot effectively detect non-visual threats. This invention constructs a three-dimensional perception network by integrating multi-modal sensors such as visible light, trace chemical sniffing, millimeter-wave penetrating imaging, and acoustic vibration monitoring. Furthermore, chemical and radioactive sensing enables the system to provide early warnings through trace volatile components or radiation signals in the air before explosives are visible, advancing the threat detection time. Penetrating imaging sensing can effectively identify the internal structure of suspicious items that are wrapped, obscured, or disguised, overcoming the physical limitations of video surveillance and reducing missed detections due to visual deception. Acoustic vibration sensing can capture vibrations or sound waves caused by abnormal placement, knocking, etc., complementing visual behavior analysis and enhancing the ability to identify intentional behavior. This synchronous acquisition and timestamp alignment of multi-source heterogeneous data lays a solid foundation for high-reliability fusion analysis in the backend, improving the system's probability of detecting complex and concealed threats.

[0013] 2. At the analysis and response level, the system accelerates the process from layer-by-layer reporting to second-level closed-loop processing, resolving issues of delayed response and reliance on experience in decision-making. In the traditional model, the process from anomaly detection to issuing handling instructions is lengthy. Through a collaborative architecture of real-time edge processing and in-depth cloud analysis, the response chain has been restructured: the edge computing layer, as the system front-end, performs AI inference near the data generation point, such as behavior recognition, object detection, and local multi-sensor feature fusion, quickly generating high-quality structured warning events; it filters massive amounts of raw data, uploading only key information, reducing bandwidth pressure and shortening the initial warning time; the cloud-based intelligent analysis platform, as the system's decision-making center, relies on powerful computing capabilities to perform decision-level fusion analysis of aggregated multimodal information from across the domain. Employing an attention-based AI model, it intelligently weighs the weight of different pieces of evidence, outputting an objective and quantifiable threat level score, replacing subjective and vague human experience-based judgment; after threat confirmation, the intelligent decision engine, based on a real-time digital twin including maps, pedestrian flow, and resource status, automatically generates a handling plan containing the optimal robot, optimal path, and standard operating procedures (SOPs). This process compresses the traditional decision-making process, which requires multi-party consultation and manual planning, into a process that can be completed in seconds, achieving an automated, intelligent, and rapid closed loop from perception to contingency plan. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the system operation process of the present invention. Detailed Implementation

[0015] To make the objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein.

[0016] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0017] In this invention, unless otherwise expressly specified and limited, "above" or "below" a second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of a second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" of a second feature includes the first feature being directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature. The terms "vertical," "horizontal," "left," "right," "above," "below," and similar expressions are for illustrative purposes only and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0018] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] This invention provides an intelligent security and explosive ordnance disposal system based on the Internet of Things, such as... Figure 1 As shown, it includes: The ubiquitous sensing layer consists of several heterogeneous IoT sensing nodes deployed within the protected area, used to collect visible light video, non-visible material components, penetrating imaging signals, and environmental disturbance signals within the protected area. The edge computing layer, consisting of edge computing gateways distributed in each logical partition of the protected area, is connected to the ubiquitous sensing layer and is used to receive and process the raw data of the sensing nodes. The cloud-based intelligent analysis platform communicates with the edge computing layer to receive and aggregate structured warning events and associated raw data samples from different edge computing gateways. It then calls a multimodal AI fusion analysis model to perform decision-level fusion analysis on the aggregated multi-dimensional data across the entire domain. This enables the identification of suspicious targets, judgment of behavioral patterns, and quantitative assessment of threat levels. Based on the threat level assessment results and the real-time situation of the built-in digital twin, the platform automatically generates a contingency plan that includes the optimal execution unit, dynamic path planning, and atomic operation sequences through an intelligent decision engine. The unmanned execution layer includes at least one security patrol robot and at least one bomb disposal robot, which are respectively connected to the cloud-based intelligent analysis platform. They are used to receive precise control instructions transformed from the disposal plan and execute the entire process from on-site reconnaissance and target confirmation to final disposal. At the same time, they transmit their own status, operation data and environmental perception data back in real time. The unified command terminal interacts with the cloud-based intelligent analysis platform to visualize the overall digital twin situation, the decision-making basis and execution details of the emergency response plan, and provides a human-machine interface for commanders to review, modify, authorize and issue emergency interventions for the plan. The edge computing layer and the cloud-based intelligent analysis platform form a collaborative processing architecture. The edge computing layer performs low-latency, high-concurrency initial perception and filtering, while the cloud-based intelligent analysis platform performs high-complexity, global in-depth analysis and decision-making. The cloud-based intelligent analysis platform and the edge computing layer are linked through bidirectional data streams of structured warning events and model parameters.

[0020] In a specific embodiment, the system is deployed by deploying several heterogeneous sensing nodes of the ubiquitous sensing layer and gateways of the edge computing layer within the physical space of the protected area; deploying a cloud-based intelligent analysis platform in the central computer room; deploying robots of the unmanned execution layer within the area; and configuring a unified command terminal in the command center. This leads to the establishment of a two-way communication link from the perception layer, edge layer, cloud platform, execution layer, and command terminal; the edge computing layer and the cloud intelligent analysis platform form a collaborative processing architecture, and agree on the data format and protocol for structured warning events and model parameters.

[0021] During system operation, namely routine monitoring and event response, the positive data flow is the process of perception and decision-making. The specific implementation process is as follows: Data acquisition and edge preprocessing: Each sensing node in the ubiquitous perception layer continuously acquires multi-dimensional raw signal data, and the raw data is sent to the corresponding edge computing gateway. Edge intelligence processing and warning generation: The edge computing gateway performs low-latency, high-concurrency preliminary perception and filtering processing on raw data, such as video analysis and signal processing; when a preset abnormal pattern is detected, it generates a structured warning event with labels such as time, location, type, and confidence level. Cloud-based aggregation and in-depth analysis: Structured warning events and related raw data samples are uploaded to the cloud-based intelligent analysis platform; the platform aggregates events from different edge gateways, calls multimodal AI models to perform highly complex, global in-depth analysis and decision-level fusion, and outputs target classification, behavior classification and threat level quantitative assessment results; Intelligent decision-making and contingency plan generation: Based on threat level assessment results and real-time situation in the digital twin, such as resource status and environmental map, the intelligent decision engine automatically generates contingency plans that include optimal execution units, dynamic path planning and atomic operation sequences. Human-machine collaboration and command issuance: The emergency response plan and its decision-making basis are pushed to the unified command terminal for visualization. After being reviewed, modified and authorized by the command personnel, the plan is transformed into precise control commands. Unmanned execution: Precise control commands are issued to designated robots in the unmanned execution layer, such as security inspection robots or bomb disposal robots; the robots perform tasks from on-site reconnaissance to final disposal.

[0022] In another data workflow, namely the execution-to-feedback process, the robot transmits its own status, work data, and environmental perception data back to the cloud platform and command terminal in real time during execution; the cloud platform updates the robot's real-time status and task progress in the digital twin.

[0023] The data flow for cloud-edge collaborative learning involves the cloud platform periodically sending updated AI model parameters to the edge computing gateway; the edge computing gateway uses local data for training and uploads encrypted model parameters incrementally to the cloud for aggregation and updates, thereby optimizing the model.

[0024] Furthermore, the heterogeneous IoT sensor nodes in the ubiquitous sensing layer specifically include: a high-definition smart camera for acquiring video streams and having a built-in moving target detection and tracking algorithm; a trace gas sensor for real-time adsorption and analysis of air samples to identify the volatile organic compound components characteristic of explosives; a millimeter-wave radar for transmitting and receiving millimeter-wave signals to generate radar images reflecting the outline, material, and internal structure of objects; and an array of acoustic and vibration sensors for acquiring abnormal acoustic signals and structural vibration signals in the environment and locating disturbance sources through direction-of-arrival estimation. All sensor nodes synchronize their clocks through a unified network time protocol to ensure that the timestamps of the original data are consistent.

[0025] In a specific embodiment, the data acquisition process is as follows: Visible light video stream: A high-definition smart camera acquires continuous video frames and calls the built-in moving target detection and tracking algorithm to output metadata with target bounding boxes and trajectories; Non-visible material composition stream: A trace gas sensor periodically adsorbs air samples and outputs time-series data of the concentration of specific chemical components through internal spectral analysis; Penetrating imaging signal stream: A millimeter-wave radar actively transmits signals and receives echoes, which are processed to generate radar image data reflecting the target outline and material contrast; Environmental disturbance signal stream: An array of acoustic and vibration sensors acquires raw waveform data, and through array signal processing, such as direction of arrival estimation, outputs the type, intensity, and azimuth data of abnormal events; During clock synchronization, the clock server in the network periodically sends Network Time Protocol (NTP) synchronization messages; all sensor nodes and edge gateways receive these messages and calibrate their local clocks; thus, each piece of raw data collected is timestamped with a timestamp provided by this unified clock source when it is generated.

[0026] Furthermore, the processing of the edge computing layer specifically includes: performing real-time human pose estimation, package-related event detection, and abnormal running and gathering behavior recognition on the video stream from the high-definition smart camera using a loaded lightweight convolutional neural network model; performing spatiotemporal alignment of radar image data from millimeter-wave radar with video frames from the high-definition smart camera, and enhancing the detection rate of suspicious items under occlusion or disguise through a feature-level fusion algorithm; and triggering the edge warning generation module when the local inference result or fusion analysis result exceeds a preset confidence threshold, packaging the time, location, sensor type, confidence level, and key evidence snapshots to form the structured warning event.

[0027] In a specific embodiment, the edge computing layer performs edge computing and feature fusion, and the edge computing gateway receives video streams from high-definition smart cameras; the video stream is input into a lightweight convolutional neural network model, the model performs real-time inference on the video stream, outputs key points of human skeleton, and then identifies human pose estimation, outputs bounding boxes of abandoned items, i.e. package abandoned event detection, and specific action classification, including but not limited to abnormal running, gathering behavior recognition, etc. The system uses a multi-source feature-level fusion data stream, where the gateway simultaneously receives radar image data from millimeter-wave radar. Based on a unified timestamp and sensor spatial calibration, the system spatiotemporally aligns video frames and radar images at the same time. The aligned video features (such as texture and shape) and radar features (such as material dielectric properties and internal structure) are input into a feature-level fusion algorithm, preferably an early fusion based on convolutional neural networks or a mid-term fusion based on feature stitching. The fusion algorithm then outputs an enhanced feature vector that integrates visual and penetration information for target discrimination, thereby enhancing the detection rate of suspicious items under occlusion or camouflage. When the confidence level of the aforementioned local inference results or fusion analysis results exceeds the preset confidence threshold, the edge warning generation module is triggered. This module packages the current time, location, types of sensors involved, event confidence level, and key evidence snapshots (such as video frames or radar charts that trigger the warning) into a structured warning event data packet with a uniform format. This data packet is uploaded to the cloud-based intelligent analysis platform through the communication interface.

[0028] Furthermore, the multimodal AI fusion analysis model in the cloud-based intelligent analysis platform adopts a Transformer-based multi-head attention mechanism fusion network. The specific steps for performing decision-level fusion analysis are as follows: video features, radar point cloud features, gas concentration time-series features, and acoustic vibration spectrum features reported from different edge gateways that are related to the same spatiotemporal event are mapped to a high-dimensional embedding space; through the multi-head attention mechanism, the relevance weights of different modal features in determining the current threat are dynamically calculated; the weighted fusion global feature vector is input to the downstream multi-task learning network, and the target classification, behavior classification, and comprehensive threat level score are output in parallel, thereby fitting and generating a threat level assessment result.

[0029] In a specific embodiment, the cloud platform receives multimodal data associated with the same spatiotemporal event from different edge gateways, including: video features, radar point cloud features, gas concentration time series features, and acoustic vibration spectrum features; the data of each modality is processed by an independent encoder on the cloud platform, such as CNN, PointNet, or LSTM, and mapped to a high-dimensional embedding space to form a set of feature vectors; The corresponding feature vector group is input into a Transformer-based multi-head attention mechanism fusion network; the attention module in the network calculates the correlation of each feature vector with respect to other vectors. Through the multi-head attention mechanism, the correlation weight of different modal features in determining the current threat is dynamically calculated. For example, when dealing with an occluded package, the network may assign higher weights to radar features. During the decision-making process, The weighted and fused global feature vector is fed into a downstream multi-task learning network. This network performs multiple classification and regression tasks in parallel: one output head classifies targets, such as explosives / flammable materials / everyday items; another output head classifies behaviors, such as placement / discarding / hiding / non-threatening behaviors; and a third output head performs regression to calculate a comprehensive threat level score, such as a continuous value between 0 and 1. These outputs together constitute the system's threat level assessment result, providing a quantitative basis for subsequent decision-making.

[0030] Furthermore, the logic by which the intelligent decision-making engine generates the contingency plan includes: Contingency plan triggering: When the comprehensive threat level score exceeds the first threshold, the contingency plan generation process is automatically triggered; Resource scheduling: Query the status, location and capabilities of all available execution units in the digital twin, and calculate and select the optimal security patrol robot to perform the first reconnaissance with multiple objectives such as shortest path, least time and best concealment; Dynamic programming: Based on the real-time 3D map and dynamic obstacle information in the digital twin, plan a collision-free optimal path for the selected robot from its current position to the target position; Task Serialization: If the threat level rises to the second threshold after reconnaissance and verification, an atomic operation sequence is generated for the bomb disposal robot. The sequence includes branch instructions for autonomous navigation to precise coordinates, robotic arm deployment, activation of X-ray fluoroscopic scanning, and selection of robotic arm grabbing and transferring to the explosion-proof container or activation of water cannon for in-situ destruction based on the scan results.

[0031] In a specific embodiment, the intelligent decision engine continuously receives the comprehensive threat level score output by the cloud analysis platform; when the score exceeds a first threshold, such as 0.7, the engine automatically triggers the contingency plan generation process. Resource scheduling process: The engine sends a query request to the digital twin module to obtain the real-time status (idle / busy), location coordinates, capabilities (reconnaissance / bomb disposal), and power information of all available execution units (robots); the engine performs calculations using the shortest path, least time, and best concealment as multi-objective optimization functions; the calculation result designates the optimal security patrol robot as the first reconnaissance unit; Dynamic programming process: The engine acquires real-time 3D map and dynamic obstacle information in the digital twin, such as simulated pedestrian flow; the path planning algorithm, preferably such as A or DLite, calculates the collision-free optimal path from the current position to the target position for the selected robot based on the above map and obstacle information, and the path information is encoded into a series of waypoints; Task serialization process: The system waits for the verification results of the first reconnaissance robot; if the threat level updated by the system rises to the second threshold after verification, for example, 0.9, the engine starts to generate an atomic operation sequence for the bomb disposal robot; the sequence generator assembles the atomic instructions in sequence, which includes branch instructions for selecting the robotic arm to grab and transfer to the explosion-proof container or to start the water cannon to carry out in-situ destruction based on the scan results.

[0032] Furthermore, the bomb disposal robot includes a modular load interface, which allows the robot's robotic arm end effector to quickly switch between different functional modules according to instructions; The functional modules include at least: a high-resolution binocular vision camera for close-range 3D modeling and visual servo control; a miniature X-ray backscatter imager for non-destructive internal imaging of suspicious items; and a high-pressure liquid water jet cannon for launching precise water bullets for cutting or detonation. The robot's controller is embedded with a local path planning and arm-hand coordination control algorithm based on reinforcement learning training, enabling it to semi-autonomously complete the atomic operation sequence.

[0033] In a specific embodiment, the controller of the bomb disposal robot receives atomic operation instructions from the cloud. The controller sends electrical control signals to the modular payload interface through the control bus. The interface drives the quick-change device at the end of the robotic arm to unload the current tool, such as a binocular camera, and load the specified functional module, such as a miniature X-ray backscatter imager. After loading is completed, the controller initializes the new module and reports the ready status. Corresponding to the servo and operation process of each functional module, when a high-resolution binocular vision camera is installed, the camera acquires images and generates a close-range 3D point cloud model of the target object through a stereo vision algorithm, which is used for the precise positioning of the robotic arm by visual servo control; when an X-ray imager is installed, the controller triggers a scanning command, the imager performs non-destructive internal imaging of the suspicious item, and the image data is transmitted back; when a high-pressure liquid water jet cannon is installed, the controller calculates the cannon attitude based on the coordinates provided by the visual servo and triggers the pressure system to launch precise water bullets for cutting or detonation; During movement or manipulation, the robot's controller incorporates a local path planning algorithm based on reinforcement learning, which processes LiDAR and depth camera data in real time to avoid dynamic obstacles not marked on the global map. Simultaneously, an arm-hand cooperative control algorithm calculates the motion trajectories of each joint of the robotic arm and the end effector, ensuring smoothness and precision when performing complex actions such as grasping and aiming. These algorithms enable the robot to semi-autonomously complete the aforementioned atomic operation sequence.

[0034] Furthermore, a federated learning framework and model update mechanism are also set up: the cloud-based intelligent analysis platform, as the central server of federated learning, periodically distributes the initialized global AI model to each edge computing gateway; each edge computing gateway uses locally generated de-identified data to train the model and uploads the trained model parameters incrementally with encryption; the cloud-based intelligent analysis platform aggregates all uploaded parameters, updates the global model, and redistributes the optimized model to each edge computing gateway in the edge computing layer.

[0035] In a specific embodiment, the cloud-based intelligent analysis platform, acting as the central server for federated learning, distributes the initialized global AI model, such as the model weight file, to each edge computing gateway after encrypting it at the start of the training cycle. Each edge computing gateway uses locally generated de-identified data, such as video clips with blurred faces and sensor data with location labels removed, to train the distributed model locally. After training, the gateway calculates the parameter increment of the local model, which is the difference between the locally updated weights and the initial weights. This parameter increment is encrypted. Each edge gateway uploads encrypted incremental training model parameters to the cloud platform. The cloud platform uses a secure aggregation algorithm to decrypt and aggregate all uploaded parameters. Using the aggregated parameter increments, the cloud platform updates the global model, generating a new version of the model. The new version of the global model is re-encrypted and distributed to each edge computing gateway in the edge computing layer, completing one federated learning iteration. Furthermore, it also includes a post-disposal effect evaluation and knowledge base construction module, which is used to automatically extract the entire chain of data from initial perception to final disposal after a complete disposal task loop, and generate structured cases; based on preset key performance indicators, automatically evaluate the system's response time, recognition accuracy, and disposal success rate; and use the structured cases as enhancement samples, label them and store them in a dedicated knowledge base for subsequent supervised incremental training of the multimodal AI fusion analysis model.

[0036] In a specific embodiment, once a task is marked as completed, the post-processing effect evaluation and knowledge base construction module is triggered. The module automatically extracts the entire chain of data from initial perception to final processing from the system log database, including original warnings, AI intermediate results, decision logs, robot control command sequences, and transmitted videos. The data is cleaned, correlated, and reorganized to generate a structured case file. The evaluation module reads preset key performance indicators, such as response time, identification accuracy, and handling success rate; the module calculates the actual values ​​of these indicators from the case files, and the evaluation results are used to generate an evaluation report; Case files and evaluation reports are submitted to an automated or semi-automated labeling process to add accurate labels, such as the final confirmed item type and the correctness of the disposal method. The labeled structured cases are stored in a dedicated knowledge base. When the system retrains the model in the future, the cases in the knowledge base are sampled as augmentation samples and added to the training dataset for supervised incremental training of multimodal AI fusion analysis models, thereby improving the model's performance in similar events in the future.

[0037] In the specific implementation process, a large-scale integrated transportation hub (such as a high-speed rail station) is used as an example to illustrate how the system operates in a real environment; During the deployment phase, the ubiquitous perception layer is deployed as follows: The following nodes are deployed in key areas of the high-speed rail station, including the central waiting hall, security checkpoints, ticket halls, commercial area passages, and luggage storage areas: High-definition smart camera: Installed above the ceiling, it covers panoramic views of various areas and details of key parts. The built-in chip supports the running of lightweight deep learning algorithms to achieve moving target tracking. Trace gas sensors: Concealed deployment at key airflow points such as vents, next to trash cans, and under seats, with the sampling port facing public areas; Millimeter-wave radar: Deployed on pillars or walls, it actively scans areas where video is easily obstructed, such as behind security inspection machines, next to large green plants, and under seats; Acoustic and vibration sensor array: integrated into smart lampposts or decorative columns at key locations to form a distributed monitoring network; all sensor nodes are connected to the station's industrial-grade IoT gateway and receive a unified NTP clock synchronization signal through this network to ensure that the timestamp error of all data is less than the preset error value; Edge computing layer deployment: Two high-performance edge computing servers are deployed in the weak current rooms on the east and west sides of the station respectively; the server on the east side is responsible for connecting to the sensors in the east area of ​​the waiting hall and the ticket hall; the server on the west side is responsible for connecting to the sensors in the west area of ​​the waiting hall, the commercial area and some passages. Deployment of cloud-based intelligent analysis platform: The platform software is deployed on a private cloud server owned by the station or built by the transportation hub itself; Deployment of unmanned execution layers: Security patrol robots conduct autonomous patrols on the east and west sides of the waiting hall daily, equipped with automatic charging capabilities and capable of rapid response during missions; Bomb disposal robots are stored in a dedicated fire / security equipment room in the station, the location of which has been entered into a digital twin map. The robots have modular payload interfaces, and their toolboxes contain three end effectors: a binocular vision camera, an X-ray backscatter imager, and a water cannon, which can be automatically swapped. Unified command terminal deployment: Located in the station's integrated command center, it consists of a main control server driving multiple spliced ​​large screens and multiple commander seats equipped with dedicated client software.

[0038] In the actual operation of the system, the scenario was set in the west area of ​​the waiting hall. A passenger left a black backpack under the seat and walked away quickly. The backpack actually contained an improvised explosive device. Phase 1 of system operation: Sensing and edge early warning: Video Trigger: A high-definition PTZ camera deployed in the area captures the abandoned behavior. The video stream is transmitted in real time to edge computing gateway A in the weak current room of the west zone; Edge AI Analysis: A lightweight CNN model in gateway A analyzes the video stream, identifies the abandoned package event, and judges it to be a deliberate placement pattern based on the behavioral trajectory, with an initial confidence level of 0.75; Multi-source Signal Trigger: A trace gas sensor about 3 meters away from the backpack detects an abnormal increase in the concentration of the TATP characteristic peak in the air; The edge computing module of this sensor uploads the concentration time series data and alarm signal to the same gateway A; Edge Fusion and Warning Generation: After receiving the video and gas alarm, gateway A starts a feature-level fusion algorithm. The algorithm determines that the two are highly correlated in time and space (consistent timestamps, adjacent locations), increasing the overall confidence level of the event to 0.88; Structured Warning Generation: Gateway A immediately packages and generates a structured warning event and transmits it to the cloud analysis platform; Phase Two of System Operation: Cloud-based In-depth Analysis and Decision-making: Upon receiving the warning, the cloud platform's data aggregation module immediately retrieves historical data from all sensors within a 10-meter radius over the past 5 minutes, focusing on the event. The system identifies that a millimeter-wave radar in the same area scanned the backpack 30 seconds before the warning, revealing an internal structure of irregular metal lines and blocky objects. The platform initiates a multimodal AI fusion analysis model. Input data includes: video behavioral characteristics from the warning, structural characteristics of the radar cache, and temporal characteristics of gas concentration. Using a Transformer-based multi-head attention mechanism, the model determines that gas characteristics (chemical threat) and radar characteristics (physical structural threat) are the most critical in this event, assigning them high weights (approximately 0.4 each). The model outputs in parallel: target classification as explosive (96% probability); behavior classification as malicious placement (91% probability); overall threat level score of 0.94. The intelligent decision-making engine is activated. The threat level score is higher than the preset first threshold. Resource scheduling: The engine accesses the digital twin and queries the data. The results show that security patrol robot A is located in the west area, 45 meters from the incident point, with 85% battery and is idle. Bomb disposal robot A is in the equipment room and is ready. The engine selects security patrol robot A to perform reconnaissance. The digital twin shows that the current passenger density in the west area is moderate. The engine calls the path planning algorithm to plan a collision-free optimal path for security patrol robot A, allowing it to travel along the wall and avoid the main passenger flow passage. The estimated arrival time is 35 seconds. The engine generates a preliminary instruction set, which is as follows: If the reconnaissance confirms the threat level, the bomb disposal robot A is instructed to execute the following sequence: Navigate to coordinates (102.5, 45.3) -> Deploy the robotic arm and switch to X-ray -> Scan -> If it is confirmed to be an IED, switch to water cannon to perform a 5-second burst of destruction. Command terminal decision support: All the above information—including the 3D situation map (highlighting the target, robot position, and planned path), threat analysis report (listing evidence and weights for each modality), and robot scheduling plan—is displayed in real time on the command center's large screen. After reviewing the information, the commander deems the AI's assessment sufficient and authorizes reconnaissance. The command is issued, and security patrol robot A automatically travels along the planned path to the target seat, adjusts the gimbal to capture images from multiple angles, and transmits high-definition images back. The image clearly shows an unclaimed black backpack. The commander confirms the threat on the terminal, and the system officially sets the threat level at 0.96, exceeding the preset second threshold. The engine compiles the prepared atomic operation sequence into robot instructions and issues them to bomb disposal robot A. Bomb disposal robot A automatically drives out of the equipment room, autonomously navigates to the predetermined work point 2 meters away from the backpack, deploys its robotic arm, and automatically removes the default gripper through the modular load interface at the end, loading a miniature X-ray backscatter imager to scan the backpack. The scanned images were transmitted back in real time, showing a circuit board, battery, and powdery material inside, confirming it as an IED (Initial Explosive Detonator). According to the pre-planned procedure, the robotic arm automatically switched to a high-pressure liquid water jet cannon. Under the control of the arm-hand collaborative control algorithm, the cannon was precisely aimed at the IED's detonation device. The commander issued the final "execute" command. Bomb disposal robot A fired a high-pressure water jet for 3 seconds, successfully destroying the detonation circuit. The on-site sensors (acoustic vibration) reported a slight muffled sound, then returned to normal. The robot's status was transmitted back throughout the process, and the digital twin was updated synchronously, completing the bomb disposal and safety work.

[0039] The IoT-based intelligent security and bomb disposal system of the present invention improves perception and early warning from a single vision to full-dimensional perspective, solving the problems of blind spots and missed detections; traditional systems rely on video surveillance, which cannot effectively detect non-visual threats. This invention constructs a three-dimensional sensing network by integrating multimodal sensors, including visible light, trace chemical sniffing, millimeter-wave penetration imaging, and acoustic vibration monitoring. Furthermore, chemical and radioactive sensing enables the system to provide early warnings through trace volatile components or radiation signals in the air before explosives are visible, advancing the threat detection timeline. Penetration imaging sensing can effectively identify the internal structure of suspicious items that are wrapped, obscured, or disguised, overcoming the physical limitations of video surveillance and reducing missed detections due to visual deception. Acoustic vibration sensing can capture vibrations or sound waves caused by abnormal placement or striking, complementing visual behavior analysis and enhancing the ability to identify intentional actions. The synchronous acquisition and timestamp alignment of this multi-source heterogeneous data lays a solid foundation for high-reliability fusion analysis in the backend, increasing the system's probability of detecting complex and concealed threats. At the analysis and response level, it accelerates the process from layer-by-layer reporting to second-level closed-loop processing, solving the problems of response lag and decision-making reliance on experience. In the traditional model, the process from detecting anomalies to issuing response instructions is lengthy. Through a collaborative architecture of real-time edge processing and in-depth cloud analysis, the response chain has been restructured: the edge computing layer, as the system front-end, performs AI inference near the data generation point, such as behavior recognition, object detection, and local multi-sensor feature fusion, quickly generating high-quality structured warning events; filtering massive amounts of raw data and uploading only key information reduces bandwidth pressure and shortens the initial warning time; the cloud-based intelligent analysis platform, as the system's decision-making center, relies on powerful computing capabilities to perform decision-level fusion analysis of the aggregated multimodal information from across the domain. Employing an attention-based AI model, it intelligently weighs the weight of different pieces of evidence, outputting an objective and quantifiable threat level score, replacing subjective and vague human experience judgment; after threat confirmation, the intelligent decision engine, based on a real-time digital twin including maps, pedestrian flow, and resource status, automatically generates a response plan containing the optimal robot, optimal path, and standard operating procedures (SOPs). This process compresses the traditional decision-making process, which requires multi-party consultation and manual planning, into a process that can be completed in seconds, achieving an automated, intelligent, and rapid closed loop from perception to contingency plan.

[0040] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An intelligent security and explosive ordnance disposal system based on the Internet of Things, characterized in that, include: The ubiquitous sensing layer consists of several heterogeneous IoT sensing nodes deployed within the protected area, used to collect visible light video, non-visible material components, penetrating imaging signals, and environmental disturbance signals within the protected area. The edge computing layer, consisting of edge computing gateways distributed in each logical partition of the protected area, is connected to the ubiquitous sensing layer and is used to receive and process the raw data of the sensing nodes. The cloud-based intelligent analysis platform communicates with the edge computing layer to receive and aggregate structured warning events and associated raw data samples from different edge computing gateways. It then calls a multimodal AI fusion analysis model to perform decision-level fusion analysis on the aggregated multi-dimensional data across the entire domain. This enables the identification of suspicious targets, judgment of behavioral patterns, and quantitative assessment of threat levels. Based on the threat level assessment results and the real-time situation of the built-in digital twin, the platform automatically generates a contingency plan that includes the optimal execution unit, dynamic path planning, and atomic operation sequences through an intelligent decision engine. The unmanned execution layer includes at least one security patrol robot and at least one bomb disposal robot, which are respectively connected to the cloud-based intelligent analysis platform. They are used to receive precise control instructions transformed from the disposal plan and execute the entire process from on-site reconnaissance and target confirmation to final disposal. At the same time, they transmit their own status, operation data and environmental perception data back in real time. The unified command terminal interacts with the cloud-based intelligent analysis platform to visualize the overall digital twin situation, the decision-making basis and execution details of the emergency response plan, and provides a human-machine interface for commanders to review, modify, authorize and issue emergency interventions for the plan. The edge computing layer and the cloud-based intelligent analysis platform form a collaborative processing architecture. The edge computing layer performs low-latency, high-concurrency initial perception and filtering, while the cloud-based intelligent analysis platform performs high-complexity, global in-depth analysis and decision-making. The cloud-based intelligent analysis platform and the edge computing layer are linked through bidirectional data streams of structured warning events and model parameters.

2. The IoT-based intelligent security and explosive ordnance disposal system according to claim 1, characterized in that, The heterogeneous IoT sensor nodes in the ubiquitous sensing layer specifically include: a high-definition smart camera for acquiring video streams and having a built-in moving target detection and tracking algorithm; a trace gas sensor for real-time adsorption and analysis of air samples to identify the volatile organic compound components characteristic of explosives; a millimeter-wave radar for transmitting and receiving millimeter-wave signals to generate radar images reflecting the outline, material, and internal structure of objects; and an array of acoustic and vibration sensors for acquiring abnormal acoustic signals and structural vibration signals in the environment and locating disturbance sources through direction-of-arrival estimation. All sensor nodes are synchronized via a unified network time protocol to ensure that the timestamps of the original data are consistent.

3. The IoT-based intelligent security and explosive ordnance disposal system according to claim 2, characterized in that, The processing of the edge computing layer specifically includes: performing real-time human pose estimation, package abandonment event detection, and abnormal running and gathering behavior recognition on video streams from high-definition smart cameras using a loaded lightweight convolutional neural network model; performing spatiotemporal alignment of radar image data from millimeter-wave radar with video frames from high-definition smart cameras, and enhancing the detection rate of suspicious items under occlusion or disguise through feature-level fusion algorithms; and triggering the edge warning generation module when the local inference result or fusion analysis result exceeds a preset confidence threshold, packaging the time, location, sensor type, confidence level, and key evidence snapshots to form the structured warning event.

4. The IoT-based intelligent security and explosive ordnance disposal system according to claim 1, characterized in that, The multimodal AI fusion analysis model in the cloud-based intelligent analysis platform adopts a Transformer-based multi-head attention mechanism fusion network. The specific steps for performing decision-level fusion analysis are as follows: video features, radar point cloud features, gas concentration time series features, and acoustic vibration spectrum features reported from different edge gateways that are related to the same spatiotemporal event are mapped to a high-dimensional embedding space; through the multi-head attention mechanism, the relevance weights of different modal features in determining the current threat are dynamically calculated; the weighted fusion global feature vector is input into the downstream multi-task learning network, which outputs target classification, behavior classification, and comprehensive threat level score in parallel, and then fits to generate a threat level assessment result.

5. The IoT-based intelligent security and explosive ordnance disposal system according to claim 1, characterized in that, The logic by which the intelligent decision-making engine generates the contingency plan includes: Contingency plan triggering: When the comprehensive threat level score exceeds the first threshold, the contingency plan generation process is automatically triggered; Resource scheduling: Query the status, location and capabilities of all available execution units in the digital twin, and calculate and select the optimal security patrol robot to perform the first reconnaissance with multiple objectives such as shortest path, least time and best concealment; Dynamic programming: Based on the real-time 3D map and dynamic obstacle information in the digital twin, plan a collision-free optimal path for the selected robot from its current position to the target position; Task Serialization: If the threat level rises to the second threshold after reconnaissance and verification, an atomic operation sequence is generated for the bomb disposal robot. The sequence includes branch instructions for autonomous navigation to precise coordinates, robotic arm deployment, activation of X-ray fluoroscopic scanning, and selection of robotic arm grabbing and transferring to the explosion-proof container or activation of water cannon for in-situ destruction based on the scan results.

6. The IoT-based intelligent security and explosive ordnance disposal system according to claim 1, characterized in that, The bomb disposal robot includes a modular payload interface, which allows the robot's robotic arm end effector to quickly switch between different functional modules according to instructions; The functional modules include at least: a high-resolution binocular vision camera for close-range 3D modeling and visual servo control; a miniature X-ray backscatter imager for non-destructive internal imaging of suspicious items; and a high-pressure liquid water jet cannon for launching precise water bullets for cutting or detonation. The robot's controller is embedded with a local path planning and arm-hand coordination control algorithm based on reinforcement learning training, enabling it to semi-autonomously complete the atomic operation sequence.

7. The IoT-based intelligent security and explosive ordnance disposal system according to claim 1, characterized in that, It also includes a federated learning framework and model update mechanism: the cloud-based intelligent analysis platform acts as the central server for federated learning, and periodically distributes initialized global AI models to each edge computing gateway. Each edge computing gateway uses locally generated de-identified data to train the model and uploads the trained model parameters incrementally with encryption. The cloud-based intelligent analysis platform aggregates all uploaded parameters, updates the global model, and redistributes the optimized model to each edge computing gateway in the edge computing layer.

8. The IoT-based intelligent security and explosive ordnance disposal system according to claim 1, characterized in that, It also includes a post-disposal effect evaluation and knowledge base construction module, which is used to automatically extract the entire chain of data from initial perception to final disposal after a complete disposal task is closed, and generate structured cases; based on preset key performance indicators, it automatically evaluates the system's response time, identification accuracy, and disposal success rate; The structured cases are used as augmented samples, labeled, and stored in a dedicated knowledge base for subsequent supervised incremental training of the multimodal AI fusion analysis model.

Images