An intelligent security inspection system based on edge computing and humanoid robots

Abstract

The application discloses an intelligent security check system based on edge computing and a humanoid robot, relates to artificial intelligence, and aims at the problem of independent case robot information in the prior art. The humanoid robot at the hardware layer is used as an edge computing node to process real-time tasks; a server is used as a non-real-time data center of the system; the perception layer is used for comprehensive digitization of the environment, personnel and articles; the decision layer is arranged in the on-board edge computing unit of the humanoid robot; and the interaction layer is used as a man-machine interface. The humanoid robot is used as a mobile edge computing node, and other case sensor data is deeply fused through a heterogeneous network architecture. The application covers an anti-occlusion visual perception algorithm based on human body key points, a risk assessment method of multi-source heterogeneous data fusion, and a real-time task cooperative scheduling technology based on priority, and solves the problems of equipment information islands and artificial operation efficiency bottlenecks in security check in high-traffic places such as subways and airports.

Description

Technical Field

[0001] This invention relates to artificial intelligence, and more particularly to an intelligent security inspection system based on edge computing and humanoid robots. Background Technology

[0002] With the increasing severity of the global public safety situation, the demand for security checks in public places such as subways and airports continues to grow. However, the existing security check system suffers from significant "information silos" and efficiency bottlenecks: 1. There is a lack of data exchange between existing security equipment (such as X-ray security scanners and metal detectors) and security personnel. The detection results of X-ray security scanners cannot be automatically transferred to personnel responsible for physical intervention, resulting in significant coordination delays between detection and handling. 2. Traditional models heavily rely on manual image interpretation and body searches. Security checks mainly depend on manual image interpretation; however, manual operation is limited by physiological functions, and prolonged, highly repetitive visual search tasks can lead to decreased vigilance, resulting in increased missed detection rates and high labor costs. 3. Current intelligent security check technologies mainly focus on AI-based image recognition algorithms (such as YOLO and DETR). These solutions only solve the "recognition" problem and fail to form a closed loop for physical intervention. Existing humanoid security robots are mostly used for patrolling or monitoring, lacking deep integration with security equipment and the ability to conduct collaborative inspections and physical guidance in security checkpoints.

[0003] Therefore, there is an urgent need for a humanoid robot security inspection system that can integrate digital perception with physical action, possess anti-occlusion perception capabilities, and can work together efficiently. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent security inspection system based on edge computing and humanoid robots to solve the problems existing in the prior art.

[0005] The intelligent security inspection system based on edge computing and humanoid robots described in this invention includes a hardware layer, a perception layer, a decision-making layer, and an interaction layer;

[0006] The hardware layer includes a humanoid robot body, an X-ray security inspection machine, a metal detection gate, and a server. The humanoid robot acts as an edge computing node to process real-time tasks, while the server acts as the system's non-real-time data center. It connects to the edge computing node via TCP / IP protocol and is responsible for storing historical security inspection logs, performing long-term data analysis, and managing the cluster of multiple humanoid robots. The perception layer is used for the comprehensive digitization of the environment, people, and objects; The decision-making layer is deployed in the onboard edge computing unit of the humanoid robot; it includes a multi-source data fusion module, a collaborative task allocation model, and a finite state machine. The interaction layer serves as a human-machine interface, enabling physical and informational interaction with passengers through speech synthesis, speech recognition, and visual alerts.

[0007] The intelligent security inspection system based on edge computing and humanoid robots described in this invention has the advantage of using humanoid robots as mobile edge computing nodes to deeply integrate data from X-ray security scanners, metal detectors, and visual sensors through a heterogeneous network architecture. This invention encompasses anti-occlusion visual perception algorithms based on human key points, risk assessment methods for multi-source heterogeneous data fusion, and priority-based real-time task collaborative scheduling technology, solving the problems of equipment "information silos" and manual operation efficiency bottlenecks in security inspections in high-traffic areas such as subways and airports. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the overall architecture of the system described in this invention.

[0009] Figure 2 This is a schematic diagram of the workflow of the object perception enhancement detection framework described in this invention.

[0010] Figure 3 This is a curve comparing the operating characteristics of the system described in this invention with those of a single-modal system. Detailed Implementation

[0011] The intelligent security inspection system based on edge computing and humanoid robots described in this invention addresses the "information silo" problem in traditional security inspection systems by providing a technical solution that uses humanoid robots as mobile edge computing nodes.

[0012] Existing technologies typically treat security inspection equipment as passive information collection terminals, while this invention innovatively constructs an active edge computing node centered on a humanoid robot. Addressing the technical challenge of resource contention arising from the conflict between image recognition computing power and motion control real-time performance within a single computing unit in existing technologies, this invention employs a hardware-level architecture design of computing power isolation and bus offloading. Unlike traditional security inspection humanoid robots used only for patrolling, the humanoid robot in this system needs to simultaneously process high-frequency image data (high bandwidth) from X-ray machines and perform precise body operations (low latency). Therefore, simply stacking hardware cannot meet the requirements; a specific heterogeneous network topology is necessary to resolve the aforementioned timing conflicts.

[0013] The overall architecture of this system is as follows: Figure 1As shown, a modular, layered design is adopted, consisting of a Hardware Layer, a Perception Layer, a Decision Layer, and an Interaction Layer, from bottom to top. To specifically implement the function of outputting physical warnings to passengers in the Interaction Layer, a specific hardware execution component, namely an intelligent warning terminal, is integrated on the humanoid robot body. The function of the intelligent warning terminal corresponds to... Figure 1 Functions within the interaction layer. This intelligent warning terminal is based on a high-reliability MCU design. Its internal protocol parser and stream processor receive high-speed data streams from the host and distribute them according to the protocol type: real-time light effect data is directly sent to the PID and aging compensation controllers to drive the PWM generator; while high-priority state switching instructions (such as emergency stop) are sent to a stacked finite state machine for logic processing. The intelligent warning terminal also integrates current / temperature sensors, sending hardware feedback signals back to the compensation controller to form a closed-loop control, ultimately driving the LED array and buzzer to emit sound and light signals strictly synchronized with the humanoid robot's movements.

[0014] The hardware layer, serving as the physical foundation, includes the humanoid robot itself, X-ray security scanner, metal detector gate, and server. The humanoid robot acts as an edge computing node, processing real-time tasks; the server, as the system's non-real-time data center, connects to the edge nodes via TCP / IP protocol, responsible for storing historical security logs, performing long-term data analysis, and managing the cluster of multiple humanoid security robots. This ensures that edge computing power is focused on millisecond-level real-time inference, achieving layered computing power collaboration between edge and cloud.

[0015] The perception layer is used for comprehensive digitization of the environment, people and objects, including a visual perception module that processes RGB and depth data, an auditory module that processes voice commands, and an external sensor data interface.

[0016] The decision-making layer is deployed in the onboard edge computing unit of the humanoid robot. This layer includes a multi-source data fusion module, a cooperative task assignment model (CTAM), and a finite state machine (FSM).

[0017] The interaction layer serves as a human-machine interface, enabling physical and informational interaction with passengers through text-to-speech (TTS), automatic speech recognition (ASR), and visual alerts (warning lights, motion guidance).

[0018] To address the issues of incompatible communication protocols between heterogeneous devices and interference between data flow and control flow, this invention abandons the traditional single-bus connection method and creatively designs a physically isolated dual-bus network topology, as follows: Real-Time Control Bus: Dedicated to the control of the humanoid robot body. Utilizing the distributed clock synchronization mechanism of the EtherCAT protocol, a hard real-time channel is constructed that serves only the servo motors. Its technical advantage lies in ensuring that even when the vision system is processing X-ray images at full load, motion control commands can still maintain a jitter accuracy of <10µs, ensuring the absolute safety of the humanoid robot's movements in crowds and preventing motion stuttering caused by fluctuations in computing power.

[0019] Event Message Bus: Dedicated to the flow of sensing data. It uses the MQTT protocol to connect the X-ray security inspection machine, the sensing layer, and the decision-making layer. The innovation of this invention lies in utilizing the asynchronous nature of the publish / subscribe model to completely decouple the bursty X-ray image data stream from the periodic motion control flow, both logically and physically.

[0020] This dual-bus architecture is not a simple application of existing protocols, but a specific technical solution for the contradiction between concurrent processing of multi-source heterogeneous data and physical security control in the specific scenario of human-machine collaboration in security inspection.

[0021] Humanoid Robot Edge Node Configuration: The humanoid robot possesses specific physical parameters adapted to the subway security inspection environment, as detailed in Table 1 below. The humanoid robot has a total of 40 degrees of freedom (DOF). Specifically, these are distributed as follows: 2 in the neck (for visual scanning), 3×2 in the shoulders, 1×2 in the elbows, 3×2 in the wrists (for flexible gesture guidance), 3×2 in the hips, 1×2 in the knees, 2×2 in the ankles, and 6×2 in the dexterous hands. This high degree of freedom design allows the humanoid robot not only to move but also to perform complex guiding gestures that conform to human social norms, such as "Please come this way" or "Please open your bag." To support real-time inference for the object perception enhancement detection framework, the humanoid robot incorporates a high-performance industrial computer configured with an Intel i9-13900 CPU, 64GB RAM, and a 500GB SSD. This configuration ensures smooth operation of YOLOv8, HRNet, and data fusion algorithms even offline, without relying on cloud computing power, thus guaranteeing data privacy and low processing latency. It adopts a 60V, 12Ah hot-swappable lithium battery pack. This design allows the humanoid robot to directly replace batteries while standing without shutting down and restarting, thus meeting the needs of 24-hour uninterrupted operation of subway security checks. The head integrates a 1080P@30fps stereo camera (RGB-D) with 1280x720 60fps depth stream acquisition capability, a horizontal field of view (H-FOV) of 360 degrees (with neck rotation), a vertical field of view of -7 degrees to 52 degrees, and a blind zone of only 0.1 meters, ensuring clear capture of nearby luggage and distant passengers.

[0022] Table 1. Hardware Platform Parameters for Humanoid Robots

[0023] The system achieves physical connection with traditional security inspection equipment through the following interfaces, as detailed in Table 2 below. The X-ray machine integration connects to the X-ray security inspection machine via an Ethernet interface (EtherCAT), using a custom TCP / IP protocol. The communication mechanism is as follows: when the photoelectric sensor is triggered, the X-ray security inspection machine sends a data packet containing Bag_ID (bag ID) and a timestamp; image data is compressed (approximately 500KB / frame) before being transmitted to the humanoid robot, with transmission latency controlled within 200ms. The metal detector gate integration connects via an RS-485 serial port. A custom serial frame format [Sync][Zone][Intensity][CRC] is used, with a parsing period of 100ms. This data is used to determine whether a passenger is carrying metal items and is associated with the humanoid robot's visual tracking ID. Conveyor belt control: connected via a relay module. The humanoid robot can send digital on / off signals to control the start and stop of the conveyor belt. When the system detects suspected contraband, it can stop the conveyor belt within 0.5 seconds to accurately locate suspicious baggage. The warning light is connected via RS-232 serial port and is triggered by CTAM using a hexadecimal command set; ensuring that the color changes within 100 milliseconds after the voice is issued.

[0024] Table 2. Description of External Device Integration Interfaces and Synchronization Mechanisms

[0025] To address the pain points of dense crowds and severe visual obstruction during subway rush hours, an object-aware enhanced (OAE) detection framework is deployed at the edge of a humanoid robot to understand the spatial relationship between people and objects and solve the occlusion problem.

[0026] like Figure 2 As shown, the object perception enhancement detection framework does not use a single-channel input, but instead constructs a parallel two-stream network structure. The visual stream is the input: an RGB image acquired by the humanoid robot. This process uses CSP-Darknet53 as the feature extractor. Its function is to extract multi-scale visual features from the image. It captures the texture, shape, and color information of objects. CSP-Darknet53, through its cross-stage local network structure, reduces computation while preserving gradients, making it suitable for edge deployment. Pose Stream: The input is also an RGB image. This stream uses a frozen-weights HRNet-W32 network. Its function is to perform human pose estimation, and the output contains a set of 17 human keypoints. (e.g., wrists, elbows, shoulders, knees, etc.). These key points provide geometric information about the human skeleton, serving as geometric anchor points for subsequent inference of the position of occluded objects.

[0027] In crowded places, luggage is often obscured by legs or people. Therefore, the system introduces a Keypoint-Guided Attention (KGA) mechanism, which uses human pose information to guide the network to focus on potentially existing object regions. In feature interaction computation, spatial location in the visual feature map... The eigenvector at that location is , No. The feature vectors of the key body points are: The system first calculates the original correlation score between the two through dot product operation. The formula is as follows: ; in, and It is a learnable linear projection matrix used to map visual features and keypoint features to the same semantic space; This is a scaling factor used to prevent the gradient from vanishing due to excessively large dot product values. The relevance scores are normalized along the keypoint dimension using the Softmax function to obtain the final attention weights. : ; This allows the model to automatically learn to give higher weights to visual regions near the detected objects and key points. This suppresses background areas. A state classifier is used to predict interaction states (e.g., handheld, carried on the back) to suppress false alarms that are physically inconsistent with human posture. Finally, an aligned depth map is used to backproject the refined 2D bounding box into the humanoid robot's 3D coordinate system (x, y, z) to achieve accurate motion guidance. To verify the practical effectiveness of the above keypoint-guided attention mechanism in anti-occlusion detection, the following comparative tests were conducted, and the results are shown in Table 3.

[0028] Table 3 Comparison of effects of different implementation methods Comparative Example 1: Only the basic CSP-Darknet53 network was used for visual feature extraction and object detection, without introducing any pose flow or attention mechanisms. This group simulated a traditional single-modal visual detection scheme, which was severely affected by occlusion (average accuracy of only 54.4%).

[0029] Comparative Example 2: An HRNet-W32 pose flow was added to the baseline, directly concatenating visual features with keypoint features through simple channel concatenation, but without using the KGA attention mechanism for feature alignment. The results showed limited accuracy improvement (57.1%), demonstrating that simply introducing pose information without an effective feature fusion strategy cannot adequately address the occlusion problem.

[0030] This embodiment simultaneously introduces a two-stream network and a KGA module, achieving deep interaction of features by calculating spatial correlation weights. Compared to control group A, the accuracy is further improved to 60.3%, and the false positive rate is significantly reduced to 4.2%, verifying the crucial role of the attention mechanism in feature alignment.

[0031] To reduce the false alarm rate, this system employs a mathematically weighted model to fuse X-ray physical attributes and visual semantic attributes. The system first needs to ensure that the person detected visually corresponds to the luggage scanned by the X-ray security scanner. This is achieved through matching timestamps and track IDs. After acquiring images from the X-ray security scanner, the average density value of the items is calculated. Because different materials (metals, liquids, plastics) have different X-ray absorption rates, density is an important basis for identifying contraband. The system uses a normalization function... The density values ​​are mapped to the [0, 1] interval. Confidence scores for visual detection are obtained from the object perception enhancement detection framework. The range is [0, 1]. To quantify risk, the system uses a weighted fusion formula to calculate the overall safety score. The formula is as follows: ; in, This is a dynamic weighting factor with a value range of [0, 1], used to adjust the relative importance of physical density features and visual semantic features (RGB) in risk assessment. Extensive experimental verification has shown that, considering the higher certainty of X-ray penetration detection in security inspection scenarios, this embodiment sets the optimal weighting. =0.6. Furthermore, to achieve accurate binary risk assessment, this invention introduces an independent safety assessment threshold. Logically, a preset safety threshold is used for the judgment. The system calculates the comprehensive score in real time. If deemed safe, the humanoid robot will allow passage and guide the passenger; if... If the condition is identified as a potential risk, the system will immediately trigger an audible and visual warning and initiate a re-inspection process.

[0032] To evaluate the discriminative power of the safety scores, the operating characteristic (ROC) curves of the three methods were compared as follows: Figure 3 As shown, visual methods only, X-ray density methods only Furthermore, this invention proposes a weighted fusion method. As shown in the ROC analysis, single-modal methods have limitations: the area under the curve (AUC) of the visual-only method is 0.76, due to occlusion; while the density-only method has an AUC of 0.81, but lacks semantic discrimination ability. The fusion method proposed in this invention significantly improves the AUC, reaching 0.92. Similarly, in the precision-recall (PR) analysis, the fusion model maintains a high precision (>0.85) even with a high recall (>0.9), which verifies that multimodal fusion can effectively reduce false positives without affecting security.

[0033] This system employs a priority-based preemptive scheduling strategy. To demonstrate the system's responsiveness to emergencies, this invention introduces mathematical analysis of worst-case response time.

[0034] Task response time model, for any security check task Its worst-case response time Defined by the following formula: ; in: For the task Worst Execution Time (WCET); For higher priority The task set; The arrival cycle for high-priority tasks; Blocking time caused by low-priority tasks.

[0035] This system categorizes tasks into 1-6 priority levels (e.g., routine patrols are level 1, passenger guidance is level 3). For the highest priority "emergency threat" tasks (…),… (Priority=6), this system employs a resource preemption mechanism to ensure... Therefore, its response time can be simplified as follows: ; in This covers context switching overhead. Mathematically, this rigorously guarantees that the system can trigger physical intervention (such as cutting off the conveyor belt power) within 100ms when a significant threat is detected.

[0036] This invention proposes an intelligent security inspection system based on edge computing and humanoid robots. Addressing the problems of "information silos," the efficiency bottleneck of manual operations, and high false negative rates in complex scenarios within existing public security inspection systems, it constructs a system solution with a humanoid robot as the core edge node. The main technical contributions and beneficial effects of this invention are summarized as follows: A high-real-time edge computing architecture was constructed. By integrating the EtherCAT real-time control bus and the MQTT event message bus, this system successfully broke down the data barriers between the X-ray security inspection machine, the metal detector gate, and the humanoid robot. This architecture achieves physical isolation and logical coordination between data flow and control flow, ensuring that the humanoid robot can maintain millisecond-level precise control of the servo motors while processing large volumes of image data, effectively solving the technical pain point of lack of linkage between traditional security inspection equipment.

[0037] This invention significantly improves the anti-occlusion perception capability in crowded environments. Addressing the severe crowd occlusion problem during peak subway hours, this invention proposes an enhanced object perception detection framework. By combining a dual-stream parallel network with a keypoint guided attention (KGA) mechanism, geometric information of human posture is used to guide visual attention, effectively suppressing background interference. Ablation experiments show that this method improves the average detection accuracy in occluded scenarios from 54.4% to 60.3%, and significantly reduces the false alarm rate.

[0038] This invention achieves accurate risk decision-making based on multi-source data fusion. It overcomes the limitations of single-modal detection and proposes a weighted risk assessment algorithm that integrates X-ray physical density attributes and visual semantic attributes. This is achieved by introducing dynamic weighting factors and independent safety judgment thresholds. The system achieved an area under the curve (AUC) of 0.92 in receiver operating characteristic (ROC) analysis, demonstrating that it can effectively reduce false alarms while ensuring high recall, which is significantly better than traditional single-modality detection methods.

[0039] This ensures millisecond-level physical response under emergency threats. Relying on the Cooperative Task Allocation Model (CTAM) and a priority-based preemptive scheduling strategy, the system mathematically guarantees the timeliness of high-priority tasks. When a major security threat is detected, the system can control the context switching overhead to within 10ms through a resource forcible preemption mechanism, and ensure that physical intervention measures such as cutting off the conveyor belt power are triggered within 100ms, achieving a rapid closed loop from digital perception to physical action.

[0040] In summary, this invention achieves a fully automated, high-precision, and physically-intervention-capable intelligent security inspection model through the deep integration of edge computing and embodied intelligence technologies. This provides a feasible technical path to reduce the manpower cost of security inspections while improving the level of public safety.

[0041] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this invention.

Claims

1. An intelligent security inspection system based on edge computing and humanoid robots, characterized in that, It includes the hardware layer, perception layer, decision-making layer, and interaction layer; The hardware layer includes a humanoid robot body, an X-ray security inspection machine, a metal detection gate, and a server. The humanoid robot acts as an edge computing node to process real-time tasks, while the server acts as the system's non-real-time data center. It connects to the edge computing node via TCP / IP protocol and is responsible for storing historical security inspection logs, performing long-term data analysis, and managing the cluster of multiple humanoid robots. The perception layer is used for the comprehensive digitization of the environment, people, and objects; The decision-making layer is deployed in the onboard edge computing unit of the humanoid robot; it includes a multi-source data fusion module, a collaborative task allocation model, and a finite state machine. The interaction layer serves as a human-machine interface, enabling physical and informational interaction with passengers through speech synthesis, speech recognition, and visual alerts. The perception layer includes a visual perception module for processing RGB and depth data, an auditory module for processing voice commands, and an external sensor data interface. The dual-bus network topology employing physical layer isolation has the following structure: Real-time control bus: Dedicated to humanoid robot body control; Utilizes the distributed clock synchronization mechanism of the EtherCAT protocol to build a hard real-time channel that serves only the servo motors; Event Message Bus: Dedicated to the flow of sensing data; uses the MQTT protocol to connect the X-ray security inspection machine, the sensing layer, and the decision-making layer; An object perception enhancement detection framework is deployed at the edge of a humanoid robot to understand the spatial relationship between people and objects; The object perception enhancement detection framework is based on a parallel two-stream network structure: The visual stream takes RGB images captured by a humanoid robot as input and uses CSP-Darknet53 as the feature extractor to extract multi-scale visual features from the images. It captures the texture, shape, and color information of objects; Pose stream, with RGB images as input; uses an HRNet-W32 network with frozen weights for human pose estimation, and outputs a set of 17 human keypoints. .

2. The intelligent security inspection system based on edge computing and humanoid robots according to claim 1, characterized in that, A keypoint-guided attention mechanism is introduced, which uses human posture information to guide the network to focus on potential object regions; in feature interaction computation, spatial location in the visual feature map... The eigenvector at that location is , No. The feature vectors of the key body points are: ; The system first calculates the original correlation score between the two through dot product operation. The formula is as follows: ； in, and It is a learnable linear projection matrix used to map visual features and keypoint features to the same semantic space; This is a scaling factor used to prevent the gradient from vanishing due to excessively large dot product values; the relevance scores are normalized along the keypoint dimension using the Softmax function to obtain the final attention weights. : ； This allows the model to automatically learn to give higher weights to visual regions near the detected objects and key points, thereby suppressing background regions.

3. The intelligent security inspection system based on edge computing and humanoid robots according to claim 2, characterized in that, A mathematical weighted model is used to fuse X-ray physical properties and visual semantic properties; After acquiring images from the X-ray security scanner, the average density value of the items is calculated. Density value is used as the basis for judging contraband; through normalization function Map the density values ​​to the interval [0, 1]; Obtaining confidence scores for visual detection from an object perception enhancement detection framework The range is [0, 1]; a weighted fusion formula is used to calculate the comprehensive safety score; the formula is as follows: ; in, It is a dynamic weighting factor with a value range of [0, 1], used to adjust the relative importance of physical density features and visual semantic features in risk assessment.

4. The intelligent security inspection system based on edge computing and humanoid robots according to claim 3, characterized in that, Introduce independent security decision thresholds ; In terms of judgment logic, a preset safety threshold is used. The system calculates the comprehensive score in real time. If deemed safe, the humanoid robot will allow passage and guide the passenger; if... If the system detects a potential risk, it will immediately trigger an audible and visual warning and initiate a re-inspection process.

5. The intelligent security inspection system based on edge computing and humanoid robots according to claim 4, characterized in that, A priority-based preemptive scheduling strategy is adopted; mathematical analysis of worst-case response time is introduced. Task response time model, for any security check task Its worst-case response time Defined by the following formula: ; in: For the task Worst execution time; For higher priority The task set; The arrival cycle for high-priority tasks; The blocking time caused by low-priority tasks.

Images