Plant dangerous area monitoring storage and intrusion behavior intelligent identification early warning system

Abstract

This invention belongs to the field of intelligent video surveillance technology and discloses a factory hazardous area monitoring storage and intrusion behavior recognition and early warning system. By incorporating a distributed hierarchical storage module, it automatically configures differentiated storage duration and multi-copy backup strategies based on the intrusion hazard level and area risk level. The first-level solid-state drive enables real-time data and second-level abnormal event retrieval and playback, the second-level mechanical hard drive stores regular video, and the third-level offline archiving medium archives high-risk event data for a long time. At the same time, it generates an immutable hash verification chain for medium and high-risk events, and automatically cleans up abnormal data in batches on a periodic basis, realizing intelligent management of security data throughout its entire life cycle. This not only significantly reduces storage costs but also ensures the complete retention and secure traceability of critical data. The multi-modal preprocessing module completes the spatiotemporal alignment and adaptive interference filtering of video, infrared thermal imaging, millimeter-wave radar, and vibration sensor data.

Description

Technical Field

[0001] This invention belongs to the field of intelligent video surveillance technology, specifically a factory area hazardous area monitoring and storage and intrusion behavior intelligent recognition and early warning system. Background Technology

[0002] Chemical, energy, and metallurgical plants typically have tank storage areas, reaction unit areas, high-voltage power distribution areas, and hazardous chemical storage areas. These areas have high control requirements for personnel access rights, abnormal stay times, and boundary crossings. Once unauthorized personnel or vehicles enter, it can easily lead to safety risks such as equipment malfunctions, material leaks, fires, and explosions.

[0003] Existing hazardous area management in factories relies heavily on continuous recording by fixed cameras, manual patrols, and ordinary motion detection alarms. Although some systems can perform video recognition or authorization verification, the recognition results, storage strategies, evidence verification, and on-site linkage are usually separate from each other. There is a lack of a unified index relationship between abnormal events and corresponding video clips, sensor data, and trajectory data.

[0004] In actual operation, changes in lighting, rainy or foggy weather, swaying vegetation, and small animal activity can easily cause false triggers; while covert intrusion behaviors such as climbing over, crawling over, and crawling through may be missed from a single video view. Because key event data is not stored and verified synchronously with the identification results, subsequent evidence collection, tracing, and joint handling are prone to problems such as incomplete data, delayed response, and interruption of target trajectory. Summary of the Invention

[0005] The purpose of this invention is to provide a factory area hazardous area monitoring and storage and intrusion behavior intelligent identification and early warning system to solve one or more of the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a factory hazardous area monitoring and storage and intrusion behavior intelligent identification and early warning system, comprising the following modules: Furthermore, the multimodal preprocessing module is deployed on the edge computing node in the hazardous area of ​​the factory. It takes real-time video footage as the core and simultaneously accesses multimodal sensing data from infrared thermal imaging, millimeter-wave radar, and vibration sensors to perform spatiotemporal alignment processing on various heterogeneous sensor data. Based on the scene adaptive algorithm, the preprocessing parameters are dynamically and adaptively adjusted according to external interference conditions, including real-time light intensity in the factory area, weather conditions, and environmental vegetation. These parameters include the exposure, contrast, and noise reduction intensity of the video image, as well as the detection sensitivity of the millimeter-wave radar and the trigger threshold of the vibration sensor. The aligned multimodal data is pre-filtered to automatically remove environmental interference information and output valid video clips with suspected intrusion features and matching sensor data.

[0007] Furthermore, the hierarchical storage module receives valid video clips, matching sensor data, and front-end intelligent analysis results transmitted by the multimodal preprocessing module. It adopts a first-level, second-level, and third-level distributed storage architecture and realizes intelligent management of the entire lifecycle of security data from collection, storage, archiving to cleanup based on the front-end intelligent analysis results. Level 1 storage stores 7 days of real-time streams and key segments of abnormal events, supporting second-level fast retrieval and playback; Level 2 storage uses mechanical hard drives to store routine factory video data without abnormalities within the range of 7 days to 3 months; Level 3 storage archives the full original data, analysis logs, and trajectory records of high-risk events over 3 months. Based on the risk level of illegal intrusion behavior, the geographical location of the event, and the risk level of the area determined by intelligent identification, the system automatically configures differentiated storage duration and multi-copy backup strategies for different types of data; it generates an immutable hash verification chain for the original video data and motion trajectory data of all medium- and high-risk intrusion events; and it automatically performs batch cleanup of ordinary surveillance video data without any abnormalities after the preset storage period ends.

[0008] Furthermore, the intrusion identification module receives valid video segments and matching sensor data transmitted by the multimodal preprocessing module, adopts a dedicated hybrid network architecture, extracts multi-dimensional spatial features of video frames through a multi-scale network, captures the appearance contour information of targets of different sizes, including people and vehicles; and extracts temporal correlation features between consecutive video frames through a temporal network to analyze the target's motion trajectory and behavior change patterns. Embedded spatial attention and channel attention mechanisms, focusing on key monitoring areas including dangerous zone boundaries, entrances and exits, and along the perimeter of walls; The system has a built-in behavioral dataset for hazardous areas within the factory, including various behavioral samples such as normal work passage, lingering at the boundary, illegally climbing over walls, crawling through, squeezing into gaps, and illegal parking of vehicles within the factory area; it also integrates pre-processed infrared thermal imaging and millimeter-wave radar auxiliary features.

[0009] Furthermore, the federated update module adopts a hierarchical federated learning architecture, which is divided into an edge computing node layer, a regional gateway layer, and a cloud server layer. The edge computing node layer uses locally collected factory video and sensor data to train the recognition model locally, and only encrypts and uploads the model gradient parameters to the regional gateway layer. The regional gateway layer aggregates, denoises, and optimizes the gradient parameters uploaded by multiple edge computing nodes, and then distributes the iteratively updated model parameters to each edge computing node to complete the local model update; the cloud server layer is used to globally identify the initial configuration of the model and periodically iterate the version, while also aggregating the model parameter features of different factory areas.

[0010] Furthermore, the early warning and handling module, based on the illegal intrusion behavior identification results output by the intrusion identification module, divides the early warning level into three levels: general early warning, relatively serious early warning, and severe early warning according to the degree of danger of the illegal intrusion behavior obtained by intelligent identification, the identity attributes of the intruder, and the inherent risk level of the incident area, and automatically executes the corresponding graded linkage handling process. Under the general warning state, the system only pops up a prompt message and displays real-time video footage in the background monitoring center; under the more severe warning state, the system automatically triggers the on-site audible and visual alarm, and simultaneously sends SMS messages and mobile APP event pushes to security management personnel; under the critical warning state, the system links the access control system to lock the entrances and exits of dangerous areas, links to cut off the power supply to high-risk production equipment, sets up a manual confirmation and recovery mechanism, and automatically dials security and emergency alarm numbers. The system supports custom configuration based on the rule engine, allowing users to set warning rules and handling procedures for different time periods, regions, and illegal intrusion behaviors. After a warning event is triggered, the system automatically generates a standardized event report, recording the event time, precise location, intrusion target information, movement trajectory, and the entire handling process.

[0011] The custom early warning rules adopt a visual configuration interface, allowing users to freely select control areas, set effective control periods, select the type of illegal intrusion behavior, and bind the corresponding early warning level and linkage response method according to the needs of factory management. The rules take effect automatically after configuration, and the system performs identification, judgment and early warning response in real time according to the custom rules.

[0012] Furthermore, the blind spot completion module constructs a topological association model of video acquisition equipment, infrared thermal imaging, millimeter-wave radar, and vibration sensor based on graph neural network. When the identification system detects an intruding target entering the monitoring range of a certain video acquisition device, the system automatically predicts the target's movement trajectory and simultaneously notifies adjacent acquisition devices to start collaborative tracking and prepare for trajectory continuation. The target trajectory prediction is based on the target's real-time speed, direction, and historical movement path. It combines fixed environmental information such as road distribution, wall orientation, and area layout in the 3D scene map of the factory area, and integrates real-time detection data from surrounding millimeter-wave radar and vibration sensors to comprehensively calculate the target's most likely movement path and next entry area. The prediction results are synchronized to the associated monitoring equipment in real time to guide the equipment to adjust the monitoring angle and focus area in advance.

[0013] When a target enters the blind spot of equipment monitoring, the system uses historical motion video data of the target, a 3D scene map of the factory area, and information from surrounding millimeter-wave radar and vibration sensors to estimate the target's location and duration of stay in the blind spot in real time. Once the target re-enters the visible monitoring range, the system completes target feature re-matching and trajectory continuation tracking.

[0014] Furthermore, the edge computing module is deployed on each edge computing node, adopting a heterogeneous computing architecture of CPU, GPU and dedicated AI chip. The system decomposes the entire process of intelligent video analysis in the factory area into tasks, assigning lightweight tasks, including video multimodal preprocessing and image decoding, to the CPU; and assigning high computing power-consuming tasks, including multi-scale feature extraction, behavior recognition reasoning and target trajectory analysis, to the GPU and dedicated AI chip. The system monitors the resource load of each computing node in real time, automatically migrates tasks to be processed from high-load nodes to low-load nodes, and simultaneously coordinates the iterative training process of the federated learning model to dynamically allocate computing resources.

[0015] The beneficial effects of this invention are as follows: 1. This invention generates an intrusion event descriptor after the intrusion identification result and uniformly binds the event number, target trajectory number, area number, behavior category, danger level, evidence data index, and hash chain index. This ensures that identification, storage, evidence preservation, and early warning handling are no longer performed in a fragmented manner, achieving the effect of retrieving video clips, sensor data, trajectory records, and handling records based on the same event. Compared with simple continuous recording or ordinary event screenshot storage methods, this structure can reduce the storage space occupied by non-abnormal videos and maintain the verifiability of data from medium-risk and high-risk events during archiving, migration, and retrieval.

[0016] 2. This invention unifies the timestamp and spatial coordinate mapping of video data, infrared thermal imaging data, millimeter-wave radar data, and vibration sensing data. Then, it combines the target trajectory, dwell time, authorized passage status, and regional risk level to determine the danger level. This achieves the effect of reducing false triggering due to environmental interference and improving the stability of recognizing covert intrusion behavior. Especially in scenarios such as rain, fog, changes in lighting, perimeter walls, entrances and exits, and local blind spots, the results of multi-source perception can be cross-verified, avoiding misjudgment or omission caused by relying on a single video image.

[0017] 3. This invention prioritizes real-time tasks such as intrusion detection, early warning linkage, and blind spot trajectory continuation through edge computing nodes, while deferred processing of data archiving, log organization, and model training as non-real-time tasks. This ensures that core security functions in hazardous areas remain operational even when computing power is limited. Furthermore, during model updates, only encrypted model parameters or gradient parameters are uploaded, without uploading original video and sensor data. This maintains model iteration capabilities while reducing the risk of leakage of sensitive factory footage and site layout information. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the overall workflow of the system of the present invention. Figure 2 This is a flowchart of the intelligent intrusion behavior recognition sub-process of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figures 1 to 2 As shown, this embodiment of the invention provides a factory hazardous area monitoring and storage and intrusion behavior intelligent identification and early warning system, including the following modules: In this embodiment of the invention, the multimodal preprocessing module is deployed on the edge computing node in the hazardous area of ​​the factory. It takes the real-time video footage as the core and simultaneously accesses multimodal sensing data from infrared thermal imaging, millimeter-wave radar, and vibration sensors. It performs spatiotemporal alignment processing on various heterogeneous sensor data to eliminate time differences and spatial perspective deviations in the acquisition process of different devices. Based on the scene adaptive algorithm, the preprocessing parameters are dynamically and adaptively adjusted according to external interference conditions, including real-time light intensity in the factory area, weather conditions, and environmental vegetation. These parameters include the exposure, contrast, and noise reduction intensity of the video image, as well as the detection sensitivity of the millimeter-wave radar and the trigger threshold of the vibration sensor. The scene adaptive algorithm collects real-time information on the external environment of the factory area at a fixed frequency, establishes a matching rule between environmental interference and equipment parameters, automatically increases video exposure and brightness compensation when the light intensity is too low, and automatically reduces contrast to avoid overexposure when the light intensity is too high; in the event of severe weather such as rain, snow, or fog, it simultaneously increases video noise reduction intensity and millimeter-wave radar detection sensitivity to enhance the ability to detect long-distance targets; for minor interference such as vegetation swaying, it automatically increases the trigger threshold of vibration sensors to reduce invalid signal triggering; through dynamic and precise parameter adaptation, it ensures that the output effective data always remains clear, stable, and free of noise.

[0021] The aligned multimodal data is pre-filtered to automatically remove environmental interference information and output valid video clips with suspected intrusion features and matching sensor data.

[0022] Multimodal data spatiotemporal alignment employs a unified timestamp synchronization mechanism, using the system time of the video acquisition equipment as a benchmark. Data from infrared thermal imaging, millimeter-wave radar, and vibration sensors are aligned on a millisecond-level time axis. Simultaneously, spatial perspective normalization is achieved through 3D coordinate mapping of the factory area, eliminating acquisition biases from different devices. Scene-adaptive interference filtering follows a process of identification-judgment-adjustment-cleaning. It first identifies environmental interference types such as sudden changes in lighting, rain and snow, vegetation swaying, and electromagnetic interference, then determines the interference intensity level, and subsequently automatically adjusts the preprocessing parameters of the corresponding devices, retaining only valid data containing the movement characteristics of personnel and vehicles. Heterogeneous sensor data fusion uses video image features as the core, binding radar detection distance, vibration sensor amplitude, infrared temperature data, and target positions in the video image one by one to form multimodal fused feature data.

[0023] In this embodiment of the invention, the hierarchical storage module receives valid video segments, matching sensor data, and front-end intelligent analysis results transmitted by the multimodal preprocessing module. It adopts a first-level, second-level, and third-level distributed storage architecture and realizes intelligent management of the entire lifecycle of security data from collection, storage, archiving to cleanup based on the front-end intelligent analysis results. The hierarchical storage module incorporates a data evidence storage, traceability audit, and tamper verification process. When storing abnormal event data, a unique identifier is generated for each single video frame and each piece of sensor data. The identifiers of adjacent data are then concatenated to form an immutable hash verification chain. The stored data and the verification chain are stored synchronously in a separate read-only storage partition. During traceability auditing, the verification chain of the target event is retrieved, and data integrity is confirmed by comparing the identifiers segment by segment. If any segment of identifier information does not match, that segment of data is marked as abnormal. The audit log records all data retrieval and verification operations, including the operator, operation time, and verification result.

[0024] The first-level storage layer uses solid-state drives (SSDs) to store real-time video streams from the past 7 days, as well as video data of all abnormal intrusion events, supporting second-level fast retrieval and playback. The second-level storage layer uses hard disk drives (HDDs) to store routine factory video data without abnormalities within the range of 7 days to 3 months. The third-level storage layer uses offline archiving media to archive and save the original videos, analysis logs, and trajectory records of high-risk intrusion events for more than 3 months. All three levels of distributed storage are equipped with independent data retrieval and retrieval functions. Data stored on the Level 1 solid-state drives can be retrieved by time, region, and event type; data stored on the Level 2 hard disk drives can be retrieved in batches by date; and data stored on the Level 3 offline archive can be retrieved by event number, hazard level, and archive time. Offline data mounting and reading are completed during retrieval.

[0025] Based on the risk level of illegal intrusion behavior, the geographical location of the event, and the risk level of the area determined by intelligent identification, the system automatically configures differentiated storage duration and multi-copy backup strategies for different types of data; it generates an immutable hash verification chain for the original video data and motion trajectory data of all medium- and high-risk intrusion events; and it automatically performs batch cleanup of ordinary surveillance video data without any abnormalities after the preset storage period ends.

[0026] The tiered storage differentiation strategy is determined based on a combination of regional risk level and intrusion hazard level. High-risk areas and high-risk intrusion events are stored permanently; medium-risk areas and medium-risk intrusion events are stored for long-term archiving; low-risk areas and low-risk intrusion events are stored for medium-term purposes; and data without anomalies is stored for short-term purposes. The multi-replica backup strategy is configured according to risk level. High-risk event data is generated into three copies and stored on different hardware devices; medium-risk event data is generated into two copies; and low-risk event data is generated into one copy. The tamper-proof hash verification chain is generated frame by frame and concatenated segment by segment. A unique hash value is generated for each segment of medium- and high-risk event data and linked sequentially. The chain is stored in an independent evidence area. Any data modification will cause the verification to fail. Data without anomalies is automatically batch-cleaned upon expiration. After cleaning, only the cleaning log is retained and it is no longer supported for retrieval. A dual trigger mechanism is used. The cleanup is automatically executed after the preset storage period is reached and the system verification shows no anomaly. A cleaning log is generated for record-keeping before cleaning.

[0027] In this embodiment of the invention, the intrusion identification module receives valid video segments and matching sensor data transmitted by the multimodal preprocessing module, adopts a dedicated hybrid network architecture, extracts multi-dimensional spatial features of video frames through a multi-scale network, captures the appearance contour information of targets of different sizes, including people and vehicles, and extracts temporal correlation features between consecutive video frames through a temporal network to analyze the target's motion trajectory and behavior change patterns. The intrusion detection module has a built-in false alarm filtering and non-target interference removal mechanism. The detection process is divided into three layers of verification. The first layer is target feature verification, which distinguishes between personnel, vehicles, animals, and non-biological interference sources by comparing contour size, morphological features, and movement speed. The second layer is trajectory behavior verification, which determines the target's movement trajectory based on continuity, dwell time, and direction of movement. The third layer is scene rule verification, which combines time period control and area permission configuration in dangerous areas to retain only valid events that meet the preset intrusion judgment conditions.

[0028] Embedded spatial attention and channel attention mechanisms, focusing on key monitoring areas including dangerous zone boundaries, entrances and exits, and along the perimeter of walls; The identification-specific hybrid network adopts a parallel connection structure of spatial feature extraction branch, temporal feature extraction branch, and dual attention enhancement branch. The multi-scale spatial network consists of 3 layers of residual convolutional blocks, with each layer having kernel sizes of 3×3, 5×5, and 7×3, respectively, outputting a 512-dimensional spatial feature map, which can be adapted to extract the contour and morphological features of intruders of different distances and sizes in hazardous areas of the factory. The temporal network employs a two-layer bidirectional Bi-LSTM structure, taking 16 consecutive frames of video feature sequences as input, with a temporal step size of 8, and outputting 256-dimensional temporal features to capture the temporal variation patterns of continuous movement behaviors such as climbing, crawling, and drilling within the factory area. Spatial attention and channel attention mechanisms are cascaded at the output of the multi-scale network. Attention weights are calculated using a Sigmoid normalization function, with a focus weight threshold set to 0.6, prioritizing the enhancement of feature weights for key areas such as hazardous zone boundaries, walls, and entrances / exits. Spatial and temporal features are fused element-wise with a fixed weight ratio of 1:1. The fused features are then input into two fully connected layers, ultimately outputting the probability of illegal intrusion and a quantitative score for the level of danger. The model training uses the cross-entropy loss function, the optimizer is Adam, the learning rate is dynamically adjusted from 1e-4 to 1e-5, the input data format is 640×640×3 standard video frames + 1D millimeter-wave radar / vibration sensor fusion features, and the output data includes three core information: 5 types of behavior classification results, intrusion danger level, and target coordinate location.

[0029] The system has a built-in behavioral dataset for hazardous areas within the factory, including various behavioral samples such as normal work passage, lingering at the boundary, illegally climbing over walls, crawling through, squeezing into gaps, and illegal parking of vehicles within the factory area; it also integrates pre-processed infrared thermal imaging and millimeter-wave radar auxiliary features.

[0030] The intrusion detection module performs feature-by-feature matching and comparison between the collected target features and the dedicated behavior dataset. It first determines whether the target behavior is normal operation or illegal intrusion, and then automatically classifies the intrusion danger level by combining key information such as the risk level of the intrusion area, the number of targets, the intrusion method, and the stay time. The danger level weight is increased for covert illegal intrusion behaviors such as climbing over, crawling, and crawling, while the danger level is reduced for behaviors such as single person accidentally entering or short stay at the boundary.

[0031] In this embodiment of the invention, the federated update module adopts a hierarchical federated learning architecture, which is divided into an edge computing node layer, a regional gateway layer and a cloud server layer. The edge computing node layer uses locally collected factory area videos and sensor data to train the recognition model locally, and only encrypts and uploads the model gradient parameters to the regional gateway layer. The regional gateway layer aggregates, denoises, and optimizes the gradient parameters uploaded by multiple edge computing nodes, and then distributes the iteratively updated model parameters to each edge computing node to complete the local model update; the cloud server layer is used to globally identify the initial configuration of the model and periodically iterate the version, while also aggregating the model parameter features of different factory areas.

[0032] The edge computing node layer uses real-time multimodal monitoring data and illegal intrusion behavior samples collected from the local factory area as training basis to independently complete the local iterative training of the recognition model. The entire training process only extracts the model gradient parameters and feature optimization data, and absolutely does not upload any sensitive privacy data such as original video footage, personnel characteristics, or real-world factory scenes. The gradient parameters are encrypted with high strength before uploading to ensure data transmission security throughout the process and avoid the risk of information leakage and tampering. After receiving encrypted gradient parameters uploaded by multiple edge computing nodes, the regional gateway layer first decrypts and verifies the validity of the parameters, and then uses an aggregation algorithm to fuse and optimize the parameters of multiple nodes, eliminating abnormal interference parameters. After optimization, the model parameters are encrypted again and sent to each edge computing node. The edge computing nodes automatically verify the integrity and adaptability of the parameters, and after confirming that there are no errors, they complete the local model weight overlay update.

[0033] The training of the hierarchical federated learning model follows a closed-loop process of local training, gradient uploading, aggregation optimization, and model distribution. The specific training steps and core parameters are as follows: Local training of edge computing nodes: The training set is multimodal monitoring data collected locally in the factory area. The batch size is set to 32, the initial learning rate is 1e-4, and the training stops after 5 local iterations. The model gradient parameters are encrypted using the AES-128 encryption algorithm and then uploaded to the regional gateway layer. No original video or sensor data is uploaded. Regional gateway layer aggregation optimization: The FedAvg federated averaging algorithm is used to weight and aggregate the gradient parameters uploaded by multiple edge computing nodes. The gradient denoising threshold is set to 0.01 to remove abnormal gradient interference and improve the model's generalization ability. Global iteration at the cloud server layer: A global model version update is completed every 24 hours. The model is initialized using ImageNet pre-trained weights. The global iteration cycle is 7 days. Behavioral feature data from different hazardous areas in different factories are collected simultaneously to optimize the model's general recognition capabilities. Edge computing node model update: The regional gateway distributes the iteratively optimized model parameters to each edge computing node, and the local model automatically completes parameter coverage and weight update.

[0034] In this embodiment of the invention, the early warning and handling module, based on the illegal intrusion behavior identification result output by the intrusion identification module, divides the early warning into three levels according to the degree of danger of the illegal intrusion behavior, the identity attributes of the intruder, and the inherent risk level of the incident area. The early warning level is divided into three levels: general early warning, relatively serious early warning, and severe early warning, and the corresponding graded linkage handling process is automatically executed. The early warning and response module has a built-in hierarchical linkage control logic that executes corresponding actions based on the level of intrusion danger. When a general early warning is triggered, a prompt will pop up on the monitoring room client and the event log will be recorded. When a more serious early warning is triggered, the audible and visual alarm device in the corresponding area will be activated, a notification will be pushed to the on-duty management personnel, and the access control channel in the corresponding area will be locked. When a severe early warning is triggered, the audible and visual alarm, access control lock, power cut-off in high-risk areas will be activated simultaneously, a preset emergency contact number will be dialed, and the on-site monitoring screen will be pushed to the management terminal.

[0035] In the general warning state, the system only pops up a prompt message and displays real-time video footage in the background monitoring center; in the more severe warning state, the system automatically triggers the on-site audible and visual alarm, and simultaneously sends SMS messages and mobile APP event pushes to security management personnel; in the critical warning state, the system links the access control system to lock the entrances and exits of dangerous areas, cuts off the power supply to surrounding high-risk production equipment, and automatically dials security and emergency alarm numbers. The system supports custom configuration based on the rule engine, allowing users to set warning rules and handling procedures for different time periods, regions, and illegal intrusion behaviors. After a warning event is triggered, the system automatically generates a standardized event report, recording the event time, precise location, intrusion target information, movement trajectory, and the entire handling process.

[0036] The system has a built-in offline operation mode and an automatic synchronization mechanism after network recovery. When the network is interrupted, the edge computing node switches to offline operation mode, stores all monitoring data locally, performs core functions of intrusion identification and early warning handling, and suspends cloud data synchronization and model iteration updates. During offline operation, all abnormal event data, device status data and operation logs are cached locally according to preset rules. When the network is restored, the edge computing node establishes a connection with the cloud server and synchronizes the cached content according to data priority, prioritizing the synchronization of abnormal event evidence data and device fault records, and then synchronizing ordinary monitoring data.

[0037] The three-tiered early warning system is based on clear judgment criteria. General warnings target non-threatening behaviors such as brief lingering at the boundary of a hazardous area or a single person accidentally entering a low-risk area. More serious warnings target dangerous behaviors such as illegally climbing over walls, entering medium-risk areas, or illegally parking vehicles. Severe warnings target high-risk behaviors such as people squeezing into gaps to enter high-risk areas, multiple people entering together, or approaching flammable or explosive equipment. The custom rule engine supports configuring five core parameters: area permissions, time period control, behavior type, warning level, and linkage method. Users can freely combine and set these parameters according to their factory management needs. Standardized event reports contain fixed fields: event number, occurrence time, precise location, area risk level, intrusion hazard level, target characteristics, trajectory information, warning level, linkage response content, response result, and evidence hash value. Reports are automatically generated and can be directly exported and archived.

[0038] In this embodiment of the invention, the blind spot completion module constructs a topological association model of video acquisition devices, infrared thermal imaging, millimeter-wave radar, and vibration sensors based on graph neural networks, so as to realize the sharing of image information and collaborative target tracking among multiple devices. When the identification system detects that an intruding target enters the monitoring range of a certain video acquisition device, the system automatically predicts the target's movement trend and trajectory, and synchronously notifies adjacent acquisition devices to start collaborative tracking and prepare for trajectory continuation. The topology association model treats each video acquisition device in the factory area as an independent network node. It constructs node connection relationships based on the physical installation coordinates of the device, the monitoring field of view, the coverage area, and the communication link status between devices. Devices with adjacent or overlapping monitoring ranges automatically establish strong association links, forming a collaborative tracking network covering the entire factory. The model updates the online status, monitoring angle, and coverage area changes of each device in real time, and automatically adjusts node association relationships and tracking priorities. When a target moves within the factory area, the topology model can predict the target's movement path and instruct associated devices to adjust their monitoring perspectives in advance to prepare for trajectory continuation.

[0039] The blind spot completion module is based on a graph neural network to construct a topological association model of the factory monitoring equipment. It adopts a 2-layer GraphSAGE graph convolutional structure. The network nodes are defined as three types of sensing terminals: factory video acquisition equipment, millimeter-wave radar, and vibration sensor. The network edge weights are jointly determined by the physical distance between devices, the stability of the communication link, and the overlap of monitoring coverage. The model input features include four dimensions: equipment 3D coordinates, monitoring field of view, real-time detection status, and regional risk level. The output consists of three data points: the probability distribution of the intruding target's position within the blind zone, the estimated dwell time, and the direction of movement. The trajectory prediction uses a Kalman filter algorithm with a process noise covariance of 0.05 and a measurement noise covariance of 0.1, matching the coordinate parameters of the 3D scene map of the factory area. The system presets a blind zone detection threshold of <10% monitoring coverage. When a target enters this area, it automatically fuses surrounding sensor data to complete position prediction. After the target re-enters the monitoring range, a feature cosine similarity matching algorithm is used to complete target identity verification and trajectory continuation.

[0040] When a target enters the blind spot of equipment monitoring, the system uses historical motion video data of the target, a 3D scene map of the factory area, and information from surrounding millimeter-wave radar and vibration sensors to estimate the target's location and duration of stay in the blind spot in real time. Once the target re-enters the visible monitoring range, the system completes target feature re-matching and trajectory continuation tracking.

[0041] The multimodal acquisition module incorporates a device self-diagnosis and health status management mechanism, periodically performing status checks on each acquisition device. These checks include image clarity, frame rate, and signal stability for video acquisition devices; detection accuracy and data transmission latency for millimeter-wave radar; sensitivity and data acquisition rate for vibration sensors; and power and network connectivity status for each device. When an abnormal device health status is detected, a device fault warning is generated, the faulty device is flagged, and a notification is pushed to the management terminal. This prompts the activation of backup devices or adjustment of the acquisition strategy. The target collaborative tracking adopts a master-slave device linkage mechanism. The master monitoring device that locks onto the target transmits the target features in real time, and the adjacent slave monitoring devices adjust the pan-tilt angle in advance to align with the target and predict the trajectory, maintaining a tracking state throughout the process. The trajectory is matched by comparing multiple features such as the target's appearance, movement speed, and body size to ensure that the target's identity is unique and accurate. The factory area's 3D scene map calls the 3D model of the corresponding monitoring area in real time, binding the sensor detection data, the target's historical trajectory, and the map coordinates to intuitively display the target's movement path and location information in the blind area.

[0042] In this embodiment of the invention, the edge computing module is deployed on each edge computing node, adopting a heterogeneous computing architecture of CPU, GPU and dedicated AI chip to achieve dynamic allocation and load balancing of computing resources. The system decomposes the entire process of intelligent video analysis in the factory area into tasks, allocating lightweight tasks, including video multimodal preprocessing and image decoding, to the CPU; and allocating high computing power-consuming tasks, including multi-scale feature extraction, behavior recognition reasoning, and target trajectory analysis, to the GPU and dedicated AI chip for operation. The system categorizes all business tasks into three priority levels: highest priority, high priority, and normal priority. Core tasks directly related to factory safety, such as real-time intrusion identification, early warning and linkage response, and blind spot trajectory tracking, are designated as highest priority. Non-real-time tasks, such as data archiving and storage, model iteration training, and log generation, are designated as normal priority. Heterogeneous computing resources are matched according to task priority and computing power requirements. Lightweight tasks with low computing power requirements are allocated to CPU processing, while high-computing-power, high-concurrency intelligent analysis tasks are allocated to GPUs and dedicated AI chips. The system monitors the load of each computing node in real time. When computing power is strained, priority is given to ensuring the resource usage of the highest priority tasks, while the computing power allocation of normal priority tasks is temporarily reduced. Normal allocation is restored after the core tasks are completed, ensuring the stable operation of the core functions of the factory's hazardous area security.

[0043] The system monitors the resource load of each computing node in real time, automatically migrates tasks to be processed from high-load nodes to low-load nodes, and simultaneously coordinates the iterative training process of the federated learning model to dynamically allocate computing resources.

[0044] The system is equipped with a full-process anomaly handling mechanism. When a sensor fails, the faulty device is automatically marked, and the system switches to other normal sensor data to continue monitoring and issues a device failure prompt. When the network is interrupted, the edge computing nodes cache data locally, and automatically synchronize it to the cloud after the network is restored. When storage capacity is insufficient, expired data without anomalies is cleaned up first to ensure that abnormal event data is stored normally. When computing power is overloaded, the priority of non-core tasks is automatically reduced to ensure the operation of core functions such as intrusion identification and early warning linkage.

[0045] Edge computing power scheduling is triggered automatically based on real-time resource load status when CPU utilization continuously exceeds the threshold, GPU computing power usage reaches the upper limit, and AI chip processing latency increases. Task migration follows a pause-packaging-transfer-resume process. First, the pending tasks on high-load nodes are paused, the task data and intermediate states are packaged and automatically transferred to idle computing power nodes, and then the task execution is resumed on the new node without interrupting the monitoring and identification process. The dynamic allocation mechanism prioritizes computing power for core tasks such as real-time intrusion identification and early warning, while allocating non-real-time tasks such as model training and data archiving to idle computing resources, and automatically reserving basic computing power during model iteration.

[0046] This embodiment provides a factory hazardous area monitoring and storage and intrusion behavior recognition and early warning system, including a perception preprocessing module, an intrusion recognition module, an event description generation module, a hierarchical storage and evidence preservation module, an early warning linkage module, a blind spot trajectory continuation module, and an edge computing power and model update module.

[0047] Each module is deployed in the factory's edge computing nodes, regional gateways, and management servers. Among them, the perception preprocessing module, intrusion identification module, event description generation module, early warning linkage module, and blind spot trajectory continuation module are preferentially deployed on the edge computing node side to ensure that intrusion events are identified and handled locally. The hierarchical storage and evidence preservation module can be jointly implemented by the edge computing nodes, the factory's local storage server, and offline archiving storage devices. The edge computing power and model update module is used to schedule resources for the above processing tasks and complete model iteration without uploading the original video data.

[0048] In this embodiment, the modules are not simply set up side by side. Instead, the intrusion event descriptor output by the event description generation module is used as a unified data index. The perception preprocessing module first generates candidate intrusion data, and the intrusion identification module then outputs the intrusion identification result based on the candidate intrusion data. The event description generation module writes the intrusion identification result, target trajectory, area risk level, authorization status, evidence data index, and early warning linkage status into the intrusion event descriptor. Subsequent hierarchical storage and evidence preservation, early warning linkage, blind zone trajectory continuation, and computing power scheduling all call the intrusion event descriptor for execution. In this way, isolated identification of video images is avoided, as well as the problem of lack of correlation between storage, evidence preservation, and alarm.

[0049] The perception preprocessing module connects to video acquisition equipment, infrared thermal imaging equipment, millimeter-wave radar, and vibration sensors. The video acquisition equipment is used to acquire visible light video images of hazardous areas in the factory area. The infrared thermal imaging equipment is used to acquire the thermal characteristics of the target. The millimeter-wave radar is used to acquire the target's distance, speed, and direction information. The vibration sensors are used to acquire vibration signals near walls, fences, the ground, or equipment foundations. The installation location, monitoring angle, and detection range of the above equipment are determined based on the factory area equipment layout diagram, hazardous area boundary diagram, and on-site installation and debugging records, and are written into the factory area 3D scene map.

[0050] The perception preprocessing module uses the system clock of the edge computing node as a unified time reference to add a unified timestamp to video frames, infrared thermal imaging frames, radar detection frames, and vibration sampling data. For cases where the sampling frequencies of different devices are inconsistent, the perception preprocessing module uses the video frame timestamp as the main time axis to map the infrared thermal imaging data, millimeter-wave radar data, and vibration sensing data into the corresponding video frame time window. The length of this time window is not a fixed, fabricated value, but is calibrated during the system initialization phase based on the sampling frequency of the field devices, network transmission latency, and edge computing node cache cycle, and stored as a system configuration parameter.

[0051] The perception preprocessing module also maps the data collected by different sensing devices to the same factory coordinate system based on the installation coordinates, monitoring field of view, detection sector, and coverage of each sensing device in the 3D scene map of the factory area. For example, the center point of the target frame in the video image is mapped to the plane coordinates of the factory area after the camera's internal and external parameters are converted; the distance and angle data of the millimeter-wave radar are converted to the detection points under the same factory coordinate system; the installation point and trigger area of ​​the vibration sensor are used as candidate areas for vibration sources. After spatial coordinate mapping, the system can determine whether the video target, radar target, and vibration trigger originate from the same actual target, thereby reducing false triggering by a single sensor.

[0052] During scene adaptive processing, the perception preprocessing module reads data from the light sensor, weather interface, video image brightness distribution, image noise level, and vegetation area movement amplitude to determine whether there is strong light, weak light, rain or fog, insufficient nighttime lighting, swaying vegetation, or disturbance from small animals. The judgment results are used to adjust video exposure compensation, contrast, noise reduction intensity, millimeter-wave radar detection sensitivity, and vibration sensor trigger threshold.

[0053] When the perception preprocessing module performs interference filtering, it first identifies the moving area in the video frame and then determines whether the moving area contains the outline of a person or vehicle. At the same time, it reads whether the millimeter-wave radar detects a moving target in the corresponding direction and distance range, and reads whether the vibration sensor forms a valid trigger at an adjacent point. If the moving area in the video only corresponds to vegetation swaying, light and shadow changes, or rain and snow reflection, and the radar and vibration data do not match, then the data is marked as environmental interference data. If the video target position, radar detection position, or vibration trigger position match each other in the time window and spatial region, then the data is retained as candidate intrusion data. Candidate intrusion data includes candidate video clips, candidate target boxes, target appearance time, target location coordinates, infrared temperature characteristics, radar distance characteristics, radar speed characteristics, and vibration amplitude characteristics.

[0054] After receiving candidate intrusion data, the intrusion detection module performs target detection and trajectory extraction on the video frames. Target detection outputs the target category, target bounding box position, and target size. The target category includes people, vehicles, and non-target interference objects. Trajectory extraction generates the target trajectory number, movement direction, movement speed, dwell time, and boundary crossing status based on the target position changes in consecutive frames. For the same target moving between adjacent video acquisition devices, the intrusion detection module matches the target's appearance features, target size, movement direction, and entry time to maintain the continuity of the target trajectory number.

[0055] The intrusion detection module includes a spatial feature extraction branch, a temporal feature extraction branch, and a sensor feature fusion branch. The spatial feature extraction branch is used to extract the contour, posture, size, and location of people or vehicles. The temporal feature extraction branch is used to analyze the target's motion direction, speed, dwell time, and boundary crossing process in consecutive video frames. The sensor feature fusion branch is used to bind infrared thermal features, millimeter-wave radar distance features, millimeter-wave radar velocity features, and vibration amplitude features to the same target trajectory number. The output results of the above three branches are used together to determine the target behavior category, avoiding misjudgment caused by simply relying on a general video detection model.

[0056] The behavior categories in this embodiment include normal passage, lingering at the boundary, climbing over a wall, crawling through gaps, crawling through, vehicle crossing the boundary, and illegal parking. Normal passage refers to the target being within the authorized area and authorized time range, and the trajectory conforming to the preset passage path; lingering at the boundary refers to the target continuously staying near the boundary of the dangerous area but not entering the interior of the dangerous area; climbing over a wall refers to the target's trajectory crossing the boundary of a wall or fence, and the target's height, posture, and direction of movement conforming to the characteristics of climbing behavior; crawling through gaps refers to the target passing through the boundary in a low posture near a fence, pipe gallery, or door gap; crawling through refers to the height change and movement speed of the personnel target conforming to the characteristics of low posture movement; vehicle crossing the boundary refers to the vehicle target entering an unauthorized area; and illegal parking refers to the vehicle continuously staying at the entrance / exit of the dangerous area, fire lane, or around high-risk equipment.

[0057] The intrusion detection module does not directly identify all moving targets as intrusion events. Instead, it makes a secondary judgment based on the area risk level and the target's access authorization status. The area risk level is determined by the plant's safety management system, hazard source distribution map, equipment layout map, and access control rules, and is divided into at least low-risk, medium-risk, and high-risk areas. The target's access authorization status can be provided by the access control system, personnel positioning tags, visitor registration records, vehicle authorization records, or duty management system. When a target has an authorized status and its trajectory is within the authorized area, the system identifies it as normal passage. When a target does not have an authorized status, or its trajectory enters an unauthorized area, or its behavior is classified as climbing over walls, crawling through gaps, crawling through, or vehicles crossing boundaries, the system identifies it as an intrusion event.

[0058] The event description generation module generates an intrusion event descriptor based on the intrusion identification result. The intrusion event descriptor is an internal data structure used by the system for association identification, storage, early warning, evidence preservation, and trajectory continuation. It includes at least the following fields: event number, event timestamp, target trajectory number, area number, behavior category, danger level, evidence data index, hash chain index, and linkage status field. Among them, the event number is used to uniquely identify an intrusion event; the target trajectory number is used to associate the continuous trajectory of the same target on different devices; the area number is used to determine the location of the event and the area risk level; the behavior category is used to determine the intrusion method; the danger level is used to determine the early warning level and storage strategy; the evidence data index is used to point to video clips, infrared thermal imaging data, radar detection data, vibration sensing data, trajectory records, and handling logs; the hash chain index is used to associate the hash verification chain of the corresponding event data; and the linkage status field is used to record the execution status of early warning, access control, audible and visual alarms, and security handling requests.

[0059] The hazard level is determined by the risk rule table, which consists of the area risk level, behavior category, target access authorization status, number of targets, and dwell time interval. Low-risk events typically correspond to a single person accidentally entering a low-risk area, brief stay at the boundary, or a vehicle briefly crossing the boundary. Medium-risk events typically correspond to personnel entering a medium-risk area, vehicles illegally parking at the entrance or exit of a hazardous area, or personnel repeatedly lingering at the perimeter boundary. High-risk events typically correspond to climbing over the perimeter wall, crawling through gaps, crawling through, multiple people entering a high-risk area, or approaching the tank area or high-risk equipment area. The area risk level, dwell time interval, and authorization conditions in the risk rule table are all derived from the plant's safety management system and the event review results during the trial operation. The system supports managers to configure and update the rules according to the actual control requirements of the plant.

[0060] The hierarchical storage and evidence preservation module determines the data storage strategy based on the intrusion event descriptor. This module includes a first-level storage layer, a second-level storage layer, and a third-level storage layer. The first-level storage layer uses solid-state storage media to cache real-time video streams, candidate intrusion data, and event segments that have triggered alerts, in order to meet the needs of fast retrieval and instant playback. The second-level storage layer uses a disk array to store routine monitoring data and low-risk event data that have not triggered alerts. The third-level storage layer uses offline archiving media or dedicated archiving storage space to store the original video, sensor data, trajectory data, alert handling records, and hash verification chains of medium-risk event data and high-risk event data.

[0061] The tiered storage and evidence preservation module does not mechanically store data according to a fixed recording cycle. Instead, it performs differentiated storage based on the danger level, area number, and evidence data index in the intrusion event descriptor. For routine monitoring data without anomalies, the system stores data according to a preset cycle. Upon expiration, it performs batch cleanup after confirming that no intrusion event descriptor is associated with the data, and retains the cleanup log. For low-risk events, the system saves the corresponding video clips, trajectory records, and basic handling logs. For medium-risk and high-risk events, the system saves the original video, infrared thermal imaging data, radar detection data, vibration sensing data, trajectory records, early warning handling records, and event reports before and after the event. The retention time of the video before and after the event is determined by the plant's safety regulations and on-site supervision requirements, and is not a fictitious fixed value in this embodiment.

[0062] For medium-risk and high-risk event data, the hierarchical storage and evidence preservation module generates a chain of hash verifications that are linked together. Specifically, the system generates data fingerprints according to the event data segments. The hash value of the current data segment is calculated by the hash value of the previous data segment, the data fingerprint of the current data segment, the event timestamp, the evidence data index, and the storage address. The hash value of the first data segment is calculated by the event number, the event timestamp, and the data fingerprint of the first data segment. Subsequent data segments are sequentially linked to the hash values ​​of the previous data segments to form a chain verification structure that is consistent with the order of the event data. If any data segment is replaced, deleted, or modified, the subsequent hash values ​​will fail the verification, thus revealing whether the event evidence data has been destroyed.

[0063] The hash verification chain is stored separately from the original event data. The original event data is stored in the corresponding storage layer, while the hash verification chain, event descriptor, and audit log are stored in an independent evidence storage partition. The independent evidence storage partition uses an append-only write method for recording. All system operations on retrieving, copying, migrating, archiving, and verifying event data are written to the audit log. The audit log includes the operation account, operation time, event number, data index, operation type, verification result, and storage location change record. Through the above structure, the system can not only save event videos but also prove the integrity of event data during storage, migration, and retrieval.

[0064] The early warning linkage module calls the early warning rule table and linkage control table based on the intrusion event descriptor. The early warning rule table is used to determine general early warnings, moderate early warnings, and severe early warnings. The linkage control table is used to record the corresponding handling actions, execution equipment, execution sequence, and confirmation conditions for different early warning levels. General early warnings correspond to pop-up prompts on the background monitoring terminal, real-time video display, and event log recording; moderate early warnings correspond to pop-up prompts on the background monitoring terminal, on-site audible and visual alarms, and push notifications from the security terminal; severe early warnings correspond to pop-up prompts on the background monitoring terminal, on-site audible and visual alarms, the access control controller locking the entrances and exits of dangerous areas, and sending a high-risk equipment safety handling request to the safety interlock controller.

[0065] For severe warnings, the system does not directly bypass the plant's safety interlock logic to forcibly cut off the power to the production equipment. Instead, it sends a safety action request to the safety interlock controller, which includes the event number, area number, hazard level, target location, and action request type. The safety interlock controller decides whether to execute shutdown, power outage, interlock, or other safety action based on the plant's existing safety interlock logic, equipment operating status, and manual confirmation conditions. This setting can avoid unplanned shutdowns of production equipment due to misjudgment from a single identification result, and it also meets the actual requirement that the handling of high-risk equipment in the plant needs to be confirmed by the safety interlock logic.

[0066] When executing each linkage action, the early warning linkage module writes the execution result into the linkage status field of the intrusion event descriptor and simultaneously writes it into the event audit log. If any action of audible and visual alarm, access control locking, security terminal push, or security handling request fails, the system records the failed device number and the reason for failure, and displays a prompt that manual handling is required on the background monitoring terminal. If the network between the edge computing node and the management server is interrupted, the edge computing node still executes the core early warning and local linkage handling according to the locally cached early warning rule table, and retransmits the event descriptor, event data, and linkage log after the network is restored.

[0067] The blind spot trajectory continuation module is used to solve the problem of trajectory interruption when the target enters the monitoring blind spot. This module uses video acquisition equipment, infrared thermal imaging equipment, millimeter-wave radar and vibration sensors as topology nodes. It generates topology edge weights based on the equipment installation coordinates, monitoring coverage overlap range, communication link status and regional risk level. Equipment with adjacent monitoring ranges, overlapping coverage areas or located in the direction the target may travel is set to higher association weights; equipment with unstable communication links, equipment failure or irrelevant coverage areas is set to lower association weights.

[0068] When an intruding target disappears from the monitoring screen of a video acquisition device, and the location where the target disappears is within the boundary of the monitoring blind zone marked by the 3D scene map of the factory area, the blind zone trajectory continuation module reads the target's movement direction, movement speed, target trajectory number, target appearance features, radar detection point, and vibration trigger point before entering the blind zone, and generates the target's position probability distribution and expected departure direction within the blind zone. The position probability distribution does not require precise unique coordinates of the target, but rather provides a probability ranking of the target's possible location in different areas within the blind zone, which is used to guide adjacent video acquisition devices to adjust their monitoring direction in advance or increase the recognition priority of the corresponding area.

[0069] When a target re-enters the visible monitoring range, the blind zone trajectory continuation module matches the appearance, size, direction of movement, time of appearance, and location of the newly appearing target with the trajectory characteristics of the target before entering the blind zone. If the matching result meets the continuation conditions set by the system, the original target trajectory number is used, and the position probability distribution, dwell time, and continuation result within the blind zone are written into the intrusion event descriptor. If the matching result does not meet the continuation conditions, a new target trajectory number is generated, and the original trajectory is marked as a blind zone loss state. The continuation conditions are determined by the trajectory verification data and equipment coverage relationship during the on-site trial operation.

[0070] The edge computing power and model update module ensures that intrusion detection, early warning linkage, and blind spot trajectory continuation can still operate with priority even under computing power constraints. This module divides business tasks into real-time tasks and non-real-time tasks. Real-time tasks include video decoding, candidate intrusion data generation, intrusion detection, hazard level determination, early warning linkage, and blind spot trajectory continuation. Non-real-time tasks include general data archiving, audit log organization, model training, and historical data statistics. The edge computing power and model update module allocates video decoding and basic preprocessing to the CPU according to task type, behavior recognition inference and trajectory analysis to GPU or AI acceleration chips, and data archiving and log organization to storage servers or low-priority computing resources.

[0071] The edge computing power and model update module continuously reads CPU utilization, GPU utilization, AI acceleration chip inference latency, memory utilization, storage write queue length, and network transmission status. When any real-time task experiences queuing delay, or when the GPU or AI acceleration chip load exceeds the system's configured threshold, the system pauses or postpones non-real-time tasks, releasing computing resources for intrusion detection, early warning linkage, and blind spot trajectory continuation. If an edge computing node remains under high load, the system will transfer transferable non-real-time tasks to adjacent low-load nodes or regional gateways for execution; for ongoing real-time tasks, the system only migrates their subsequent processing queues to avoid interrupting the handling process of already triggered early warning events.

[0072] Model updates are performed using edge-local training and parameter upload. Edge computing nodes incrementally train the recognition model using locally verified intrusion event samples, false alarm samples, and normal passage samples. During training, edge computing nodes do not upload raw video data, infrared thermal imaging data, radar detection data, or vibration sensing data; they only upload encrypted model parameters or gradient parameters. The regional gateway performs integrity verification, abnormal parameter removal, and aggregation updates on the parameters uploaded by multiple edge computing nodes and distributes the updated model parameters to the edge computing nodes. After receiving the new model parameters, the edge computing nodes first verify them on their local verification sample set. If the verification is successful, the online recognition model is replaced; if the verification fails, the original model is retained, and a model update failure log is recorded.

[0073] To prevent model updates from impacting real-time security operations, the edge computing power and model update module incorporates business protection logic during model training and replacement. When intrusion detection, early warning linkage, or blind spot trajectory continuation tasks are under high load, model training tasks are automatically downgraded or paused. When new model parameters are ready to be replaced, the system retains the previous version model as a fallback model and automatically restores the previous version model if the new model malfunctions. These measures ensure that model iteration is isolated from the factory's real-time security operations, preventing model training from consuming core computing power and causing early warning delays.

[0074] This embodiment also includes a device health status management mechanism. The system periodically checks the image clarity, frame rate, and signal stability of the video acquisition device, the imaging status of the infrared thermal imaging device, the data transmission delay and detection status of the millimeter-wave radar, and the triggering and sampling status of the vibration sensor. If a device malfunctions, the system marks the faulty device in the 3D scene map of the factory area and adjusts the topology weights and data fusion strategies of adjacent devices. If the video acquisition device malfunctions, the system increases the data weights of adjacent video acquisition devices, millimeter-wave radar, and vibration sensors. If the radar or vibration sensor malfunctions, the system retains the video and infrared thermal imaging recognition results and marks the incomplete perception data status in the early warning event report.

[0075] During a specific operation, when a person approached the fence of the chemical plant's tank area, the video acquisition equipment detected a moving target near the fence boundary, the millimeter-wave radar detected the moving target in the corresponding direction, and the fence vibration sensor generated a trigger signal. The perception preprocessing module mapped the above data to the same time window and the same plant area coordinate system, and generated candidate intrusion data after removing interference from light and shadow and vegetation swaying. The intrusion identification module identified the intrusion as a fence-climbing behavior based on the target's posture, direction of movement, and boundary crossing process. Combining the high-risk area attribute of the tank area and the target's unauthorized status, it determined it to be a high-risk event. The event description generation module generated an intrusion event descriptor, writing the event number, target trajectory number, behavior category, hazard level, evidence data index, and linkage status fields into the system.

[0076] Subsequently, the hierarchical storage and evidence preservation module saves the original video, infrared thermal imaging data, radar detection data, vibration sensing data, and trajectory records before and after the intrusion event descriptor, and generates a corresponding hash verification chain. The early warning linkage module triggers a severe warning based on high-risk events, pushes real-time images and event locations to the backend monitoring terminal, activates on-site audible and visual alarms, controls the access control system to lock the entrances and exits of the storage tank area, and sends a safety handling request to the safety interlock controller. If the target enters the video surveillance blind spot after climbing over, the blind spot trajectory continuation module generates a position probability distribution based on the target's movement direction before entering the blind spot, radar detection points, and vibration trigger points, and notifies adjacent video acquisition devices to increase the recognition priority of the corresponding area. After the target reappears in the adjacent video frame, the system completes trajectory continuation based on appearance characteristics, appearance time, and movement direction, and writes the continuation result into the same intrusion event descriptor.

[0077] Through the above implementation methods, multimodal perception, intrusion identification, event description, hierarchical storage, hash-based evidence storage, hierarchical early warning, blind spot continuation, and computing power scheduling can be formed into a closed loop.

[0078] Unlike systems that trigger alarms solely based on video motion detection, this system does not simply determine whether there is a moving target in the frame; unlike systems that only store video data using hashes, it binds the hash chain to intrusion event descriptors, target trajectories, early warning and response records, and evidence data indexes; unlike systems that only perform centralized cloud analysis, it completes core identification and coordinated response at the edge computing node level, and does not upload original on-site data when updating the model. Therefore, those skilled in the art can implement the technical solution of this application based on the module connection relationships, data processing flow, rule sources, storage and evidence preservation methods, and coordinated response logic disclosed in this embodiment.

[0079] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0080] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A plant dangerous area monitoring storage and intrusion behavior intelligent identification early warning system, characterized in that, Includes the following modules: The perception preprocessing module is deployed at the edge computing node in the hazardous area of ​​the factory. It connects video acquisition equipment, infrared thermal imaging equipment, millimeter-wave radar and vibration sensors. It is used to perform unified timestamp calibration, spatial coordinate mapping and interference data filtering on video data, infrared thermal imaging data, radar detection data and vibration sensing data to generate candidate intrusion data. An intrusion identification module, connected to the perception preprocessing module, is used to perform target detection, target trajectory extraction and behavior category identification on the candidate intrusion data, and output the intrusion identification result. The event description generation module, connected to the intrusion identification module, is used to generate an intrusion event descriptor based on the intrusion identification result, the regional risk level of the target's location, and the target's access authorization status. The intrusion event descriptor includes an event number, an event timestamp, a target trajectory number, a region number, a behavior category, a danger level, an evidence data index, a hash chain index, and a linkage status field. A hierarchical storage and evidence preservation module, connected to the event description generation module, is used to determine the storage level, storage duration and number of copies of the corresponding event data based on the intrusion event descriptor, and to generate a hash verification chain that links the medium-risk event data and the high-risk event data. The early warning linkage module is connected to the event description generation module. It is used to call the early warning rule table according to the intrusion event descriptor, generate a general early warning, a more serious early warning or a severe early warning, and send the corresponding linkage control instructions to the background monitoring terminal, the sound and light alarm device, the access control controller and the security interlock controller. The blind spot trajectory continuation module is connected to the event description generation module. It is used to estimate the position of the target entering the monitoring blind spot based on the target trajectory number, the three-dimensional scene map of the factory area and the topological relationship of adjacent sensing devices, and to complete the trajectory continuation after the target re-enters the visible monitoring range. The edge computing power and model update module is deployed on the edge computing node. It is used to allocate video decoding, data preprocessing, behavior recognition, trajectory continuation, early warning linkage, data archiving and model training tasks to CPU, GPU and AI acceleration chip according to task priority, and complete the parameter update of the recognition model without uploading the original video data.

2. The intelligent identification and early warning system for monitoring, storing, and identifying intrusion behavior in hazardous areas of a factory, as described in claim 1, is characterized in that... The perception preprocessing module uses the system clock of the edge computing node as a unified time reference to map infrared thermal imaging data, radar detection data, and vibration sensing data to the same timeline of the video image. Based on the installation coordinates, monitoring field of view, and detection range of each sensing device in the 3D scene map of the factory area, the data collected by different sensing devices are mapped to the same factory area coordinate system.

3. The intelligent identification and early warning system for monitoring, storing, and identifying intrusion behavior in hazardous areas of a factory, as described in claim 2, is characterized in that... The perception preprocessing module adjusts the video image exposure parameters, noise reduction parameters, millimeter-wave radar detection sensitivity, and vibration sensor trigger threshold according to light intensity, rainfall status, fog status, and vegetation swaying status. When the target position in the video image matches the millimeter-wave radar detection position or vibration sensor trigger position within the time window and spatial region, the corresponding data is retained as candidate intrusion data; otherwise, the corresponding data is marked as environmental interference data.

4. The intelligent identification and early warning system for monitoring, storing, and identifying intrusion behavior in hazardous areas of a factory, as described in claim 3, is characterized in that... The intrusion detection module includes a spatial feature extraction branch, a temporal feature extraction branch, and a sensor feature fusion branch; The spatial feature extraction branch is used to extract the target contour, target size, and target location from consecutive video frames; The temporal feature extraction branch is used to generate motion direction, movement speed and dwell time based on the target position changes in consecutive video frames; The sensor feature fusion branch is used to bind infrared temperature features, radar distance features, and vibration amplitude features to the target location; The intrusion detection module outputs the behavior category and danger level based on the fusion result of spatial features, temporal features and sensor features.

5. The intelligent identification and early warning system for monitoring, storing, and identifying intrusion behavior in hazardous areas of a factory, as described in claim 4, is characterized in that... The behavior categories include normal passage, lingering at the boundary, climbing over walls, crawling through gaps, crawling through, vehicles crossing boundaries, and illegal parking. The danger level is determined by a risk rule table, which includes the area risk level, behavior category, target passage authorization status, number of targets, and time interval of stay. The event description generation module classifies intrusion events into low-risk events, medium-risk events, and high-risk events according to the risk rule table.

6. The intelligent identification and early warning system for monitoring, storing, and identifying intrusion behavior in hazardous areas of a factory, as described in claim 5, is characterized in that... The hierarchical storage and evidence preservation module includes a primary storage layer, a secondary storage layer, and a tertiary storage layer; The primary storage layer uses solid-state storage media to cache real-time video streams, candidate intrusion data, and event segments that have triggered warnings. The secondary storage layer uses a disk array to store routine monitoring data and low-risk event data that have not triggered warnings. The third-level storage layer uses offline archiving media or dedicated archiving storage space to store the original video, sensor data, trajectory data, early warning and handling records, and hash verification chains of medium-risk event data and high-risk event data; Before medium-risk and high-risk event data are written to the third-level storage layer and verified, event copies are retained in the first-level storage layer.

7. The intelligent identification and early warning system for monitoring, storing, and identifying intrusion behavior in hazardous areas of a factory, as described in claim 6, is characterized in that... The hash verification chain is generated according to the event data segments. The hash value of the current data segment is calculated from the hash value of the previous data segment, the data fingerprint of the current data segment, the event timestamp, the evidence data index, and the storage address. When the event data is retrieved, copied, migrated, or archived, the hierarchical storage and evidence preservation module verifies the hash verification chain segment by segment and writes the verification results into the event audit log.

8. The intelligent identification and early warning system for monitoring, storing, and identifying intrusion behavior in hazardous areas of a factory, as described in claim 7, is characterized in that... The early warning linkage module includes an early warning rule table and a linkage control table; Generally, early warnings are accompanied by pop-up notifications on the backend monitoring terminal and event log recordings. More severe warnings are responded to by pop-up notifications on the backend monitoring terminal, on-site audible and visual alarms, and push notifications from security terminals. A severe warning will trigger a pop-up notification on the backend monitoring terminal, an on-site audible and visual alarm, and the access control controller will lock the entrances and exits of the dangerous area. It will also send a request for safe handling of high-risk equipment to the safety interlock controller. The safety handling request is executed after manual confirmation or confirmation by the plant area safety interlock logic.

9. The intelligent identification and early warning system for monitoring, storing, and identifying intrusion behavior in hazardous areas of a factory, as described in claim 8, is characterized in that... The blind spot trajectory continuation module uses video acquisition equipment, infrared thermal imaging equipment, millimeter-wave radar and vibration sensors as topology nodes, and generates topology edge weights based on equipment installation coordinates, monitoring coverage overlap range, communication link status and regional risk level. When a target enters a monitoring blind zone, the blind zone trajectory continuation module generates the target's position probability distribution and expected departure direction within the blind zone based on the target's movement direction, speed, trajectory number, and real-time detection results from adjacent sensing devices before entering the blind zone, and writes the position probability distribution and expected departure direction into the intrusion event descriptor.

10. A factory area hazardous area monitoring and storage and intrusion behavior intelligent identification and early warning system according to claim 9, characterized in that, The edge computing power and model update module sets intrusion recognition, early warning linkage, and blind spot trajectory continuation as real-time tasks, and data archiving, audit log organization, and model training as non-real-time tasks. When the load of the CPU, GPU, or AI acceleration chip exceeds the corresponding load threshold, the non-real-time tasks are paused or postponed, and the real-time tasks are given priority. The parameter update of the recognition model is performed by edge local training and parameter upload. The edge computing node uploads encrypted model parameters or gradient parameters, but does not upload original video data, infrared thermal imaging data, radar detection data, and vibration sensing data.

Images