A multi-modal community security sensing data aggregation and early warning system

Abstract

The application provides a kind of multimodal community security sensing data convergence and early warning system, it is related to Internet of Things and intelligent security technology field.The system includes data acquisition layer, edge computing layer, fog computing convergence early warning layer and blockchain storage module.Data acquisition layer deploys multiple heterogeneous sensors;Edge computing layer carries out local preprocessing and feature extraction;Fog computing convergence early warning layer integrates multimodal features through adaptive weighted fusion algorithm, uses risk situation map to carry out dynamic analysis and generates graded warning;The blockchain storage module synchronously stores the reliable evidence of the early warning event.The application realizes the efficient fusion and intelligent analysis of multi-source security data through end-edge-fog-chain collaborative architecture, significantly improves the accuracy of community risk early warning, response timeliness and the reliability of post-event evidence.

Description

Technical Field

[0001] This invention relates to the fields of Internet of Things and smart security technology, and in particular to a multimodal community security sensor data aggregation and early warning system. Background Technology

[0002] The construction of modern urban community security systems increasingly relies on IoT sensing devices. Currently, various security sensing subsystems are independently built by different manufacturers. Video surveillance, perimeter alarms, and access control systems each use different data formats and communication protocols, creating isolated data silos. This prevents unified aggregation and correlation analysis of various sensor data. Security personnel must operate multiple unconnected control interfaces simultaneously, making it difficult to grasp the overall security situation of the community from a holistic perspective and to comprehensively analyze various security information, thus creating blind spots in security management.

[0003] Current community security early warning systems largely rely on fixed threshold triggering mechanisms from single sensors. They lack intelligent perception of environmental context and fail to achieve multi-source data fusion and verification. This makes them susceptible to environmental interference, equipment malfunctions, and triggering by small animals, resulting in numerous false alarms. A large number of invalid alarms not only consume significant security personnel and resources but also reduce staff's alertness to genuine alarm signals, creating a "crying wolf" effect. Furthermore, single sensors have limited sensing capabilities, insufficient ability to identify complex security events such as loitering, tailing, group conflicts, falls, and fainting, resulting in significant blind spots and a lag in response to sudden risks.

[0004] When a security alarm is triggered in the community, the existing system's response process heavily relies on manual operation. Alarm information is only presented as a basic audio-visual alert at the duty center, requiring staff to manually retrieve video and data from multiple devices for verification before notifying on-site personnel via traditional communication methods such as telephones and walkie-talkies. This process involves numerous intermediate steps, resulting in low information transmission efficiency and making it difficult to seize the optimal response time in time-sensitive emergency scenarios. Furthermore, the system lacks intelligent contingency plan matching and resource scheduling capabilities, failing to quickly activate corresponding standardized response procedures based on the nature, severity, and scope of the incident.

[0005] Furthermore, information such as sensor data, alarm logs, and response records from community security systems are all stored on centralized servers or local storage on devices. The storage and management of this data lacks effective anti-tampering mechanisms, posing a technical risk of manipulation by internal personnel or unauthorized alteration or deletion through system vulnerabilities. In the event of a security incident or related case, the originality, integrity, and non-repudiation of this data as electronic evidence cannot be effectively guaranteed, failing to meet the stringent requirements of the judiciary for electronic evidence. This constitutes a key technical weakness in the closed-loop management of community security. Summary of the Invention

[0006] To address the technical problems in existing community security technologies, such as isolated data from various subsystems that cannot be uniformly aggregated and analyzed, high false alarm rates from single-sensor warnings and insufficient ability to identify complex events, reliance on manual handling of alarms leading to low response efficiency, and centralized storage of security data that is easily tampered with and lacks credibility of electronic evidence, this invention provides a multimodal community security sensor data aggregation and early warning system.

[0007] The technical solution provided by this invention is as follows: This invention provides a multimodal community security sensor data aggregation and early warning system, comprising: Data acquisition layer, edge computing layer, fog computing convergence and early warning layer, and blockchain evidence storage module; The data acquisition layer is deployed in the community's public areas and unit entrances, and includes multiple modal sensing units for collecting heterogeneous security data; The edge computing layer is communicatively connected to the data acquisition layer and includes multiple edge computing nodes. Each edge computing node is used to connect to one or more geographically adjacent modal sensing units to preprocess and extract features from the raw sensing data and output a standardized feature vector. The fog computing convergence and early warning layer is deployed in a community data center or regional center and communicates with all edge computing nodes, including: The data aggregation and fusion module is used to receive multimodal feature vectors from different edge computing nodes, and to perform cross-modal feature fusion using an adaptive weighted fusion algorithm to generate a unified fused feature representation. The risk situation analysis engine is used to perform real-time analysis based on the fused feature representation, call the preset risk identification model, and construct a dynamic community risk situation map. The graded early warning and contingency plan execution module is used to generate graded early warning instructions based on the risk level and impact range of the risk situation map, and automatically trigger or recommend corresponding contingency plans. The blockchain evidence storage module, embedded in the fog computing convergence early warning layer, is used to generate evidence storage transactions for key feature information of the early warning event and upload them to the blockchain network for trusted evidence storage at the same time as the tiered early warning instruction is generated.

[0008] Furthermore, the plurality of modal sensing units include at least three of the following: a visible light camera, a thermal imaging infrared camera, a wide-area acoustic sensor, a millimeter-wave radar sensor, an intelligent access control terminal, and a perimeter vibration fiber optic sensor.

[0009] Furthermore, the edge computing node's preprocessing and feature extraction of the raw sensing data specifically includes: For video stream data, extract key points of human skeleton, motion trajectory, and feature vectors of abnormal behavior; For acoustic data, extract abnormal voiceprint features and feature vectors of specific frequency events; For radar data, extract the trajectory of moving targets and the feature vectors of subtle vital signs. The extracted modal feature vectors are then time-stamped and spatially calibrated.

[0010] Furthermore, the adaptive weighted fusion algorithm is specifically as follows: Given a set of feature vectors for N modalities after time alignment. ; Assign a dynamic weight to each mode i Weight The context-aware module determines the modality based on the current environmental state S and the real-time signal-to-noise ratio of the modality data. Together, we can calculate that the calculation function is: ,in is the environmental state encoding function, g is the weight calculation function, and the current environmental state S includes time, illumination, and weather; Fusion features The calculation formula is: ,in Pre-trained feature transformation matrices for each modality; The context-aware module is implemented using a lightweight neural network, whose input is an environment state code. The output is the initial weights for each mode. Then, fine-tune based on the real-time signal-to-noise ratio: .

[0011] Alternatively, as an alternative implementation, the context-aware module can also be implemented based on preset expert rules or lookup tables. These expert rules directly map different environmental states S (such as "night + rainy day") to a set of preset initial weight combinations based on historical experience, without requiring real-time calculations through a neural network. This implementation is suitable for scenarios with extremely limited computing resources or extremely high response speed requirements.

[0012] Furthermore, the method by which the risk situation analysis engine constructs a dynamic community risk situation map is as follows: Each identified potential risk event is abstracted as a graph node v, with node attributes including event type, location, time, and confidence level. ; The spatiotemporal and causal relationships between events are abstracted into graph edges. Its weight Indicates the strength of the association; The graph is updated and inferred in real time using a graph neural network. The update formula for each node is as follows: ; in Let v be the feature representation of node v at layer l. Let v be the set of neighbors of node v. For trainable parameters, For activation functions; By calculating node importance scores and community risk indices ,in For a set of nodes, The importance score of node v is used to identify core risk nodes, risk transmission paths, and potential risk convergence areas.

[0013] It should be noted that the above-described aggregation method based on concatenation (CONCAT) and summation is only one implementation of the graph neural network. In other embodiments of the present invention, the graph neural network may also employ other aggregation functions, such as weighted summation of neighbor node features, application of graph attention mechanism (GAT) to dynamically assign attention coefficients to different neighbors, or the use of gating mechanisms, to achieve more complex reasoning capabilities.

[0014] Furthermore, the tiered early warning and contingency plan execution module classifies the early warning level into: alert level, based on the overall community risk index R and the impact range of the core risk nodes. On-site handling level Community-level collaboration , Report from higher authorities ; Different warning levels correspond to different information notification scopes, emergency response plan databases, and resource allocation strategies.

[0015] Furthermore, the blockchain evidence storage module specifically performs the following steps: Receive early warning event trigger signals and associated fused feature representations ; Extract the immutable feature information of the event and generate a structured evidence storage data packet D; Use community private key Sign the evidence storage data packet: The signed and stored data packet is then broadcast to a consortium blockchain network consisting of multiple community nodes or management agency nodes. After the consortium blockchain network reaches a consensus, it will store the hash value of the evidence data packet. And metadata records are recorded on the blockchain.

[0016] In a preferred embodiment of the present invention, the blockchain network adopts a consortium blockchain architecture and uses a practical Byzantine Fault Tolerance (PBFT) consensus algorithm to balance storage efficiency and security. However, those skilled in the art will understand that the present invention is not limited thereto. Depending on the actual size of the community, the degree of mutual trust between nodes, and performance requirements, other types of blockchain networks, such as private blockchains, or other consensus mechanisms, such as Proof-of-Authority (PoA) and Raft, can also be used, as long as they can achieve tamper-proof data storage.

[0017] Furthermore, the fog computing convergence early warning layer also includes a continuous learning and optimization module, which is used for: Collect feedback results on the handling of historical early warning events. ; Based on the feedback results, the parameters of the risk identification model and the adaptive weighted fusion algorithm are adjusted. Incremental learning and optimization are performed, with the optimization objective being to minimize the loss function: ,in For the model's predicted output, The regularization coefficient is used. Optimized model parameters Securely synchronize to each edge computing node.

[0018] Furthermore, the system also includes a visual interactive terminal for: The dynamic community risk situation map and risk heat map of each area are displayed in real time. Receive and display the tiered early warning instructions and response plans; It provides an interface for querying evidence storage information, allowing users to retrieve and verify corresponding trusted evidence storage records from the blockchain network by inputting an event number or time range.

[0019] Furthermore, encrypted communication links are used for data transmission between the edge computing node and the fog computing convergence and early warning layer, as well as between the fog computing convergence and early warning layer and the blockchain evidence storage module.

[0020] The beneficial effects of the technical solution provided by this invention include at least the following: (1) In this invention, by constructing a three-level collaborative processing architecture of end-edge-fog, sensor data of different modalities can be standardized and preprocessed and feature extracted at the edge. Then, by using the context-aware adaptive weighted fusion algorithm of the fog computing layer, the fusion weight is dynamically adjusted in combination with the environmental status and real-time data quality, realizing the efficient aggregation and intelligent fusion of multi-source security data, breaking down the data barriers of each subsystem, forming a global community security view, and effectively reducing the false alarm problem caused by a single sensor through cross-verification of multi-source information, thereby improving the comprehensive identification capability of complex risk events.

[0021] (2) In this invention, a dynamic community risk situation map is constructed by introducing a risk situation analysis engine. The spatiotemporal correlation and causal correlation of risk events are analyzed in real time using a graph neural network to quantitatively assess the overall risk level of the community. Combined with the graded early warning and automatic execution module, the accurate division of early warning levels and the automatic matching and triggering of emergency response plans are realized, replacing the traditional manual verification and scheduling mode. An automated closed loop from risk perception to instruction execution is constructed, which greatly improves the timeliness of emergency response and the standardization of the handling process.

[0022] (3) In this invention, by deeply integrating the blockchain evidence storage module with the early warning generation process, the key feature information of the early warning event is encrypted and uploaded to the consortium blockchain network to complete the evidence storage while the hierarchical early warning instruction is generated. The immutability of the blockchain is used to realize the trusted storage of security data, effectively prevent the risk of data tampering caused by centralized storage, and improve the originality and integrity of electronic evidence. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the architecture of a multimodal community security sensor data aggregation and early warning system provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the multimodal fusion and risk map generation process in a multimodal community security sensor data aggregation and early warning system provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0026] Reference manual attached Figure 1 The diagram shows an architecture diagram of a multimodal community security sensor data aggregation and early warning system provided by an embodiment of the present invention.

[0027] This invention provides a multimodal community security sensor data aggregation and early warning system, the processing flow of which may include the following steps: The system consists of a data acquisition layer, an edge computing layer, a fog computing convergence and early warning layer, and a blockchain evidence storage module.

[0028] The data acquisition layer is deployed in the community's public areas and unit entrances, and includes multiple modal sensing units for collecting heterogeneous security data.

[0029] The data acquisition layer is physically deployed throughout the community's public areas, including roads, gardens, parking lots, and entrances to each building unit. This layer consists of a large number of heterogeneous IoT sensors that connect to the community network via wired or wireless communication protocols, continuously collecting raw physical signals reflecting the community's security status. Each sensor has a unique network identifier and geolocation code, ensuring that the data streams it generates can be uniquely traced and located.

[0030] The edge computing layer communicates with the data acquisition layer and includes multiple edge computing nodes. Each edge computing node is used to connect to one or more geographically nearby modal sensing units to preprocess and extract features from the raw sensing data and output a standardized feature vector.

[0031] The edge computing layer consists of several embedded computing gateways distributed throughout the community. Each gateway is deployed as an edge computing node near the network aggregation point of the sensor device group. Each edge computing node establishes a stable connection with a group of geographically proximate modal sensing units through a dedicated data interface protocol, receiving the raw sensor data streams uploaded by them in real time. This node has a built-in lightweight computing unit responsible for running standardized preprocessing programs to clean, convert formats, and perform preliminary calculations on the raw data. It also executes feature extraction algorithms to transform raw data from devices of different manufacturers and protocols into a unified, standardized sequence of feature vectors, thereby breaking down system barriers at the data level.

[0032] The fog computing convergence and early warning layer is deployed in community data centers or regional centers, and communicates with all edge computing nodes, including: The data aggregation and fusion module is used to receive multimodal feature vectors from different edge computing nodes, and to perform cross-modal feature fusion using an adaptive weighted fusion algorithm to generate a unified fused feature representation.

[0033] The data aggregation and fusion module is deployed on the server of the fog computing aggregation and early warning layer. This module maintains a real-time data buffer, continuously receiving standardized multimodal feature vector streams uploaded from different edge computing nodes. The core of the module is an adaptive weighted fusion processor. This processor first performs alignment verification on different modal features from the same spatiotemporal window, then calls the internal fusion algorithm model to dynamically calculate the fusion weights for each set of aligned multimodal features and performs feature fusion operations. Finally, it outputs a unified, high-information-density fused feature representation for use by the subsequent analysis engine.

[0034] The risk situation analysis engine is used to perform real-time analysis based on fused feature representations and call pre-built risk identification models to build a dynamic community risk situation map.

[0035] The risk situation analysis engine operates as the core analysis unit of the fog computing convergence and early warning layer. This engine is loaded with a pre-trained multi-dimensional risk identification model library. After receiving the continuous fused feature representation stream from the data convergence and fusion module, the engine first inputs it into the risk identification model for real-time inference to determine the existence and type of potential risk events. The engine further maintains a dynamic graph-structured database, instantiating each identified risk event as a node in the graph, and dynamically establishing or updating the edges between nodes based on the event's spatiotemporal and semantic attributes, thereby constructing and updating a panoramic, dynamic community risk situation map in real time.

[0036] The tiered early warning and contingency plan execution module is used to generate tiered early warning instructions based on the risk level and impact range of the risk situation map, and automatically trigger or recommend corresponding contingency plans.

[0037] The tiered early warning and contingency plan execution module is a strategy execution and scheduling system. This module monitors the graph status and risk indicators output by the risk situation analysis engine in real time. The module has a built-in tiered early warning strategy rule base, which defines the mapping relationship between different risk levels and graph status parameters. When a risk indicator reaches a preset threshold, the module automatically generates a corresponding tiered early warning instruction based on the rule base, and simultaneously matches and activates a preset response plan process from the contingency plan library, automatically distributing the instruction and plan to the designated terminal or responsible person.

[0038] The blockchain evidence storage module, embedded in the fog computing convergence and early warning layer, is used to generate evidence storage transactions for key feature information of early warning events at the same time as the generation of hierarchical early warning instructions, and upload them to the blockchain network for trusted evidence storage.

[0039] The blockchain evidence storage module is embedded in the fog computing server in hardware as a secure cryptographic coprocessor or software service. Its operation is automatically triggered by the generation of tiered warning commands. Once triggered, the module immediately captures a snapshot of key data associated with the current warning event, including a summary of fused feature representations, a timestamp, relevant device identifiers, and the warning level. The module then constructs this key information into a standardized evidence storage data packet, digitally signs it using a locally stored community private key, and broadcasts the signed evidence storage data packet as a transaction to pre-configured consortium blockchain network nodes, completing the instantaneous evidence preservation of the warning event.

[0040] In one possible implementation, the multiple modal sensing units include at least three of the following: a visible light camera, a thermal imaging infrared camera, a wide-area acoustic sensor, a millimeter-wave radar sensor, a smart access control terminal, and a perimeter vibration fiber optic sensor.

[0041] The selection and deployment of modal sensing units follow the principles of multi-source complementarity and redundant coverage. Visible light cameras are used to acquire high-resolution color images and video streams. Thermal imaging infrared cameras are used to detect live targets and abnormal heat sources in low-light or smoky environments. Wide-area acoustic sensors are used to collect ambient sound and locate abnormal acoustic events. Millimeter-wave radar sensors are used to accurately detect the distance, speed, and angle of moving targets and have the ability to penetrate non-metallic obstructions. Smart access control terminals integrate card readers, fingerprint or facial recognition modules to collect personnel identification and entry / exit events. Perimeter vibration fiber optic sensors are laid on walls or fences to sense vibration signals generated by physical intrusion. At least three of the above types of sensors are deployed collaboratively, forming the basic data source of the community security sensor network.

[0042] In one possible implementation, the edge computing node preprocesses and extracts features from the raw sensing data, specifically including: For video stream data, extract key points of human skeleton, motion trajectory, and feature vectors of abnormal behavior; For acoustic data, extract abnormal voiceprint features and feature vectors of specific frequency events; For radar data, extract the trajectory of moving targets and the feature vectors of subtle vital signs. The extracted modal feature vectors are then time-stamped and spatially calibrated.

[0043] Edge computing nodes run dedicated feature extraction algorithms. For video stream data, a deep learning-based human pose estimation algorithm is used to extract the two-dimensional or three-dimensional coordinate sequences of key points on the human skeleton in real time, forming motion trajectories, and outputting feature vectors representing abnormal behaviors through a behavior recognition model. For acoustic data, Fast Fourier Transform and voiceprint feature extraction are performed to identify abnormal event feature vectors of specific frequency patterns, such as glass breaking and cries for help. For radar data, clustering and tracking algorithms are used to extract multi-target trajectories, and a vital sign detection algorithm is used to separate micro-motion feature vectors from the micro-Doppler effect. All feature vectors are appended with a high-precision timestamp and the node's own geographical location information before output.

[0044] In one possible implementation, the adaptive weighted fusion algorithm is specifically as follows: Given a set of feature vectors for N modalities after time alignment. ; Assign a dynamic weight to each mode i Weight The context-aware module determines the modality based on the current environmental state S and the real-time signal-to-noise ratio of the modality data. Together, we can calculate that the calculation function is: ,in is the environmental state encoding function, g is the weight calculation function, and the current environmental state S includes time, illumination, and weather; Fusion features The calculation formula is: ,in Pre-trained feature transformation matrices for each modality; The context-aware module is implemented using a lightweight neural network, whose input is an environment state code. The output is the initial weights for each mode. Then, fine-tune based on the real-time signal-to-noise ratio: .

[0045] The adaptive weighted fusion algorithm is implemented as a software module in the fog computing layer. The context-aware module incorporates a lightweight fully connected neural network. This network takes encoded environmental state vectors (time, light intensity, weather type, etc.) as input and outputs a set of initial weights reflecting the theoretical reliability of each modality's data under the current environment. Subsequently, the algorithm calculates the signal-to-noise ratio (SNR) of each modality's data stream in real time and multiplies the initial weights by the proportion of that modality's SNR to the sum of all modality SNRs, thus dynamically fine-tuning the weights. Finally, the feature vectors of each modality are linearly weighted and summed with their corresponding dynamic weights and a pre-trained feature transformation matrix to generate a fused feature representation. The feature transformation matrix, obtained through offline training before system deployment, is used to map features from different modalities to a unified semantic space.

[0046] In one possible implementation, such as Figure 2 As shown, the method by which the risk situation analysis engine constructs a dynamic community risk situation map is as follows: Each identified potential risk event is abstracted as a graph node v, with node attributes including event type, location, time, and confidence level. ; The spatiotemporal and causal relationships between events are abstracted into graph edges. Its weight Indicates the strength of the association; The graph is updated and inferred in real time using a graph neural network. The update formula for each node is as follows: ; in Let v be the feature representation of node v at layer l. Let v be the set of neighbors of node v. For trainable parameters, For activation functions; By calculating node importance scores and community risk indices ,in For a set of nodes, The importance score of node v is used to identify core risk nodes, risk transmission paths, and potential risk convergence areas.

[0047] The risk situation analysis engine maps the community's physical space into a logical graph structure. Each identified risk event, such as gatherings, rapid running, or abnormal intrusions, is created as a graph node. Node attributes include event type code, latitude and longitude coordinates, occurrence timestamp, and the confidence level of the model output. Based on preset spatiotemporal association rules, the system automatically calculates the distance, time difference, and semantic relevance between different event nodes, thereby establishing directed edges between nodes and assigning weights representing the strength of the association. The graph neural network employs a graph convolutional network or graph attention network architecture, continuously updating node feature representations through multi-layer message passing and aggregation. Node importance scores are obtained by calculating their centrality index within the entire graph, while the community risk index is calculated as the weighted sum of the confidence and importance scores of all nodes, thus identifying key risk sources and diffusion paths.

[0048] In one possible implementation, the tiered early warning and contingency plan execution module classifies the early warning level into: alert level, warning level, and warning level, based on the overall community risk index R and the impact range of core risk nodes. On-site handling level Community-level collaboration , Report from higher authorities ; Different warning levels correspond to different information notification scopes, emergency response plan databases, and resource allocation strategies.

[0049] The tiered early warning and contingency plan execution module incorporates built-in rules for quantifying early warning levels. The system sets three incremental threshold parameters: T1, T2, and T3. When the calculated community risk index R is between 0 and T1, it is classified as a warning level, generating a log record only on the security duty terminal. When R is between T1 and T2, it is classified as an on-site handling level, automatically sending alarm information and location to the mobile terminal of the nearest patrol post. When R is between T2 and T3, it is classified as a community linkage level, simultaneously sending alarms to the property management center, multiple patrol posts, and backup forces, and activating the regional linkage contingency plan. When R is greater than T3, it is classified as a superior notification level, automatically generating a standardized report based on community linkage and uploading it to the superior street or security supervision platform. Each level is associated with different information templates, notification recipient lists, and handling process guidelines.

[0050] In one possible implementation, the blockchain evidence storage module specifically performs the following steps: Receive early warning event trigger signals and associated fused feature representations ; Extract the immutable feature information of the event and generate a structured evidence storage data packet D; Use community private key Sign the evidence storage data packet: The signed and stored data packet is then broadcast to a consortium blockchain network consisting of multiple community nodes or management agency nodes. After the consortium blockchain network reaches a consensus, it will store the hash value of the evidence data packet. And metadata records are recorded on the blockchain.

[0051] The operation of the blockchain evidence storage module begins with a system callback that generates an early warning instruction. The module first extracts and serializes key immutable information about the current early warning event, including the event's unique ID, the hash value represented by the fused feature, the trigger time, a list of associated device IDs, and the early warning level, forming a structured evidence storage data packet. Subsequently, it uses a private key from a pre-built encryption hardware module or software keystore to digitally sign the data packet. The signed data packet is then encapsulated into a transaction conforming to the consortium blockchain network protocol format and immediately broadcast to multiple pre-configured consensus nodes. The consortium blockchain network employs a practical Byzantine fault-tolerant consensus algorithm. After a majority of nodes verify the signature and format validity, the transaction's hash value, timestamp, and block number, among other metadata, are recorded on the blockchain, completing the immutable evidence storage process. The Byzantine fault-tolerant consensus algorithm is one of the conventional consensus mechanisms used in this field to achieve consensus on transaction order and state among multiple mutually trusted nodes in a consortium blockchain. Those skilled in the art can select and configure specific consensus algorithm instances and their parameters according to the community node scale and performance requirements.

[0052] In one possible implementation, the fog computing convergence early warning layer also includes a continuous learning and optimization module, which is used for: Collect feedback results on the handling of historical early warning events. ; Based on the feedback results, the parameters of the risk identification model and the adaptive weighted fusion algorithm are adjusted. Incremental learning and optimization are performed, with the optimization objective being to minimize the loss function: ,in For the model's predicted output, The regularization coefficient is used. Optimized model parameters Securely synchronize to each edge computing node.

[0053] The continuous learning and optimization module runs as a background service. This module monitors the follow-up manual handling feedback of early warning events, converting feedback results such as "false alarm," "confirmed effective," and "handling completed" into quantifiable labels. The module periodically collects early warning records and their feedback labels within a certain time window, forming an incremental training dataset. The system employs an online learning or fine-tuning strategy, using the mean squared error between the predicted output and the feedback label as the primary loss, and incorporating L2 regularization constraints on changes in model parameters to prevent overfitting. The optimization process adjusts the parameters in the risk identification model and fusion algorithm using the gradient descent algorithm. After version management and integrity verification, the updated model parameters are distributed in batches through a secure channel to each edge computing node and fog computing server for hot updates, achieving incremental optimization of the overall system performance.

[0054] In one possible implementation, the system further includes a visual interactive terminal for: Real-time display of dynamic community risk situation map and risk heat map of each area; Receive and display tiered early warning instructions and response plans; It provides an interface for querying evidence storage information, allowing users to retrieve and verify corresponding trusted evidence storage records from the blockchain network by inputting an event number or time range.

[0055] The visual interactive terminal is a software application running on a PC or large-screen command center. This terminal maintains a real-time data connection with the fog computing convergence and early warning layer via a secure API interface. Its main interface displays a two-dimensional or three-dimensional electronic map of the community, overlaid with a dynamic community risk situation map, using different icons and colors to represent risk nodes and related edges. Simultaneously, the terminal generates a risk heat map of the entire community, visually displaying the risk density distribution. When an early warning command is issued, the terminal provides a strong alert through a combination of pop-ups, sound, and flashing icons, and displays detailed information including the warning level, location, associated video stream entry point, and system-recommended response plan. The terminal also provides a certificate of record verification function. After the user enters query conditions, the terminal initiates a query request to the blockchain network and displays the on-chain time, block height, and data hash value of the certificate of record in a graphical interface for verification.

[0056] In one possible implementation, encrypted communication links are used for data transmission between edge computing nodes and the fog computing convergence and early warning layer, as well as between the fog computing convergence and early warning layer and the blockchain evidence storage module.

[0057] The system enforces end-to-end encryption on all cross-layer data transmission links. A secure communication protocol based on TLS or national standard SSL is used between edge computing nodes and the fog computing convergence and early warning layer to establish a two-way authentication and encrypted data transmission channel. Communication between the fog computing convergence and early warning layer and the blockchain evidence storage module, as well as between the blockchain evidence storage module and consortium blockchain network nodes, also employs high-strength encryption protocols. All key materials are uniformly managed and periodically rotated by the system security module. Encrypted communication ensures the confidentiality, integrity, and non-repudiation of sensor feature data, early warning commands, model parameters, and evidence storage transactions during transmission, meeting information security level protection requirements.

[0058] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: (1) In this invention, by constructing a three-level collaborative processing architecture of end-edge-fog, sensor data of different modalities can be standardized and preprocessed and feature extracted at the edge. Then, by using the context-aware adaptive weighted fusion algorithm of the fog computing layer, the fusion weight is dynamically adjusted in combination with the environmental status and real-time data quality, realizing the efficient aggregation and intelligent fusion of multi-source security data, breaking down the data barriers of each subsystem, forming a global community security view, and effectively reducing the false alarm problem caused by a single sensor through cross-verification of multi-source information, thereby improving the comprehensive identification capability of complex risk events.

[0059] (2) In this invention, a dynamic community risk situation map is constructed by introducing a risk situation analysis engine. The spatiotemporal correlation and causal correlation of risk events are analyzed in real time using a graph neural network to quantitatively assess the overall risk level of the community. Combined with the graded early warning and automatic execution module, the accurate division of early warning levels and the automatic matching and triggering of emergency response plans are realized, replacing the traditional manual verification and scheduling mode. An automated closed loop from risk perception to instruction execution is constructed, which greatly improves the timeliness of emergency response and the standardization of the handling process.

[0060] (3) In this invention, by deeply integrating the blockchain evidence storage module with the early warning generation process, the key feature information of the early warning event is encrypted and uploaded to the consortium blockchain network to complete the evidence storage while the hierarchical early warning instruction is generated. The immutability of the blockchain is used to realize the trusted storage of security data, effectively prevent the risk of data tampering caused by centralized storage, and improve the originality and integrity of electronic evidence.

[0061] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multimodal community security sensor data aggregation and early warning system, characterized in that, include: Data acquisition layer, edge computing layer, fog computing convergence and early warning layer, and blockchain evidence storage module; The data acquisition layer is deployed in the community's public areas and unit entrances, and includes multiple modal sensing units for collecting heterogeneous security data; The edge computing layer is communicatively connected to the data acquisition layer and includes multiple edge computing nodes. Each edge computing node is used to connect to one or more geographically adjacent modal sensing units to preprocess and extract features from the raw sensing data and output a standardized feature vector. The fog computing convergence and early warning layer is deployed in a community data center or regional center and communicates with all edge computing nodes, including: The data aggregation and fusion module is used to receive multimodal feature vectors from different edge computing nodes, and to perform cross-modal feature fusion using an adaptive weighted fusion algorithm to generate a unified fused feature representation. The dynamic weights of the adaptive weighted fusion algorithm are calculated by a context-aware module, which is implemented through a lightweight neural network. Its input is the environmental state code, and its output is the initial weights of each modality. The weights are then fine-tuned based on the real-time signal-to-noise ratio of each modality's data. The environmental state code includes at least time, illumination, and weather information. The risk situation analysis engine is used to perform real-time analysis based on the fused feature representation, call the preset risk identification model, and construct a dynamic community risk situation map. The risk situation analysis engine constructs a dynamic community risk situation map and updates and infers the map in real time through a graph neural network to identify core risk nodes, risk transmission paths and potential risk convergence areas. The graded early warning and contingency plan execution module is used to generate graded early warning instructions based on the risk level and impact range of the risk situation map, and automatically trigger or recommend corresponding contingency plans. The blockchain evidence storage module, embedded in the fog computing convergence early warning layer, is used to generate evidence storage transactions for key feature information of the early warning event and upload them to the blockchain network for trusted evidence storage at the same time as the tiered early warning instruction is generated.

2. The multimodal community security sensor data aggregation and early warning system according to claim 1, characterized in that, The plurality of modal sensing units include at least three of the following: a visible light camera, a thermal imaging infrared camera, a wide-area acoustic sensor, a millimeter-wave radar sensor, an intelligent access control terminal, and a perimeter vibration fiber optic sensor.

3. The multimodal community security sensor data aggregation and early warning system according to claim 1, characterized in that, The edge computing node performs preprocessing and feature extraction on the raw sensor data, specifically including: For video stream data, extract key points of human skeleton, motion trajectory, and feature vectors of abnormal behavior; For acoustic data, extract abnormal voiceprint features and feature vectors of specific frequency events; For radar data, extract the trajectory of moving targets and the feature vectors of subtle vital signs. The extracted modal feature vectors are then time-stamped and spatially calibrated.

4. The multimodal community security sensor data aggregation and early warning system according to claim 1, characterized in that, The adaptive weighted fusion algorithm is specifically as follows: Given a set of feature vectors for N modalities after time alignment. ; Assign one of the dynamic weights to each mode i Weight The context-aware module determines the modality based on the current environmental state S and the real-time signal-to-noise ratio of the modality data. Together, we can calculate that the calculation function is: ,in is the environmental state encoding function, g is the weight calculation function, and the current environmental state S includes time, illumination, and weather; Fusion features The calculation formula is: ,in Pre-trained feature transformation matrices for each modality; The context-aware module is implemented using a lightweight neural network, whose input is an environment state code. The output is the initial weights for each mode. Then, fine-tune based on the real-time signal-to-noise ratio: .

5. The multimodal community security sensor data aggregation and early warning system according to claim 1, characterized in that, The method by which the risk situation analysis engine constructs a dynamic community risk situation map is as follows: Each identified potential risk event is abstracted as a graph node v, with node attributes including event type, location, time, and confidence level. ; The spatiotemporal and causal relationships between events are abstracted into graph edges. Its weight Indicates the strength of the association; The graph is updated and inferred in real time using the graph neural network. The update formula for each node is as follows: ； in Let v be the feature representation of node v at layer l. Let v be the set of neighbors of node v. For trainable parameters, For activation functions; By calculating node importance scores and community risk indices ,in For a set of nodes, The importance score of node v is used to identify core risk nodes, risk transmission paths, and potential risk convergence areas.

6. The multimodal community security sensor data aggregation and early warning system according to claim 1, characterized in that, The tiered early warning and contingency plan execution module classifies the early warning levels into: alert level, warning level, and warning level, based on the overall community risk index R and the impact range of core risk nodes. On-site handling level Community-level collaboration , Report from higher authorities ; Different warning levels correspond to different information notification scopes, emergency response plan databases, and resource allocation strategies.

7. A multimodal community security sensor data aggregation and early warning system according to claim 1, characterized in that, The blockchain evidence storage module specifically performs the following steps: Receive early warning event trigger signals and associated fused feature representations ; Extract the immutable feature information of the event and generate a structured evidence storage data packet D; Use community private key Sign the evidence storage data packet: The signed and stored data packet is then broadcast to a consortium blockchain network consisting of multiple community nodes or management agency nodes. After the consortium blockchain network reaches a consensus, it will store the hash value of the evidence data packet. And metadata records are recorded on the blockchain.

8. A multimodal community security sensor data aggregation and early warning system according to claim 1, characterized in that, The fog computing convergence early warning layer also includes a continuous learning and optimization module, which is used for: Collect feedback results on the handling of historical early warning events. ; Based on the feedback results, the parameters of the risk identification model and the adaptive weighted fusion algorithm are adjusted. Incremental learning and optimization are performed, with the optimization objective being to minimize the loss function: ,in For the model's predicted output, The regularization coefficient is used. Optimized model parameters Securely synchronize to each edge computing node.

9. A multimodal community security sensor data aggregation and early warning system according to claim 1, characterized in that, The system also includes a visual interactive terminal for: The dynamic community risk situation map and risk heat map of each area are displayed in real time. Receive and display the tiered early warning instructions and response plans; It provides an interface for querying evidence storage information, allowing users to retrieve and verify corresponding trusted evidence storage records from the blockchain network by inputting an event number or time range.

10. A multimodal community security sensor data aggregation and early warning system according to claim 1, characterized in that, The edge computing node and the fog computing convergence and early warning layer, as well as the fog computing convergence and early warning layer and the blockchain evidence storage module, all use encrypted communication links for data transmission.

Images