Community intelligent monitoring and emergency linkage method and system fusing BIM spatial semantics

Abstract

The invention discloses a community intelligent monitoring and emergency linkage method and system fusing BIM spatial semantics, and the method comprises the steps: constructing a BIM scene map, and obtaining the attributes and mutual relationships of components and spatial regions in a BIM model; mapping a dynamic target detected in video monitoring into the BIM model, and obtaining spatial semantic information of the dynamic target; based on BIM spatial semantic information of a dynamic target, a target-environment interaction graph is constructed, a graph neural network model is used for training and reasoning, and specific complex events related to spatial contexts are recognized; taking the BIM model as a space-time reference, fusing multi-source heterogeneous data, and reconstructing by adopting a graph-based event association algorithm to form a complete event chain containing an atomic event sequence and an association relationship; and when an emergency event or an event chain is detected to indicate an emergency state, combining BIM preset information and real-time sensor data, dynamically generating an optimal emergency plan, and performing visual commanding and dispatching through a BIM three-dimensional scene and augmented reality.

Description

Technical Field

[0001] The present invention relates to the field of intelligent security technology, and in particular to a community intelligent monitoring and emergency linkage method and system integrating BIM spatial semantics. Background Art

[0002] With the acceleration of urbanization and the advancement of smart community construction, community security monitoring and emergency management face increasingly severe challenges. Traditional video surveillance systems primarily rely on two-dimensional image features for event detection, lacking an understanding of three-dimensional spatial structure and the deep semantics of scene objects. For example, the system struggles to distinguish between an entry into an ordinary office and a critical power distribution room, or accurately determine whether a person lingering for an extended period at an evacuation entrance poses a potential risk. These limitations result in insufficient monitoring systems' ability to recognize complex behaviors, leading to high rates of false alarms and missed detections.

[0003] Secondly, community surveillance typically involves multiple cameras and heterogeneous data sources such as access control, fire protection, and environmental sensors. Effectively integrating this multimodal data to achieve continuous cross-camera target tracking, event correlation analysis, and comprehensive situation assessment is a pressing technical challenge. Existing systems often process this data in isolation, making it difficult to form a unified, holistic understanding.

[0004] Furthermore, when emergencies such as fires and unauthorized intrusions occur, it is crucial to quickly generate targeted emergency response plans and conduct efficient, visual command and dispatch. Traditional emergency response plans are often static text or fixed processes, making them difficult to adapt to dynamic changes. While BIM (Building Information Modeling) technology provides a wealth of 3D geometric and semantic information about buildings, fully leveraging this information for intelligent plan generation and dynamic dispatch is still under exploration. For example, dynamically planning optimal evacuation routes based on the real-time location of the fire and the distribution of personnel, and intuitively communicating this information to on-site personnel and the command center, is key to improving emergency response efficiency.

[0005] Therefore, how to integrate the rich spatial semantics provided by BIM, improve the accuracy of video surveillance in identifying complex behaviors and potential risks, achieve effective fusion of multimodal information and event association, and build intelligent emergency plan generation and visual scheduling capabilities on this basis, is a technical problem that urgently needs to be solved in the current field of community intelligent monitoring and emergency linkage. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a community intelligent monitoring and emergency linkage method and system that integrates BIM spatial semantics. It aims to use BIM's spatial semantic information to guide video event detection and understanding, especially to identify context-related complex events through a graph neural network model, and to achieve deep fusion and comprehensive situation analysis of multi-source heterogeneous data through a refined event chain reconstruction algorithm, thereby supporting the dynamic generation of emergency plans and AR / VR visual command, thereby improving the intelligence level of community monitoring and the efficiency and accuracy of emergency response.

[0007] To achieve the above objectives, the present invention provides a community intelligent monitoring and emergency linkage method integrating BIM spatial semantics, comprising the following steps: S1: Constructing a community building information model (BIM) scene graph, wherein the BIM scene graph includes BIM component information, spatial region information and their respective semantic attribute information, and topological relationship information between the BIM components and spatial regions; S2: Acquire video surveillance data, detect dynamic targets therefrom, map the dynamic targets to the three-dimensional space corresponding to the BIM scene graph through coordinate transformation and geometric projection, and obtain BIM space semantic information of the dynamic targets; S3: Based on the BIM spatial semantic information of the dynamic target, a local interaction graph is constructed that includes the dynamic target node and its neighboring BIM element nodes. The local interaction graph is analyzed using a pre-trained graph neural network (GNN) event recognition model, or the behavior of the dynamic target is analyzed in combination with a preset event rule library to identify specific events related to the spatial context. The node features of the GNN model include the kinematic features of the dynamic target and the functional and state features of the BIM spatial elements, and the edge features represent the interactive relationships between nodes. S4: Using the BIM as a unified spatiotemporal reference, integrating multi-source heterogeneous data acquired by at least two different types of sensors, performing spatiotemporal alignment and correlation analysis, and forming an atomic event set; S5: Based on the topological relationship information in the BIM scene graph, the motion trajectory of the dynamic target in the BIM space, and the set of atomic events formed in S4, an event chain construction algorithm is used to screen and connect related atomic events by evaluating the time series relationship, spatial proximity relationship, object consistency, and predefined logical association rules between atomic events, and reconstruct a complete event chain with time sequence and logical association; S6: When an emergency event is identified in S3 or the event chain reconstructed in S5 indicates an emergency state, the optimal emergency plan for the current emergency event is dynamically calculated and generated based on the emergency resource information, evacuation path information, hazard source information preset in the BIM scene graph and the real-time sensor data integrated in S4; S7: The optimal emergency plan is visualized in the BIM three-dimensional scene, and navigation and instructions can be provided to on-site personnel through augmented reality (AR) equipment, or immersive situational awareness and command and dispatch support can be provided to command center personnel through virtual reality (VR) equipment.

[0008] In a second aspect, the present invention provides a community intelligent monitoring and emergency linkage system integrating BIM spatial semantics, comprising: A BIM semantic parsing module is used to construct a community building information model (BIM) scene graph, wherein the BIM scene graph includes BIM component information, spatial area information and their respective semantic attribute information, and topological relationship information between the BIM components and spatial areas; A video analysis and mapping module is used to acquire video surveillance data, detect dynamic targets, and map the dynamic targets into the three-dimensional space corresponding to the BIM scene graph to obtain its BIM space semantic information; A semantic event detection module is used to construct a local interaction graph containing the dynamic target node and its neighboring BIM element nodes based on the BIM spatial semantic information of the dynamic target, and analyze the local interaction graph using a pre-trained graph neural network (GNN) event recognition model, or combine it with a preset event rule library to identify specific events related to the spatial context; A multi-source data fusion module is used to fuse multi-source heterogeneous data acquired by at least two different types of sensors using the BIM as a unified spatiotemporal reference, perform spatiotemporal alignment and correlation analysis, and form an atomic event set; An event reasoning and warning module is used to reconstruct a complete event chain with temporal and logical associations using an event chain construction algorithm based on the topological relationship information in the BIM scene graph, the motion trajectory of dynamic targets in the BIM space, and the resulting atomic event set, and to issue a warning when an emergency event is identified or the event chain indicates an emergency state; An emergency plan generation module is used to dynamically calculate and generate an optimal emergency plan for the current emergency event based on the emergency resource information, evacuation route information, hazard source information preset in the BIM scene map, and integrated real-time sensor data after receiving an early warning; The 3D visualization and interaction module is used to visualize the optimal emergency plan in the BIM 3D scene, and can optionally provide navigation and instructions to on-site personnel through augmented reality (AR) equipment, or provide immersive situational awareness and command and dispatch support to command center personnel through virtual reality (VR) equipment.

[0009] The technical solution provided by the present invention can have the following beneficial effects: This invention discloses a community intelligent monitoring and emergency linkage method and system that integrates BIM spatial semantics. The method aims to address the problems of traditional monitoring's lack of spatial semantic understanding, difficulty fusing multi-source data, and low intelligence in emergency response plans. The method includes: constructing a BIM scene graph to obtain the attributes and relationships of components and spatial regions in the BIM model; mapping dynamic targets detected during video surveillance into the BIM model to obtain their spatial semantic information; constructing a target-environment interaction graph based on the BIM spatial semantic information of the dynamic targets; and using graph neural network (GNN) models, such as graph attention networks (GAT), for training and inference to identify specific complex events related to spatial context. Using the BIM model as a spatiotemporal reference, the method fuses multi-source heterogeneous data and, based on spatiotemporal proximity, object consistency, and predefined logical rules, employs a graph-based event association algorithm to reconstruct a complete event chain containing atomic event sequences and their relationships. When an emergency event is detected or the event chain indicates an emergency state, the method dynamically generates an optimal emergency response plan by combining BIM preset information with real-time sensor data. Visual command and dispatch are then performed through BIM three-dimensional scenes, augmented reality (AR), or virtual reality (VR). The present invention can significantly improve the intelligence level of community monitoring and emergency response efficiency.

[0010] Compared to existing combustion optimization methods, this invention offers at least the following outstanding substantive features and significant advancements: First, by introducing a GNN model to analyze the dynamic interaction graph between the target and the BIM environment, it can more accurately and robustly identify complex context-dependent events that are difficult for traditional methods to handle, such as subtle behavioral anomalies or multi-target collaborative anomalies. Second, a refined event chain reconstruction algorithm not only connects atomic events related in time and space but also imparts high-level semantics to event chains through logical rules and scenario templates, thereby providing a deeper understanding of the nature and development trends of events and providing a more comprehensive basis for early warning and decision-making. Third, using BIM as a unified spatiotemporal reference, it effectively integrates heterogeneous data such as video, access control, and sensors, breaking down information silos and providing a more comprehensive and dynamic picture of community safety through event chain reconstruction. Fourth, it can dynamically generate optimal emergency response plans based on real-time events (such as fire locations and personnel distribution) and BIM model information, such as intelligently planning evacuation routes and deploying emergency resources, significantly improving the targeted and effective nature of emergency responses. Fifth, by combining BIM three-dimensional scenes with AR / VR technology, intuitive and immersive situational information and command guidance are provided to on-site personnel and command centers, improving collaborative efficiency and handling capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0012] Figure 1This is a functional module architecture diagram of a community intelligent monitoring and emergency linkage system that integrates BIM spatial semantics, provided by an embodiment of the present invention.

[0013] Figure 2 This is a flow chart of a community intelligent monitoring and emergency linkage method integrating BIM spatial semantics provided by an embodiment of the present invention.

[0014] Figure 3 Schematic diagram of a BIM scene graph in an embodiment of the present invention.

[0015] Figure 4 Schematic diagram of video target to BIM space mapping and semantic acquisition in an embodiment of the present invention.

[0016] Figure 5 This is a schematic diagram of constructing a local interaction graph based on a GNN-based event recognition model in an embodiment of the present invention.

[0017] Figure 6 Schematic diagram of the event chain construction process in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] To further understand the content of the present invention, the present invention is described in detail with reference to the accompanying drawings and examples. The present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0019] Example 1 Reference Figure 1 This embodiment provides a community intelligent monitoring and emergency linkage system 100 that integrates BIM spatial semantics. The system 100 is designed to achieve intelligent monitoring of the community environment and rapid emergency response to emergencies. The system 100 mainly includes: 1. BIM semantic parsing module: This module is responsible for parsing, semantic processing, and graph construction of the original BIM model (usually in IFC format or native formats of software such as Revit). It includes: BIM data interface submodule 111: used to read and parse BIM model files, extract the geometric information (location, size, shape), physical properties (material), and functional properties (purpose) of building components (such as walls, doors, windows, columns, beams, and floor slabs), equipment (such as fire hydrants, fire extinguishers, cameras, smoke detectors, and access control card readers), and spaces (such as rooms, corridors, stairs, power distribution rooms, and pump rooms). Semantic Enhancement and Ontology Mapping Submodule 112: This submodule addresses potential semantic gaps or irregularities in BIM models by semantically enriching and standardizing extracted information using a pre-built architectural ontology library (e.g., containing spatial function classifications, equipment type specifications, and security level definitions). For example, a space labeled simply "Room" can be further precisely labeled as a "Fire Control Room" or "High-Voltage Distribution Room" based on its internal equipment and connections, and assigned corresponding security levels and access permissions.

[0020] Scene Graph Construction and Storage Submodule 113: This constructs a BIM scene graph using semantically defined BIM components, devices, and spaces as nodes, and their physical connections (e.g., a door connecting two rooms), spatial topological relationships (e.g., room A adjoins room B, corridor C contains camera D), and functional affiliations (e.g., a fire hydrant belongs to a fire zone) as edges. This graph is preferably stored and managed in a graph database (e.g., Neo4j) to facilitate efficient querying of complex spatial relationships and semantic information. Figure 3 A simplified BIM scene graph is shown, where nodes represent BIM elements (such as room 201, door D1, camera C1), and edges represent the relationships between them (such as "located in" and "connected").

[0021] 2. Video Analysis and Mapping Module: This module is responsible for processing real-time video streams from various surveillance cameras in the community, detecting dynamic targets and locating them in the BIM 3D space. It includes: Video stream access and decoding submodule 121: accesses the video stream of the network camera through protocols such as RTSP and ONVIF, and decodes it.

[0022] Dynamic object detection and tracking submodule 122: This module uses deep learning object detection algorithms (such as YOLOv5 and Faster R-CNN) to detect and classify dynamic objects (primarily people and vehicles) in video frames in real time. It also combines object tracking algorithms (such as DeepSORT and FairMOT) to continuously track the trajectory of the same object within a single camera.

[0023] Camera Calibration and Coordinate Transformation Submodule 123: The system pre-calibrates the intrinsic and extrinsic parameters of each surveillance camera (focal length, principal point, distortion coefficient, and position and orientation in the BIM world coordinate system). This submodule uses these calibration parameters to map the coordinates of target pixels detected in the 2D video image (e.g., the center point of the target bounding box or the plantar point) to rays in the BIM 3D space or to specific 3D coordinates through a projective transformation (e.g., pinhole camera model backprojection).

[0024] BIM space semantic association submodule 124: associates the three-dimensional coordinates of the target in BIM with the scene graph constructed by the BIM semantic parsing module 110, and queries the BIM space area where the target is currently located (such as "corridor A01", "distribution room P02"), nearby BIM components (such as "door M03", "fire hydrant F04") and their related semantic information (such as area function "no entry" and equipment status "available"). Figure 4 It illustrates the process of detecting a target (person) from a video image and mapping it to a specific room in the BIM model.

[0025] 3. Semantic Event Detection Module: This module identifies specific events based on the spatial semantic information of the target in the BIM. It includes: The local interaction graph construction submodule 131a is responsible for constructing the input graph of the GNN model in real time. After the video analysis and mapping module detects a dynamic target and maps it to the BIM space, this submodule, centered on the target, combines the BIM scene graph 110 to extract BIM elements (such as doors, windows, walls, cameras, fire hydrants, and specific area boundaries) within a certain spatial neighborhood (e.g., within a radius of R meters, or topologically within 1-hop or 2-hop neighborhoods). These dynamic targets and BIM elements together constitute the nodes of the local interaction graph. Node feature generation involves calculating a feature vector for each dynamic target node, such as [type (one-hot), normalized velocity_x, normalized velocity_y, relative BIM area position_x, relative BIM area position_y, heading angle, accumulated dwell time] A feature vector is also calculated for each BIM element node, such as [type (one-hot), function ID (one-hot), safety level, real-time status (one-hot, e.g., door open / closed, device normal / damaged)] .

[0026] Edge definition and feature generation: Define different types of edges, such as "target-target" (features: relative distance, direction), "target-BIM element" (features: distance, contact, interaction direction), and "BIM element-BIM element" (inherited from the BIM scene graph, feature: connectivity).

[0027] GNN event recognition model submodule 132: A pre-trained graph neural network model is used, such as a heterogeneous graph neural network with multiple graph attention layers (GATv2Conv). Each GATv2Conv layer aggregates information from neighboring nodes through an attention mechanism to update the representation of the central node. The model may also include a graph pooling layer (such as TopKPooling or MeanPooling) to obtain a global representation of the entire local interaction graph. Finally, through one or more fully connected layers and a softmax function, the model outputs the probabilities of predefined event categories (such as normal passage, illegal intrusion into a designated area, tailgating, abnormal wandering, items left in a dangerous area, fighting, falls, etc.). This graph neural network model is trained using a large amount of annotated data. The training data includes video clips of normal behavior and various abnormal events, as well as corresponding BIM model interaction data (target trajectory, BIM element state changes, etc.). The loss function is typically cross-entropy loss.

[0028] Event rule engine submodule 131: For simple events that are insufficiently covered by the GNN model or have low confidence, or as pre- / post-processing of the GNN, an IF-THEN based rule engine can be used.

[0029] 4. Multi-source data fusion module: This module is responsible for integrating data from different sensors and fusing them based on BIM. It includes: Multi-source data access submodule 141: receives data from the access control system (card swiping records, door opening status), fire protection system (smoke and temperature alarms), environmental sensors (gas leakage, water immersion), security sensors (infrared beam, window and door magnets), etc.

[0030] Spatiotemporal Alignment and Correlation Submodule 142: The physical locations of all sensors are pre-annotated in the BIM model. When sensor data is received, its BIM location and event timestamp are combined to align it with the target information output by the video analysis and mapping module 120. For example, if the access control system reports that a door was opened with a card swipe, and a nearby camera captures a person passing through, the two can be correlated to confirm the person's identity (if the card swipe information contains the identity). If a smoke alarm sounds in a certain area and a camera in that area also captures smoke, the authenticity of the fire alarm can be cross-verified.

[0031] 5. Event Reasoning and Warning Module: This module performs in-depth analysis on the fused atomic events (GNN output from the semantic event detection module 130, sensor events from the multi-source data fusion module 140, etc.), reconstructs the event chain, and issues warnings. It includes: Cross-camera target re-identification submodule 151: Utilizes the spatial connectivity in the BIM scene graph (such as corridors connecting multiple rooms and stairs connecting different floors) as well as the appearance features and movement patterns of pedestrians to achieve the association of targets between different camera fields of view, namely target re-identification (Re-ID). BIM path constraints are used to reduce the Re-ID search space, thereby constructing the complete movement trajectory of the target in the entire community.

[0032] Event Chain Construction and Management Submodule 152: This submodule connects events identified by a single camera, events associated with multiple sensors, and target behaviors tracked across multiple cameras, chronologically and logically (e.g., causal relationships and coherence) to form a complete event chain. For example, "Person A enters the east gate at 08:00 (access control and video confirmation) -> is captured by camera Cam01 in the east corridor at 08:01 -> enters the power distribution room at 08:03 (video semantic event detection) -> smoke alarm in the power distribution room at 08:05 (sensor data)." This submodule specifically includes: Atomic event library management: Receives and stores atomic events from various modules. Each atomic event has a standard format: `{event_id, timestamp, bim_location_id, event_type, source_module, involved_objects_ids: [], confidence, description}`.

[0033] Correlation Rules Engine: A set of built-in configurable event correlation rules, including: Time correlation: `AFTER(e1, e2, min_dt, max_dt)`, `CONCURRENT(e1, e2, window_t)`.

[0034] Spatial association: `ADJACENT_BIM(e1, e2, hops)` (BIM topological adjacency), `WITHIN_DISTANCE_BIM(e1, e2, dist)`.

[0035] Object association: `SAME_TARGET(e1, e2, target_id)`, `INTERACTS_WITH_SAME_BIM_ELEMENT(e1, e2, bim_element_id)`.

[0036] Logical / causal relationship: `IF (type(e1) == T1 AND condition1) THEN LIKELY_LEADS_TO (type(e2) == T2)` (e.g., "illegal entry" may appear after "breaking window sound").

[0037] Dynamic event chain graph generation: New atomic events are added as nodes to a dynamic "potential event chain graph" in real time. If the new event is associated with an existing event in the graph, as determined by the association rule engine, a directed edge is added between them. The edge can also carry a weight (indicating the strength or type of the association).

[0038] Event Chain Extraction and Scoring: In this diagram, a path search algorithm (e.g., heuristic search) is run periodically or triggered by high-priority atomic events. The search goal is to find paths that satisfy a specific pattern (e.g., a "template event chain": a predefined sequence of atomic event types and the expected connections between them), or paths whose weight (a combination of atomic event confidence, connection strength, path length, and other factors) exceeds a threshold. These paths are identified as event chains.

[0039] Event chain semantic labeling and alerting: The extracted event chains are matched against a higher-level "situation template library" (for example, a "theft attempt" template might include the event chain pattern of "wandering around a restricted area at night -> attempting to unlock -> triggering the door sensor alarm -> escaping"). Successfully matched event chains are assigned an overall semantic label and trigger an alert level based on their severity.

[0040] Situation Assessment and Warning Submodule 153: This module performs a comprehensive situation assessment based on the severity and development trend of the input structured event chain and the preset warning threshold. When the assessment result reaches a warning level (e.g., "illegal intrusion and fire risk"), it sends a warning signal and related event information to the Emergency Response Plan Generation Module 160 and the 3D Visualization and Interaction Module 170.

[0041] 6. Emergency plan generation module: After receiving an early warning, this module dynamically generates an emergency response plan. It includes: Emergency resource matching submodule 161: Based on the event type (fire, intrusion, medical assistance, etc.) and the location of the incident (obtained from the BIM scene map), query and match the most appropriate emergency resources in the BIM model, such as the nearest and available fire hydrants, fire extinguishers, first aid kits, security personnel, etc.

[0042] Dynamic Path Planning Submodule 162: For scenarios requiring evacuation (e.g., fire), based on pre-defined evacuation routes and emergency exits in the BIM, and combined with real-time sensor data (e.g., fire location, smoke spread, occupant density, and damaged or blocked passages), a path planning algorithm (e.g., a modified A* algorithm, where path costs take into account distance, congestion, and danger) is used to dynamically calculate optimal evacuation routes for personnel at different locations. For rescue forces, the optimal route to the incident site is planned.

[0043] The emergency plan instruction generation submodule 163 integrates the matched resources, planned paths, and necessary action instructions (such as blocking specific passages, starting smoke exhaust fans, and notifying personnel at specific posts) into a structured emergency plan.

[0044] 7. 3D visualization and interaction module: This module is responsible for visualizing and interactively displaying monitoring information, event status, and emergency plans in a 3D environment. It includes: BIM 3D scene rendering engine 171: Renders the BIM model in real time and overlays dynamic targets (from module 120), event information (from modules 130 and 150), sensor status (from module 140) and emergency plans (from module 160) on the model.

[0045] AR / VR interface and rendering submodule 172: AR augmented reality: Projecting plan instructions (such as evacuation arrows, danger zone highlights, and target equipment indications) into the real field of view of on-site personnel through AR glasses (such as Microsoft HoloLens) to provide intuitive navigation. Figure 5 The diagram shows the evacuation guidance arrows displayed in AR glasses superimposed on the real environment corridor.

[0046] VR: Provide command center personnel with access to VR headsets (such as HTC Vive, Oculus Quest), allowing them to immersively navigate the BIM 3D scene from a first-person perspective, observe real-time dynamics, simulate the effects of different treatment plans, and conduct remote command and dispatch.

[0047] Interactive control submodule 173: allows operators to interact on the BIM visualization interface, such as clicking to query component information, manually adjusting the camera PTZ, simulating the triggering of specific equipment (such as remotely starting a fire pump), and manually intervening in emergency plans.

[0048] Modules communicate and collaborate via an internal bus or network. For example, the output of the video analysis and mapping module 120 serves as input to the semantic event detection module 130; the results of the multi-source data fusion module 140 support the analysis of the event reasoning and early warning module 150; and the results of the emergency plan generation module 160 are ultimately presented in the 3D visualization and interaction module 170.

[0049] Example 2 Reference Figure 2 This embodiment provides a detailed process of a community intelligent monitoring and emergency linkage method integrating BIM spatial semantics: Step S201: Construct a BIM scene graph and initialize the system. The BIM scene graph includes BIM component information, spatial region information, their respective semantic attribute information, and topological relationship information between the BIM components and spatial regions. This step corresponds to the work of the BIM semantic parsing module 110 in the system.

[0050] First, load the community's BIM model (for example, an IFC file). The parsing engine extracts basic attributes, including geometry, ID, name, and type, for all building components (walls, floors, doors, windows, etc.), spatial objects (rooms, corridors, halls, stairwells, etc.), and equipment in the MEP (mechanical, electrical, and plumbing) system (fire hydrants, smoke detectors, cameras, access control readers, etc.).

[0051] Secondly, semantic enhancement is performed. By combining predefined community functional ontology libraries (for example, defining "distribution room" as a high-risk, restricted area and "evacuation staircase" as a life-safety passage), BIM objects are assigned more precise semantic labels, such as room function (office, conference room, distribution room), safety level, and management responsibility area. For example, a room simply named "RM-101" in the BIM can be labeled as a "high-voltage distribution room" by analyzing its internal equipment (such as high-voltage switchgear) or design document associations.

[0052] Next, a BIM scene graph is constructed. All BIM objects and their semantic attributes are considered nodes in the graph, and spatial relationships between objects (e.g., room A contains equipment B, door C connects room D to corridor E, and staircase F connects floors G and H) and logical relationships (e.g., fire hydrant I serves area J) are considered edges. This graph is stored in a graph database.

[0053] At the same time, other modules in the system are initialized, such as loading the video analysis model, calibrating camera parameters, and establishing connections with various sensors.

[0054] Step S202: Real-time monitoring and multi-source data collection.

[0055] The video analysis and mapping module 120 continuously obtains real-time video streams from various surveillance cameras in the community. At the same time, the multi-source data fusion module 140 receives real-time data from access control systems, fire alarm systems, environmental sensors, etc.

[0056] Step S203: Dynamic target detection, tracking and BIM space mapping, mapping the dynamic target to the three-dimensional space corresponding to the BIM scene graph through coordinate transformation and geometric projection, and obtaining the BIM space semantic information of the dynamic target.

[0057] The video analysis and mapping module 120 processes the received video stream: 1. Object detection: Use deep learning models such as the YOLO series to detect dynamic objects of interest, mainly people or vehicles, in each frame and obtain their 2D bounding boxes.

[0058] 2. Single-camera tracking: For detected targets, a tracking algorithm such as DeepSORT is used to assign a unique ID within the field of view of a single camera and continuously track its movement trajectory.

[0059] 3. BIM Spatial Mapping: Obtain the pre-calibrated parameters of each camera (intrinsic and extrinsic parameters, with extrinsic parameters referring to the camera's position and posture in the BIM world coordinate system). The target's pixel coordinates in the 2D image (typically taking the midpoint of the bottom edge of the bounding box as the foot point) are converted to the BIM 3D world coordinate system using a backprojection algorithm (e.g., based on a homography matrix or direct linear transformation). By querying the BIM scene graph, the 3D coordinates are determined to lie within a BIM spatial object (e.g., room, corridor) or to which BIM component (e.g., door, window) they are near. This provides dynamic targets with rich BIM spatial semantic context, such as "Person Zhang (target ID: P001) is located in Room 305 (function: office) on the third floor, near window W003."

[0060] Step S204: Event recognition based on BIM semantics.

[0061] The semantic event detection module 130 performs event judgment based on the BIM space semantic information obtained in step S203: 1. Construct a local interaction graph: For each (or selected key) dynamic target, use itself as the central node. The BIM scene graph queries BIM elements (such as room entities, doors, windows, corners, fire hydrants, and restricted zone boundaries) within a certain radius (e.g., 5 meters) or topologically adjacent (e.g., 1-hop or 2-hop) of its current location as environmental nodes. These elements together form a local interaction graph centered on the target. Each node (target node, BIM element node) is assigned a feature vector, such as target type, speed, direction; and BIM element type, function, and status. Edges between nodes (target-target, target-BIM element, and BIM element-BIM element) are assigned features, such as relative distance, direction, and interaction type (e.g., target moving toward a door, target crossing a zone boundary).

[0062] 2. GNN Model Inference: The constructed local interaction graph (which may be a sequence of graphs within a time window to capture dynamic changes) is input into a pre-trained GNN model (such as the GAT model mentioned above). The GNN model uses graph convolution and attention mechanisms to learn the complex spatiotemporal interaction patterns between the target and the environment, and outputs an event classification result for the current scene (such as "normal passage", "illegal intrusion into the finance room", "wandering at the evacuation staircase for more than 2 minutes", "item left in the fire escape") and its confidence level.

[0063] 3. Rule engine supplement: For scenarios that the GNN model fails to cover or has low classification confidence, preset BIM semantic-based rules (such as "a person enters a 'distribution room' that is 'prohibited from entering' and has no authorized access control record") can be used for supplementary judgment.

[0064] The output of this step is the video event with semantic labels and confidence, which is input to the next step as an atomic event.

[0065] Step S205: spatiotemporal fusion and correlation analysis of multi-source data.

[0066] The multi-source data fusion module 140 performs the following steps: 1. Data alignment: This unifies the timestamps of the various sensor data collected in step S202 (e.g., access control card swipe records: card number, time, door ID; smoke alarms: sensor ID, time, status). The sensor's physical ID is associated with its pre-annotated location in the BIM model.

[0067] 2. Data Association: Using the BIM space as a link, data from different sources is associated. For example, if a card swipe occurs at a certain access control point (such as D001), the BIM scene graph is checked to see if there is a camera near D001. Video clips from that camera before and after the card swipe are retrieved. Combined with the target detection results from S203, facial comparison or behavior verification is performed to determine whether the operation was made by the cardholder or whether there is any anomaly such as being tailed. For another example, if a smoke sensor in a certain area (such as R101) sounds an alarm, the video footage from cameras within or nearby R101 is checked to confirm whether there is visible smoke or flames.

[0068] Step S206: Event chain reconstruction and comprehensive situation analysis. The event reasoning and early warning module 150 performs: 1. Atomic Event Collection and Standardization: Aggregate atomic events generated by all information sources, including video semantic events from S204, sensor alarms (such as smoke detectors, door sensors, and infrared beams) fused from S205 (formerly S205, now understood as the result of multi-source data fusion), and access control records (success, failure, and duress codes). Unify the data format, including timestamp, BIM spatial location ID, event type, source, list of participating object IDs, and confidence level.

[0069] 2. Construct a candidate event association graph: Maintain a dynamic graph structure where nodes represent atomic events. When a new atomic event is generated, it is added to the graph. Then, based on a predefined set of association rules (temporal association, spatial association, object consistency, logical / causal association, and configurable weights within the rule base), the potential association between the new event and existing events in the graph is evaluated. If one or more rules are met, one or more directed edges are created between the corresponding atomic event nodes. Edge attributes may include the association type and association strength (calculated from the rule weights and the atomic event confidence).

[0070] 3. Event Chain Extraction: Periodically or whenever a high-priority atomic event (such as a fire alarm or intrusion alarm) occurs, a path search algorithm is executed within the candidate event association graph. For example, a template-based search can be designed to find subgraphs or paths that match a predefined "scenario template" (e.g., a typical theft process contains a sequence of atomic event types and the expected relationships between them). Alternatively, a scoring-based search can be employed to find meaningful event sequences whose total weight in the graph (representing the credibility or importance of the entire chain) exceeds a certain threshold. The extracted path is a reconstructed event chain. An event chain can be represented as an ordered list of atomic events and the clear relationships between them.

[0071] 4. Event Chain Semantic Annotation and Situation Assessment: The extracted event chain is matched to a higher-level "situation ontology library" (for example, an "early stage fire" situation might consist of the event chain "smoke alarm -> temperature alarm -> smoke appears in the corresponding area video -> evacuation"). This gives the entire event chain a comprehensive semantic description and risk level. If the event chain indicates an urgent or potentially severe risk, an alert is triggered, and the process proceeds to step S207.

[0072] Step S207: Determine whether it is an emergency event.

[0073] If S204 identifies a high-risk event (such as a fire alarm or violent break-in) or the situation assessment result of S206 reaches the preset emergency threshold, it is determined to be an emergency event and step S208 is executed; otherwise, if it is a general event, a log is recorded, relevant management personnel are notified, and then the process returns to step S202 to continue monitoring.

[0074] Step S208: Dynamically generate an emergency plan. The emergency plan generation module 160 is activated: Information acquisition: Retrieve static information related to the incident location (such as evacuation routes, safety exit locations, firefighting facility distribution, and hazard source information) from the BIM scene map, and obtain real-time dynamic information (such as the precise location of the fire point / intruder, smoke spread range, population distribution density, and the status of available / damaged facilities) from the multi-source data fusion module.

[0075] Resource Matching and Route Planning: Automatically match emergency resources based on the type of incident. For example, in the event of a fire, the nearest available fire hydrant and fire extinguisher are located; in the event of an injury, the nearest first aid kit and AED are located. Using algorithms such as A* or Dijkstra, the optimal evacuation route to the nearest safe exit is planned for trapped personnel on the BIM road network model (accounting for real-time obstacles such as fire zones, thick smoke areas, and closed passages). The fastest access route to the incident site is also planned for rescuers.

[0076] 3. Plan integration: Integrate evacuation instructions, rescue instructions, resource allocation plans, and control measures to be taken (such as starting smoke exhaust and cutting off power to specific areas) into a complete, dynamic emergency plan for the current situation.

[0077] Step S209: Visualization command and AR / VR linkage scheduling. The 3D visualization and interaction module 170 performs: BIM visualization presentation: On the command center's BIM three-dimensional visualization platform, the location of the incident, the scope of the incident, the dynamically generated evacuation path, the location of emergency resources, the rescue team's route, etc. are highlighted.

[0078] AR navigation and instructions send information such as evacuation routes, danger warnings, and operating instructions via wireless networks to AR glasses worn by on-site personnel, and display them superimposed on their real-time field of view in the form of arrows, icons, and text to provide accurate navigation.

[0079] VR immersive perception: Commanders can use VR helmets to enter the virtual BIM accident scene, observe the overall situation from a first-person or God's perspective, conduct simulations and remote commands.

[0080] Command issuance and feedback: Commanders issue commands through the platform, and on-site personnel receive and provide feedback on the execution status through mobile terminals or AR devices, forming a closed-loop command.

[0081] Step S210: Event handling is completed and normal monitoring is resumed.

[0082] After the emergency incident is handled, the system cancels the warning state, clears the temporary emergency information, and the relevant modules return to normal monitoring, and returns to step S202.

[0083] Example 3 Suppose a fire breaks out in a residential building in a smart community.

[0084] 1. Fire detection and preliminary location (S202-S204): The smoke sensor (Sensor_Smoke_502) in Room 502 on the 5th floor was triggered first, sending an alarm signal to the system's multi-source data fusion module 140 via the IoT gateway. Almost simultaneously, the video analysis and mapping module 120 of the smart camera (Cam_Corridor_5F) installed in the 5th-floor corridor detected smoke pouring out of the crack in Room 502's door using its video analysis algorithm. The video analysis module mapped the smoke's location (2D image coordinates) to the BIM model, confirming the smoke's source as Room 502. The semantic event detection module 130, combining the function of Room 502 ("bedroom") in the BIM scene graph, the smoke sensor alarm, and the smoke confirmed by the video, determined a "fire in Room 502" event. Before the fire officially broke out, the GNN model, analyzing the video from the corridor camera near Room 502, might have detected an event: a person (possibly a resident) anxiously lingering near the door of Room 502 and repeatedly attempting to open it (through interactive analysis of the target and the BIM door model). This video semantic event will be associated with the later "smoke alarm in room 502" and the "internal opening record of the access control system in room 502 (if it is a smart door lock) (if the resident escapes)" in the event chain reconstruction step, forming a more complete picture of the initial development of the fire, which will help determine the cause of the fire or whether there are people trapped.

[0085] 2. Multi-source confirmation and situation escalation (S205-S206): Event reasoning and early warning module 150 receives a smoke alarm and video confirmation of a fire. The system queries the BIM model and discovers that there is no thermal imaging camera in room 502. However, Cam_Corridor_5F, located in the 5th-floor corridor, begins capturing residents (targets P_Res01 and P_Res02) running from their respective rooms (e.g., 501 and 503) and moving along the corridor toward the stairwell. Through cross-camera tracking, the system detects that some residents choose the east staircase, while others choose the west staircase. The BIM scene map indicates that room 502 is located near the east staircase. A preliminary event chain is formed: "Sensor_Smoke_502 alarm -> Cam_Corridor_5F detects smoke at the door of room 502 -> P_Res01 and P_Res02 ​​evacuate from the 5th floor toward the staircase." Based on the severity of the fire, the system immediately identifies it as an emergency.

[0086] 3. Dynamic emergency plan generation (S208): The emergency plan generation module 160 starts: Hazard assessment: Based on the location of Room 502 and the ventilation design in the BIM, it is preliminarily determined that the fire and smoke may spread preferentially to the east staircase.

[0087] Resource Location: Querying the BIM scene map reveals a fire hydrant (FH-5E, FH-5W) and a fire extinguisher box at each end of the 5th floor corridor. The nearest fire control room is on the 1st floor.

[0088] Evacuation Path: For residents on floors 5 and above, the system dynamically calculates evacuation paths. Because the fire is near the east staircase, the system temporarily reduces the access weight of the east staircase (Stair_E) at that floor's entrance or marks it as "high risk." Residents on floors 5 and above are prioritized to evacuate using the west staircase (Stair_W). Residents on floors 4 and below, if not affected by smoke, can use the nearest staircase. The system also checks the status of normally closed fire doors marked in the BIM model (assuming sensors provide feedback).

[0089] Rescue Path: Plan a route from the ground level entrance to the 5th floor fire for arriving firefighters, giving priority to hydrants close to the fire but upwind (such as FH-5W, if the wind direction is known or default).

[0090] Command generation: Generated commands include: notifying the community micro-fire station to go out; activating the fire broadcast and sound and light alarms on the 5th floor and above; recommending that residents on the 5th floor and above evacuate from the west staircase first; notifying the property control center to remotely start the smoke exhaust fan (if any and link it with the system).

[0091] 4. Visualization Command and AR / VR Linkage (S209): The 3D visualization and interaction module 170 performs: Command Center Large Screen: In the BIM 3D model, Room 502 is marked as a red fire point, and the simulated smoke spread area (based on preliminary CFD estimates or empirical models) is displayed semi-transparently. The recommended evacuation route is marked in the model with dynamic green arrows. The live feed from camera Cam_Corridor_5F is embedded in the display. Nearby fire hydrants and fire extinguishers are highlighted.

[0092] On-site AR guidance: For example, if a resident on the fifth floor (wearing community-issued AR glasses or using a mobile AR app) is preparing to evacuate via the east staircase, the AR interface will immediately display a red warning: "Large fire ahead, please change route!" and superimpose a green navigation arrow pointing to the west staircase. The first team members of the community micro-fire station to arrive will wear AR glasses, which will highlight the location of the FH-5W fire hydrant and provide brief operating instructions.

[0093] VR remote command: Fire commanders wearing VR headsets in the command center can immerse themselves in the fire scene on the fifth floor (a virtual scene synthesized from the BIM model and real-time sensor data), assess the fire's progress, and direct on-site firefighting and search and rescue operations. If the camera can see part of Room 502, that video footage will be projected onto the corresponding location in the VR scene.

[0094] Information linkage: If a resident reports their trapped location through a mobile app (for example, via GPS or manually selecting on a BIM map), the location will be displayed in real time on the command center's large screen and on the rescuers' AR devices.

[0095] 5. Incident Handling and Recovery: Firefighters arrive and conduct firefighting and rescue operations according to the emergency plan and on-site instructions. Once the fire is extinguished, all personnel have been evacuated, and there is no risk of re-ignition, the command center declares the emergency over. The system records the entire emergency process for subsequent analysis and plan optimization. The system then resumes normal monitoring.

[0096] In summary, the embodiment of the present invention discloses a community intelligent monitoring and emergency linkage method and system that integrates BIM spatial semantics, including: constructing a BIM scene graph to obtain the attributes and mutual relationships of components and spatial areas in the BIM model; mapping dynamic targets detected in video surveillance to the BIM model to obtain their spatial semantic information; constructing a target-environment interaction graph based on the BIM spatial semantic information of the dynamic targets, using a graph neural network model for training and reasoning to identify specific complex events related to the spatial context; using the BIM model as a spatiotemporal benchmark, integrating multi-source heterogeneous data, and using a graph-based event association algorithm based on spatiotemporal proximity, object consistency and predefined logical rules to reconstruct a complete event chain containing atomic event sequences and association relationships; when an emergency event is detected or the event chain indicates an emergency state, the optimal emergency plan is dynamically generated by combining BIM preset information and real-time sensor data, and visual command and dispatch are performed through BIM three-dimensional scenes and augmented reality.

[0097] Through the above embodiments, it can be seen that the method and system of the present invention can significantly improve the community's monitoring and early warning capabilities, emergency response speed and command and dispatch efficiency when facing emergencies such as fires, thereby maximizing the protection of residents' lives and property.

[0098] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A community intelligent monitoring and emergency linkage method integrating BIM spatial semantics, characterized by: The following steps are involved: S1: Constructing a community building information model (BIM) scene graph, wherein the BIM scene graph includes BIM component information, spatial region information and their respective semantic attribute information, and topological relationship information between the BIM components and spatial regions; S2: Acquire video surveillance data, detect dynamic targets therefrom, map the dynamic targets to the three-dimensional space corresponding to the BIM scene graph through coordinate transformation and geometric projection, and obtain BIM space semantic information of the dynamic targets; S3: Based on the BIM spatial semantic information of the dynamic target, a local interaction graph is constructed that includes the dynamic target node and its neighboring BIM element nodes. The local interaction graph is analyzed using a pre-trained graph neural network (GNN) event recognition model, or the behavior of the dynamic target is analyzed in combination with a preset event rule library to identify specific events related to the spatial context. The node features of the GNN model include the kinematic features of the dynamic target and the functional and state features of the BIM spatial elements, and the edge features represent the interactive relationships between nodes. S4: Using the BIM as a unified spatiotemporal reference, integrating multi-source heterogeneous data acquired by at least two different types of sensors, performing spatiotemporal alignment and correlation analysis, and forming an atomic event set; S5: Based on the topological relationship information in the BIM scene graph, the motion trajectory of the dynamic target in the BIM space, and the set of atomic events formed in S4, an event chain construction algorithm is used to screen and connect related atomic events by evaluating the time series relationship, spatial proximity relationship, object consistency, and predefined logical association rules between atomic events, and reconstruct a complete event chain with time sequence and logical association; S6: When an emergency event is identified in S3 or the event chain reconstructed in S5 indicates an emergency state, the optimal emergency plan for the current emergency event is dynamically calculated and generated based on the emergency resource information, evacuation path information, hazard source information preset in the BIM scene graph and the real-time sensor data integrated in S4; S7: The optimal emergency plan is visualized in the BIM three-dimensional scene, and navigation and instructions can be provided to on-site personnel through augmented reality (AR) equipment, or immersive situational awareness and command and dispatch support can be provided to command center personnel through virtual reality (VR) equipment.

2. The method according to claim 1, characterized in that The steps of constructing the BIM scene graph in S1 specifically include: Parse the IFC (Industry Foundation Classes) file or native format file of the BIM model to extract the geometric information, physical properties and functional properties of the BIM components; Perform semantic enhancement on the extracted BIM components and spaces, and supplement or modify their semantic labels in terms of functional use, safety level, and belonging area; By using graph database technology, BIM components and spatial areas are taken as nodes, and the inclusion, adjacency or connectivity relationships between them are taken as edges to construct the BIM scene graph.

3. The method according to claim 1, characterized in that The graph neural network (GNN) event recognition model in S3 is a graph attention network (GAT) or a relational graph convolutional network (R-GCN). The nodes of the input local interaction graph of the model include: dynamic target nodes, whose feature vectors include target category, normalized speed, movement direction, and relative position in the BIM space; BIM element nodes, whose feature vectors include element type, functional semantics, safety level, and current status; the edges of the local interaction graph are defined as: dynamic target-dynamic target edges, which represent the relative distance and speed between targets; dynamic target-BIM element edges, which represent the distance between the target and the BIM element, whether contact or crossing behavior occurs, and the interaction type; BIM element-BIM element edges, which inherit the topological relationship of the BIM scene graph.

4. The method according to claim 3, characterized in that The training process of the GNN event recognition model includes: collecting annotated video data containing multiple predefined event scenarios and synchronized BIM interaction data; constructing a corresponding local interaction graph sequence for each annotated event scenario; using the local interaction graph sequence and its event labels as training samples, and adopting a supervised learning method to optimize the model parameters by minimizing the cross-entropy loss function between the predicted event category and the true event label.

5. The method according to claim 1, wherein The event chain construction algorithm in S5 specifically includes: a. Define atomic events: Video semantic events identified by S3, sensor alarm events fused by S4, and access control records are uniformly represented as atomic events containing a timestamp, BIM spatial location ID, event type, a list of participating object IDs, and a confidence score. b. Construct a candidate event association graph: Using atomic events as nodes, directed edges are added between pairs of atomic events that meet the conditions according to preset association rules. The association rules include: temporal order rules, spatial proximity rules, object consistency rules, and causal inference rules; c. Event chain extraction: In the candidate event association graph, a weighted path search algorithm is used to extract one or more atomic event sequences as event chains. The weight of the path comprehensively considers the confidence of the atomic event and the strength of the associated edge; d. Event chain semantic annotation: Match the extracted event chain with a predefined complex event scenario template library to give the event chain a high-level semantic description.

6. The method according to claim 1, characterized in that The step of dynamically generating the optimal emergency plan in S6 specifically includes: Based on the fire location, hazardous gas leakage range, or intruder location determined by real-time sensor data, the access weights of relevant channels in the evacuation path are dynamically adjusted or marked as impassable in the BIM scene graph; Based on the current personnel distribution information and the updated evacuation path information, the Dijkstra algorithm is used to calculate the optimal evacuation path for each trapped person to the safe exit; Based on the type and location of the emergency event, it automatically finds and recommends the nearest available fire hydrants, fire extinguishers, first aid kits and other emergency resources marked in the BIM scene map.

7. A community intelligent monitoring and emergency linkage system integrating BIM spatial semantics, characterized by: include: A BIM semantic parsing module is used to construct a community building information model (BIM) scene graph, wherein the BIM scene graph includes BIM component information, spatial area information and their respective semantic attribute information, and topological relationship information between the BIM components and spatial areas; A video analysis and mapping module is used to acquire video surveillance data, detect dynamic targets, and map the dynamic targets into the three-dimensional space corresponding to the BIM scene graph to obtain its BIM space semantic information; A semantic event detection module is used to construct a local interaction graph containing the dynamic target node and its neighboring BIM element nodes based on the BIM spatial semantic information of the dynamic target, and analyze the local interaction graph using a pre-trained graph neural network (GNN) event recognition model, or combine it with a preset event rule library to identify specific events related to the spatial context; A multi-source data fusion module is used to fuse multi-source heterogeneous data acquired by at least two different types of sensors using the BIM as a unified spatiotemporal reference, perform spatiotemporal alignment and correlation analysis, and form an atomic event set; An event reasoning and warning module is used to reconstruct a complete event chain with temporal and logical associations using an event chain construction algorithm based on the topological relationship information in the BIM scene graph, the motion trajectory of dynamic targets in the BIM space, and the resulting atomic event set, and to issue a warning when an emergency event is identified or the event chain indicates an emergency state; An emergency plan generation module is used to dynamically calculate and generate an optimal emergency plan for the current emergency event based on the emergency resource information, evacuation route information, hazard source information preset in the BIM scene map, and integrated real-time sensor data after receiving an early warning; The 3D visualization and interaction module is used to visualize the optimal emergency plan in the BIM 3D scene, and can optionally provide navigation and instructions to on-site personnel through augmented reality (AR) equipment, or provide immersive situational awareness and command and dispatch support to command center personnel through virtual reality (VR) equipment.

8. The system according to claim 7, characterized in that The BIM semantic parsing module is specifically configured to parse BIM model files through the built-in IFC parsing engine and semantic ontology library, extract and enrich the semantics of BIM components and spaces, and use a graph database to store and manage BIM scene graphs.

9. The system according to claim 7, wherein: The graph neural network (GNN) event recognition model in the semantic event detection module has an architecture that includes multiple graph attention layers for learning weighted message passing between nodes, a graph pooling layer for generating graph-level representations, and a fully connected output layer for event classification. The model uses the Adam optimizer and the cross-entropy loss function during training.

10. The system according to claim 7, wherein: The event reasoning and warning module contains an event chain knowledge base for storing predefined atomic event types, association rules and complex event scenario templates, and is equipped with an event chain reasoning engine. The engine dynamically builds and updates event chains based on real-time generated atomic events and association rules, and matches them with the template library for high-level semantic annotation and warning judgment.

Images