An indoor fire evacuation guiding system and method based on air-ground cooperation

Abstract

The application discloses an indoor fire evacuation guiding system and method based on air-ground cooperation, and belongs to the technical field of public safety. The system comprises a distributed infrared monitoring subsystem, a central dispatching subsystem, a multi-machine cooperative flight subsystem and a digital twin terminal. The application constructs a UWB positioning and thermal imaging monitoring network through fixed nodes. The unmanned aerial vehicle uses UWB+ToF+IMU multi-modal fusion navigation technology to solve the navigation problem of visual failure in the indoor environment without GPS and under the condition of fire smoke. The system adopts a hierarchical response mechanism, identifies the fire through end-cloud cooperative AI, actively guides personnel evacuation through the sound and light signals of the unmanned aerial vehicle, and finally realizes full-factor three-dimensional visual command in the digital twin terminal. The application significantly improves the response speed and environmental adaptability of indoor fire emergency rescue.

Description

Technical Field

[0001] This invention belongs to the field of public safety technology, specifically relating to an indoor fire evacuation guidance system and method based on air-ground coordination. Background Technology

[0002] As urban buildings become increasingly large and complex, and the functionality and density of indoor spaces continue to rise, emergency response to fires and other emergencies faces severe challenges. In enclosed and complex indoor environments, traditional fire protection and evacuation systems have significant limitations in multiple aspects, including perception, positioning, navigation, and command, seriously affecting the timeliness, accuracy, and safety of emergency response.

[0003] Currently, common indoor fire monitoring mainly relies on point-type fire detectors such as smoke and heat detectors. Although these detectors can provide basic alarms, their sensing dimensions are limited, providing only a binary signal of "whether an alarm has been triggered." They cannot obtain crucial information such as the location of the fire source, temperature distribution, and the trend of fire spread. Furthermore, they lack the ability to verify alarm information through images, making them prone to false alarms (such as cooking smoke or dust) or missed alarms (such as smoldering fires), leading to resource misallocation or response delays.

[0004] In terms of indoor emergency navigation and guidance, two major challenges exist. Firstly, indoor positioning and navigation are difficult: the Global Navigation Satellite System (GPS) suffers severe signal attenuation or even complete failure indoors. Furthermore, vision-based Simultaneous Localization and Mapping (SLAM) technology faces difficulties in feature point extraction due to dense smoke and open flames, leading to a sharp decline in visual sensor performance and preventing mobile platforms such as drones from achieving stable positioning and path tracking. Secondly, evacuation guidance methods are passive: existing systems largely rely on fixed evacuation signs, emergency broadcasts, or handheld guides, failing to dynamically plan safe routes based on real-time changes in the fire situation or proactively and accurately guide each trapped individual away from dangerous areas, resulting in low guidance efficiency in chaotic environments.

[0005] Furthermore, existing command systems lack global visualization and collaborative command capabilities. Commanders typically rely on 2D drawings, scattered sensor alarm information, and personnel reports to understand the situation on-site, making it difficult to grasp key elements such as the three-dimensional temperature distribution of the fire, the precise location of trapped personnel, and the dynamics of rescue forces in real time and intuitively. This leads to delayed decision-making, suboptimal resource allocation, and insufficient coordination efficiency.

[0006] Therefore, there is an urgent need for an indoor fire emergency evacuation system that can integrate multi-dimensional perception, anti-interference positioning and navigation, active intelligent guidance, and global three-dimensional visualization command capabilities to improve the overall efficiency of fire emergency rescue in complex building environments. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide an indoor fire evacuation guidance system and method based on air-ground collaboration, which uses "fixed sentinels" to provide perception and positioning benchmarks, and "mobile worker bees" to achieve accurate verification and guidance through multimodal fusion navigation.

[0008] To achieve the above objectives, the present invention provides the following technical solution: An indoor fire evacuation guidance system based on air-ground collaboration includes: a distributed infrared thermal imaging monitoring subsystem, a central dispatch subsystem, and a multi-drone collaborative flight guidance subsystem. The distributed infrared thermal imaging monitoring subsystem consists of several fixed monitoring nodes deployed in various independent indoor spaces, configured to construct a real-time temperature field across the entire indoor area and establish a high-precision indoor positioning reference network. The central dispatch subsystem is communicatively connected to the fixed monitoring nodes and the multi-drone collaborative flight guidance subsystem, configured to perform multi-source data fusion, task decision-making, and drone swarm scheduling. The multi-drone collaborative flight guidance subsystem includes at least one guidance drone stationed within the central dispatch subsystem, configured to respond to dispatch commands to perform fire situation verification and personnel evacuation guidance tasks. The fixed monitoring nodes preferably use infrared thermal imaging sensors (such as the AMG8833) as core sensing elements, outputting a temperature matrix. They also integrate a UWB positioning base station module as the main base station for indoor positioning.

[0009] The main control chip for the guided drone is preferably a high-performance dual-core processor (such as the ESP32-S3), responsible for sensor data fusion and attitude calculation. Visual perception is achieved using a visual sensor (such as the OV2640). Positioning and ranging are achieved using a UWB positioning tag and a ToF (Time-of-Flight) laser ranging sensor (such as the VL53L0X). The ToF sensor provides accurate ground altitude data when smoke interference causes visual altitude hold to fail. The interaction module integrates an intelligent voice module and a full-color LED light strip.

[0010] As a further preferred embodiment of the present invention, it also includes a digital twin visualization command terminal; the digital twin visualization command terminal is communicatively connected to the central dispatch subsystem, and is used to receive fused data and to map the indoor fire distribution, personnel location and drone flight trajectory in real time in a virtual three-dimensional space.

[0011] Distributed Infrared Thermal Imaging Monitoring Subsystem: Monitoring nodes integrating infrared array sensors and UWB positioning base stations are deployed in each room of the building. They serve as both "sentinels" for heat source detection and "anchors" for the indoor positioning system.

[0012] Central Dispatch Subsystem: As the edge computing core of the system (physical form can be a smart charging dock or server), it has a built-in high-performance AI acceleration unit, receives data to perform deep learning inference, and plans safe routes to avoid high-temperature areas based on BIM maps and UWB coordinate systems.

[0013] Multi-drone cooperative flight guidance subsystem: Composed of several guided UAVs, it possesses unique anti-smoke navigation logic. In smoke-free environments, it hovers using optical flow; when dense smoke obscures vision, it automatically switches to a fusion navigation mode of "IMU + UWB (XY axis positioning) + ToF (Z axis altitude hold)".

[0014] Digital twin visualization command terminal: Based on UWB coordinate data, drone icons and heat maps are accurately mapped onto the building BIM 3D model to achieve virtual-real synchronization.

[0015] As a further preferred embodiment of the present invention, the fixed monitoring node includes an infrared array temperature sensor configured to output multi-pixel temperature matrix data; the fixed monitoring node integrates an ultra-wideband (UWB) positioning base station module, and several of the nodes jointly construct a positioning coordinate system in the indoor space; the fixed monitoring node has a built-in microprocessor configured to, by comparing the temperature gradient changes of adjacent frames, trigger a level one alarm and send abnormal data containing its own UWB coordinates to the central dispatch subsystem when the temperature or temperature rise rate of any pixel exceeds a preset threshold.

[0016] As a further preferred embodiment of the present invention, the guided UAV is equipped with an onboard main control unit, a visual acquisition module, an intelligent voice module, a UWB positioning tag, and a laser rangefinder (ToF) sensor. The onboard main control unit is configured to execute multimodal fusion navigation logic: in a first mode, hovering and flight are achieved by combining optical flow data acquired by the visual acquisition module with inertial measurement unit (IMU) data; in a second mode, when the ambient smoke concentration is detected, causing the confidence level of the visual data to be lower than a preset value, the system automatically switches to an anti-smoke navigation mode, acquires planar coordinates (X, Y axis data) using the UWB positioning tag, acquires relative ground altitude (Z axis data) using the laser rangefinder, and performs closed-loop flight control by combining the IMU data.

[0017] The system workflow is as follows: Anomaly Detection (Level 1 Response): When a fixed monitoring node detects that the temperature exceeds a preset threshold (e.g., 60℃), it immediately packages the alarm data and its own UWB coordinates and sends them to the central dispatch subsystem. Precise Arrival and Mode Switching: The central dispatch subsystem directs the drone to fly to the alarm coordinates. The drone monitors the image quality of the visual sensor in real time: Normal mode: If the image is clear, it flies using the optical flow module combined with the IMU. Anti-smoke mode: If it enters a dense smoke area, the optical flow feature points are lost. The main control chip immediately initiates multi-sensor fusion logic: It uses UWB tags to obtain the absolute coordinates of the plane (X, Y) and corrects position drift; it uses ToF lidar to lock the altitude (Z-axis) and measures the distance to the ground through the smoke; Fusion algorithm: It adopts the extended Kalman filter (EKF) algorithm to fuse the UWB position observation, the ToF altitude observation, and the IMU acceleration / angular velocity prediction values ​​to calculate the drone's three-dimensional pose in the case of complete visual failure, achieving "blind flight" hovering and navigation. Edge-cloud collaborative verification: The drone transmits the collected key frames back to the central dispatch subsystem, where the high-computing AI model on the server determines whether it is an open fire. Cluster guidance (secondary response): After confirming a fire, the drone searches for trapped personnel. The system uses LED light strips to emit a green light stream pointing towards the safety exit, and uses UWB positioning to maintain a preset distance (e.g., 1-2 meters) in front of personnel, thus achieving proactive evacuation guidance.

[0018] As a further preferred embodiment of the present invention, the system adopts an "edge-cloud collaborative" computing architecture: the UAV runs a motion detection algorithm on the edge side, and when a dynamic target is captured, it extracts key frame images and sends them back to the central scheduling subsystem; the central scheduling subsystem is equipped with an AI inference unit, which performs high-precision inference on the returned key frames to identify flames, smoke and human targets. The drone is also equipped with a controllable light source array. When performing guidance tasks, it emits directional guidance light streams according to the planned path and controls the voice module to play directional evacuation commands.

[0019] As a further preferred embodiment of the present invention, the digital twin visualization command terminal includes a map rendering module and a multi-source data fusion module. The map rendering module constructs an indoor three-dimensional base model based on building BIM information. The multi-source data fusion module is configured to generate a dynamic thermal layer from the temperature matrix uploaded by fixed monitoring nodes using an interpolation algorithm, and overlay it onto the three-dimensional base model. Simultaneously, based on UWB coordinate data, it maps the real-time video stream transmitted by the UAV and the UAV virtual model to the corresponding coordinates on the three-dimensional map. By reading data from the central dispatch subsystem, the command terminal interpolates the temperature matrix of the monitoring nodes into a continuous thermal layer and overlays it on the BIM map. At the same time, it maps the UWB real-time coordinates of the UAV to three-dimensional space, allowing the commander to intuitively see the direction of fire spread and the progress of rescue efforts.

[0020] As a further preferred embodiment of the present invention, the microprocessor is used to execute the following anomaly detection logic: Time series analysis is performed on the temperature matrix data of multiple consecutive frames to calculate the temperature gradient and temperature rise acceleration of each pixel region. When the temperature gradient of any pixel region is continuously positive and the temperature rise acceleration exceeds the first threshold, but the absolute temperature does not reach the second threshold, it is determined to be a potential smoldering fire source and an early warning signal is triggered. When the absolute temperature exceeds the second threshold or the temperature rise rate exceeds the third threshold, a fire alarm signal is triggered.

[0021] As a further preferred embodiment of the present invention, when the guided UAV is performing personnel evacuation guidance tasks, the onboard main control unit is used to execute the following cooperative guidance logic: The visual acquisition module identifies the outline of the nearest person and calculates the person's direction of movement and speed. Based on its own coordinates, the planned safe path exit coordinates, and the identified personnel location provided by the ultra-wideband positioning tag, the flight attitude and speed are dynamically adjusted to always maintain a preset distance ahead of the personnel's direction of travel. The controllable light source array is synchronously controlled to project a dynamic optical path pointing to the next navigation node, and the voice module is controlled to play voice guidance consistent with the direction of the optical path.

[0022] As a further preferred embodiment of the present invention, the central scheduling subsystem includes a path planning module and a hierarchical response strategy module; The path planning module is used to perform the following operations: based on the real-time dynamic heat map provided by the digital twin visualization command terminal, the load-bearing structure and evacuation channel data from the building information model, and the personnel distribution information identified by the guiding drone, it generates a dynamic risk map, predicts the direction and speed of fire spread, and plans a path for the drone that avoids the current high temperature zone, predicts the fire spread risk to be lower than the threshold and the structural safety redundancy to be higher than the threshold, and plans differentiated evacuation paths for different drones in the same area. The graded response strategy module is used to perform the following operations: dynamically dispatching a drone cluster based on the received alarm level and review results, including dispatching a single drone for review when an early warning signal is received, and dispatching multiple drones to form a formation and assigning different guidance zones when a fire is confirmed.

[0023] As a further preferred embodiment of the present invention, the onboard main control unit of the guided UAV also performs the following cooperative guidance operations: By analyzing the video stream of the visual acquisition module, a lightweight neural network model is used to calculate the direction and speed of the person's movement, and to make a preliminary judgment on the behavior of the person in front, including walking normally, running, crawling forward or remaining still. When a person is detected to be stationary or behaving abnormally, the guidance strategy is adjusted, including: The controllable light source array is switched to a high-frequency flashing mode and its brightness is increased in an attempt to wake the user or attract attention. Control the voice module to play more urgent and explicit voice instructions; Send an abnormal personnel location report to the central dispatch subsystem, requesting additional attention or dispatching other drones for assistance.

[0024] An indoor fire evacuation guidance method based on air-ground coordination includes the following steps: S1: Utilize distributed fixed monitoring nodes to scan the indoor temperature field in real time and construct a UWB indoor positioning network; S2: When a monitoring node detects an abnormal temperature, it sends an alarm signal and its own coordinates to the central dispatch subsystem. S3: The central dispatch subsystem plans a path based on the alarm coordinates and dispatches and guides the drone to the target area for visual verification. S4: The guided drone assesses environmental visibility in real time during flight and switches between pure visual navigation mode and anti-smoke navigation mode based on visibility. S5: If a fire is confirmed, trigger a system-wide emergency state and dispatch a cluster of drones to perform zoned evacuation guidance.

[0025] As a further preferred embodiment of the present invention, the anti-smoke navigation mode described in step S4 specifically includes: when the visual sensor fails due to smoke obstruction, the system abandons the optical flow positioning data; reads the absolute position coordinates (X, Y) calculated by communication between the UWB tag and the indoor base station; reads the ground altitude data (Z) measured by the ToF laser sensor; fuses the acceleration and angular velocity data of the IMU, and calculates the UAV attitude through the Kalman filter algorithm to achieve blind flight and hovering in environments without GPS and without vision.

[0026] The beneficial effects of this invention are as follows: 1. Achieves multi-dimensional, proactive, and precise early fire detection: Addressing the shortcomings of background technologies that suffer from "single perception dimension (lack of image verification)," this invention constructs a real-time indoor temperature field across the entire area through a distributed infrared thermal imaging monitoring subsystem, outputting temperature matrix data. Combined with anomaly detection logic, it can identify potential smoldering fire sources based on temperature gradients and temperature rise acceleration, achieving early warning. This overcomes the limitations of traditional point detectors that only provide binary alarms, offering multi-dimensional information such as fire source location, temperature distribution, and evolution trends, significantly improving the accuracy and timeliness of alarms and saving valuable time for subsequent verification and response.

[0027] 2. Overcoming the Challenges of Reliable Indoor Positioning and Autonomous Navigation in Complex Fire Environments: Addressing the challenge of "difficult indoor positioning and navigation (no GPS signal indoors and smoke interfering with visual SLAM navigation)" in the background technology, this invention provides the system with a high-precision position reference unconstrained by GPS signals through the construction of a UWB indoor positioning reference network. In particular, through a defined multimodal fusion navigation scheme, the guided UAV can adaptively switch between pure visual navigation and smoke-resistant navigation modes. When smoke causes visual failure, it can seamlessly switch to a fusion navigation mode based on UWB (planar positioning), laser ranging (altitude hold), and IMU, achieving stable "blind flight" in harsh environments without GPS and with visual interference, ensuring the mission reliability of the system's core mobile platform (UAV).

[0028] 3. Provides dynamic, proactive, and intelligent personnel evacuation guidance: Addressing the problem of "passive evacuation guidance" in the background technology, this invention achieves a transformation from passive instruction to proactive guidance through the defined functions of the guidance drone. The drone can project dynamic light flow using a controllable light source array according to the safe path planned by the central dispatch subsystem, and play directional instructions in conjunction with a voice module, providing clear and intuitive forward guidance for personnel. Furthermore, through the aforementioned personnel status perception and interaction logic, the drone can initially determine the behavioral status of personnel and adopt enhanced wake-up and guidance strategies for stationary or abnormally moving personnel, achieving differentiated and humanized intelligent guidance, greatly improving evacuation efficiency and success rate in chaotic environments.

[0029] 4. A global, real-time, and virtual-real integrated 3D visualization command center was constructed: Addressing the deficiency of "lack of global visualization" in the background technology, this invention integrates multi-source information such as temperature data from dispersed fixed monitoring nodes, real-time location and video streams of drones, and building BIM models through a digital twin visualization command terminal, generating a 3D real-scene map overlaid with dynamic thermal layers. Commanders can simultaneously and intuitively grasp all elements of information, including the fire spread, the distribution of trapped personnel, and the drone operation trajectory, in a virtual 3D space, achieving "one map for overall view, virtual-real synchronization," significantly improving the global perspective, real-time performance, and scientific rigor of command decisions, and optimizing the coordinated dispatch of emergency resources.

[0030] 5. A highly efficient operational system with tiered response and edge-cloud collaboration has been established: This invention, through its constructed system architecture and logic, achieves a complete closed loop from anomaly detection, drone scheduling, fire situation verification to cluster guidance. The system adopts a tiered response mechanism, intelligently scheduling resources according to the alarm level (early warning / fire alarm) to avoid overreaction. Simultaneously, it employs an edge-cloud collaborative computing architecture, with the drone end focusing on real-time motion detection and navigation, and the cloud focusing on high-computing-power-demand AI recognition and global decision-making. This optimizes the allocation of computing power and maximizes the overall system efficiency, ensuring a rapid, orderly, and efficient process from early warning to evacuation. Attached Figure Description

[0031] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is a schematic diagram of the overall system topology of the present invention; Figure 2 This is a block diagram showing the hardware module connection of the multi-aircraft cooperative flight guidance UAV of the present invention; Figure 3 A schematic diagram of the interface of a digital twin visualization command terminal; Figure 4 System hierarchical response and logic control flowchart. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail and completely below with reference to the accompanying drawings and preferred embodiments. The embodiments described in this section are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0033] like Figures 1-4 As shown, this invention discloses an indoor fire evacuation guidance system and method based on air-ground coordination.

[0034] The system hardware architecture and deployment are as follows: like Figure 1 and Figure 2 As shown, this system comprises three core hardware components: fixed monitoring nodes, a swarm of guided drones, and a central dispatch subsystem (which can be physically represented as a smart charging dock or an edge server).

[0035] 1. Hardware implementation of fixed monitoring nodes: Fixed monitoring nodes act as the system's "fixed sentinels," deployed in key indoor areas (such as room ceilings and both ends of corridors). Each node contains: Core sensing unit: Employs an infrared array temperature sensor (such as AMG8833), which outputs 8x8 pixel temperature matrix data, with each pixel corresponding to the temperature value of a region. The sampling frequency can reach 10Hz, thereby constructing a discrete temperature field for the region.

[0036] Positioning and Communication Unit: The Decawave DW1000 series ultra-wideband (UWB) chip is integrated as a positioning base station (anchor). All nodes are connected to the central dispatch subsystem via wired or wireless networks (such as LoRa or Wi-Fi). At the same time, the UWB base station modules of each node synchronize with each other by ranging and clock, and jointly establish a global UWB positioning coordinate system indoors.

[0037] Processing unit: Employs a low-power microprocessor (such as the STM32 series) to read and preprocess temperature matrix data in real time, execute the aforementioned anomaly detection algorithm, and, when an alarm is triggered, package and send the alarm information (type, level) along with the precise UWB coordinates of this node.

[0038] This hardware deployment scheme achieves the integrated construction of a sensing and positioning reference network. The nodes not only provide continuous temperature field data, overcoming the shortcomings of traditional point detectors in terms of limited information, but also establish a high-precision indoor positioning reference covering the entire area and unrestricted by GPS signals, providing fundamental support for UAV navigation.

[0039] 2. Hardware implementation for guiding unmanned aerial vehicles (UAVs): The guided drone, acting as the system's "mobile worker bee," has the following hardware configuration: Figure 2 As shown, it mainly includes: Main control and navigation unit: Main control chip: It adopts a high-performance processor (such as ESP32-S3) and is responsible for the data fusion of all sensors and the calculation of flight control logic.

[0040] Visual sensor: Equipped with a global shutter camera (such as OV2640) for optical flow calculation.

[0041] Positioning and elevation module: Equipped with a UWB tag (such as DW1000) to receive signals from a fixed base station to calculate its own (X, Y) plane coordinates; Equipped with a ToF laser ranging sensor (such as VL53L1X) to accurately measure its altitude (Z) above the ground.

[0042] Inertial Measurement Unit (IMU): Integrated six-axis IMU (MPU6050) provides acceleration and angular velocity data.

[0043] Task execution unit: Controllable light source array: The underside of the unit is equipped with a programmable RGB LED light strip, which can dynamically display light flow with different colors, brightness and flow patterns.

[0044] Voice module: integrates a speaker and an audio decoding chip, and can play pre-stored or real-time voice commands.

[0045] Communication unit: Equipped with a Wi-Fi or 5G module for high-speed data interaction with the central dispatch subsystem.

[0046] This hardware configuration provides the physical foundation for multimodal fusion navigation and active interactive guidance. A rich array of sensors ensures that at least one reliable positioning and altitude-keeping solution is available even in complex environments, while the audio-visual interaction module enables a leap from static signage to dynamic guidance.

[0047] Construction of a digital twin visualization command terminal: The digital twin visualization command terminal is the system's visual interactive interface. Its core function is to fuse and map multi-source, heterogeneous data from the physical world into a unified, real-time virtual 3D model, achieving virtual-real synchronization and a comprehensive overview.

[0048] Terminal architecture and data flow: Command terminal software runs on high-performance workstations or servers, and its logical architecture includes: Data Access Layer: Subscribes to the central scheduling subsystem's data bus in real time via dedicated communication protocols (such as MQTT / WebSocket), continuously receiving three types of core data streams: Situational data stream: Temperature matrix data and its UWB coordinates from all fixed monitoring nodes.

[0049] Dynamic data stream: real-time UWB position coordinates, flight status, and key video frames transmitted back from all guided UAVs.

[0050] Event data stream: alarm signals, task instructions, and AI recognition results (such as "open flame confirmed in area A" and "trapped personnel found at point B") from the central dispatch subsystem.

[0051] Core processing layer: 3D Engine and Map Rendering Module: Using 3D engines such as Unity3D or Unreal Engine, import the building's BIM (Building Information Model) file (such as IFC format) to generate a high-fidelity indoor 3D base model, including precise geometric and semantic information such as rooms, corridors, stairs, doors and windows.

[0052] Dynamic generation of heatmaps: Spatial interpolation is performed using the inverse distance weighted (IDW) interpolation algorithm on the received discrete temperature matrix data. For each vertex V(x,y,z) of the 3D model surface to be valued, its temperature value is... Calculated from data from N surrounding monitoring nodes with known temperatures: in, Let be the temperature of the i-th node. Let V be the spatial distance from vertex V to the node, and P be a power parameter (usually 2). Through real-time calculation, the entire 3D model surface is rendered as a continuous dynamic thermal layer from blue (low temperature) to red (high temperature) and updated animatedly over time.

[0053] Entity mapping and synchronization: A virtual model (such as a drone icon) is created for each guided UAV in a 3D scene. Through coordinate transformation, the actual UWB coordinates of the UAV are mapped to the target model. It can drive the position and orientation of its virtual model in real time. At the same time, the real-time video stream transmitted back by the drone can be bound next to its virtual model in a "picture-in-picture" format, or projected onto the wall texture corresponding to the 3D model.

[0054] Interactive presentation layer: Provides users (commanders) with a graphical interface that supports rotation, scaling, and translation of 3D scenes, as well as layer control (such as showing / hiding heatmaps, personnel markers, drone trajectories, etc.).

[0055] Workflow and Collaboration: Initial loading: After the system starts, the command terminal loads the BIM model and presents a static three-dimensional digital twin of the building.

[0056] Real-time fusion: As data from fixed nodes is continuously transmitted, the terminal updates the overall heat map in real time, showing the origin of the fire, the high-temperature core area, and its spread trend. After a drone takes off, its virtual model immediately appears on the 3D map and moves along the planned path. Commanders can click on any virtual drone at any time to view its first-person view video, remaining battery power, current mission, and other information. When the AI ​​identifies flames or personnel, the corresponding location will be highlighted in the 3D model (e.g., a flame symbol or a personnel icon).

[0057] Command and Decision Support: Commanders can plan interventions based on the overall situation: directly clicking or selecting on the 3D map to assign new target points or paths to drones. Resource Allocation: Clearly understanding the operational status and location of each drone, and instructing specific drones to support key areas. Situation Simulation: Combining historical heat map data to help determine the possible direction of fire spread.

[0058] This system integrates the scattered drawings, sensor lists, and video surveillance footage from traditional fire command into a unified, spatially accurate 3D visualization environment. Its core value lies in: Intuition: Transforming abstract coordinate data and alarm information into intuitive, spatially accurate 3D images (heat maps, drone icons), significantly reducing the information comprehension threshold and command decision-making time. Integration: Achieving spatiotemporal synchronization and overlay of multi-dimensional data such as temperature fields, personnel locations, drone status, and video streams, providing unprecedented overall situational awareness capabilities. Collaboration: Providing commanders with a virtual interface to interact with the physical system, enabling remote command, precise dispatch, and collaborative intervention, truly achieving intelligent command.

[0059] Graded Fire Sensation Detection and Early Warning: The microprocessor within the fixed monitoring node executes the following anomaly detection algorithm: Data preprocessing: Filter the temperature matrix T(t) of consecutive frames (e.g., the most recent 10 frames) to eliminate noise.

[0060] Feature calculation: Calculate the temperature gradient of each pixel (i,j) within the time window Δt. Calculate the acceleration due to temperature rise .

[0061] Grading judgment: Early warning: If a certain pixel area meets the following conditions >0 (continuous positive gradient) and (Acceleration exceeds the first threshold, such as 5°C / s²), and at the same time If the absolute temperature is below the second threshold, such as 60°C, it is identified as a potential smoldering ignition source, triggering an "early warning signal". Fire alarm: If a certain pixel area meets the requirements (Absolute temperature greater than or equal to the second threshold) or If the rate of temperature rise exceeds the third threshold, such as 20°C / s, a fire alarm signal will be triggered immediately.

[0062] Information reporting: Regardless of the type of alarm triggered, the alarm type, level, and UWB coordinates of this node will be reported. Send to the central dispatch subsystem.

[0063] This algorithm, through time series analysis, achieves a shift from a binary judgment of "whether the temperature is too high" to intelligent early warning of "how quickly it is approaching danger." In particular, the early warning function can detect risks in the smoldering stage (before visual and smoke detection triggers), buying valuable time for response and greatly enhancing the system's foresight.

[0064] Anti-smoke multimodal fusion navigation: The onboard main control unit of the guided drone executes the following navigation logic, the process of which is as follows: Figure 4 As shown: Environmental assessment: Real-time analysis of the number of feature points and contrast in visual sensor images to calculate "visual confidence level". Set a threshold. .

[0065] Mode switching decision: like Enter pure visual navigation mode: integrate optical flow displacement increments Using IMU data, stable hovering and flight are achieved through PID control.

[0066] like (Indicates entering a smoke zone) Immediately switch to anti-smoke navigation mode.

[0067] Detailed process of anti-smoke navigation mode: Data acquisition: Communicate with at least three base stations via UWB tags to calculate the absolute planar coordinates of the UAV. The altitude above the ground is obtained through a ToF sensor. .

[0068] Read the acceleration a and angular velocity ω from the IMU.

[0069] Kalman filter fusion (formula explanation): Define state vector It includes position and velocity.

[0070] State prediction (based on IMU): in, Here is the state transition matrix. The control input (displacement increment) is obtained by integrating the IMU. The input matrix is ​​given.

[0071] Observation Updates (based on UWB and ToF): Observation equations ,in For the observation matrix, To observe noise.

[0072] Kalman gain calculation and state update: in, To predict covariance, To observe the noise covariance, For Kalman gain. Final output. This refers to the smooth and reliable 3D pose estimation after fusion.

[0073] Control output: The fused pose estimate is input into the flight control algorithm to generate motor control commands, enabling precise hovering and path tracking.

[0074] This innovative solution combines the absolute positioning of UWB, the precise altitude measurement of Time-of-Flight (ToF), and the short-term high dynamic range of an IMU, and then optimizes the fusion using Kalman filtering. Even in dense smoke where vision is completely impaired, the drone can still fly stably, fundamentally solving the navigation problem for drones in the core area of ​​a fire.

[0075] Dynamic risk path planning and cluster scheduling: The path planning module and the hierarchical response strategy module of the central dispatch subsystem work together: 1. Dynamic Risk Map Construction: The system receives temperature matrices uploaded from all fixed nodes and uses bilinear interpolation to generate a continuous dynamic thermal layer H(x,y,t) covering the entire floor. It extracts structural information layers S(x,y) from the building BIM model, including load-bearing walls, evacuation routes, and door / window locations. It also receives personnel distribution layers P(x,y,t) reported by drones. By fusing the above information, a dynamic risk map R(x,y,t)=f(H,S,P) is generated, where high-risk areas are those with high temperatures, fragile structures, or dense populations.

[0076] 2. Intelligent Path Planning: When planning a path for a drone to a target point, the objective function is to minimize the path integral risk. It also constrains path distance and turning angle. Prediction and differentiation: Based on historical data from heat maps, it predicts the direction of fire spread and plans differentiated evacuation routes for different drones in space and time to avoid congestion at exits.

[0077] 3. Tiered Response and Cluster Scheduling: Level 1 Response (Early Warning): Upon receiving an early warning signal, a single drone is dispatched to the alarm coordinates for detailed infrared thermography and close-range image capture verification. Level 2 Response (Confirmed Fire): When verification confirms an open flame or a fire alarm signal is received directly, an emergency state is triggered. Multiple drones are dispatched from the charging dock. Zoning: Based on the risk map and personnel distribution, the area surrounding the fire scene is divided into several guidance zones. Allocation: Each drone is assigned a zone and an optimal safe path. Coordination: Drones locate and guide personnel within their respective zones, while the central dispatch subsystem monitors the overall situation and dynamically adjusts zone boundaries and drone tasks.

[0078] This solution upgrades path planning from a static "shortest path" to a dynamic "safest path." By integrating real-time fire conditions, building structure, and personnel dynamics, it achieves intelligent risk avoidance and predictive adjustments to navigation paths. A tiered response and cluster scheduling mechanism enables efficient and precise allocation of system resources, avoiding resource waste and ensuring sufficient guidance coverage during real fires.

[0079] Personnel Status Perception and Intelligent Interaction: When a guided UAV performs a guidance mission, its onboard main control unit has the following logic: Personnel Status Recognition: A lightweight convolutional neural network (such as MobileNet SSD) is run on the video stream from the visual acquisition module to detect personnel bounding boxes in real time. By tracking the changes in the center point position of the bounding boxes in consecutive frames, the movement speed and direction of the personnel are calculated. Based on the movement speed threshold and posture classification model, the personnel status is classified as: walking normally, running, crawling, and stationary.

[0080] Adaptive guidance strategy: For moving personnel (normal / running / crawling): The drone stays in front of their direction of travel (e.g., 1.5 meters), projects a green flowing light strip pointing to the next turning point, and plays regular voice prompts such as "Please follow the light".

[0081] For stationary / abnormal individuals: Trigger the enhanced interaction strategy: a. Light stimulation: Control the light source array to switch to a high-frequency (e.g., 5Hz) red and blue alternating flashing mode, increase the brightness to the maximum, and attempt to wake up or attract strong attention.

[0082] b. Voice Enhancement: Play more directive and urgent voice prompts, such as "Danger is XX meters ahead! Please get up immediately and follow the flashing lights to evacuate in the XX direction!"

[0083] c. Information Reporting: Immediately send the person's precise coordinates and status to the central dispatch subsystem, marking them as "requiring close monitoring." The central dispatch subsystem can then dispatch another nearby drone to assist or notify rescue personnel.

[0084] This plan represents a leap from "path guidance" to "person guidance." Differentiated strategies are employed for individuals in different states, with enhanced stimulation and reporting for those who may be unconscious or panicked. This significantly improves the human element and ultimate success rate of evacuation operations, ensuring the rescue objectives are met.

[0085] Overall system workflow: Combination Figure 4 The flowchart shows the entire process of the system from startup to completion of evacuation: Initialization and Network Construction (S1): The system starts up, all fixed monitoring nodes go online, the UWB base station network self-calibrates, and the indoor positioning coordinate system is established. The digital twin terminal loads the BIM model and presents the initial 3D scene.

[0086] Anomaly Detection and Alarm (S2): Fixed nodes continuously monitor. When a node detects an anomaly, it triggers an alarm signal and sends it along with its own coordinates to the central dispatch subsystem.

[0087] Scheduling and Verification (S3, S4): Upon receiving an alarm, the central scheduling subsystem immediately invokes the path planning module to generate a safe path from the drone nest to the alarm point. A standby drone is scheduled to fly along the planned path. During flight, the drone adaptively switches navigation modes to reliably reach the target area. After hovering, the drone captures on-site images via its camera and transmits keyframes back. The AI ​​inference unit of the central scheduling subsystem (e.g., deploying a YOLOv5 model) identifies the images to determine the presence of flames / smoke.

[0088] Confirmation and Cluster Guidance (S5): If the AI ​​verifies and confirms the fire, the central dispatch subsystem triggers a system emergency. Multiple drones are dispatched to form a formation, and guidance zones and paths are assigned. Each drone flies to its designated zone, actively searching for and identifying the situation using audio-visual signals, and guiding personnel within that zone to evacuate along safe routes. Throughout the process, the digital twin command terminal integrates all data in real time, dynamically displaying heat maps, drone positions, trajectories, and personnel gathering points in a 3D model, providing commanders with global situational awareness.

[0089] The aforementioned complete workflow systematically integrates all innovative aspects from perception, decision-making, control to feedback, forming a complete technological closed loop from "early detection" to "precise arrival," then to "efficient guidance" and "full visibility." Through organic coordination via a central dispatch subsystem, it ultimately achieves a revolutionary improvement in indoor fire emergency evacuation response speed, environmental adaptability, and overall efficiency.

[0090] The above embodiments are merely preferred embodiments of the present invention. Any non-substantial modifications or substitutions made by those skilled in the art after understanding the core ideas of the present invention should be included within the protection scope of the present invention.

Claims

1. An indoor fire evacuation guidance system based on air-ground coordination, characterized in that, include: Distributed infrared thermal imaging monitoring subsystem, central dispatch subsystem, and multi-aircraft cooperative flight guidance system; The distributed infrared thermal imaging monitoring subsystem includes multiple fixed monitoring nodes deployed indoors, used to monitor and construct an indoor temperature field in real time, and to jointly construct an indoor positioning reference network based on the positioning base station modules built into each fixed monitoring node. The central dispatch subsystem is communicatively connected to the distributed infrared thermal imaging monitoring subsystem and the multi-aircraft cooperative flight guidance subsystem, and is used to receive and fuse monitoring data, generate mission decisions, and dispatch UAV clusters. The multi-aircraft cooperative flight guidance subsystem includes multiple guidance drones, which are used to respond to the dispatch instructions of the central dispatch subsystem and fly to the target area to perform fire situation verification and personnel evacuation guidance tasks.

2. The indoor fire evacuation guidance system based on air-ground coordination according to claim 1, characterized in that: It also includes a digital twin visualization command terminal; the digital twin visualization command terminal is communicatively connected to the central dispatch subsystem and is used to receive multi-source information including temperature data, positioning data and video data, and to map and display the indoor fire distribution, personnel location and drone flight trajectory in real time in a virtual three-dimensional scene constructed based on building information model; The digital twin visualization command terminal includes a map rendering module and a data fusion module; The map rendering module is used to generate an indoor 3D model based on the building information model; The data fusion module is used to interpolate the temperature matrix data uploaded by the fixed monitoring node to generate a dynamic thermal layer, and superimpose it onto the three-dimensional model. At the same time, it maps the real-time coordinates of the guided UAV and its returned video stream to the corresponding spatial position in the three-dimensional model.

3. The indoor fire evacuation guidance system based on air-ground coordination according to claim 2, characterized in that: The fixed monitoring nodes include: Infrared array temperature sensor, used to output temperature matrix data of the area it is in; Ultra-wideband positioning base station module, used to collaboratively construct an indoor positioning coordinate system with other nodes and drones; The microprocessor is used to process the temperature matrix data. When the temperature or temperature rise rate is detected to exceed a preset threshold, an alarm is triggered and an alarm message containing the coordinates of the node is sent to the central scheduling subsystem.

4. The indoor fire evacuation guidance system based on air-ground coordination according to claim 1, characterized in that: The guided UAV is equipped with an onboard main control unit, a visual acquisition module, an ultra-wideband positioning tag, a laser rangefinder, and an inertial measurement unit. The airborne main control unit is used to execute multimodal fusion navigation, including a pure vision navigation mode and an anti-smoke navigation mode. In pure vision navigation mode, the optical flow data acquired by the vision acquisition module is fused with the data from the inertial measurement unit to achieve flight control; In the anti-smoke navigation mode, when the confidence level of visual data is lower than the preset value due to ambient smoke, the system switches to the anti-smoke navigation mode. In this mode, the system integrates the planar coordinates provided by the ultra-wideband positioning tag, the ground altitude data provided by the laser ranging sensor, and the data from the inertial measurement unit to achieve flight and hovering control without visual reference. The motion detection algorithm for the guided UAV is used to capture dynamic targets and extract images to be transmitted back to the central scheduling subsystem. The central dispatch subsystem is equipped with an AI inference unit, which is used to identify the returned images and determine whether there are flames, smoke or human targets. The guided drone is also equipped with a controllable light source array and a voice module. When performing guidance tasks, it controls the light source array to emit directional light streams according to the planned path and controls the voice module to play directional evacuation prompts.

5. An indoor fire evacuation guidance system based on air-ground coordination according to claim 3, characterized in that: The microprocessor is used to execute the following anomaly detection logic: Time series analysis is performed on the temperature matrix data of multiple consecutive frames to calculate the temperature gradient and temperature rise acceleration of each pixel region. When the temperature gradient of any pixel region is continuously positive and the temperature rise acceleration exceeds the first threshold, but the absolute temperature does not reach the second threshold, it is determined to be a potential smoldering fire source and an early warning signal is triggered. When the absolute temperature exceeds the second threshold or the temperature rise rate exceeds the third threshold, a fire alarm signal is triggered.

6. The indoor fire evacuation guidance system based on air-ground coordination according to claim 4, characterized in that: When the guided UAV is performing personnel evacuation guidance tasks, the onboard main control unit is used to execute the following cooperative guidance logic: The visual acquisition module identifies the outline of the nearest person and calculates the person's direction of movement and speed. Based on its own coordinates, the planned safe path exit coordinates, and the identified personnel location provided by the ultra-wideband positioning tag, the flight attitude and speed are dynamically adjusted to always maintain a preset distance ahead of the personnel's direction of travel. The controllable light source array is synchronously controlled to project a dynamic optical path pointing to the next navigation node, and the voice module is controlled to play voice guidance consistent with the direction of the optical path.

7. An indoor fire evacuation guidance system based on air-ground coordination according to claim 5, characterized in that: The central scheduling subsystem includes a path planning module and a hierarchical response strategy module; The path planning module is used to perform the following operations: based on the real-time dynamic heat map provided by the digital twin visualization command terminal, the load-bearing structure and evacuation channel data from the building information model, and the personnel distribution information identified by the guiding drone, it generates a dynamic risk map, predicts the direction and speed of fire spread, and plans a path for the drone that avoids the current high temperature zone, predicts the fire spread risk to be lower than the threshold and the structural safety redundancy to be higher than the threshold, and plans differentiated evacuation paths for different drones in the same area. The graded response strategy module is used to perform the following operations: dynamically dispatching a drone cluster based on the received alarm level and review results, including dispatching a single drone for review when an early warning signal is received, and dispatching multiple drones to form a formation and assigning different guidance zones when a fire is confirmed.

8. An indoor fire evacuation guidance system based on air-ground coordination according to claim 6, characterized in that: The onboard main control unit of the guided UAV also performs the following cooperative guidance operations: By analyzing the video stream of the visual acquisition module, a lightweight neural network model is used to calculate the direction and speed of the person's movement, and to make a preliminary judgment on the behavior of the person in front, including walking normally, running, crawling forward or remaining still. When a person is detected to be stationary or behaving abnormally, the guidance strategy is adjusted, including: The controllable light source array is switched to a high-frequency flashing mode and its brightness is increased in an attempt to wake the user or attract attention. Control the voice module to play more urgent and explicit voice instructions; Send an abnormal personnel location report to the central dispatch subsystem, requesting additional attention or dispatching other drones for assistance.

9. A method for guiding indoor fire evacuation based on air-ground coordination, characterized in that, Using the indoor fire evacuation guidance system based on air-ground coordination as described in any one of claims 1-8, the method includes the following steps: S1: Through distributed fixed monitoring nodes, the temperature of various indoor areas is continuously scanned, and an indoor positioning network is built based on each fixed monitoring node; S2: When any fixed monitoring node detects that the temperature or temperature rise rate exceeds the preset threshold, it sends an alarm signal containing its own coordinates to the central dispatch subsystem. S3: The central dispatch subsystem plans a safe flight path based on the received alarm coordinates and dispatches and guides the UAV to the corresponding area for visual verification. S4: The guided UAV assesses environmental visibility in real time during flight and adaptively switches between pure visual navigation mode and anti-smoke navigation mode based on the assessment results. S5: If the fire is confirmed upon verification, the system emergency response is triggered, and the central dispatch subsystem dispatches multiple guided drones to jointly perform the evacuation guidance task for the area.

10. The indoor fire evacuation guidance method based on air-ground coordination according to claim 9, characterized in that: The specific process of switching to the anti-smoke navigation mode as described in S4 includes: When the visual acquisition module is determined to be malfunctioning due to smoke obstruction, the reliance on optical flow data for positioning is stopped; the absolute plane coordinates calculated by the ultra-wideband positioning tag carried by the guided UAV are read; the ground altitude data measured in real time by the laser rangefinder is read; the acceleration and angular velocity data output by the inertial measurement unit are fused, and the data fusion and pose calculation are performed through the Kalman filter algorithm to achieve stable flight and hovering control in environments without GPS signals and with visual failure.

Images