Fire hidden danger visual identification method

Abstract

The invention discloses a fire hidden danger visual identification method, and belongs to the technical field of intelligent security and disaster early warning. According to the technical scheme, the fire hidden danger visual identification method specifically comprises the following steps that S1, multi-modal data collection is conducted, specifically, mid-infrared and far-infrared dual-band thermal imaging data of a target area are obtained through a quantum dot infrared sensor array, the mid-infrared band is 3-5 microns, and the far-infrared band is 8-12 microns; two-waveband thermal imaging data are obtained through the quantum dot infrared sensor array, multi-source information is synchronously collected, high-precision space-time alignment is achieved through the Beidou third time service module, a dynamic feature fusion network is constructed to achieve multi-modal data intelligent weighting, layered early warning is achieved in combination with edge calculation and a federated learning framework, and the system is high in reliability and high in reliability. And finally, the unmanned aerial vehicle cooperative response is triggered through the block chain smart contract. The method has the advantages that the early fire recognition precision is improved, the false report and missing report rate is reduced, and second-level emergency response is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent security and disaster warning, and specifically relates to a method for visually identifying fire hazards. Background Art

[0002] Fire, one of the most devastating disasters worldwide, poses a serious threat to life and property, the ecological environment, and socioeconomic stability. Early identification and rapid response to fire hazards are particularly crucial in complex environments such as forests, hydropower stations, and photovoltaic power plants. Current mainstream fire monitoring technologies primarily consist of single-sensor monitoring, manual inspections, and traditional data fusion methods, but these approaches all have significant drawbacks. Single-sensor monitoring relies on single-modality devices such as visible light cameras, thermal imagers, or smoke detectors. Visible light imaging is susceptible to interference from lighting conditions, thermal imaging is insensitive to smoldering fires, and smoke detectors are prone to false alarms due to dust and fog in open environments. Manual inspections are inefficient and cannot achieve all-weather, real-time monitoring, especially in complex terrain, where blind spots exist. While traditional data fusion methods attempt to combine multi-source data, they suffer from insufficient temporal and spatial alignment accuracy and low feature fusion efficiency, resulting in high rates of missed detections and false alarms.

[0003] Existing technologies face significant bottlenecks in early fire identification, multi-device coordination and response delays, and adaptability to complex environments. Traditional detection technologies struggle to effectively capture smoldering fires due to their low temperatures and slow smoke diffusion. Existing AI models are poorly adaptable to low-visibility scenarios and lack the ability to perform dynamic time-series analysis. "Information silos" exist between monitoring devices, leading to significant command transmission delays and inefficient resource scheduling. Early warning classification standards are vague, and response levels are not dynamically adjusted based on fire spread models. Equipment deployment is difficult in complex terrain areas such as mountainous and forested areas, and traditional fixed sensor networks have insufficient coverage. Existing systems have weak privacy protection mechanisms, making monitoring in residential areas prone to leaking sensitive information, while fuzzy algorithms can obscure key fire characteristics.

[0004] While recent efforts to overcome these bottlenecks have included multimodal data fusion, edge computing and layered detection, and the integration of drones and machine vision, these technologies still face limitations, such as weight allocation issues in dynamic environments, a lack of time series analysis and multi-device coordination mechanisms, and response delays caused by reliance on cloud-based analysis. These technical shortcomings severely restrict the accuracy and timeliness of fire hazard identification, necessitating an innovative solution that efficiently integrates multimodal data, accurately identifies early fires, and enables intelligent, coordinated response. Summary of the Invention

[0005] The present invention provides a method for visually identifying fire hazards to solve at least one of the above technical problems.

[0006] The technical solution adopted in the present invention is:

[0007] A method for visually identifying fire hazards, comprising the following steps:

[0008] S1. Multimodal data acquisition: Quantum dot infrared sensor arrays are used to acquire mid-infrared and far-infrared dual-band thermal imaging data of the target area. The mid-infrared band is 3-5μm, and the far-infrared band is 8-12μm. Visible light images and environmental parameter data are also collected simultaneously. Environmental parameter data include wind speed, CO concentration, and PM2.5.

[0009] S2. Spatiotemporal data alignment: Time synchronization of multi-source data is achieved based on the BeiDou-3 timing module, with a time synchronization error of less than 10ns. Spatial registration of thermal and visible light images is achieved through point cloud SLAM algorithm and SIFT feature matching, with a spatial registration error of less than 0.3 pixels.

[0010] S3. Dynamic feature fusion: Build a feature fusion network based on a multi-head attention mechanism, dynamically assign weights to thermal imaging, visible light, and smoke features according to environmental parameters, and generate a joint feature vector.

[0011] S4. Hierarchical analysis and early warning: A lightweight MobileNet model is used at the edge for initial screening, with a processing delay of less than 50ms. Suspected fire points are uploaded to the cloud via a federated learning framework, and multi-model voting and verification are performed using ResNet50 and 3D-CNN models.

[0012] S5. Collaborative response and disposal: When the fire is confirmed, the nearest fire-fighting drone is automatically dispatched based on the blockchain smart contract, the optimal fire-fighting path is calculated based on the fire spread model, and the dry powder spraying or fire-extinguishing bomb throwing action is triggered.

[0013] Furthermore, the present application also proposes that the deployment of the quantum dot infrared sensor array meets the following conditions:

[0014] Condition 1: Sensor spacing Among them, S is the area of ​​the monitoring area, and N is the number of sensors;

[0015] Condition 2: Sensitivity reaches 0.1°C resolution, and effective detection distance ≥ 500 meters;

[0016] Condition 3: Support event-driven mode, and start data acquisition only when the temperature gradient change rate is detected to be greater than 2℃ / min.

[0017] Furthermore, the present application also proposes that the specific implementation of the dynamic feature fusion network includes the following sub-steps:

[0018] S31: Extracting latent feature vectors from thermal imaging data using variational autoencoders (VAEs).

[0019] S32: Use the improved channel attention module CBAM to spatially enhance visible light image features;

[0020] S33: Using the environmental parameter weight matrix W evn =σ(MLP(v wind , c co ))Dynamically weight the multimodal features, where σ is the Sigmoid function.

[0021] Furthermore, this application also proposes that the working steps of the federated learning framework include:

[0022] In the first step, the edge node uploads the model gradient parameters, which are aggregated in the cloud to generate a global model;

[0023] In the second step, differential privacy technology is used to add Gaussian noise N (0, 0.1 2 );

[0024] The third step is to perform model synchronization every 24 hours with a synchronization delay of less than 5 minutes.

[0025] Furthermore, the present application also proposes that the blockchain smart contract contains the following logic:

[0026] Logic 1: Define the fire level judgment function:

[0027]

[0028] Logic 2: Automatically trigger response strategies based on fire severity:

[0029] Level 1: Dispatch 3 heavy drones to drop fire bombs;

[0030] Level 2: dispatch two fire-fighting drones to provide dry powder coverage;

[0031] Level 3: Activate audible and visual alarms and notify manual review.

[0032] Furthermore, this application also proposes the following privacy protection steps:

[0033] Step 1: Perform federated learning feature desensitization on the face / license plate area in the visible light image;

[0034] Step 2: Sensitive data is encrypted using a homomorphic encryption algorithm at the edge computing node, and the encryption key is dynamically updated using the quantum key distribution (QKD) protocol.

[0035] Step 3: When storing data, add Gaussian blur to non-fire related areas, σ = 2.0. The larger the σ value, the higher the degree of blur. When σ = 2.0, the coverage is wider and suitable for medium-intensity smoothing.

[0036] Furthermore, the present application also proposes that the control method of the drone includes:

[0037] S01. Use LiDAR to construct a 3D point cloud model of the fire scene in real time with a resolution of 10cm;

[0038] S02, using the improved ant colony algorithm to plan the fire extinguishing path, the objective function is:

[0039] Among them, t i is the flight time, A i is the fire intensity, α, β, k are weight coefficients;

[0040] S03. The calculation of the throwing angle of the fire extinguishing bomb meets the following requirements:

[0041] Among them, v0 is the initial velocity, g is the acceleration due to gravity, and (x, y) is the target coordinate.

[0042] Furthermore, this application also proposes a system self-checking and fault-tolerance mechanism:

[0043] Mechanism 1: Perform sensor health checks every 6 hours and automatically isolate nodes with deviations greater than 10%;

[0044] Mechanism 2: When communication is interrupted, the edge node switches to local decision-making mode and uses a lightweight LSTM model to predict the direction of fire spread;

[0045] Mechanism 3: Dual redundant power supply design is adopted to seamlessly switch to supercapacitor backup power supply when the main power supply fails.

[0046] Furthermore, the present application also proposes that the installation arrangement of the multimodal data acquisition unit includes:

[0047] Method 1: Hexagonal cellular topology deployment: Dynamically calculate sensor spacing based on terrain height differences

[0048] Where R is the effective detection radius of the sensor, H max and H avg The maximum and average terrain height differences in the region, respectively, and the overlap rate of the fields of view of adjacent units ≥ 20%;

[0049] Method 2: 3D gradient arrangement, with layered deployment every 100 meters in mountainous areas and adaptive pitch angle adjustment In the forest scene, sensor arrays are arranged vertically in the canopy layer, shrub layer, and ground layer.

[0050] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

[0051] Due to the adoption of the above technical solution, the beneficial effects achieved by the present invention are as follows:

[0052] 1. Through the above-mentioned technical solutions, this application effectively solves the problem of difficulty in early fire identification in complex environments and improves the detection sensitivity of smoldering fire points. The dynamic feature fusion mechanism overcomes the problem of misjudgment caused by environmental interference and realizes the efficient and comprehensive utilization of multi-source data. The layered processing architecture takes into account the dual needs of real-time processing and high-precision analysis, avoiding resource waste. The automated response system shortens the time delay from fire identification to disposal action, forming a complete intelligent prevention and control closed loop.

[0053] 2. When deploying the quantum dot infrared sensor array, first calculate the spacing parameters based on the area of ​​the monitoring area and the number of sensors to ensure that the coverage and density of the sensor network adapt to the actual scenario requirements. The sensor unit has built-in quantum dot materials, which generate thermal radiation signals through photoelectric conversion. Its sensitivity design can distinguish temperature differences of 0.1°C and achieve high-precision thermal imaging acquisition within a detection distance of 500 meters. When the temperature gradient changes rapidly in the monitoring area, the sensor automatically switches to event-driven mode and starts data collection only when the temperature change rate reaches the set threshold. It remains in a low-power standby state for the rest of the time. It solves the problem of monitoring blind spots caused by unreasonable sensor deployment density in complex scenarios, improves the ability to identify early smoldering fire points, and reduces energy consumption and data processing burden through intelligent triggering mechanisms, achieving a balance between accuracy and sustainability of the fire monitoring system.

[0054] 3. This application addresses the rigid weight distribution problem of traditional fire identification methods when fusing multimodal data, effectively reducing the risk of feature fusion failure caused by sudden changes in environmental parameters. In scenarios with strong winds or high dust concentrations, it can dynamically enhance the correlation between smoke diffusion characteristics and abnormal areas in thermal imaging, significantly improving fire detection accuracy in low-visibility environments.

[0055] 4. The edge computing node first encrypts the gradient parameters generated by training and transmits them to the cloud server via a secure communication link. The cloud server then performs a weighted average calculation on the multiple gradient parameters received to generate updated global model parameters. During the gradient aggregation process, random noise conforming to a Gaussian distribution is superimposed on each gradient parameter, making it impossible to reverse-engineer the model parameters to derive the original data distribution characteristics. To achieve unified management of model status, a fixed time window is set to trigger synchronization. The cloud server then distributes the latest global model parameters to all edge nodes, while monitoring network latency to ensure synchronization timeliness.

[0056] This application effectively prevents the leakage of sensitive information such as faces and license plates during fire monitoring, ensuring that the model training process complies with privacy protection regulations. Adding noise to the gradient parameters maintains the stability of the model performance while blocking potential data reverse engineering attacks. A regular model synchronization mechanism enables edge devices distributed across complex environments to collaboratively maintain a unified fire identification standard, avoiding the risk of misjudgment due to local environmental differences.

[0057] 5. Once a fire is confirmed, the smart contract automatically analyzes the temperature, smoke, and environmental parameters uploaded by the sensors and calculates the current fire severity using a level determination function. If it is determined to be Level 1, the drone management module's coordinate information is used to generate an instruction set containing target coordinates and trajectory parameters, which is then sent to the heavy-duty drone fleet. If it is determined to be Level 2, the dry powder firefighting drone closest to the fire source is automatically matched, and the spray angle is adjusted based on real-time wind speed data. For Level 3, the audible and visual alarms are triggered simultaneously, and a verification request is sent to the monitoring center.

[0058] This application seamlessly connects fire confirmation and response actions, avoiding the time lost in traditional hierarchical approval processes. It precisely matches fire severity to different levels of response resources, preventing minor fires from overconsumerizing heavy equipment. A pre-set review mechanism ensures the reliability of high-level alarms and reduces resource waste caused by false triggers.

[0059] 6. This solution achieves dynamic key updates through quantum key distribution, significantly improving the security of the key system. Existing fuzzy algorithms generally use fixed-strength processing, but this solution, through the appropriate setting of the σ value, protects privacy while retaining the identifiability of key fire characteristics.

[0060] Through the above technical solutions, this application effectively prevents the risk of sensitive information leakage during data collection, transmission, and storage. Federated learning feature desensitization avoids the direct transmission of raw image data, homomorphic encryption ensures security during data transmission, and Gaussian blurring further eliminates privacy risks during storage. The dynamic key update mechanism enhances the encryption system's anti-attack capabilities, and multi-level privacy protection measures achieve comprehensive data security without compromising fire identification accuracy.

[0061] 7. During the 3D point cloud modeling phase of the fire scene, the LiDAR sensor scans the fire area at a fixed frequency, generating a point cloud dataset with spatial coordinate information. A noise reduction filter algorithm is then used to eliminate smoke interference. During the firefighting path planning process, an improved ant colony algorithm incorporates fire intensity parameters into the pheromone update rules, allowing the path search process to dynamically adapt to changing fire trends. During the fire bomb launch phase, the relative positional relationship between the target point coordinates and the drone's flight state is calculated in real time. The optimal launch angle is calculated by combining gravity acceleration and initial velocity parameters. A Kalman filter is also used to compensate for the effects of wind speed on the trajectory.

[0062] This application can achieve centimeter-level modeling accuracy of the three-dimensional structure of the fire scene, improve the obstacle avoidance capability of the UAV in a dense smoke environment; dynamically adjust the flight path weight coefficient so that the UAV prioritizes extinguishing areas with high fire intensity; calculate the projection parameters in real time based on physical kinematic equations, reduce the trajectory deviation of the fire extinguishing bomb in a complex airflow environment, and ultimately shorten the fire response time and improve the initial fire extinguishing success rate.

[0063] 8. The system self-check and fault-tolerance mechanism ensures system reliability through collaborative operations at three levels. The sensor health detection module periodically collects the working parameters of each node, for example, using a dynamic comparison algorithm between the sensor output value and the reference value. When the deviation exceeds the preset threshold, the isolation instruction is triggered to prevent erroneous data from entering the subsequent processing link. In the case of abnormal communication, the edge computing device automatically calls the locally stored prediction model, which generates the decision-making basis required for emergency response by performing time-series modeling on the direction of fire spread. The power management unit monitors the status of the main power supply in real time. When a voltage anomaly or power-off signal is detected, the load is switched to the backup power supply through the relay control circuit. The high power density characteristics of the supercapacitor group ensure continuous power supply to key equipment during the switching process.

[0064] This application solves the problem of missed fire detection caused by insufficient equipment reliability in complex environments, reduces the impact of communication interruptions on system continuity, realizes the uninterrupted operation capability of key equipment in power failure scenarios, and provides technical guarantee for the long-term stable operation of fire monitoring systems in harsh environments.

[0065] 9. Specifically, in open terrain, hexagonal cellular topology deployment is preferred. The coverage range and device density are balanced by dynamically calculating the sensor spacing. For example, a uniform cellular structure is used in flat areas, and the cell spacing is adjusted according to the height difference in undulating terrain. For mountainous scenarios, the sensor array is deployed in layers according to altitude. The spacing between each layer can be set to a specific height value based on the actual slope. The pitch angle is adjusted to ensure the continuous connection of the field of view of each layer of sensors. In forest scenarios, sensors are installed at the top of the tree canopy, in the shrub area, and on the ground layer. Temperature anomaly signals at different heights are captured through multi-layer vertical layout. The overlapping part of the field of view of adjacent sensors is set to a coverage ratio of no less than 20%, so that monitoring relay can be achieved by adjacent units in the event of a single point failure.

[0066] This application solves the technical problems of low sensor deployment coverage and large monitoring blind spots in complex terrain environments, achieving comprehensive capture of multi-dimensional spatial fire signals. In scenes with obvious vertical structures such as forests, a layered deployment effectively identifies fire characteristics at different heights, avoiding missed detections caused by ground sensors being obscured by upper vegetation. Compared to fixed installation, the dynamically adjusted deployment method saves 15%-30% of the equipment required, reducing system construction costs while ensuring monitoring quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a principle diagram of a specific embodiment of the present invention;

[0068] Figure 2 This is a schematic diagram of the quantum dot infrared sensor array method in the present invention;

[0069] Figure 3 This is a schematic diagram of the dynamic feature network in the present invention;

[0070] Figure 4 Schematic diagram of the federated learning framework in the present invention;

[0071] Figure 5 This is a schematic diagram of the blockchain smart contract principle in the present invention.

[0072] The drawings described herein are used to provide further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention. DETAILED DESCRIPTION

[0073] In order to more clearly illustrate the overall concept of the present invention, a detailed description is given below in an exemplary manner in conjunction with the accompanying drawings.

[0074] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0075] In addition, in the description of the present invention, it should be understood that the terms "top", "bottom", "inside", "outside", "axial", "radial", "circumferential", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0076] In the present invention, unless otherwise expressly specified or limited, terms such as "mounted," "connected," "connect," and "fixed" should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection, or communication; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0077] In the present invention, unless otherwise clearly specified and limited, a first feature "above" or "below" a second feature may be that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. In the description of this specification, the description with reference to the terms "implementation method", "embodiment", "one embodiment", "example" or "specific example" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in an appropriate manner in any one or more embodiments or examples.

[0078] Example 1: Reference Figure 1 A method for visually identifying fire hazards, characterized in that it specifically includes the following steps:

[0079] S1. Multimodal Data Acquisition: A quantum dot infrared sensor array acquires dual-band thermal imaging data of the target area, covering mid-infrared (3-5μm) and far-infrared (8-12μm). Visible light images and environmental parameter data, including wind speed, CO concentration, and PM2.5, are simultaneously acquired. The quantum dot infrared sensor array utilizes the size effect of quantum dot materials to adjust absorption wavelengths, covering both the mid-infrared (3-5μm) and far-infrared (8-12μm) bands. The mid-infrared band is suitable for detecting high-temperature fire sources (such as flames), while the far-infrared band can identify low-temperature thermal radiation (such as smoldering fires). The high thermal sensitivity of quantum dot materials (0.1°C resolution) combined with InGaAs detectors can capture minute temperature differences from 500 meters away. RGB images are simultaneously acquired along with wind speed, CO concentration, and PM2.5 data. Visible light images are used to identify smoke patterns, while environmental parameters assist in determining fire spread trends (e.g., high wind speeds accelerate fire spread).

[0080] S2. Spatiotemporal Data Alignment: Time synchronization of multi-source data is achieved using the BeiDou-3 timing module, with a time synchronization error of less than 10ns. Spatial registration of thermal and visible light images is achieved through point cloud SLAM algorithms and SIFT feature matching, with a spatial registration error of less than 0.3 pixels. BeiDou-3 Time Synchronization: Utilizing BeiDou-3's nanosecond timing capability (error <10ns), the data timestamps of infrared, visible light, and environmental sensors are consistent. For example, within an area of ​​1 square kilometer, the coordinate offset caused by time synchronization error is negligible. Point Cloud SLAM and SIFT Feature Matching: A 3D point cloud map of the environment is generated using the SLAM algorithm. The SIFT algorithm extracts key points (such as edges and corners) from the visible light image and performs sub-pixel registration with the thermal imaging feature points (error <0.3 pixels). For example, in a forest scene, the thermal image outline of a tree trunk precisely matches the outline of the visible light image.

[0081] S3. Dynamic feature fusion: Build a feature fusion network based on a multi-head attention mechanism, dynamically assign weights to thermal imaging, visible light, and smoke features according to environmental parameters, and generate a joint feature vector.

[0082] Multi-head attention mechanism: Build a fusion network, where each attention head independently learns feature weights for different modalities. For example, when wind speed exceeds 5m / s, the visible light image generates noise due to swaying leaves, and the network automatically reduces its weight (from 0.6 to 0.3). When CO concentration exceeds 50ppm, the weight of thermal imaging features is increased to 0.8, enhancing fire spot identification.

[0083] S4. Hierarchical analysis and early warning: A lightweight MobileNet model is used at the edge for initial screening, with a processing delay of less than 50ms. Suspected fire points are uploaded to the cloud via a federated learning framework, and multi-model voting and verification are performed using ResNet50 and 3D-CNN models.

[0084] MobileNet initial screening at the edge: The MobileNet v3 model is compressed to less than 2MB and processed in real time on edge computing nodes (such as the NVIDIA Jetson Nano). For example, for a 640×480 pixel image, the initial screening latency is controlled at 48ms, filtering 90% of the non-fire point areas.

[0085] Cloud-based multi-model voting: Suspected fire spot data is uploaded to the cloud via a federated learning framework. ResNet50 performs image classification, while 3D-CNN analyzes spatiotemporal changes (such as heat diffusion). If both models determine a fire is occurring, an alarm is triggered.

[0086] S5. Collaborative response and disposal: When the fire is confirmed, the nearest fire-fighting drone is automatically dispatched based on the blockchain smart contract, the optimal fire-fighting path is calculated based on the fire spread model, and the dry powder spraying or fire-extinguishing bomb throwing action is triggered.

[0087] Among them, the quantum dot infrared sensor array refers to a collection of infrared detectors constructed using quantum dot materials as photosensitive elements. It can be implemented using a focal plane array based on quantum dots such as PbS, PbSe, or HgCdTe, such as an uncooled or cooled quantum dot infrared detector array. Its main purpose is to obtain high-sensitivity thermal imaging data in specific bands, especially for the detection of mid-infrared and far-infrared bands, so as to capture smoldering fire points that are difficult for traditional sensors to detect. The Beidou-3 timing module refers to a hardware unit that integrates the timing function of the Beidou-3 satellite navigation system. It can be implemented using a high-precision timing chip or module, such as a timing receiver that supports nanosecond-level timing accuracy. Its main purpose is to achieve high-precision time synchronization of multi-source heterogeneous sensor data to ensure the timeliness and accuracy of subsequent data fusion. Point cloud SLAM algorithm and SIFT feature matching refer to image feature matching methods that combine three-dimensional point cloud simultaneous positioning and mapping technology with scale-invariant feature transformation algorithms. They can be implemented by acquiring point cloud data based on lidar or depth cameras and combining them with visual images for joint processing. For example, the graph-optimized SLAM algorithm and the RANSAC robust matching method are mainly used to complete the precise spatial registration of images collected by different sensors (such as thermal imagers and visible light cameras) and eliminate data misalignment caused by differences in perspective and position. The feature fusion network of the multi-head attention mechanism refers to a deep learning network structure that contains multiple attention heads and can learn different representations of input features in parallel and perform weighted combinations. It can be implemented based on the Transformer architecture or an improved attention module, such as the self-attention mechanism or the cross-attention mechanism. Its main purpose is to achieve dynamic and efficient fusion of multimodal features and to adjust the contribution weights of different modal features according to external environmental parameters. The federated learning framework refers to a distributed machine learning paradigm that allows multiple participants to collaboratively train models without sharing raw data. This can be implemented using federated averaging algorithms based on model parameter or gradient exchange, such as the FedAvg algorithm. Its primary purpose is to train or verify cloud-based models using edge data while protecting data privacy, thereby improving the model's generalization capabilities. Blockchain smart contracts are computer programs deployed on a blockchain that can automatically execute predefined terms. These can be implemented using smart contracts developed on blockchain platforms such as Ethereum and Hyperledger Fabric. For example, automated contracts for resource scheduling are primarily designed to enable automated and trusted scheduling of response resources (such as firefighting drones) after fire confirmation, reducing manual intervention and delays.A fire spread model refers to a mathematical model or simulation model used to predict the direction, speed, and scope of fire development. It can be implemented using models based on physical principles, empirical formulas, or machine learning methods, such as the FARSITE model or a model based on cellular automation. Its main purpose is to combine real-time fire data to predict fire dynamics and provide a decision-making basis for fire extinguishing path planning and resource scheduling.

[0088] The core innovation of this application lies in the high-precision spatiotemporal alignment of specific-band thermal imaging data, visible light images, and environmental parameters obtained by the quantum dot infrared sensor array and the dynamic feature fusion based on the multi-head attention mechanism, and the hierarchical analysis and early warning mechanism combined with lightweight initial screening on the edge and multi-model verification of federated learning on the cloud, and ultimately achieving intelligent collaborative response through blockchain smart contracts, thereby solving the problems of insufficient early fire identification capabilities, delayed collaborative response of multiple devices, and poor adaptability to complex environments in existing technologies, achieving the effect of improving the accuracy and timeliness of fire hazard identification and optimizing resource scheduling efficiency.

[0089] Specifically, the method first acquires multiple data sources from the target area through a multimodal data acquisition step. These include mid-infrared and far-infrared dual-band thermal imaging data from a quantum dot infrared sensor array, visible light images, and environmental parameters such as wind speed, CO concentration, and PM2.5. In the spatiotemporal data alignment step, these data are synchronized with high precision using the BeiDou-3 timing module. A point cloud SLAM algorithm and SIFT feature matching are then used to precisely spatially register the thermal and visible light images. The aligned multimodal data then enters the dynamic feature fusion step. A multi-head attention feature fusion network dynamically adjusts the weights of different modal features based on real-time environmental parameters, generating a richer and more discriminative joint feature vector. These joint feature vectors are first quickly screened at the edge using a lightweight MobileNet model in the hierarchical analysis and early warning step to reduce latency. Suspected fire points identified in this initial screening are uploaded to the cloud using a federated learning framework, where they are jointly validated using multiple models, including ResNet50 and 3D-CNN, to improve recognition accuracy and robustness. Once a fire is confirmed, a coordinated response process is triggered. Pre-set dispatch logic is automatically executed based on blockchain smart contracts, mobilizing the nearest firefighting drone. The optimal firefighting path is calculated based on a fire spread model, and the drone ultimately executes firefighting actions such as spraying dry powder or dropping fire extinguishing bombs. The entire process forms a closed-loop system from data collection, processing, analysis, to response, enabling early, accurate identification of potential fire hazards and intelligent, efficient response.

[0090] As a preferred embodiment, the solution of this application is specifically implemented as follows: Multiple edge computing nodes integrated with quantum dot infrared sensors, visible light cameras, environmental sensors, and BeiDou-3 timing modules are deployed in the monitoring area. Each node is equipped with an embedded AI processor that runs the MobileNet model for local initial screening. The nodes are connected via a wireless network and communicate with a cloud server. The cloud server deploys a federated learning platform, ResNet50 and 3D-CNN models, and blockchain nodes. After the edge nodes collect data, they first perform time synchronization and spatial registration. The aligned multimodal data is then input into a locally running feature fusion network, which dynamically adjusts the fusion weights of thermal imaging, visible light, and smoke features based on data such as wind speed and CO concentration collected by environmental sensors. The fused feature vector is input into the MobileNet model for rapid identification. If a suspected fire spot is identified, the edge node uploads the relevant feature data (not the original image) via an encrypted channel to the cloud-based federated learning platform. The cloud platform aggregates the suspected fire spot features from multiple edge nodes and uses the ResNet50 and 3D-CNN models for joint inference and voting to improve recognition accuracy. Once a fire is confirmed, a cloud-based blockchain node triggers a smart contract. This contract queries the drone's location, combines it with a fire spread model to calculate the optimal dispatch plan and firefighting path, and then sends instructions to the designated firefighting drone. Upon receiving the instructions, the drone autonomously flies to the target area and carries out the firefighting mission.

[0091] Through the above scheme, this application can effectively solve the problems of the existing technology in early fire identification, such as insufficient sensitivity to smoldering fire points and susceptibility to environmental interference leading to false alarms and missed alarms, thereby improving the accuracy of identification. Through high-precision spatiotemporal alignment and dynamic feature fusion, the complementary advantages of multi-source data are fully utilized to enhance the ability to identify fires in complex environments. The layered analysis and early warning combines the real-time nature of edge computing with the accuracy of cloud computing, reducing processing delays and improving response speed. The cloud verification mechanism based on federated learning improves the robustness of the model while protecting privacy. The application of blockchain smart contracts realizes the automation and trusted scheduling of response resources, improving the efficiency and reliability of collaborative disposal.

[0092] Example 2: In the fire hazard visual identification method, through the steps of multimodal data acquisition, spatiotemporal data alignment, dynamic feature fusion, hierarchical analysis and early warning, and coordinated response and disposal, efforts are made to achieve accurate identification and rapid response to early fire hazards. However, in actual applications, the deployment method of the quantum dot infrared sensor array directly affects the monitoring range, sensitivity and response speed. Unreasonable deployment may lead to monitoring blind spots, insufficient sensitivity or response delays, thereby affecting the performance of the entire fire hazard visual identification system. Therefore, how to optimize the deployment of the quantum dot infrared sensor array to improve monitoring efficiency and accuracy is a technical problem that needs to be solved. In some of the above-mentioned schemes of this application, a quantum dot infrared sensor array is proposed to obtain mid-infrared and far-infrared dual-band thermal imaging data of the target area. However, in this process, unreasonable deployment of the sensor array will lead to monitoring blind spots, insufficient sensitivity or response delays.

[0093] In this regard, the present application proposes that the deployment of the quantum dot infrared sensor array meets the following conditions:

[0094] Condition 1: Sensor spacing Among them, S is the area of ​​the monitoring area, and N is the number of sensors;

[0095] Derivation principle: Based on the hexagonal close packing theory, each sensor covers a regular hexagonal area, and the total area After formula optimization, the overlap rate of the fields of view of adjacent sensors is ensured to be ≥20%, eliminating blind spots.

[0096] Example: When the monitoring area S = 10,000 m2, if N = 50, the calculated spacing D ≈ 18.26 meters, and the total coverage area reaches 10,800 m2 (including overlapping areas).

[0097] Condition 2: Sensitivity reaches 0.1°C resolution, and effective detection distance ≥ 500 meters;

[0098] 0.1°C resolution: The carrier mobility of quantum dot materials is extremely high, and changes in thermal radiation energy cause significant current fluctuations. Weak signals can be extracted by combining phase-locked amplification technology.

[0099] 500m detection distance: adopts f / 1.0 large aperture optical design to improve signal-to-noise ratio, in foggy and hazy weather (PM2.5≤200μg / m 3 ) can still identify the fire point.

[0100] Condition 3: Supports event-driven mode, initiating data acquisition only when a temperature gradient change rate > 2°C / min is detected. Temperature gradient triggering: The sensor's built-in FPGA calculates the temperature change rate in real time. For example, when the temperature in a certain area rises from 30°C to 32°C (rate of change = 2°C / min), data acquisition begins, and the sensor remains dormant. Data acquisition is initiated only when the temperature change rate > 2°C / min, reducing power consumption by 80%.

[0101] Condition 1 defines the relationship between sensor spacing, the area of ​​the monitoring area, and the number of sensors. The calculated sensor spacing guides the spatial distribution of sensors within the monitoring area, aiming to maximize coverage of the monitoring area and minimize blind spots for a given number of sensors.

[0102] Condition 2 specifies the performance level that sensors must meet. A sensitivity of 0.1°C resolution means the sensor can detect very small temperature changes, which is crucial for early detection of fire hazards in their infancy, before temperatures rise significantly. An effective detection range of 500 meters or greater ensures that a single sensor can cover a wide area, potentially reducing the total number of sensors required while maintaining effective monitoring, thereby lowering deployment costs and complexity.

[0103] Condition three requires the sensor to operate in an event-driven mode and set trigger conditions for initiating data collection. This means the sensor does not continuously collect data, but instead only activates when it detects an abnormal temperature increase (temperature gradient rate greater than 2°C / min). This mode significantly reduces sensor energy consumption during normal operating conditions, extending battery life or reducing power requirements. It also avoids collecting large amounts of redundant background data, reduces the burden on data transmission and backend processing, and improves the system's response to critical events.

[0104] Specifically, the solution of this application enhances the data acquisition capabilities of a fire hazard visual identification method by optimizing the deployment of a quantum dot infrared sensor array, thereby improving the overall performance of the method. By satisfying the sensor spacing relationship set by condition one, the sensor array ensures reasonable spatial coverage within the monitoring area, enabling the method to acquire dual-band mid-infrared and far-infrared thermal imaging data covering the entire target area, avoiding monitoring blind spots caused by improper deployment and providing a comprehensive raw data foundation for subsequent steps such as spatiotemporal data alignment and dynamic feature fusion. Meeting the high sensitivity and long detection range requirements of condition two enables the sensor to capture weaker temperature anomaly signals and detect potential fire hazards at greater distances, directly improving the method's ability to detect early, subtle fires during the data acquisition phase. This high-quality, long-range sensing capability enables subsequent feature fusion to utilize richer, earlier thermal anomaly information, allowing layered analysis and early warning to be based on more reliable input, thereby improving the timeliness and accuracy of early warnings. Furthermore, through the event-driven mode introduced in condition three, the sensor only initiates data acquisition when it detects rapid temperature changes. This allows data collection to be more focused on potential anomaly areas and reduces the generation of invalid data. This reduces the processing burden of subsequent data alignment, feature fusion, and hierarchical analysis, improving processing speed and shortening the overall time from hidden danger occurrence to system response, thereby increasing the efficiency of coordinated response and disposal. Therefore, by optimizing the deployment of the sensor array, the solution of this application improves the efficiency and accuracy of the entire fire hidden danger visual identification method from the source of the data, effectively solving the problems of monitoring blind spots, insufficient sensitivity, and response delays caused by unreasonable sensor deployment.

[0105] As a preferred embodiment, the solution of this application is implemented as follows: Assume that an area of ​​1,000,000 square meters (i.e., S = 1,000,000) needs to be monitored, and 50 quantum dot infrared sensors (i.e., N = 50) are planned to be deployed. Based on condition one, the calculated sensor spacing D = 2 * 1,000,000 / 50 = 40,000. In actual deployment, this calculation result can be used to adjust the specific sensor locations based on the specific terrain and obstacle conditions, striving to achieve a similar spatial coverage density. The sensors themselves use highly sensitive quantum dot infrared detectors, such as those using specific materials and cooling technologies, and are equipped with optimized lenses. This allows them to distinguish temperature differences as low as 0.1°C, and their effective detection range for typical fire targets within a clear field of view can reach or exceed 500 meters, meeting the requirements of condition two. Each sensor integrates a low-power microprocessor and a temperature change monitoring module. This module continuously monitors the maximum temperature or the temperature of critical areas within the sensor's field of view. The microprocessor runs firmware to calculate the rate of temperature change over time. When the calculated temperature gradient change rate exceeds the threshold of 2°C / min for a continuous period of time, the microprocessor triggers the sensor to enter full-speed data acquisition mode, starts capturing high-resolution mid-infrared and far-infrared thermal imaging data, and sends the data to the edge computing node or cloud platform through the communication module for further processing, thereby meeting the requirements of condition three.

[0106] Through the above solution, this application can effectively avoid monitoring blind spots caused by improper sensor deployment and ensure full coverage of the target area. It can detect small temperature anomalies earlier and more accurately, improving the ability to identify early fire hazards. It can significantly reduce the energy consumption of sensors in non-abnormal conditions, extend the working time of equipment, and reduce the collection and transmission of invalid data, thereby improving data processing efficiency. As a result, the overall monitoring efficiency, accuracy and response speed of the fire hazard visual identification system are improved.

[0107] Example 3, in the fire hazard visual identification method, by constructing a feature fusion network based on a multi-head attention mechanism, the weights of thermal imaging, visible light and smoke features can be dynamically assigned according to environmental parameters to generate a joint feature vector. However, in actual applications, there are differences in the amount of information and noise levels contained in different modal data. Simply performing weight assignment may not fully utilize the advantages of each modal data, resulting in the fused feature vector being inaccurate, which in turn affects the accuracy of fire identification. Therefore, a more refined dynamic feature fusion method is needed to improve the accuracy and robustness of fire identification. In this regard, the present application proposes that the specific implementation of the dynamic feature fusion network includes the following sub-steps:

[0108] S31: Extracting latent feature vectors from thermal imaging data using a variational autoencoder (VAE). Principle: The VAE encodes thermal imaging data into a latent space vector (dimension = 128), capturing nonlinear characteristics of the temperature distribution (such as the shape of hot areas). The decoder reconstructs the data, retaining key fire point information and filtering out noise.

[0109] S32: An improved channel attention module (CBAM) is used to spatially enhance visible light image features. CBAM generates a spatial attention map through max pooling and average pooling to enhance smoke edge features. For example, after CBAM processing, the edge clarity of blurred smoke outlines in visible light images is improved by 30%.

[0110] S33: Using the environmental parameter weight matrix W evn =σ(MLP(v wind , c co )) Dynamically weight the multimodal features, where σ is the Sigmoid function; for example, the input wind speed (v wind ) and CO concentration (c co ), MLP outputs weight coefficients, which are normalized by Sigmoid function. For example, when v wind =8m / s, and c co =60ppm, the thermal imaging weight increases to 0.7.

[0111] Among them, the variational autoencoder (VAE) refers to a generative model that maps input data to a latent space through an encoder and reconstructs the data using a decoder. Specifically, a multi-layer convolutional neural network can be used to implement the encoder and decoder structure, which is used to extract latent feature vectors with low-dimensional representation capabilities from thermal imaging data. The improved channel attention module (CBAM) refers to a convolutional neural network module that combines channel attention and spatial attention. Specifically, it can be implemented using a dual-path feature aggregation method of global average pooling and maximum pooling to enhance the spatial feature response of key areas in visible light images. The environmental parameter weight matrix refers to a weight distribution matrix dynamically generated based on parameters such as wind speed, CO concentration, and PM2.5. Specifically, it can be constructed by combining a fully connected layer with a Sigmoid activation function to adaptively adjust the contribution of different modalities during multimodal feature fusion.

[0112] Specifically, thermal imaging data undergoes nonlinear dimensionality reduction using a variational autoencoder (VAE) to generate a latent vector containing temperature distribution features. Visible light images are processed using an improved channel attention module (CBAM) to prioritize the spatial features of smoke diffusion areas or flame outlines. An environmental parameter weight matrix dynamically calculates the fusion weights of different modalities based on real-time wind speed and particulate matter concentration data, and constrains the weights between 0 and 1 using a sigmoid function. Finally, thermal imaging features, visible light features, and smoke features are weighted and fused according to dynamically assigned weights to form a joint feature vector. This fully exploits the complementary information of multimodal data while avoiding the problem of fixed-weight fusion being inadequate to complex environmental changes.

[0113] Compared to existing technologies, existing methods typically use fixed weights or simple linear weighting for feature fusion, failing to adjust the importance of different modalities based on dynamic environmental changes. For example, traditional methods fail to consider the impact of wind speed on smoke diffusion, resulting in a mismatch between the weight distribution of smoke features and the actual fire situation. This solution introduces an environmental parameter weight matrix, enabling the feature fusion process to respond to environmental parameter changes in real time. Furthermore, by combining a variational autoencoder and an attention mechanism, it enhances feature representation capabilities, thereby achieving more accurate multimodal information fusion in dynamic scenes.

[0114] Through the above technical solution, this application solves the rigid weight distribution problem existing in traditional fire identification methods when fusing multimodal data, effectively reducing the risk of feature fusion failure caused by sudden changes in environmental parameters. In scenes with strong winds or high dust concentrations, it can dynamically enhance the correlation between smoke diffusion characteristics and abnormal areas in thermal imaging, significantly improving the accuracy of fire detection in low-visibility environments.

[0115] In Example 4, the working steps of the federated learning framework include:

[0116] In the first step, the edge node uploads the model gradient parameters, which are aggregated in the cloud to generate a global model;

[0117] In the second step, differential privacy technology is used to add Gaussian noise N (0, 0.1 2 );

[0118] The third step is to perform model synchronization every 24 hours with a synchronization delay of less than 5 minutes.

[0119] This embodiment adopts the FedAvg algorithm with gradient aggregation, and adds N(0,0.1 2 ) noise, the model synchronization cycle is 24 hours, thereby reducing the risk of privacy leakage from 5% to 0.3%, and the model update delay is less than 5 minutes.

[0120] The federated learning framework refers to a distributed machine learning paradigm implemented through a model parameter sharing mechanism. This allows edge devices to participate in model training without uploading original data, thereby addressing data privacy issues. Differential privacy technology is a privacy protection method that injects random noise into data. Specifically, it uses a Gaussian noise distribution to perturb gradient parameters, making it impossible for attackers to infer the original data information from model updates. The model synchronization mechanism refers to the periodic updating of global model parameters. Specifically, it uses a time-triggered strategy to ensure that each edge node maintains the same model version as the cloud, avoiding model drift caused by local updates.

[0121] Specifically, the edge computing node first encrypts the gradient parameters generated by training and transmits them to the cloud server via a secure communication link. The cloud performs a weighted average calculation on the multiple gradient parameters received to generate updated global model parameters. During the gradient aggregation process, random noise conforming to a Gaussian distribution is superimposed on each gradient parameter, making it impossible to reversely deduce the original data distribution characteristics from the model parameters. To achieve unified management of the model state, a fixed time window is set to trigger the synchronization operation. The cloud will then send the latest global model parameters to all edge nodes, while monitoring network latency to ensure synchronization timeliness.

[0122] Compared to existing technologies, traditional centralized training requires uploading all monitoring data to the cloud, which poses a risk of sensitive information leakage. This solution, however, localizes data processing through a federated learning framework and combines it with differential privacy technology to form a dual protection barrier. Existing methods typically use a fixed-frequency model update strategy, which struggles to balance communication overhead and model consistency. This solution, by setting a synchronization time threshold and incorporating a delay control mechanism, optimizes resource utilization while ensuring model accuracy.

[0123] Through the above technical solution, this application effectively prevents the leakage of sensitive information such as faces and license plates during fire monitoring, ensuring that the model training process complies with privacy protection regulations. The operation of adding noise to the gradient parameters not only maintains the stability of model performance but also blocks potential data reverse engineering attacks. A regular model synchronization mechanism enables edge devices distributed in complex environments to collaboratively maintain a unified fire identification standard, avoiding the risk of misjudgment due to local environmental differences.

[0124] In Example 5, the blockchain smart contract includes the following logic:

[0125] Logic 1: Define the fire level judgment function:

[0126]

[0127] Logic 2: Automatically trigger response strategies based on fire severity:

[0128] Level 1: Dispatch 3 heavy drones to drop fire bombs;

[0129] Level 2: dispatch two fire-fighting drones to provide dry powder coverage;

[0130] Level 3: Activate audible and visual alarms and notify manual review.

[0131] Blockchain smart contracts are self-executing agreements deployed on a distributed ledger. The Ethereum Virtual Machine (EVM) is used to implement smart contract code deployment, with execution triggered by pre-set conditions. The fire severity determination function is a quantitative assessment model based on temperature gradients, smoke concentrations, and diffusion rates. A normalized weighted algorithm is used to map multi-dimensional parameters to discrete severity values. The response strategy trigger mechanism matches predefined response plans to fire severity, employing an event-driven architecture for multi-device coordinated control.

[0132] Specifically, once a fire is confirmed, the smart contract automatically analyzes the temperature, smoke, and environmental parameters uploaded by the sensors and calculates the current fire level using a level determination function. If it is determined to be Level 1, the coordinate information of the drone management module is called up, and a command set containing target coordinates and trajectory parameters is generated and sent to the heavy drone fleet. If it is determined to be Level 2, the dry powder fire-fighting drone closest to the fire source is automatically matched, and the spray angle is adjusted based on real-time wind speed data. For Level 3, the sound and light alarm system is triggered simultaneously, and a review request is sent to the monitoring center.

[0133] Compared to existing technologies, traditional fire response systems rely on manual judgment of response plans, which can lead to decision-making delays and subjective bias. This solution, through pre-built smart contract logic, establishes a deterministic mapping between fire severity and response strategies, eliminating manual intervention. While existing technologies require a central control system to coordinate multiple devices, this solution leverages the distributed nature of blockchain to enable parallel response across device nodes.

[0134] Through the above technical solution, this application achieves a seamless connection between fire confirmation and disposal actions, avoiding the time loss caused by the traditional hierarchical approval process. It accurately matches different levels of disposal resources according to fire intensity, preventing minor fires from excessively consuming heavy equipment. A pre-set review mechanism ensures the reliability of high-level alarms and reduces the waste of resources caused by false triggers.

[0135] Example 6 further includes a privacy protection step:

[0136] Step 1: Perform federated learning feature desensitization on the face / license plate area in the visible light image;

[0137] Step 2: Sensitive data is encrypted using a homomorphic encryption algorithm at the edge computing node, and the encryption key is dynamically updated using the quantum key distribution (QKD) protocol.

[0138] Step 3: When storing data, add Gaussian blur to non-fire related areas, σ = 2.0. The larger the σ value, the higher the degree of blur. When σ = 2.0, the coverage is wider and suitable for medium-intensity smoothing.

[0139] Among them, federated learning feature desensitization processing refers to model training without sharing the original data, and privacy protection is achieved through gradient parameter exchange. Specifically, it can be achieved by using model parameter perturbation technology based on differential privacy. Homomorphic encryption algorithm refers to an encryption method that supports data calculations in an encrypted state. Specifically, the Paillier semi-homomorphic encryption algorithm can be used to achieve secure communication between edge nodes and the cloud. Quantum key distribution protocol refers to the use of quantum uncloning properties to generate dynamic keys. Specifically, the BB84 protocol can be used to achieve secure key transmission. Gaussian blur processing refers to performing convolution operations on images to reduce the information resolution of local areas. Specifically, the GaussianBlur function of the OpenCV library can be used to achieve pixel blurring in non-fire areas.

[0140] Specifically, at the data collection end, the facial and license plate areas of visible light images are desensitized at the feature level. Through the federated learning framework, only the desensitized feature parameters are uploaded, avoiding the transmission of the original sensitive data. During the data preprocessing phase, edge nodes encrypt the desensitized feature data using homomorphic encryption technology, with the key updated hourly via the quantum channel. The encrypted data undergoes secondary processing in the storage phase, using a Gaussian kernel function to smooth the pixel values ​​in non-fire areas, preserving the clarity of fire-related areas. This layered processing mechanism protects privacy throughout the entire lifecycle of data collection, transmission, and storage.

[0141] Compared to existing technologies, traditional methods typically rely on single data encryption or image blurring techniques, failing to address the security requirements of the entire data processing process. The static encryption keys used in existing technologies pose a risk of brute force attacks. This solution, however, significantly improves the security of the key system by enabling dynamic key updates through quantum key distribution. Existing fuzzy algorithms generally employ fixed-strength processing, while this solution, through the appropriate setting of the σ value, protects privacy while maintaining the recognizability of key fire characteristics.

[0142] Through the above technical solutions, this application effectively prevents the risk of sensitive information leakage during data collection, transmission, and storage. Federated learning feature desensitization avoids the direct transmission of raw image data, homomorphic encryption ensures security during data transmission, and Gaussian blurring further eliminates privacy risks during storage. The dynamic key update mechanism enhances the encryption system's anti-attack capabilities, and multi-level privacy protection measures achieve comprehensive data security without compromising fire identification accuracy.

[0143] In embodiment 7, the control method of the drone includes:

[0144] S01. Use LiDAR to construct a 3D point cloud model of the fire scene in real time with a resolution of 10cm;

[0145] S02, using the improved ant colony algorithm to plan the fire extinguishing path, the objective function is:

[0146] Among them, t i is the flight time, A i is the fire intensity, α, β, k are weight coefficients;

[0147] S03. The calculation of the throwing angle of the fire extinguishing bomb meets the following requirements:

[0148] Among them, v0 is the initial velocity, g is the acceleration due to gravity, and (x, y) is the target coordinate.

[0149] Among them, LiDAR real-time construction of a three-dimensional point cloud model of the fire scene refers to the generation of three-dimensional spatial data by scanning the fire scene environment through LiDAR. Specifically, this can be achieved by using a multi-beam LiDAR sensor in conjunction with a real-time point cloud processing algorithm. Its function is to accurately perceive the distribution of terrain obstacles in the fire scene and the spatial diffusion form of the fire. Improved ant colony algorithm for planning fire-fighting paths refers to optimizing the convergence speed of the traditional ant colony algorithm in path search. Specifically, this can be achieved by introducing a dynamic weight coefficient adjustment mechanism. Its function is to balance the influence of flight time and fire intensity on path selection. The calculation of the throwing angle of the fire-extinguishing bomb to meet the physical kinematic equation refers to determining the optimal projection parameters based on the ballistic trajectory model. Specifically, this can be achieved by using real-time solution of the parabolic motion equation combined with an environmental disturbance compensation algorithm. Its function is to improve the hit accuracy of the fire-extinguishing bomb in complex airflow environments.

[0150] Specifically, during the three-dimensional point cloud modeling phase of the fire scene, the LiDAR sensor scans the fire area at a fixed frequency, generating a point cloud dataset with spatial coordinate information. A noise reduction filter algorithm is then used to eliminate smoke interference. During the firefighting path planning process, an improved ant colony algorithm incorporates fire intensity parameters into the pheromone update rules, allowing the path search process to dynamically adapt to changing fire trends. During the fire bomb launch phase, the relative positional relationship between the target point coordinates and the drone's flight state is calculated in real time, combining gravity acceleration and initial velocity parameters to calculate the optimal launch angle. A Kalman filter is also used to compensate for the effects of wind speed on the trajectory.

[0151] Compared with existing technologies, traditional drone firefighting solutions rely on visible light imagery or low-resolution infrared data to construct two-dimensional fire scene models, which cannot accurately identify the distribution of three-dimensional obstacles. Path planning algorithms only consider the shortest distance and fail to incorporate fire intensity weights, resulting in irrational allocation of firefighting resources. Fire bomb delivery relies on manual empirical formulas and lacks real-time physical model calculations. This solution effectively addresses the issues of insufficient navigation accuracy and low firefighting efficiency in complex fire environments through high-precision three-dimensional modeling, dynamic weighted path optimization, and ballistic equation solving.

[0152] Through the above technical solution, this application can achieve centimeter-level modeling accuracy of the three-dimensional structure of the fire scene, improve the obstacle avoidance capability of the UAV in a dense smoke environment; dynamically adjust the flight path weight coefficient so that the UAV prioritizes extinguishing areas with high fire intensity; calculate the projection parameters in real time based on the physical kinematic equations, reduce the trajectory deviation of the fire extinguishing bomb in a complex airflow environment, and ultimately shorten the fire response time and improve the initial fire extinguishing success rate.

[0153] Example 8 also includes system self-checking and fault tolerance mechanisms:

[0154] Mechanism 1: Perform sensor health checks every 6 hours and automatically isolate nodes with deviations greater than 10%;

[0155] Mechanism 2: When communication is interrupted, the edge node switches to local decision-making mode and uses a lightweight LSTM model to predict the direction of fire spread;

[0156] Mechanism 3: Dual redundant power supply design is adopted to seamlessly switch to supercapacitor backup power supply when the main power supply fails.

[0157] Sensor health monitoring refers to the process of periodically evaluating the operating status of sensors. This can be achieved by using a preset deviation threshold to trigger a node isolation mechanism. By comparing the difference between the sensor output value and the standard reference value, it is determined whether the device is in an abnormal state. Local decision-making mode refers to the autonomous operation of edge computing nodes when the communication link is interrupted. Specifically, the fire spread function can be implemented through a pre-deployed lightweight LSTM model that can learn the spatiotemporal characteristics of fire spread based on historical data. Dual redundant power supply design refers to a structure with two independent power supply units: primary and backup. Specifically, supercapacitors can be used as the energy storage medium of the backup power supply, leveraging their rapid charging and discharging characteristics to achieve seamless power switching.

[0158] Specifically, the system self-checking and fault-tolerance mechanism ensures system reliability through collaborative operations at three levels. The sensor health detection module periodically collects the working parameters of each node. For example, it uses a dynamic comparison algorithm between the sensor output value and the reference value. When the deviation exceeds the preset threshold, the isolation instruction is triggered to prevent erroneous data from entering the subsequent processing link. In the case of abnormal communication, the edge computing device automatically calls the locally stored prediction model, which generates the decision-making basis required for emergency response by performing time-series modeling on the direction of fire spread. The power management unit monitors the status of the main power supply in real time. When a voltage anomaly or power-off signal is detected, the load is switched to the backup power supply through the relay control circuit. The high power density characteristics of the supercapacitor group ensure continuous power supply to key equipment during the switching process.

[0159] Compared with existing technologies, traditional systems lack proactive self-checking mechanisms, making them unable to promptly identify sensor anomalies, increasing the risk of misjudgment of fire conditions. Existing edge devices often cease functioning when disconnected from the internet, rendering them unable to maintain basic decision-making capabilities. This mechanism, however, leverages local models to achieve continuous warning capabilities even in offline conditions. Conventional single-power systems are prone to service interruptions during power failures. The dual-redundancy design significantly improves operational stability through hardware-level backup.

[0160] Through the above technical solution, this application solves the problem of missed fire detection caused by insufficient equipment reliability in complex environments, reduces the impact of communication interruptions on system continuity, and realizes the uninterrupted operation capability of key equipment in power failure scenarios, providing technical guarantee for the long-term stable operation of the fire monitoring system in harsh environments.

[0161] As a preferred embodiment of the present application, the installation arrangement of the multimodal data acquisition unit includes:

[0162] Method 1: Hexagonal cellular topology deployment: Dynamically calculate sensor spacing based on terrain height differences

[0163] Where R is the effective detection radius of the sensor, Hmax and H avg The maximum and average terrain height differences in the region, respectively, and the overlap rate of the fields of view of adjacent units ≥ 20%;

[0164] Method 2: 3D gradient arrangement, with layered deployment every 100 meters in mountainous areas and adaptive pitch angle adjustment In the forest scene, sensor arrays are arranged vertically in the canopy layer, shrub layer, and ground layer.

[0165] Among them, hexagonal cellular topology deployment refers to the use of a honeycomb grid structure for sensor layout. Specifically, it can be achieved by using a dynamic matching algorithm between terrain elevation data and the effective coverage radius of the sensor. This deployment method ensures monitoring continuity and redundancy by optimizing the unit spacing. Three-dimensional gradient arrangement refers to the layered configuration of sensor groups according to the vertical dimension of space. Specifically, it can be achieved by using a combination of positioning technology of laser altimeter and inertial navigation system. This arrangement method improves the monitoring capability of three-dimensional space through vertical dimension coverage. The field of view overlap rate refers to the proportion of the intersection area of ​​adjacent sensor monitoring areas. Specifically, it can be controlled by adjusting the installation angle and height. This indicator directly affects the multi-angle verification capability of abnormal areas. Pitch angle adaptive adjustment refers to the automatic adjustment of the sensor installation angle with the slope of the terrain. Specifically, it can be achieved by using a gyroscope and servo motor collaborative control system. This function ensures that the sensor maintains the optimal detection angle on sloping terrain.

[0166] Specifically, hexagonal cellular topology deployment is preferred in open terrain, and the coverage range and device density are balanced by dynamically calculating the sensor spacing. For example, a uniform cellular structure is used in flat areas, and the unit spacing is adjusted according to the height difference in undulating terrain. For mountainous scenarios, the sensor array is deployed in layers according to altitude. The spacing between each layer can be set to a specific height value according to the actual slope, and the pitch angle is adjusted to ensure the continuous connection of the field of view of each layer of sensors. In forest scenarios, sensors are installed at the top of the tree canopy, in the bush area, and on the ground layer. Temperature anomaly signals at different heights are captured through multi-layer layout in the vertical direction. The overlapping part of the field of view of adjacent sensors is set to a coverage ratio of not less than 20%, so that monitoring relay can be achieved by adjacent units in the event of a single point failure.

[0167] Compared with existing technologies, traditional fixed-spacing deployment methods are prone to blind spots in complex terrain. For example, in mountainous areas with elevation variations exceeding 15%, horizontally arranged sensor networks have a coverage rate of less than 70%. This solution increases monitoring coverage to over 95% by dynamically adjusting sensor spacing and three-dimensional gradient layout. Existing single-plane deployment methods are unable to effectively capture vertical fire signals in forest scenarios. However, this solution, through a three-dimensional deployment across the canopy, shrub layer, and ground layer, significantly improves the ability to simultaneously monitor both smoldering ground fires and crown fires.

[0168] Through the above technical solutions, this application solves the technical problems of low sensor deployment coverage and large monitoring blind spots in complex terrain environments, and realizes the comprehensive capture of multi-dimensional spatial fire signals. In scenes with obvious vertical structures such as forests, the layered deployment effectively identifies the characteristics of fires at different heights, avoiding the problem of missed detection caused by ground sensors being blocked by upper vegetation. The dynamically adjusted deployment method saves 15%-30% of the number of equipment compared to fixed installation, reducing system construction costs while ensuring monitoring quality.

[0169] As a preferred embodiment of the present application, a computer-readable storage medium stores a computer program, which implements the steps of the method in the above embodiment when executed by a processor.

[0170] A computer-readable storage medium refers to a physical medium capable of persistently storing program instructions. Specifically, this can be achieved using a solid-state drive, USB flash drive, or optical disk. Its purpose is to ensure the integrity of the stored program code even when the power is off. The execution of a computer program by a processor refers to the execution of an instruction set written in a programming language within a computing device. Specifically, this can be achieved using the C++ or Python programming languages ​​in conjunction with a compiler. Its purpose is to convert the method flow described in the claims into executable machine instructions.

[0171] Specifically, when the program code stored in the storage medium is executed, it first controls the quantum dot infrared sensor array to collect dual-band thermal imaging data, while also acquiring visible light images and environmental parameters. After eliminating temporal and spatial deviations in the multi-source data using a spatiotemporal alignment algorithm, a dynamic feature fusion network is used to generate a joint feature vector. Edge computing nodes perform an initial lightweight model screening, and suspected fire point data is uploaded to the cloud via an encrypted channel for multi-model verification. Ultimately, the verification results trigger drone dispatch commands and firefighting operations, with the entire process recorded in a blockchain operation log.

[0172] Compared to existing technologies, which only support static storage of a single algorithm model, this solution programmatically encapsulates the entire process of multimodal data collection, layered analysis, and coordinated response, enabling offline operation. While traditional technologies rely on real-time cloud-based data transmission, this solution allows edge devices to independently perform initial screening and encrypted storage without a network connection, while ensuring the localization of private data.

[0173] Through the above technical solution, this application achieves portable deployment of fire identification algorithms, enabling rapid loading of standardized program modules across diverse hardware devices. A self-checking mechanism built into the storage medium regularly verifies program integrity, preventing misoperations caused by data corruption. Privacy protection steps integrated into the program code automatically desensitize sensitive information during execution, preventing personal information from being leaked during storage medium transfer.

[0174] Anything not described in the present invention can be achieved by adopting or drawing on existing technologies.

[0175] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0176] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A method for visually identifying fire hazards, characterized in that: The specific steps include: S1. Multimodal data acquisition: Quantum dot infrared sensor arrays are used to acquire mid-infrared and far-infrared dual-band thermal imaging data of the target area. The mid-infrared band is 3-5μm, and the far-infrared band is 8-12μm. Visible light images and environmental parameter data are also collected simultaneously. Environmental parameter data include wind speed, CO concentration, and PM2.

5. S2. Spatiotemporal data alignment: Time synchronization of multi-source data is achieved based on the BeiDou-3 timing module, with a time synchronization error of less than 10ns. Spatial registration of thermal and visible light images is achieved through point cloud SLAM algorithm and SIFT feature matching, with a spatial registration error of less than 0.3 pixels. S3. Dynamic feature fusion: Build a feature fusion network based on a multi-head attention mechanism, dynamically assign weights to thermal imaging, visible light, and smoke features according to environmental parameters, and generate a joint feature vector. S4. Hierarchical analysis and early warning: A lightweight MobileNet model is used at the edge for initial screening, with a processing delay of less than 50ms. Suspected fire points are uploaded to the cloud via a federated learning framework, and multi-model voting and verification are performed using ResNet50 and 3D-CNN models. S5. Collaborative response and disposal: When the fire is confirmed, the nearest fire-fighting drone is automatically dispatched based on the blockchain smart contract, the optimal fire-fighting path is calculated based on the fire spread model, and the dry powder spraying or fire-extinguishing bomb throwing action is triggered.

2. A fire hazard visual identification method according to claim 1, characterized in that: The deployment of the quantum dot infrared sensor array meets the following conditions: Condition 1: Sensor spacing Among them, S is the area of ​​the monitoring area, and N is the number of sensors; Condition 2: Sensitivity reaches 0.1°C resolution, and effective detection distance ≥ 500 meters; Condition 3: Support event-driven mode, and start data acquisition only when the temperature gradient change rate is detected to be greater than 2℃ / min.

3. A fire hazard visual identification method according to claim 1, characterized in that: The specific implementation of the dynamic feature fusion network includes the following steps: S31: Extracting latent feature vectors from thermal imaging data using variational autoencoders (VAEs). S32: Use the improved channel attention module CBAM to spatially enhance visible light image features; S33: Using the environmental parameter weight matrix W evn =σ(MLP(v wind , c co ))Dynamically weight the multimodal features, where σ is the Sigmoid function.

4. The method for visually identifying fire hazards according to claim 1, characterized in that: The working steps of the federated learning framework include: In the first step, the edge node uploads the model gradient parameters, which are aggregated in the cloud to generate a global model; In the second step, differential privacy technology is used to add Gaussian noise N (0, 0.1 2 ); The third step is to perform model synchronization every 24 hours with a synchronization delay of less than 5 minutes.

5. The method for visually identifying fire hazards according to claim 3, characterized in that: The blockchain smart contract contains the following logic: Logic 1: Define the fire level judgment function: Logic 2: Automatically trigger response strategies based on fire severity: Level 1: dispatch 3 heavy drones to drop fire bombs; Level 2: dispatch two fire-fighting drones to provide dry powder coverage; Level 3: Activate audible and visual alarms and notify manual review.

6. The method for visually identifying fire hazards according to claim 1, characterized in that: Also includes privacy protection steps: Step 1: Perform federated learning feature desensitization on the face / license plate area in the visible light image; Step 2: Sensitive data is encrypted using a homomorphic encryption algorithm at the edge computing node, and the encryption key is dynamically updated using the quantum key distribution (QKD) protocol. Step 3: When storing data, add Gaussian blur to non-fire related areas, σ = 2.

0. The larger the σ value, the higher the degree of blur. When σ = 2.0, the coverage is wider and suitable for medium-intensity smoothing.

7. The method for visually identifying fire hazards according to claim 2, characterized in that: The control method of the drone includes: S01. Use LiDAR to construct a 3D point cloud model of the fire scene in real time with a resolution of 10cm; S02, using the improved ant colony algorithm to plan the fire extinguishing path, the objective function is: Among them, t i is the flight time, A i is the fire intensity, α, β, k are weight coefficients; S03. The calculation of the throwing angle of the fire extinguishing bomb meets the following requirements: Among them, v0 is the initial velocity, g is the acceleration due to gravity, and (x, y) is the target coordinate.

8. The method for visually identifying fire hazards according to claim 1, characterized in that: It also includes system self-checking and fault-tolerance mechanisms: Mechanism 1: Perform sensor health checks every 6 hours and automatically isolate nodes with deviations greater than 10%; Mechanism 2: When communication is interrupted, the edge node switches to local decision-making mode and uses a lightweight LSTM model to predict the direction of fire spread; Mechanism 3: Dual redundant power supply design is adopted to seamlessly switch to supercapacitor backup power supply when the main power supply fails.

9. The method for visually identifying fire hazards according to claim 1, characterized in that: The installation arrangement of the multimodal data acquisition unit includes: Method 1: Hexagonal cellular topology deployment: Dynamically calculate sensor spacing based on terrain height differences Where R is the effective detection radius of the sensor, H max and H avg The maximum and average terrain height differences in the region, respectively, and the overlap rate of the fields of view of adjacent units ≥ 20%; Method 2: 3D gradient arrangement, with layered deployment every 100 meters in mountainous areas and adaptive pitch angle adjustment In the forest scene, sensor arrays are arranged vertically in the canopy layer, shrub layer, and ground layer.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images