A multi-modal dynamic weighting-based ancient building fire detection system

Abstract

This invention relates to the field of computer vision technology, specifically to a multimodal dynamic weighted fire detection system for ancient buildings. The system includes a multimodal data acquisition and collaborative calibration unit for acquiring visible light, thermal imaging, infrared, and environmental monitoring data, and for collaboratively calibrating each acquisition device based on the fixed structural characteristics of the ancient building. This invention effectively compensates for the performance limitations of single-modal detection in complex environments such as nighttime and smoke by combining visible light, thermal imaging, and infrared multimodal data and dynamically adjusting the weights of each modality, thus adapting to the variable lighting and weather conditions of ancient buildings. Through temporal behavior analysis, it captures the dynamic changes in the location, size, and temperature of the fire source. Combined with temperature feature calibration and contextual knowledge specific to ancient buildings, it accurately distinguishes between sacrificial fires and real fires, solving the problem of misjudgment caused by visual similarity.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a fire detection system for ancient buildings based on multimodal dynamic weighting. Background Technology

[0002] Fire detection in ancient buildings needs to balance timely early warning with scene adaptability. Multimodal fusion technology, by integrating different types of sensor data such as visible light, thermal imaging, and infrared, overcomes the limitations of single-modal detection in complex environments and has become the mainstream technology in this field. Its core logic is to utilize complementary multi-dimensional features to improve the accuracy and environmental adaptability of fire detection, so as to match the complex structural layout of ancient buildings, diverse usage scenarios, and special protection needs of precious cultural relics.

[0003] In existing technologies, such as the remote fire alarm system disclosed in CN120318965A, a multimodal perception and dynamic weighted data fusion strategy is used to calculate the fire probability by adjusting the weights based on the stability of sensor data. However, it does not consider the special characteristics of ancient building scenarios, nor does it adapt to the visual similarity between sacrificial fire sources and real fires in ancient buildings, the complexity of wooden structure backgrounds, and the temporal evolution characteristics of fire sources. Its weighting mechanism only focuses on the reliability of the sensors themselves, making it difficult to solve the specific detection challenges in ancient building scenarios.

[0004] Existing fire detection systems and similar technologies suffer from several problems in ancient building scenarios: they fail to effectively distinguish between sacrificial fire sources and real fires, and lack adaptability to the complex environments of ancient buildings. On one hand, general computer vision detection systems do not consider the unique structure and usage of ancient buildings, making it difficult to differentiate between stable fire sources like incense and braziers used in sacrificial activities and the spreading fire sources of real fires, resulting in a high false alarm rate. On the other hand, single-modal detection relies on a single type of data, leading to significant performance degradation in complex environments such as nighttime or smoke-covered conditions. Furthermore, static detection ignores the temporal behavior of fire development, failing to capture the dynamic differences between stable and spreading fire sources. Additionally, existing multimodal fusion technologies often employ simple weighting methods, failing to dynamically adjust the weights of each modality according to the characteristics of ancient building scenarios, thus hindering the full utilization of the complementary advantages of multimodal data and impacting detection accuracy and early warning effectiveness. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a multimodal dynamic weighted ancient building fire detection system, which solves the problems of weak single-modal adaptability to complex environments, inability to distinguish between sacrificial fires and real fires, and high rates of missed detection and false alarms for small targets compared to existing technologies.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a fire detection system for ancient buildings based on multimodal dynamic weighting, comprising:

[0007] The multimodal data acquisition and collaborative calibration unit is used to acquire visible light, thermal imaging, infrared and environmental monitoring data, and to perform collaborative calibration of each acquisition device based on the fixed structural characteristics of ancient buildings.

[0008] Cross-modal temporal-spatial alignment unit is used to perform spatial coordinate calibration, channel semantic mapping, scale unification, and temporal delay correction on multimodal data;

[0009] The dynamic weighted fusion and feature adjudication unit is used to dynamically weight and fuse multimodal features based on scene features, temporal dynamics, urgency, feature importance, and feature conflict adjudication results, and output cross-modal enhanced features.

[0010] The fire source temporal behavior evolution analysis unit is used to extract the location trajectory stability, size expansion rate and temperature change gradient of the fire source, and identify the behavior pattern of the fire source.

[0011] The ancient building dynamic context knowledge base unit is used to store and update information on the structural model, functional zoning, material distribution, and human activity patterns of ancient buildings.

[0012] The fire source-wood structure material interaction feature extraction unit is used to extract the temperature conduction rate, material thermal response characteristics and smoke contact diffusion mode of the wood structure around the fire source.

[0013] A hierarchical multi-feature decision unit is used to integrate cross-modal enhancement features, fire source temporal behavior features, contextual knowledge, and fire source-material interaction features to make hierarchical decisions to generate a fire risk score.

[0014] The graded early warning unit is used to execute graded early warning responses based on fire risk scores and the real-time status of ancient buildings.

[0015] Furthermore, the specific processing method of the multimodal data acquisition and collaborative calibration unit is as follows:

[0016] Visible light acquisition equipment, thermal imaging sensing equipment, infrared acquisition equipment, and environmental monitoring equipment are deployed. The visible light acquisition equipment covers the sacrificial area, the dense wooden structure area, and the ancient book storage area. The thermal imaging sensing equipment and infrared acquisition equipment are used to capture temperature distribution and cope with low visibility scenes, respectively.

[0017] Environmental monitoring equipment simultaneously collects data on temperature, humidity, wind speed and direction, and air quality.

[0018] All equipment is concealed and uses beam-column joints or door and window outlines as reference points to calibrate the acquisition angle, timestamp, and data accuracy of each device in real time.

[0019] Furthermore, the specific processing method of the cross-modal temporal-spatial alignment unit is as follows:

[0020] Using the fixed structure of ancient buildings as a reference point, spatial alignment of multimodal data is performed through perspective transformation matrix;

[0021] Thermal imaging and infrared feature distribution are mapped to the visible light feature space through adaptive linear transformation;

[0022] Unify all modal features to the same resolution;

[0023] Based on the synchronous analysis of fire source dynamic trajectory and timestamp, the acquisition delay of different modal data is corrected.

[0024] Furthermore, the specific processing method of the dynamic weighted fusion and feature adjudication unit is as follows:

[0025] A lightweight feature extraction network with a preset computational threshold is used to extract scene features of lighting, weather, and smoke, and scene-adaptive weights are assigned.

[0026] The detection accuracy of each modality in the preceding frame is tracked using an LSTM network, and time-series dynamic weights are assigned.

[0027] Based on the rate of temperature change and the intensity of thermal radiation, urgency-driven weights are assigned.

[0028] Based on the correlation between each modal feature and fire source classification and location, feature importance weights are assigned;

[0029] When different modal features contradict each other, feature combinations are selected by combining fire source-material interaction features, temporal evolution features and contextual information through multi-dimensional voting and confidence verification.

[0030] By introducing channel attention and spatial attention mechanisms, the core features of the fire source are highlighted and background interference information is filtered out.

[0031] Furthermore, the specific processing method of the fire source temporal behavior evolution analysis unit is as follows:

[0032] A dynamic feature extraction model for fire sources is constructed to obtain the stability of the fire source location trajectory, the rate of size expansion, the temperature change gradient, the shape retention, and the continuity of the combustion state.

[0033] A behavior pattern recognition engine is built using an LSTM network to distinguish between stable patterns, diffusion patterns, and intermittent patterns.

[0034] A fire source evolution trend prediction module was constructed by using sliding window analysis and modeling with multiple preset time scales. Combining the distribution of ancient building materials and environmental parameters, the future location changes, size expansion and temperature rise trends of the fire source were predicted.

[0035] An abnormal change detection module for fire source behavior has been added to increase the detection priority for situations such as sudden temperature rises or size jumps.

[0036] Furthermore, the specific processing method of the ancient building dynamic context knowledge base unit is as follows:

[0037] Establish a three-dimensional structural model and material distribution map of the ancient building, and divide it into sacrificial area, dense wooden structure area and fire-fighting weak area;

[0038] Based on a pre-defined deep learning algorithm model, environmental scene classification and human activity pattern recognition are achieved.

[0039] Identify fire sources occurring during abnormal time periods through temporal context analysis;

[0040] The risk level of each area is updated in real time based on changes in environmental parameters and the intensity of human activities.

[0041] Furthermore, the specific processing method of the fire source-wood structure material interaction feature extraction unit is as follows:

[0042] Extract the temperature conduction rate, material thermal response characteristics, and smoke-smoke diffusion patterns of the wooden structure surrounding the fire source;

[0043] By combining a multi-scale edge enhancement module and a dual-level attention enhancement module, the extraction effect of weak temperature rise signals and local material thermal radiation change features at the cracks in the wooden beam is enhanced.

[0044] Furthermore, the system also includes a dynamic noise intelligent suppression unit, which is used to build a scene adaptive noise model, identify the noise types of wood texture, tourist movement and object heat radiation through deep learning algorithms, and generate a noise filter mask;

[0045] For dynamic noise, time-series behavior analysis is used to distinguish between mobile interference and stationary fire sources;

[0046] For static noise, non-fire source edge features are filtered out through multi-scale edge separation.

[0047] Furthermore, the system also includes a cross-scene migration adaptive unit, which is used to perform online fine-tuning based on a small amount of measured data from target ancient buildings, on the basis of pre-training on a general ancient building fire dataset, and adaptively adjust the feature extraction parameters and decision weights.

[0048] A feature library of ancient building scenes is constructed. When the system is deployed to a new scene, the scene model parameters are automatically matched with the preset scene similarity threshold.

[0049] Furthermore, the specific processing method of the hierarchical multi-feature decision unit is as follows:

[0050] A hierarchical mechanism is adopted, which includes local feature decision-making, global feature fusion, and dynamic result verification.

[0051] Local feature decision-making makes a preliminary determination of the core features of each modality;

[0052] Global feature fusion uses a combination of expert rules and machine learning to assign weights and calculate a comprehensive risk score.

[0053] The dynamic result verification module combines historical detection data, similar scenario decision cases, and fire source evolution trend prediction results to retrospectively verify the current decision;

[0054] A triple verification mechanism is introduced, which includes temperature threshold calibration, behavior pattern matching, and material interaction verification.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0056] This invention effectively compensates for the performance shortcomings of single-modal detection in complex environments such as nighttime and smoke by combining visible light, thermal imaging and infrared multimodal data and dynamically adjusting the weights of each modality, thus adapting to the changing lighting and weather conditions of ancient buildings.

[0057] By capturing the dynamic changes in the location, size, and temperature of the fire source through temporal behavior analysis, and combining temperature feature calibration with contextual knowledge specific to ancient buildings, the system can accurately distinguish between sacrificial fires and real fires, thus solving the problem of misjudgment caused by visual similarity.

[0058] By employing multi-scale edge enhancement, a two-level attention mechanism, and a four-level feature pyramid design, the detection capability of small and slender fire sources is strengthened, avoiding missed detection of small targets and background interference such as wood texture.

[0059] By further improving the reliability of detection through a hierarchical multi-feature decision-making and graded early warning mechanism, early warning of fires can be achieved. While ensuring the fire safety of ancient wooden buildings, the impact of false alarms on normal religious and cultural activities can be minimized, balancing the needs of cultural relic protection and daily use, and providing practical technical reference for fire detection in similar scenarios. Attached Figure Description

[0060] Figure 1 This is a diagram of the core system architecture of the present invention;

[0061] Figure 2 This is a flowchart of the multimodal data processing of the present invention;

[0062] Figure 3 This is a flowchart of the fire source temporal behavior evolution analysis of the present invention;

[0063] Figure 4 This is a flowchart illustrating the hierarchical multi-feature decision-making process of the present invention.

[0064] Figure 5 This is a flowchart of the graded early warning response of the present invention;

[0065] Figure 6 This is a flowchart of the auxiliary optimization unit of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Example 1

[0068] Please see Figure 1-6 This invention provides a multimodal dynamic weighted fire detection system for ancient buildings, comprising:

[0069] The multimodal data acquisition and collaborative calibration unit is used to acquire visible light, thermal imaging, infrared and environmental monitoring data, and to perform collaborative calibration of each acquisition device based on the fixed structural characteristics of ancient buildings.

[0070] Specifically, the first step is to conduct multimodal data acquisition and collaborative calibration, deploying visible light acquisition equipment, thermal imaging sensors, infrared acquisition equipment, and environmental monitoring equipment. The visible light acquisition equipment uses high-definition cameras with 4K resolution and low-light enhancement capabilities, focusing on covering the sacrificial area, densely wood-structured areas, and ancient book storage areas. These areas are critical for ancient building fires and have concentrated cultural relic value, ensuring comprehensive monitoring of key risk points. The thermal imaging sensors employ sensors with a temperature measurement range of -20℃ to 550℃ and an accuracy of ±2℃ to accurately capture temperature distribution information, providing data support for distinguishing between low-temperature stable fire sources and high-temperature spreading fire sources. The infrared acquisition equipment is specifically designed to handle low-visibility scenarios such as nighttime and smoke obscuration, compensating for the detection limitations of visible light equipment in special environments. The environmental monitoring equipment simultaneously collects temperature, humidity, wind speed and direction, and air quality data; these environmental parameters will provide important references for subsequent fire source evolution trend prediction and risk assessment. All equipment is installed in a concealed manner. Without damaging the original appearance and style of the ancient buildings, the acquisition angle, timestamp and data accuracy of each device are calibrated in real time based on the beam and column nodes or the outline of doors and windows as reference points. This ensures the consistency of multimodal data in time and space, laying a foundation for subsequent data fusion and analysis.

[0071] In one specific embodiment, the cross-modal temporal-spatial alignment unit is used to perform spatial coordinate calibration, channel semantic mapping, scale unification, and temporal delay correction on multimodal data;

[0072] Specifically, cross-modal temporal-spatial alignment is performed, using the fixed structure of ancient buildings as a reference point, and spatial alignment of multimodal data is achieved through perspective transformation matrices. Let the modalities be... ( The set of reference points (where RGB represents visible light, IR represents infrared, and T represents thermal imaging) is: Four fixed structural points, such as the four corners of doors and windows of ancient buildings, which do not change over time, are selected as reference points to calculate the modal characteristics. Perspective transformation matrix to RGB mode The expression is ,in This is a function to calculate the perspective transformation matrix, which is used to transform the modes. Affine transformation of feature maps ,in For modality The original feature map, and These represent the height and width of the RGB modal feature maps, respectively, to minimize the positional deviation of the same fire source under different modalities. An adaptive linear transformation maps the thermal imaging and infrared feature distribution to the visible light feature space; the transformation formula is as follows: ,in The channel semantic mapping weight matrix, For bias vectors, and RGB mode and mode respectively The number of feature channels is determined, and optimal parameters are learned through network training to achieve semantic consistency across different modal channels. All modal features are unified to the same resolution using bilinear interpolation. This ensures the matching of multimodal features across scales. Based on the synchronous analysis of fire source dynamic trajectory and timestamps, the acquisition delay of different modal data is corrected. By calculating the positional offset of the fire source center point in consecutive frames, the timestamps of each modal data are adjusted to ensure accurate alignment of multimodal data in the time dimension, avoiding feature fusion deviations caused by acquisition delays.

[0073] In one specific embodiment, the dynamic weighted fusion and feature adjudication unit is used to dynamically weight and fuse multimodal features based on scene features, temporal dynamics, urgency, feature importance and feature conflict adjudication results, and output cross-modal enhanced features;

[0074] Specifically, dynamic weighted fusion and feature adjudication are performed, and scene features such as lighting, weather, and smoke are extracted using the lightweight feature extraction network MobileNetV2 with a preset computational threshold. ,in Light intensity, Weather type For smoke concentration, assign scene-adaptive weights based on scene features. The scene mapping rules are set based on the actual environmental characteristics of the ancient buildings and weather types. Days are categorized into four types: sunny, rainy, windy, and foggy, which will be related to the intensity of sunlight. Smoke concentration They participate in the weighting process; for example, on a clear day with no smoke. and , , , When there is no smoke at night and , , , ; Smoke scene , , , Rainy day scenes Based on the above, the weight of the visible light mode is reduced by 0.1, and the weight of the thermal imaging mode is increased by 0.1 to adapt to the reduced visible light imaging quality in rainy weather. These weight values ​​were obtained through statistical analysis of a large amount of ancient building scene data, and can adapt to the differences in the effectiveness of each mode under different scenarios. The detection accuracy of each mode in the previous 10 frames is tracked using an LSTM network, and temporal dynamic weights are assigned accordingly. , of which The temporal dynamic weight of a frame is denoted as ,and For the same technical feature, the calculation formula is: ,in For the previous frame mode Temporal weights, For modality Continuous detection accuracy indicator (1 indicates continuous accuracy, 0 indicates otherwise). For modality False positive flag (1 indicates false positive, 0 indicates otherwise). This is the weighting factor. To reduce the coefficient for weights, the weight range is constrained. To prevent a single mode from completely failing, this parameter setting ensures dynamic adjustment of sensitivity based on weights while preventing excessive weight fluctuations from affecting detection stability. Based on temperature change rate... With thermal radiation intensity Assign urgency-driven weights Temperature change rate ,in and These are the fire source temperatures for the current frame and the previous frame, respectively. The urgency weight is calculated using the following formula, where the time interval is specified:

[0075]

[0076] in For the Sigmoid activation function, when or At this stage, the thermal imaging and infrared modal weights can reach up to 2.0, prioritizing the capture of temperature anomalies. This design can quickly respond to the temperature surge characteristics of initial fires. Based on the correlation between each modal feature and fire source classification and location, feature importance weights are assigned. The weight is determined by calculating the mutual information value between the feature and the fire source label; the higher the mutual information value, the greater the weight, ensuring that features contributing significantly to fire source detection are given sufficient attention. When different modal features contradict each other, feature combinations are selected through multi-dimensional voting and confidence checks, combining fire source-material interaction features, temporal evolution features, and contextual information. For example, when the RGB modality detects a suspected fire source but the thermal imaging modality does not detect a temperature anomaly, temporal features are combined to determine if the suspected area has a size expansion trend. Simultaneously, the contextual knowledge base is consulted to determine if the area is a sacrificial activity area, making a final feature selection based on comprehensive information from multiple aspects. Channel attention and spatial attention mechanisms are introduced. Channel attention calculates and weights the importance scores of each feature channel, while spatial attention generates a spatial attention map to highlight the core fire source area, effectively filtering background interference information and improving the effectiveness of feature fusion.

[0077] In one specific embodiment, the fire source temporal behavior evolution analysis unit is used to extract the location trajectory stability, size expansion rate and temperature change gradient of the fire source, and identify the behavior pattern of the fire source.

[0078] Specifically, when conducting temporal behavior evolution analysis of fire sources, a dynamic feature extraction model for the fire source is first constructed to obtain the stability of the fire source's location trajectory, size expansion rate, temperature change gradient, shape retention, and continuity of combustion state. The stability of the location trajectory is determined by calculating the mean Euclidean distance between the fire source's center point and the fire source in consecutive frames; a smaller mean value indicates higher stability. The size expansion rate... ,in and These represent the fire source areas in the current frame and the previous frame, respectively; the temperature gradient is calculated using the Sobel operator to determine the gradient value of the temperature field. ,in This is the normalized temperature characteristic map. and The Sobel operators are applied in the x and y directions, respectively. Shape preservation is obtained by calculating the Hu moment similarity of the fire source contours in consecutive frames; the higher the similarity, the better the shape preservation. The continuity of combustion state is determined by judging the overlap rate of the fire source region in consecutive frames. A behavior pattern recognition engine is constructed using an LSTM network to distinguish between stable, spreading, and intermittent modes. The input to the LSTM network is the extracted dynamic feature sequence, and the output is the probability value of each behavior mode. When the probability of the stable mode is the highest, it is identified as a stable fire source such as a sacrificial fire, characterized by fixed position, stable size, and small temperature fluctuations. When the probability of the spreading mode is the highest, it is identified as a real fire, characterized by positional movement, size expansion, and temperature rise. When the probability of the intermittent mode is the highest, it is identified as a fire source in a specific sacrificial activity, characterized by periodic appearance and rapid extinguishing. A fire source evolution trend prediction module is constructed using sliding window analysis and modeling with multiple preset time scales. The sliding window size is set to 30 frames to cover the short-term behavioral changes of the fire source. Multiple time scales, including 1s, 3s, and 5s, correspond to different fire development rates. Combining the distribution of ancient building materials and environmental parameters, a time-series prediction model is used to predict the future location changes, size expansion, and temperature rise trends of the fire source. For example, in areas with dense wooden structures, considering the flammable properties of wood, the predicted fire source expansion rate will be higher than in other areas. An abnormal abrupt change detection module for fire source behavior is added to increase the detection priority for sudden temperature rises or size jumps. When a temperature change rate is detected... or the rate of dimensional expansion² In such cases, the area is marked as a high-priority detection area to expedite subsequent processing and ensure a rapid response to sudden fires.

[0079] In one specific embodiment, the ancient building dynamic context knowledge base unit is used to store and update the structural model, functional zoning, material distribution and human activity patterns of the ancient building.

[0080] Specifically, when constructing a dynamic contextual knowledge base for ancient buildings, the first step is to establish a three-dimensional structural model and material distribution map of the ancient buildings. Three-dimensional point cloud data of the ancient buildings is acquired through laser scanning technology. After preprocessing, registration, and modeling, a high-precision three-dimensional structural model is generated. Simultaneously, the location and material information of wooden structures, brick and stone structures, and decorative components are marked by on-site surveys to form a material distribution map. Functional zones such as sacrificial areas, densely wood-structured areas, and fire-prone areas are clearly defined. For example, the area around the incense burner in the main hall of a temple is designated as a sacrificial area, the area with concentrated wooden beams on the roof is designated as a densely wood-structured area, and narrow passageways are designated as fire-prone areas. Environmental scene classification is achieved based on a pre-set deep learning algorithm model. A lightweight CNN network is used to classify images of ancient building scenes, including indoor and outdoor scenes such as halls, courtyards, and corridors. The classification results provide a scene basis for subsequent modality weight adjustment. Human activity patterns are identified through object detection and behavior analysis algorithms, distinguishing between normal sacrificial activities, tourist visits, maintenance work, and other human activities. For example, when a crowd gathers around an incense burner and burns incense, it is determined to be a normal sacrificial activity, and the system adjusts the fire source detection threshold for that area accordingly to reduce false alarms. Temporal context analysis identifies fire sources occurring during abnormal time periods. Combining this with the usage patterns of ancient buildings, the system statistically analyzes the intensity of human activity and the probability of fire sources occurring at different times. For instance, fire sources occurring between 11:00 PM and 5:00 AM are considered abnormal fire sources, and the system raises the warning level. The risk level of each area is updated in real time based on changes in environmental parameters and the intensity of human activity. The risk level is divided into low, medium, and high levels. When environmental humidity decreases, wind speed increases, or the intensity of human activity increases, the risk level of the corresponding area is adjusted upwards, and vice versa, providing a dynamic risk reference for subsequent decision-making.

[0081] In one specific embodiment, the fire source-wood structure material interaction feature extraction unit is used to extract the temperature conduction rate, material thermal response characteristics and smoke contact diffusion mode of the wood structure surrounding the fire source.

[0082] Specifically, when extracting the interaction features between the fire source and the wooden structure, the temperature conduction rate, material thermal response characteristics, and smoke diffusion patterns of the wooden structure surrounding the fire source are extracted. The temperature conduction rate is obtained by calculating the ratio of the temperature change difference of the wooden structure at different distances from the fire source to time. Different types of wood have different thermal conductivity, resulting in varying temperature conduction rates; for example, pine has a lower thermal conductivity than hardwood, and its temperature conduction rate is relatively slower. Material thermal response characteristics are obtained by analyzing the temperature change curve of the wooden structure under fire radiation, including parameters such as the heating rate and the time to reach pyrolysis temperature. The smoke diffusion patterns of the wooden structure are extracted through image analysis, showing the propagation path and diffusion speed of smoke on the wooden structure surface. A multi-scale edge enhancement module (MEFE) and a two-level attention enhancement module (BRAC) are combined to enhance the extraction of weak temperature rise signals and local material thermal radiation change features at cracks in the wooden beams. The MEFE module strengthens the edge features of the fire source through a three-level process of "low-frequency suppression - multi-scale edge separation - edge weighted enhancement," and its core formula includes low-frequency feature extraction. ,in For the input feature map, The pooling kernel size, Step size, For padding; edge separation Edge weight optimization ,in For the Sigmoid function, The operation is a 1×1 convolution; enhanced features are output after multi-scale edge fusion. The BRAC module accurately locates the elongated fire source region through a two-level attention mechanism of "channel-level semantic focusing + spatial-level location localization". At the channel level, the sampling position is dynamically adjusted through deformable convolution, and at the spatial level, semantically relevant regions are filtered through Top-K routing, effectively enhancing the extraction effect of weak signals.

[0083] In one specific embodiment, the system further includes a dynamic noise intelligent suppression unit, used to construct a scene-adaptive noise model. This model uses deep learning algorithms to identify noise types related to wood texture, visitor movement, and thermal radiation from objects, generating a noise filter mask. The scene-adaptive noise model is trained on a large amount of noisy ancient building scene data and can automatically identify noise types based on current scene characteristics. For example, in areas with dense wood structures, the main noise is wood texture, while in areas with many visitors, the main noise is visitor movement. For dynamic noise, temporal behavior analysis is used to distinguish between moving interference and fixed fire sources. By calculating the target's trajectory and speed, when the target's trajectory is irregular and its speed exceeds a preset threshold, it is determined to be dynamic noise and filtered out. For static noise, multi-scale edge separation filters non-fire source edge features. Utilizing the multi-scale edge separation technology in the MEFE module, the edge features of static noise such as wood texture are distinguished from fire source edge features, retaining effective fire source edge features and filtering static noise.

[0084] In one specific embodiment, the system further includes a cross-scene transfer adaptive unit, used for online fine-tuning based on a general ancient building fire dataset and a small amount of measured data from the target ancient building. This unit adaptively adjusts feature extraction parameters and decision weights. The general ancient building fire dataset contains fire, sacrificial fire, and background samples from various types of ancient buildings, covering different scene conditions. After pre-training on this dataset, the model can learn the general features of ancient building fires. For the target ancient building, a small amount of measured data is collected, including multimodal data from different scenes. Based on this data, the pre-trained model is fine-tuned, and the model parameters are updated using a mini-batch gradient descent algorithm to adapt the model to the specific structural and environmental characteristics of the target ancient building. An ancient building scene feature library is constructed, containing scene feature parameters for various typical ancient buildings. When the system is deployed to a new scene, the similarity between the new scene and each scene in the scene feature library is calculated. The system automatically matches the scene model parameters with a preset scene similarity threshold. The similarity calculation uses the cosine similarity formula. ,in For the feature vector of the new scene, The similarity vector is a scene feature vector in the scene feature library. When the similarity is greater than the preset threshold, the model parameters of the scene are directly called. Otherwise, it is fine-tuned based on a small amount of actual test data to improve the deployment efficiency and adaptability of the system.

[0085] In one specific embodiment, a hierarchical multi-feature decision unit is used to integrate cross-modal enhancement features, fire source temporal behavior features, contextual knowledge, and fire source-material interaction features to make hierarchical decisions to generate a fire risk score.

[0086] Specifically, a hierarchical multi-feature decision-making process is implemented, employing a hierarchical mechanism of local feature decision-making, global feature fusion, and dynamic result verification. Local feature decision-making makes preliminary judgments based on the core features of each modality: visible light modality is judged based on visual features such as flame color and shape; thermal imaging modality is judged based on temperature features; and infrared modality is judged based on thermal radiation features. Each modality outputs a preliminary judgment result and confidence level. Global feature fusion uses a combination of expert rules and machine learning to assign weights and calculate a comprehensive risk score. The weight allocation considers the importance of static features, dynamic features, and contextual features. Static features include visual detection confidence, thermal feature consistency, and location rationality; dynamic features include behavioral pattern scores and development trend assessments; and contextual features include regional risk level, temporal rationality, and correlation with human activities. Machine learning algorithms learn the weight coefficients of each feature, ensuring that the comprehensive risk score accurately reflects the true risk level of the fire source. The dynamic result verification module combines historical detection data, similar scenario decision cases, and fire source evolution trend prediction results to retrospectively verify the current decision. When there are significant differences between the current decision and historical detection data or similar scenario decision cases, the weights of each feature and the judgment results are re-evaluated to ensure the reliability of the decision. A triple verification mechanism is introduced, consisting of temperature threshold calibration, behavior pattern matching, and material interaction verification. Temperature threshold calibration is based on the temperature characteristics of ancient building fire sources, setting reasonable temperature thresholds to distinguish between sacrificial fires and real fires. Behavior pattern matching verifies the rationality of the judgment results by matching the current fire source behavior pattern with known sacrificial fire and fire behavior patterns. Material interaction verification combines the interaction characteristics between the fire source and the surrounding wooden structure to determine whether the fire source will cause the wooden structure to burn, further verifying the accuracy of the decision results.

[0087] In one specific embodiment, the graded early warning unit is used to execute graded early warning responses based on fire risk scores and the real-time status of ancient buildings.

[0088] Furthermore, the risk score ranges from 0 to 1.0. When the comprehensive risk score is 0 to 0.3, it is considered a normal state, the system maintains a regular monitoring frequency, and only stores the monitoring data in the system log without triggering an early warning. When the comprehensive risk score is 0.3 to 0.5, a Level 1 early warning (monitoring level) is triggered, and the response measures are to increase the monitoring frequency, record and analyze relevant data, and notify only the system log records. When the comprehensive risk score is 0.5 to 0.7, a Level 2 early warning (observation level) is triggered, and the response measures are to manually confirm and prepare for on-site inspections, and notify the on-duty personnel. When the comprehensive risk score is 0.7 to 0.85, a Level 3 early warning (action level) is triggered, and the response measures are to conduct on-site verification and implement preliminary emergency response, and notify the security personnel. When the comprehensive risk score is greater than 0.85, a Level 4 early warning (emergency level) is triggered, and the response measures are to activate the emergency plan, and notify all relevant personnel. Through the tiered early warning mechanism, corresponding countermeasures can be taken according to the severity of the fire risk, ensuring timely handling of fires while avoiding resource waste and unnecessary panic caused by excessive early warnings.

[0089] In summary, this implementation method effectively solves the problems of traditional fire detection technologies in ancient building scenarios, such as difficulty in distinguishing sacrificial fire sources from real fires, poor adaptability to complex environments, high false alarm rates, and insufficient early warning capabilities. Multimodal data acquisition and collaborative calibration ensure the comprehensiveness and consistency of the data; cross-modal temporal-spatial alignment solves the misalignment problem of multimodal features; dynamic weighted fusion and feature adjudication achieve effective fusion and conflict resolution of features from various modalities; fire source temporal behavior evolution analysis can accurately identify fire source behavior patterns; the dynamic contextual knowledge base of ancient buildings provides scene-adaptive information for detection decisions; fire source-wood structure material interaction feature extraction enhances the detection capability of weak signals; dynamic noise intelligent suppression reduces background interference; cross-scene migration and adaptive design improve the system's scene adaptability; hierarchical multi-feature decision-making ensures the accuracy and reliability of decisions; and the graded early warning mechanism enables reasonable response to fires.

[0090] Example 2

[0091] To enable those skilled in the art to fully understand and implement this invention, the specific implementation principle of this invention is further explained below in conjunction with a specific application scenario.

[0092] This embodiment selects a wooden temple from the Ming and Qing dynasties as the application scenario. The temple includes multiple buildings such as the main hall, the scripture pavilion, and the bell and drum towers. The interior of the buildings has a dense wooden structure and numerous decorative components. There are daily sacrificial activities, and sacrificial fire sources such as incense and braziers are frequently seen. At the same time, there is a large flow of tourists. Fire detection faces multiple challenges such as complex background interference, confusion between sacrificial fire sources and real fires, and adaptation to multiple environmental conditions. The application of this invention in this scenario can effectively address these problems.

[0093] When deploying the system at the temple, in accordance with the requirements of multimodal data acquisition and collaborative calibration, 4K resolution visible light high-definition cameras were deployed in key areas such as around the incense burner in the main hall, under the wooden beams, and near the ancient book storage shelves in the scripture pavilion to ensure no blind spots in key areas. Thermal imaging sensors were deployed in the corners of the temple courtyards and at the intersections of passageways to cover a wide area for temperature monitoring. Infrared acquisition devices were deployed under the eaves of the main hall and inside the windows of the scripture pavilion to meet the detection needs in nighttime and smoky environments. Environmental monitoring devices were deployed in different areas of the temple to collect real-time data on temperature, humidity, wind speed, wind direction, and air quality. All devices were installed discreetly, for example, by embedding visible light cameras into the wooden decorations and installing thermal imaging sensors on the sides of beams and columns, to avoid damaging the appearance of the ancient buildings. At the same time, the fixed outlines of doors and windows and beam and column nodes in the temple were used as reference points to regularly calibrate the acquisition angle, timestamp, and data accuracy of each device to ensure the synchronization and accuracy of multimodal data.

[0094] During system operation, the cross-modal temporal-spatial alignment unit continuously processes the acquired multimodal data. It uses a perspective transformation matrix to spatially align thermal imaging and infrared data with visible light data, ensuring the consistent position of the same fire source in different modal data. It uses adaptive linear transformation to map thermal imaging and infrared features to the visible light feature space, achieving channel semantic unification. It unifies all modal data to 4K resolution to ensure consistency of feature scale. Based on timestamp synchronization analysis, it corrects the acquisition delay of different modal data, enabling precise matching of multimodal data in the time dimension.

[0095] The dynamic weighted fusion and feature adjudication unit dynamically adjusts the weights of each modality based on the actual scene of the temple. During the day, when there is sufficient light and no smoke, the visible light modality has a higher weight, fully utilizing its clear visual features for fire source detection. At night or in the early morning when light is insufficient, the weights of the infrared and thermal imaging modalities are automatically increased, relying on thermal radiation and temperature characteristics to detect fire sources. During sacrificial activities, when there is a lot of incense smoke, the system enhances the weight of the thermal imaging modality, distinguishing sacrificial fire from real fires by temperature differences. When different modal features contradict each other, for example, the visible light modality detects a suspected fire source but the thermal imaging modality does not detect abnormal temperatures, the system combines the fire source-material interaction features to determine whether there is a thermal response in the wooden structure in the area. It also refers to the contextual knowledge base to determine whether the area is a sacrificial activity area, making an accurate judgment based on multiple factors to avoid false alarms.

[0096] The fire source temporal behavior evolution analysis unit extracts the dynamic characteristics of fire sources within the temple in real time. For incense and candle flames in the incense burner, their fixed position, stable size, and small temperature fluctuations are identified as a stable mode, and the system does not trigger high-level warnings. For accidentally dropped sparks, their small size, low temperature, and lack of obvious expansion trend, combined with temporal characteristics, indicate a risk-free fire source. If the fire source moves, continuously expands in size, and rapidly rises in temperature, the system identifies it as a diffusion mode and promptly issues a warning signal. Simultaneously, this unit can also predict the evolution trend of fire sources. For fire sources in densely wooded areas within the temple, it predicts a faster expansion rate, allowing for advance emergency preparation.

[0097] The ancient building's dynamic contextual knowledge base stores the temple's 3D structural model, material distribution map, and functional zoning information, clearly identifying the locations of key areas such as the sacrificial area, densely wood-structured areas, and ancient book storage areas. The system combines this information for scene understanding and risk assessment. During sacrificial activities, the system identifies normal human sacrificial behavior and adjusts the fire source detection threshold for that area accordingly. At night, when no one is around, if a fire source is detected, the system classifies it as an abnormal situation and raises the warning level. Simultaneously, based on changes in environmental parameters and the intensity of human activity within the temple, the risk level of each area is updated in real time. For example, during the dry spring season, the risk level of various areas within the temple is appropriately increased; during peak tourist seasons, the risk level of the sacrificial area is adjusted upwards.

[0098] The fire source-wood structure material interaction feature extraction unit extracts the interaction features between the fire source and the surrounding wooden structure within the temple. For fire sources near the wooden beams in the main hall, the system analyzes the temperature conduction rate and thermal response characteristics of the wooden beams to determine whether the wooden structure faces a fire risk. When smoke is present, the system analyzes the diffusion pattern of smoke on the surface of the wooden structure to predict the fire spread path, providing a reference for emergency response. Combined with the MEFE and BRAC modules, the system can effectively detect weak temperature rise signals at cracks in the wooden beams, identifying potential fire hazards in advance.

[0099] The dynamic noise intelligent suppression unit effectively filters out various interfering noises within the temple. For the complex texture of the wooden structure, the system uses multi-scale edge separation technology to distinguish it from the edge features of the fire source. For the movement of tourists and the reflection of clothing, the system uses temporal behavior analysis to determine them as dynamic noise and filters them out. For the heat radiation of objects in the temple, the system uses temperature characteristics and spatial location analysis to distinguish it from the fire source, ensuring the accuracy of fire source detection.

[0100] In the initial stage of system deployment, the cross-scene migration adaptive unit uses a model pre-trained on a general ancient building fire dataset to quickly adapt to the temple scene. Subsequently, it fine-tunes the model by collecting a small amount of measured data from the temple to make the model more closely match the specific structure and environmental characteristics of the temple. When new sacrificial facilities are added or the architectural structure undergoes minor changes, the system automatically adjusts the model parameters through scene feature matching, eliminating the need for large-scale retraining and improving the system's adaptability and maintenance efficiency.

[0101] The hierarchical multi-feature decision-making unit integrates multimodal enhanced features within the temple, temporal behavior features of the fire source, contextual knowledge, and fire source-material interaction features to make decisions. It first makes preliminary judgments on each modal feature through local feature decision-making, then calculates a comprehensive risk score through global feature fusion, and finally ensures the accuracy of the decision through dynamic result verification. A triple verification mechanism further validates the decision results: temperature threshold calibration ensures accurate differentiation between the temperature difference of sacrificial fires and real fires; behavior pattern matching verifies the rationality of the fire source behavior; and material interaction verification determines whether the wooden structure faces a fire risk. Through a multi-level decision-making process, the system ensures accurate and reliable judgments.

[0102] The tiered early warning unit executes corresponding early warning responses based on the comprehensive risk score. For normal incense and candle fires during sacrificial activities, the system only records relevant information and does not trigger high-level early warnings. For suspected fire sources, the system notifies on-duty personnel for manual confirmation. For confirmed fire hazards, security personnel are promptly notified to conduct on-site verification and preliminary handling. If a real fire occurs, the system immediately activates the emergency plan, notifying all relevant personnel to take emergency measures to minimize fire losses.

[0103] Through the above-mentioned application scenarios, this invention can accurately distinguish between sacrificial fire sources and real fires, effectively cope with the challenges of complex background interference and different environmental conditions, realize early warning and accurate detection of fires in ancient buildings, and provide reliable fire safety protection for ancient buildings while protecting the original appearance of ancient buildings and normal religious and cultural activities, demonstrating good practicality and scenario adaptability.

[0104] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0105] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A fire detection system for ancient buildings based on multimodal dynamic weighting, characterized in that, include: The multimodal data acquisition and collaborative calibration unit is used to acquire visible light, thermal imaging, infrared and environmental monitoring data, and to perform collaborative calibration of each acquisition device based on the fixed structural characteristics of ancient buildings. Cross-modal temporal-spatial alignment unit is used to perform spatial coordinate calibration, channel semantic mapping, scale unification, and temporal delay correction on multimodal data; The dynamic weighted fusion and feature adjudication unit is used to dynamically weight and fuse multimodal features based on scene features, temporal dynamics, urgency, feature importance, and feature conflict adjudication results, and output cross-modal enhanced features. The fire source temporal behavior evolution analysis unit is used to extract the location trajectory stability, size expansion rate and temperature change gradient of the fire source, and identify the behavior pattern of the fire source. The ancient building dynamic context knowledge base unit is used to store and update information on the structural model, functional zoning, material distribution, and human activity patterns of ancient buildings. The fire source-wood structure material interaction feature extraction unit is used to extract the temperature conduction rate, material thermal response characteristics and smoke contact diffusion mode of the wood structure around the fire source. A hierarchical multi-feature decision unit is used to integrate cross-modal enhancement features, fire source temporal behavior features, contextual knowledge, and fire source-material interaction features to make hierarchical decisions to generate a fire risk score. The graded early warning unit is used to execute graded early warning responses based on fire risk scores and the real-time status of ancient buildings.

2. The ancient building fire detection system based on multimodal dynamic weighting according to claim 1, characterized in that, The specific processing method of the multimodal data acquisition and collaborative calibration unit is as follows: The system deploys visible light acquisition equipment, thermal imaging sensing equipment, infrared acquisition equipment, and environmental monitoring equipment. The visible light acquisition equipment covers the sacrificial area, the dense wooden structure area, and the ancient book storage area. The thermal imaging sensing equipment and infrared acquisition equipment are used to capture temperature distribution and cope with low visibility scenarios, respectively. Environmental monitoring equipment simultaneously collects data on temperature, humidity, wind speed and direction, and air quality. All equipment is concealed and uses beam-column joints or door and window outlines as reference points to calibrate the acquisition angle, timestamp, and data accuracy of each device in real time.

3. The ancient building fire detection system based on multimodal dynamic weighting according to claim 1, characterized in that, The specific processing method of the cross-modal temporal-spatial alignment unit is as follows: Using the fixed structure of ancient buildings as a reference point, spatial alignment of multimodal data is performed through perspective transformation matrix; Thermal imaging and infrared feature distribution are mapped to the visible light feature space through adaptive linear transformation; Unify all modal features to the same resolution; Based on the synchronous analysis of fire source dynamic trajectory and timestamp, the acquisition delay of different modal data is corrected.

4. The ancient building fire detection system based on multimodal dynamic weighting according to claim 1, characterized in that, The specific processing method of the dynamic weighted fusion and feature adjudication unit is as follows: A lightweight feature extraction network with a preset computational threshold is used to extract scene features of lighting, weather, and smoke, and scene-adaptive weights are assigned. The detection accuracy of each modality in the preceding frame is tracked using an LSTM network, and time-series dynamic weights are assigned. Based on the rate of temperature change and the intensity of thermal radiation, urgency-driven weights are assigned. Based on the correlation between each modal feature and fire source classification and location, feature importance weights are assigned; When different modal features contradict each other, feature combinations are selected by combining fire source-material interaction features, temporal evolution features and contextual information through multi-dimensional voting and confidence verification. By introducing channel attention and spatial attention mechanisms, the core features of the fire source are highlighted and background interference information is filtered out.

5. The ancient building fire detection system based on multimodal dynamic weighting according to claim 1, characterized in that, The specific processing method of the fire source temporal behavior evolution analysis unit is as follows: A dynamic feature extraction model for fire sources is constructed to obtain the stability of fire source location trajectory, size expansion rate, temperature change gradient, shape retention, and continuity of combustion state. A behavior pattern recognition engine is built using an LSTM network to distinguish between stable patterns, diffusion patterns, and intermittent patterns. A fire source evolution trend prediction module was constructed by using sliding window analysis and modeling with multiple preset time scales. Combining the distribution of ancient building materials and environmental parameters, the future location changes, size expansion and temperature rise trends of the fire source were predicted. An abnormal change detection module for fire source behavior has been added to increase the detection priority for situations such as sudden temperature rises or size jumps.

6. The ancient building fire detection system based on multimodal dynamic weighting according to claim 1, characterized in that, The specific processing method of the ancient building dynamic context knowledge base unit is as follows: Establish a three-dimensional structural model and material distribution map of the ancient building, and divide it into sacrificial area, dense wooden structure area and fire-fighting weak area; Based on a pre-defined deep learning algorithm model, environmental scene classification and human activity pattern recognition are achieved. Identify fire sources occurring during abnormal time periods through temporal context analysis; The risk level of each area is updated in real time based on changes in environmental parameters and the intensity of human activities.

7. The ancient building fire detection system based on multimodal dynamic weighting according to claim 1, characterized in that, The specific processing method of the fire source-wood structure material interaction feature extraction unit is as follows: Extract the temperature conduction rate, material thermal response characteristics, and smoke-smoke diffusion patterns of the wooden structure surrounding the fire source; By combining a multi-scale edge enhancement module and a dual-level attention enhancement module, the extraction effect of weak temperature rise signals and local material thermal radiation change features at the cracks in the wooden beam is enhanced.

8. The ancient building fire detection system based on multimodal dynamic weighting according to claim 1, characterized in that, The system also includes a dynamic noise intelligent suppression unit, which is used to build a scene-adaptive noise model and identify the noise types of wood texture, tourist movement and object heat radiation through deep learning algorithms to generate a noise filter mask; For dynamic noise, time-series behavior analysis is used to distinguish between mobile interference and stationary fire sources; For static noise, non-fire source edge features are filtered out through multi-scale edge separation.

9. The ancient building fire detection system based on multimodal dynamic weighting according to claim 1, characterized in that, The system also includes a cross-scene migration adaptive unit, which is used to fine-tune the feature extraction parameters and decision weights online based on a small amount of measured data from a general ancient building fire dataset, and adaptively adjust the parameters based on the data. A feature library of ancient building scenes is constructed. When the system is deployed to a new scene, the scene model parameters are automatically matched with the preset scene similarity threshold.

10. The ancient building fire detection system based on multimodal dynamic weighting according to claim 1, characterized in that, The specific processing method of the hierarchical multi-feature decision unit is as follows: A hierarchical mechanism is adopted, which includes local feature decision-making, global feature fusion, and dynamic result verification. Local feature decision-making makes a preliminary determination of the core features of each modality; Global feature fusion uses a combination of expert rules and machine learning to assign weights and calculate a comprehensive risk score. The dynamic result verification module combines historical detection data, similar scenario decision cases, and fire source evolution trend prediction results to retrospectively verify the current decision; A triple verification mechanism is introduced, which includes temperature threshold calibration, behavior pattern matching, and material interaction verification.

Images