Ancient building group fire early warning method and system based on Internet platform

Abstract

The invention discloses an ancient building group fire early warning method and system based on an internet platform, and relates to the technical field of fire early warning, and the specific implementation scheme is as follows: obtaining a visible light image, and determining a scene type based on smoke detection confidence; the method comprises the following steps: after triggering to execute registration operation on an original infrared image, extracting an infrared feature map and a visible light feature map, generating an attention weight map according to the infrared feature map and the visible light feature map, and generating a preliminary fusion feature by combining the infrared feature map and the visible light feature map; performing dimension reduction compression processing on the preliminary fusion feature to obtain a low-dimensional feature vector, and inputting the low-dimensional feature vector into a classifier to obtain a fire occurrence probability of the current monitoring grid so as to select an early warning scheme; and marking the current monitoring grid as a risk source grid, calculating a directional propagation coefficient, updating a fire occurrence probability of an adjacent target grid, and adjusting an early warning scheme. According to the technical scheme of the invention, the problem of poor effectiveness of fire early warning of the ancient building group caused by abnormal fire behavior recognition is solved.

Description

Technical Field

[0001] This invention relates to the field of fire early warning technology, and in particular to a fire early warning method and system for ancient building complexes based on an Internet platform. Background Technology

[0002] Ancient building complexes are mostly constructed using flammable materials such as wood, and their structures generally suffer from low fire resistance ratings, insufficient fire separation distances, and complex internal spaces. With the acceleration of urbanization and increased tourism development, many ancient building complexes are being used as cultural exhibition venues, religious activity sites, or commercial spaces, resulting in frequent personnel flow, increased use of electrical equipment, and a significant increase in fire risk.

[0003] Currently, the most widely used detectors are smoke, temperature, and light detectors based on physical principles. These devices typically determine whether a fire has occurred by detecting the concentration of smoke particles in the air, abnormal increases in ambient temperature, or light radiation of specific wavelengths emitted by flames.

[0004] With the increasing prevalence of video surveillance, fire identification using computer vision technology has gradually become a research hotspot. This technology achieves automatic fire identification by analyzing surveillance video footage in real time and extracting the spectral response features of flames (such as red and yellow spectra), texture features (such as the flickering changes at the edge of flames), and dynamic motion features of smoke (such as the diffusion and drifting patterns of smoke).

[0005] Furthermore, by transmitting video streams and sensor data collected on-site to the monitoring center via wired or wireless networks (such as 4G / 5G or fiber optic private networks), remote centralized monitoring of ancient buildings scattered in different geographical locations was achieved. Some systems incorporate geographic information systems, displaying the real-time status of monitoring points on electronic maps, facilitating managers' understanding of the overall situation.

[0006] For example, Chinese invention patent CN119181197B discloses a fire early warning method and system based on joint feature analysis of smoke images, including: collecting smoke concentration distribution data and infrared thermal images of the indoor environment based on smoke sensor arrays and infrared sensor arrays, and collecting visible light images of the indoor environment based on cameras; extracting spatiotemporal features of smoke concentration distribution using 3D convolutional neural networks, extracting texture and shape features of visible light images and temperature distribution features of infrared thermal images using 2D convolutional neural networks; weighted fusing the three features to generate a joint feature representation; representing the fused joint features as a time-order-based feature sequence, and obtaining the fire risk level of the current time window based on a trained multilayer perceptron classification model.

[0007] For example, Chinese invention patent application CN114419835A discloses a method and system for monitoring fires in historical buildings based on digital twin technology. The method includes: receiving building data to create a historical building model and visualizing the model; receiving real-time video data from the monitored historical building to identify fires in the video data, and visualizing the ignition point information obtained from the fire identification in the historical building model; performing impact analysis on the simulated fire situation in the historical building model to obtain the fire impact range, determining the optimal fire route based on the fire impact range, and pushing fire messages to the corresponding rescue and resettlement points.

[0008] The existing technology has at least the following technical problems: In the field of ancient building preservation, there are various permitted or customary fire-use behaviors, such as cooking for warmth in ancient villages and celebrations during specific festivals. The smoke, high temperatures, or light radiation generated by these behaviors are physically very similar to the initial signals of a fire, constituting typical non-fire interference sources. Current technologies lack the ability to universally identify and effectively distinguish between these diverse fire-use behaviors and dangerous fire-use behaviors.

[0009] In the existing process of fire behavior recognition, when data from different modalities (e.g., visible light images, infrared thermal imaging, smoke sensor data, etc.) are processed simultaneously, there is complementary information between different modalities that can provide unique discrimination information (e.g., infrared thermal imaging can penetrate smoke to capture heat source features, making up for the defect of visible light being ineffective under smoke), as well as a large amount of repetitive or similar redundant information (e.g., the same object captured by two visible light cameras in adjacent locations).

[0010] Furthermore, existing fire early warning systems primarily focus on detecting and alarming areas already on fire, with the warning range limited to the physical space covered by sensors or the field of view of cameras. However, the spread of fire is a dynamic process influenced by multiple factors, particularly meteorological conditions (such as wind direction, wind speed, air humidity, and temperature gradients), which have a decisive impact on the propagation path and speed of fire between different areas. This leads to the problem of poor effectiveness of fire early warning systems for ancient building complexes due to the identification of abnormal fire-related behaviors. Summary of the Invention

[0011] This invention provides a method and system for fire early warning of ancient building complexes based on an internet platform, which can improve the accuracy of fire identification and thus enhance the effectiveness of fire early warning for ancient building complexes. The technical solution provided by this application is as follows: According to a first aspect of this application, a method for fire early warning of ancient building complexes based on an internet platform is provided. The method includes: S1, acquiring a visible light image of the current monitoring grid within the ancient building complex; using a preset target detection network to detect smoke regions in the visible light image and generating a confidence score for smoke detection in the visible light image; determining the scene type based on the generated confidence score; if the scene type is an interference scene, triggering a registration operation on the original infrared image; otherwise, directly issuing a fire early warning based on the infrared image; S2, in response to an instruction to trigger a registration operation on the original infrared image, acquiring the original infrared image of the current monitoring grid within the ancient building complex; using the visible light image as a reference, performing a registration operation on the original infrared image to generate a registered infrared image; and using a feature extractor to extract the infrared feature map and... The visible light feature map of the visible light image is used to generate an attention weight map based on the infrared and visible light feature maps. A preliminary fusion feature map is then generated by combining the infrared and visible light feature maps. The feature extractor is characterized as a feature extractor composed of a two-stream convolutional neural network. S3: The preliminary fusion feature map is dimensionality-reduced and compressed to obtain a low-dimensional feature vector. This low-dimensional feature vector is input into a classifier to obtain the fire occurrence probability of the current monitoring grid. Based on the fire occurrence probability, an early warning scheme is selected. The early warning scheme includes at least triggering a first-level alarm signal and triggering a second-level alarm signal. S4: In response to the command to trigger a first-level alarm signal, the current monitoring grid is marked as a risk source grid. The directional propagation coefficient of adjacent target grids relative to the risk source grid is calculated. The fire occurrence probability of adjacent target grids is updated based on the directional propagation coefficient, and the early warning scheme is adjusted.

[0012] According to another aspect of this application, a fire early warning system for ancient building complexes based on an internet platform is provided. This system includes: a smoke detection module for acquiring visible light images of the current monitoring grid within the ancient building complex; using a preset target detection network to detect smoke regions in the visible light images and generate a confidence score for smoke detection in the visible light images; determining the scene type based on the generated confidence score; if the scene type is an interference scene, triggering a registration operation on the original infrared image; otherwise, directly issuing a fire early warning based on the infrared image; and an image registration module for responding to an instruction to trigger a registration operation on the original infrared image, acquiring the original infrared image of the current monitoring grid within the ancient building complex, performing a registration operation on the original infrared image using the visible light image as a reference, generating a registered infrared image, and using a feature extractor to extract the infrared feature map and visible light feature map of the registered infrared image, respectively. The visible light feature map of the light image is used to generate an attention weight map based on the infrared and visible light feature maps. This is then combined with the infrared and visible light feature maps to generate preliminary fused features. The feature extractor is represented as a feature extractor composed of a two-stream convolutional neural network. The fire early warning module performs dimensionality reduction and compression on the preliminary fused features to obtain a low-dimensional feature vector. This low-dimensional feature vector is input into a classifier to obtain the fire occurrence probability of the current monitoring grid. Based on the fire occurrence probability, an early warning scheme is selected, which includes at least triggering a level one alarm signal and triggering a level two alarm signal. The ancient building complex early warning linkage module responds to the command to trigger a level one alarm signal by marking the current monitoring grid as a risk source grid, calculating the directional propagation coefficient of adjacent target grids relative to the risk source grid, updating the fire occurrence probability of adjacent target grids based on the directional propagation coefficient, and adjusting the early warning scheme.

[0013] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: 1. By rapidly detecting smoke areas and assessing their reliability through a target detection network, the existence of potential fire hazards can be preliminarily determined. By distinguishing between interfering and non-interfering scenarios, false alarms caused by direct infrared image-based early warnings under interfering conditions are avoided, improving the accuracy of early warnings. Then, after triggering the registration process, the infrared and visible light images are spatially aligned, providing a foundation for feature fusion. Through a dual-stream convolutional neural network, the unique features of both infrared and visible light images can be fully extracted. The preliminary fusion features combine the advantages of both infrared and visible light images, providing richer feature basis for subsequent accurate assessment of fire probability. Next, the preliminary fusion features are subjected to dimensionality reduction and compression to reduce the amount of data. The classifier accurately determines the fire probability based on the low-dimensional feature vectors. Appropriate early warning schemes are selected according to different probabilities, enabling timely and effective responses to different levels of fire risk. Finally, the risk source grid is marked to clarify the possible source location of the fire. The directional propagation coefficient takes into account the directional factors of fire propagation, making the fire probability updates of adjacent grids more consistent with the actual situation. Adjusting the early warning scheme can take more appropriate countermeasures in a timely manner according to changes in fire propagation risk, improving the effectiveness of fire early warning and prevention.

[0014] 2. The original infrared image is processed by a preset gradient operator to generate a gradient magnitude map, which highlights areas with drastic temperature changes in the infrared image. The dynamic segmentation threshold can be adaptively determined according to the actual situation of the gradient magnitude map, avoiding the inaccurate segmentation problem that may occur in different images due to a fixed threshold, thus improving the accuracy of edge detection. Then, the gradient magnitude map is binarized according to the dynamic segmentation threshold to generate an edge mask. The edge mask clearly separates the edge and non-edge regions in the image. Next, in the edge regions, an edge enhancement coefficient is introduced to highlight the features of the edge regions, so that the features of the edge regions can be better identified and matched in subsequent image matching, improving the accuracy and stability of feature matching. In the non-edge regions, a contrast threshold is set to filter out pixels with stronger features, avoiding the extraction of irrelevant features and improving the efficiency and accuracy of feature extraction. Finally, through feature matching and homography matrix calculation, accurate registration of the infrared image and the visible light image is achieved, providing an accurate image foundation for subsequent applications such as fire early warning.

[0015] 3. By introducing a directional propagation coefficient to correct the basic fire probability, the likelihood of fire propagating to adjacent target grids after considering wind direction and distance factors can be more accurately reflected. This makes the assessment of fire occurrence probability more in line with reality. Then, the updated fire occurrence probability is compared with a second fire occurrence probability threshold, and timely warnings are issued to target grids where fires may occur, reducing the losses and impacts of fires. Finally, existing technologies only consider local factors of fire occurrence, such as the flammability of the grid itself, or simply consider the impact of distance factors on fire propagation. This invention, by weighting and fusing wind direction consistency factors and distance attenuation factors, can more comprehensively and accurately reflect the characteristics and probability of fire propagation. The wind direction factor makes the assessment of fire propagation direction more scientific and reasonable, and the distance factor considers the attenuation law of fire propagation intensity with distance. By combining the two key factors of wind direction and distance, the propagation of fire between different grids can be predicted more accurately, providing more reliable fire early warning information.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0017] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein: Figure 1 This is a flowchart of a fire early warning method for ancient building complexes based on an Internet platform, provided by an embodiment of the present invention; Figure 2 This is a flowchart for determining the scene type provided in an embodiment of the present invention; Figure 3 This is a flowchart of the image registration operation provided in an embodiment of the present invention; Figure 4 This is a flowchart of attention weight adjustment provided in an embodiment of the present invention. Detailed Implementation

[0018] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0019] This invention provides a fire early warning method for ancient building complexes based on an internet platform. This method is primarily applied to complex environments with numerous sources of interference and extremely high protection requirements, such as national or provincial key cultural relics protection units (e.g., wooden ancient building complexes) and religious sites (e.g., temples and Taoist temples). By constructing an intelligent fire early warning platform based on the Internet of Things, it can not only intelligently identify and eliminate environmental interference such as incense burning and water vapor diffusion, but also has the ability to predict fire spread, thereby achieving efficient fire early warning for ancient building complexes through the internet platform. Figure 1 The flowchart shown is a method for early warning of fires in ancient building complexes based on an internet platform. The processing flow of this method may include the following steps: S1. Acquire visible light images of the current monitoring grid within the ancient building complex. Utilize a pre-defined target detection network to detect smoke areas in the visible light images and generate a confidence score for smoke detection. Based on the generated confidence score, determine the scene type. If the scene type is an interference scene, trigger registration operations on the original infrared image; otherwise, directly issue a fire warning based on the infrared image. The monitoring grid represents dividing the ancient building complex area into individual grid units, facilitating refined monitoring and management of different areas. The target detection network represents a deep learning model used to automatically detect specific targets in the image, such as the smoke area model in this application. Through learning from a large amount of labeled data, it can identify whether a target exists in the image and the target's location. The confidence score indicates the degree of confidence the target detection network has in identifying that the detected smoke area belongs to a real target; a higher confidence score indicates a more reliable detection result. An interference scene represents a scenario where the presence of smoke in the image may lead to misjudgments by the target detection network. By quickly detecting smoke areas and assessing their confidence through the target detection network, a preliminary judgment can be made regarding the existence of fire hazards. Distinguishing between interference and non-interference scenes avoids misjudgments caused by directly issuing warnings based on infrared images under interference conditions, thus improving the accuracy of warnings. The registration process is triggered to prepare for more accurate early warning using various image information.

[0020] The training process of the object detection network is as follows: First, visible light images of the ancient building complex under different environments are collected, and the category and bounding box position of the smoke target are labeled to construct a training dataset containing positive and negative samples, such as real fire smoke images, smoke from burning incense, water vapor, and dust clouds. Next, data augmentation (such as rotation, flipping, brightness adjustment, and Mosaic stitching) is performed on the visible light images, and they are input into a pre-set deep convolutional neural network (such as YOLO or Faster R-CNN). The object detection network extracts image features and outputs the confidence score and bounding box coordinates of the smoke. By calculating the loss function between the predicted value and the true label (including confidence loss, localization loss, and classification loss), the network weights are iteratively updated using the gradient descent algorithm until the model converges. Finally, the model performance is evaluated on the test set, and the optimal smoke detection threshold is determined based on precision and recall to obtain the final object detection network.

[0021] like Figure 2 The flowchart shown illustrates the process of determining the scene type. Further, based on the generated confidence score, the scene type is determined, and the specific process is as follows: The confidence score of smoke detection in visible light images is compared with a preset smoke detection threshold. The smoke detection threshold is a pre-set numerical limit used to determine whether smoke interference may exist. This allows for a quick preliminary assessment of whether smoke interference exists in the current scene, which may affect fire detection. This provides a basis for decision-making in subsequent processing procedures and improves the accuracy and efficiency of fire early warning.

[0022] If the confidence score of smoke detection in the visible light image is greater than the preset smoke detection threshold, the current scene is determined to be an interfering scene, triggering a registration operation on the original infrared image. Image registration aligns the visible light image and the infrared image to comprehensively utilize the information from both images, more accurately determining whether a fire exists and avoiding misjudgments due to interference from a single image. Otherwise, the current scene is determined to be a non-interfering scene, and fire warning is directly based on the infrared image, reducing computation and processing time and improving the real-time performance of fire warnings.

[0023] Fire early warning based on infrared images, specifically: Infrared feature maps are extracted from the original infrared image using a feature extractor. Based on these infrared feature maps, statistical and texture features are extracted to construct an infrared feature vector. Statistical features represent features obtained through statistical analysis of various numerical characteristics of the target region in the infrared image, including at least the highest temperature value and the average temperature value. The highest temperature value reflects the temperature of the hottest point in the target region (i.e., the image area covered by the monitoring grid currently being analyzed and processed), while the average temperature value reflects the overall temperature level of the target region. Texture features represent features describing the surface texture structure of the target region in the infrared image, including at least the local binary pattern histogram and area of ​​the infrared image. The local binary pattern histogram describes the texture by statistically analyzing the binary pattern distribution of a local region (i.e., a circular or square neighborhood within a fixed radius, such as 3 pixels, centered on each pixel in the target region). The infrared feature map represents an image extracted from the original infrared image that reflects the characteristic information of the flame in the image. The infrared feature vector represents a vector formed by combining the extracted statistical and texture features, used to represent the comprehensive feature information of the target in the infrared image, so that it can be input into a classifier for classification. Infrared feature maps can focus on key information in an image. The extracted statistical and texture features describe the characteristics of the target in the infrared image from different perspectives. The constructed infrared feature vector comprehensively and accurately reflects the characteristic information of the target, providing strong support for subsequent accurate determination of whether a flame exists.

[0024] Infrared feature vectors are input into an infrared classifier, which outputs flame probability. Flame probability represents the probability of a flame being present in the current infrared image. After training, the infrared classifier can analyze and classify the input infrared feature vectors and output the probability of a flame being present in the current infrared image, providing a quantitative basis for fire detection and improving the accuracy and reliability of fire detection.

[0025] Feature extractors refer to convolutional neural networks, such as ResNet-50 or MobileNetV2 backbone networks. These networks consist of a series of convolutional layers, activation functions (such as ReLU), and pooling layers. In the convolutional layers, multiple convolutional kernels perform sliding window operations on the input image region to extract basic features such as edges and textures at different scales. The activation function introduces non-linearity, and pooling operations (such as max pooling) reduce the size of the feature map, enhancing the translation invariance of the features while preserving the main feature information. After processing by the above multi-layer network, a multi-dimensional tensor is finally output. If the input is an infrared image, the output is the infrared feature map corresponding to the monitoring grid. This infrared feature map is a three-dimensional matrix (containing width, height, and number of channels), and the feature vector at each location (corresponding to a spatial location in the original image) encodes the temperature radiation intensity at that location and the surrounding texture, structure, and other complex information. If the input is a visible light image, the output is a visible light feature map, which is also a three-dimensional data structure, and its spatial location corresponds to the infrared feature map. Figure 1 There is a one-to-one correspondence, but its channel content mainly includes the scene's texture structure, spectral color, and visible light semantic information.

[0026] The infrared classifier employs a Support Vector Machine (SVM). The specific training and workflow are as follows: Infrared images of the ancient building complex are acquired, and the flame and background areas are manually labeled. The images are preprocessed to extract statistical and texture features, constructing infrared feature vectors and their corresponding ground truth labels (0 or 1). These feature vectors are then input into the infrared classifier, and a radial basis function (RBF) kernel is used to map the samples to a high-dimensional feature space. The optimal hyperplane weight parameters are obtained by solving a quadratic programming problem that maximizes the classification margin. Grid search and cross-validation are used to determine the optimal penalty coefficient and kernel function parameters to optimize model performance. Finally, the standard SVM outputs a class determination. This scheme uses the Platt scaling method, employing a Sigmoid function to map the SVM's decision output to a value between 0 and 1, representing the flame probability. The performance of the SVM is evaluated on a test set to determine the optimal segmentation threshold as the infrared alarm threshold.

[0027] If the calculated probability of a fire is greater than the preset infrared alarm threshold, a fire is directly determined to have occurred in the current monitoring grid, triggering a Level 1 alarm signal. Otherwise, it is determined that no fire has occurred in the current monitoring grid, and monitoring continues. The infrared alarm threshold is a preset numerical limit used to determine whether a fire has occurred. By setting the threshold and comparing it, it is possible to clearly determine whether a fire has occurred, trigger an alarm signal in a timely manner so that appropriate measures can be taken. At the same time, monitoring continues even when no fire has occurred, ensuring continuous and effective monitoring of the ancient building complex and protecting the safety of the buildings.

[0028] S2, in response to the instruction to perform a registration operation on the original infrared image, acquires the original infrared image of the current monitoring grid in the ancient building complex. Using the visible light image as a reference, it performs a registration operation on the original infrared image to generate a registered infrared image. A feature extractor is used to extract the infrared feature map of the registered infrared image and the visible light feature map of the visible light image, respectively. An attention weight map is generated based on the infrared and visible light feature maps, and preliminary fusion features are generated by combining the infrared and visible light feature maps. The feature extractor is characterized as a feature extractor composed of a two-stream convolutional neural network, capable of extracting features from the infrared and visible light images respectively. The registration operation involves spatially aligning the images acquired by the visible light and infrared cameras, making the same object in the two images correspond in spatial position, so that subsequent feature fusion operations can be performed. The attention weight map is used to represent the importance of features at different positions to the final fusion result during feature fusion. It is generated through an attention mechanism, enabling the two-stream convolutional neural network to pay more attention to important feature information. The preliminary fusion features represent the features obtained by combining the infrared and visible light feature maps. Registration aligns infrared and visible light images spatially, providing a foundation for feature fusion. Visible light images offer rich scene detail, while infrared images are sensitive to heat and can detect early, often undetectable, fire hazards. Combining the two allows for more accurate detection of smoke areas, reducing false positives and false negatives, and improving the accuracy and reliability of fire warnings. Generating an attention weight map further enhances the quality of the fused features, focusing on those crucial for fire detection, thereby improving the accuracy of fire probability assessment and ultimately enhancing the accuracy of fire warnings for ancient building complexes.

[0029] like Figure 3 The flowchart shown illustrates the image registration operation. Further, the registration operation is performed on the original infrared image, specifically including: Using a preset gradient operator, such as the Sobel operator, the gradient components of the original infrared image in the horizontal and vertical directions are calculated. The gradient magnitude is obtained by performing square and square root operations on the gradient components in the horizontal and vertical directions, generating a gradient magnitude map to reflect the temperature change region. The gradient component table of the original infrared image in the horizontal and vertical directions, that is, the rate of change of the original infrared image in the horizontal and vertical directions, reflects the brightness or temperature change of the image in the horizontal and vertical directions, respectively. The gradient magnitude map represents an image composed of pixel values ​​with gradient magnitude, which is used to intuitively reflect the distribution of temperature change regions in the image.

[0030] The mean and standard deviation of the gradient magnitude map are calculated. Based on the mean and standard deviation of the gradient magnitude map and the preset adjustment coefficient, a dynamic segmentation threshold is obtained. This threshold can automatically adjust the sensitivity to edges according to the statistical characteristics of the current image, thereby ensuring the accuracy and stability of edge mask generation. The dynamic segmentation threshold is the result of summing the product of the preset adjustment coefficient and the standard deviation of the gradient magnitude map, and the mean of the gradient magnitude map. The preset adjustment coefficient is set in advance by the preset personnel based on practical experience.

[0031] The gradient magnitude map is binarized according to a dynamic segmentation threshold to generate an edge mask. Binarization involves dividing the pixel values ​​in the gradient magnitude map into two values, typically 0 and 1, based on the dynamic segmentation threshold, transforming the gradient magnitude map into a black-and-white binary image. Regions with a pixel value of 1 in the edge mask are marked as edge regions. A preset edge enhancement coefficient is introduced, and the feature response values ​​of pixels within the edge regions are multiplied by this coefficient. Infrared feature points and their corresponding infrared feature descriptors are then re-extracted based on the weighted feature response values ​​within a preset scale space. The preset edge enhancement coefficient represents a pre-defined coefficient designed to enhance the features of edge regions. The Harris corner detection algorithm can be used to obtain the feature response values. The preset scale space represents a series of image spaces at different scales, used to extract image features at different scales. Infrared feature points and their corresponding infrared feature descriptors represent representative points extracted from the infrared image and vectors describing the features of these points, used for image matching. A scale-invariant feature transform algorithm can be used to obtain the infrared feature descriptors. By marking edge regions with edge masks and introducing preset edge enhancement coefficients to weight the feature response values, the response intensity of edge structures in infrared images is significantly enhanced, enabling the originally weak edge features to be significantly extracted, thereby increasing the number of feature points available for matching.

[0032] The region with a pixel value of 0 in the edge mask is marked as a non-edge region. The feature response value of the pixels in the non-edge region is calculated, and the pixels with the feature response value greater than the preset contrast threshold are selected and the corresponding infrared feature descriptor is generated. The preset contrast threshold is a pre-set threshold used to select pixels in the non-edge region. This avoids extracting unstable noise as features in the non-edge region (such as a smooth wall or sky) and causing mismatch.

[0033] The extracted infrared feature descriptors are matched with the feature descriptors of the visible light image. A random consistency algorithm is used to eliminate mismatched point pairs. From the retained matching point pairs, a predetermined number of coordinate points (e.g., 4 sets) are selected. Based on these coordinate points, a system of linear equations is constructed. The unknowns of the equations are the 8 degrees of freedom parameters in the homography matrix. The 9th parameter is usually normalized to 1. The system of equations is solved using the least squares method to obtain the specific value of the homography matrix. The homography matrix is ​​calculated, and the original infrared image is transformed based on the homography matrix to achieve image registration. The random consistency algorithm is an iterative algorithm used to estimate mathematical model parameters from a dataset containing a large number of outliers, often used to eliminate mismatched point pairs in image matching. The homography matrix describes the projection transformation relationship between two planes, used to map points on one plane to another. This effectively filters out erroneous matches caused by sudden changes in illumination, heat source interference, or pedestrian movement. The final infrared image transformation can be precisely aligned with the visible light image at the pixel level, providing an accurate spatial reference for subsequent multimodal image fusion (such as superimposing visible light smoke texture using infrared temperature measurement information), and ensuring the accuracy of the fire early warning system's judgment.

[0034] Specifically, mismatch phenomena include at least: semantically different matching errors, structurally similar but geographically different matching errors, and matching errors caused by repeated texture interference. Semantically different matching errors refer to matching point pairs that are inconsistent in physical semantics. Specifically, feature points detected in infrared images correspond to heat source targets in the scene, such as light bulbs or exhaust pipes, which emit high-temperature radiation, while the matched feature points in visible light images correspond to reflective areas in the background, such as glass reflections or metallic luster. Structurally similar but geographically different matching errors refer to the presence of multiple geometrically similar targets in the scene, making it difficult for the algorithm to distinguish their specific spatial locations. Specifically, the scene contains target objects with similar shapes, sizes, and heights, such as two... For windows with identical fan structures, due to the minimal difference in the numerical values ​​of their feature descriptors, the algorithm incorrectly pairs a feature point located on the left side of the infrared image with a feature point located on the right side of the visible light image, resulting in a mismatch in spatial location. Matching errors caused by repetitive texture interference refer to the difficulty in distinguishing local descriptions of feature points due to the high repetition of background textures. Specifically, in areas where the textures are regularly arranged or have large areas of similarity, such as grid walls, grass, or brick walls, the feature descriptors are very close in numerical value. In this case, the matching algorithm is prone to confusing adjacent feature points, incorrectly matching a feature point in the infrared image to its adjacent or neighboring position in the visible light image, rather than its corresponding true position.

[0035] Furthermore, the infrared feature map and the visible light feature map are used to generate an attention weight map, specifically including: The infrared feature map is mapped to a query vector. The infrared feature map represents thermal radiation information, including at least the specific temperature value of each point and the temperature change rate between adjacent pixels. The query vector is used to query relevant information in the visible light feature map. The information of the infrared feature map is transformed into a vector form suitable for attention calculation, which prepares for subsequent information interaction with the visible light feature map. Key information in the infrared feature map is extracted for querying relevant parts in the visible light feature map.

[0036] The visible light feature map is mapped into a key vector and a value vector. The visible light feature map represents the spectral response characteristics of the flame and the texture structure characteristics of the smoke. The key vector is used to calculate the similarity with the query vector to determine the attention weight. The value vector carries the specific information of the visible light feature map, which will be weighted and summed according to the attention weight, so that the information of the visible light feature map can participate in the calculation of the attention mechanism.

[0037] The infrared feature map is mapped to a query vector, and the visible light feature map is mapped to key and value vectors. Specifically, a linear transformation is used to convert the high-dimensional convolutional feature tensor into a specific vector space representation required for the attention mechanism's computation. Independent convolutional kernels (typically 1×1 convolutions) or fully connected layer parameter matrices are configured for the infrared and visible light feature maps, denoted as the weight matrices corresponding to the query vector, key vector, and value vector, respectively. A linear projection operation is performed on the infrared feature map using the weight matrix corresponding to the query vector. This operation maps the original feature channels to the query subspace, generating the query vector. Then, linear projection operations are performed on the visible light feature map using the weight matrices corresponding to the key and value vectors, respectively, to obtain the key and value vectors.

[0038] The similarity between the query vector and the key vector is calculated by dot product operation. The calculated similarity results are scaled and normalized (such as by Softmax operation) to generate an initial attention weight map, which initially reflects the correspondence and attention level between the infrared feature map and the visible light feature map, providing a foundation for subsequent optimization and feature fusion.

[0039] The initial attention weight map is optimized using redundancy removal and complementarity extraction mechanisms to obtain an attention weight map that more accurately reflects the relationship between infrared and visible light features. The redundancy removal mechanism reduces the weights of regions in the infrared feature map below a preset temperature threshold and regions in the visible light feature map below a preset feature variance threshold in the attention weight map. Specifically, it calculates the mean response value of the infrared feature map in the spatial dimension to obtain an infrared thermal saliency map. For locations below the preset temperature threshold (i.e., the room temperature region), the mask value is set to 0, while the mask value for other regions is set to 1. Simultaneously, the local variance of the visible light feature map at each location is calculated to obtain the texture complexity. For regions below the preset feature variance threshold, the mask value is set to 0, while the mask value for other regions is set to... The result is set to 1. Finally, the infrared saliency mask and the visible light texture complexity mask are weighted and summed. The result is then input into the Sigmoid activation function to obtain the redundancy removal mask. The redundancy removal mask is multiplied by the initial attention weight map to remove redundancy and suppress the redundant interference of background noise, thereby improving the accuracy and effectiveness of feature fusion. The preset temperature threshold and the preset feature variance threshold are pre-set numerical limits. The complementary mechanism is used to increase the weight of regions in the visible light feature map that contain feature information lacking in the infrared feature map in the attention weight map. The feature information lacking in the infrared feature map includes the texture structure features of smoke or the spectral response features of flames. That is, the pixel feature descriptions of the infrared feature map and the visible light feature map are compared position by position. For each spatial location, the degree of difference between the feature descriptions of the two is calculated to obtain a feature difference map. The feature difference map is then normalized using the Sigmoid activation function to obtain complementary enhancement weights. When the complementary enhancement weight is greater than the preset complementary enhancement weight (pre-set by the personnel based on practical experience), it indicates that the descriptions of the infrared feature map and the visible light feature map are significantly different, and it is determined that the location contains complementary information. For example, the infrared feature map shows a uniform high-temperature hot spot, while the visible light feature map shows complex granular textures or specific colors at the same location, indicating that the visible light image contains smoke texture details or flame color features that are missing in the infrared image. Finally, the complementary enhancement weights are applied to the attention weight map after redundancy removal to obtain the final attention weight map. By increasing the weight of the complementary feature regions, the expressive power of the fused features for flame and smoke features is enhanced.

[0040] The value vector of the visible light feature map is weighted and summed using an attention weight map. This involves multiplying the weight of each position in the attention weight map by the corresponding value in the value vector of the visible light feature map, and then summing all the results to obtain the weighted visible light feature. The value vector of the visible light feature map is selectively fused based on the attention weight map, highlighting important features related to the infrared feature map and weakening irrelevant or redundant parts, making the weighted visible light feature more in line with the fusion requirements.

[0041] The weighted visible light features and infrared feature maps are subjected to residual concatenation and layer normalization to generate preliminary fused features. Residual concatenation means directly adding the weighted visible light features and infrared feature maps to prevent information loss during fusion and accelerate network training convergence. Layer normalization means normalizing the features after residual concatenation, such as calculating the mean and variance of the features and then standardizing them. This further ensures the stability and effectiveness of the fused features, enabling the preliminary fused features to more comprehensively and accurately represent the characteristics of flames and smoke, providing stronger feature support for subsequent flame and smoke detection and recognition tasks.

[0042] S3. The initial fused features are reduced in dimension and compressed to obtain a low-dimensional feature vector. The low-dimensional feature vector is then input into the classifier to obtain the fire occurrence probability of the current monitoring grid. Based on the fire occurrence probability, an early warning scheme is selected to respond to different levels of fire risk in a timely and effective manner. The early warning scheme includes at least triggering a first-level alarm signal and triggering a second-level alarm signal. Dimension reduction and compression means reducing the dimension of the feature vector and reducing the amount of data to improve the efficiency and accuracy of subsequent classification and other operations.

[0043] Furthermore, early warning schemes are selected based on the probability of fire occurrence, specifically including: If the probability of a fire occurring is greater than the first fire probability threshold, a Level 1 alarm signal is triggered. The fire probability threshold is a pre-set critical value used to determine the likelihood of a fire. The Level 1 alarm signal represents the highest level of alarm, indicating an extremely high fire risk. If the probability of a fire occurring is between the second and first fire probability thresholds, a Level 2 warning signal is triggered. The Level 2 warning signal represents the second highest level of warning, indicating a high fire risk. If the probability of a fire occurring is less than the second fire probability threshold, no alarm signal is triggered. The first fire probability threshold is greater than the second fire probability threshold. By setting different levels of alarm signals, corresponding countermeasures can be taken according to different degrees of fire risk, improving the accuracy and effectiveness of fire early warning.

[0044] The probability of a fire occurring is obtained using the following method: The low-dimensional feature vector is input into a pre-defined classifier to obtain the fire occurrence probability of the current monitoring grid. The classifier is used to predict the fire occurrence probability based on the input feature vector and includes a fully connected layer and an activation function. The fully connected layer maps the low-dimensional feature vector to a fire risk score, and the activation function normalizes the fire risk score, outputting the fire occurrence probability of the current monitoring grid. The low-dimensional feature vector is a feature representation after feature extraction and dimensionality reduction, which is used to input the classifier to predict the fire occurrence probability. By predicting the low-dimensional feature vector through the classifier, the probability of fire occurrence is obtained, providing a basis for subsequent early warning decisions.

[0045] The classifier employs a multilayer perceptron architecture, specifically including an input layer, hidden layers, activation functions, and an output layer. The input layer is configured to receive low-dimensional feature vectors obtained through reparameterized sampling. The hidden layer contains at least two fully connected layers. The first fully connected layer maps the input 128-dimensional feature vector into a high-dimensional feature space, such as 256 dimensions. The second fully connected layer performs dimensionality reduction processing on the high-dimensional features, for example, compressing them to 64 dimensions. The output layer has a fully connected node used to map the 64-dimensional features output by the hidden layer into a real scalar, which is the fire risk score. The output layer is followed by a sigmoid activation function, which normalizes the fire risk score, mapping its numerical range to between 0 and 1, thereby directly obtaining the probability of fire occurrence in the current monitoring grid.

[0046] The classifier training process is as follows: Visible light and infrared image pairs from the ancient building complex monitoring scene are collected as raw samples. Using the image registration, feature fusion, and reparameterized sampling methods described in Example 1, each raw sample is processed to extract the corresponding low-dimensional feature vector. Simultaneously, manual labeling is performed based on the actual content of the raw samples: if the image contains open flames or smoldering fire, the ground truth label is "1," indicating a fire; if it is a normal background or interference source (such as high-temperature equipment or sunlight reflection), the ground truth label is "0," indicating no fire. All samples are divided into training, validation, and test sets according to a preset ratio, such as 8:1:1. The low-dimensional feature vectors from the training set are then input into the classifier. The data passes through the input layer and enters the hidden layer. After weighted summation, bias adjustment, and nonlinear transformation, the output layer generates the predicted fire probability. A binary cross-entropy loss function is used as the optimization objective to calculate the difference between the predicted probability and the manually labeled ground truth. Finally, the Adam algorithm is used to calculate the gradient of each parameter in the model layer by layer from the output layer to the input layer based on the calculated loss value. The weight parameters and bias parameters in the model are corrected and updated according to the calculated gradient. In the early stage of training, a larger learning rate is set to speed up the convergence speed. As the number of training rounds increases, the learning rate is gradually reduced to ensure that the model can stably converge to the optimal solution in the later stage of training.

[0047] The low-dimensional feature vector is obtained as follows: The initial fused features are input into the bottleneck fully connected layer, which outputs a mean vector and a variance vector. The bottleneck fully connected layer encodes the initial fused features and outputs statistical parameters representing the feature distribution. The bottleneck fully connected layer is configured to linearly transform and compress the dimension of the input initial fused features from the first dimension to the second dimension, which is smaller than the first dimension, in order to extract the fire features from the initial fused features. The bottleneck fully connected layer encodes the initial fused features, extracts the fire features, and reduces the feature dimension, while outputting statistical parameters representing the feature distribution, namely the mean vector and the variance vector. The bottleneck fully connected layer is the preprocessing module of the classifier.

[0048] By employing a reparameterized sampling method, sampling is performed based on the mean and variance vectors to obtain a compressed low-dimensional feature vector, providing a suitable feature representation for subsequent classifier input. Specifically, a noise factor is sampled from a standard normal distribution, and the compressed feature vector is calculated by combining the mean and variance. This operation transforms the determined distribution parameters into a specific feature representation, thereby obtaining a low-dimensional feature vector that meets the requirements of the classifier input.

[0049] S4, in response to the command to trigger a Level 1 alarm signal, marks the current monitoring grid as a risk source grid, calculates the directional propagation coefficient of adjacent target grids relative to the risk source grid, updates the fire occurrence probability of adjacent target grids based on the directional propagation coefficient, and adjusts the early warning scheme. The directional propagation coefficient is used to measure the probability and extent of fire propagation from the risk source grid to adjacent grids, taking into account the influence of factors such as wind direction on fire propagation. After triggering a Level 1 alarm signal, calculating the directional propagation coefficient of adjacent grids and updating the fire occurrence probability enables a more comprehensive assessment of fire risk, allowing for early warning measures to be taken for adjacent grids that may be affected by fire, effectively preventing the spread of fire, and improving the fire prevention and control capabilities of the entire ancient building complex.

[0050] Furthermore, the fire occurrence probability of adjacent target grids is updated based on the directional propagation coefficient, specifically including: The fire occurrence probability of adjacent target grids is obtained and recorded as the base fire probability. A directional propagation coefficient is introduced to correct the base fire probability of the target grids. That is, the directional propagation coefficient is multiplied by the base fire probability to update the fire occurrence probability of adjacent target grids. Adjacent target grids refer to the grid areas that are spatially adjacent to the risk source grids in the fire propagation model and may be affected by fire propagation. The base fire probability represents the original fire occurrence probability of adjacent target grids. By introducing the directional propagation coefficient to correct the base fire probability, the possibility of fire propagating to adjacent target grids after considering wind direction and distance factors can be more accurately reflected, making the fire occurrence probability assessment more in line with the actual situation.

[0051] If the updated fire probability is not less than the preset second fire probability threshold, the target grid is determined to be in a fire warning state and an alarm message is generated; otherwise, the target grid is determined to be in a normal state and monitoring continues. The alarm message includes at least the specific geographical location of the target grid in the fire warning state, the fire probability, the location of the risk source grid, and the specific time the warning was triggered. Warnings are issued to target grids that may be prone to fire in a timely manner to provide relevant departments and personnel with a basis for decision-making so that corresponding preventive measures can be taken to reduce the losses and impacts of fire.

[0052] The directional propagation coefficient is obtained as follows: Obtain current wind direction and speed data through meteorological monitoring equipment or other data sources; wind direction and speed are important factors affecting the direction and speed of fire spread, providing key information for subsequent analysis of fire spread characteristics.

[0053] The cosine of the angle between the direction of the line connecting the risk source grid and the target grid and the current wind direction is calculated as the wind direction consistency factor. The wind direction consistency factor reflects the degree of fit between the fire propagation direction and the wind direction. The closer the cosine of the angle is to 1, the more consistent the two directions are, and the easier it is for the fire to spread along that direction, providing a quantitative indicator for assessing the directionality of fire propagation.

[0054] The distance between the risk source grid and the target grid, i.e., the straight-line length between the two points, is calculated using the Euclidean distance formula. A distance attenuation factor is then calculated using an exponential attenuation function based on a preset attenuation constant. This preset attenuation constant is set by experienced personnel. The distance attenuation factor reflects the law that the intensity of fire propagation decreases with increasing distance. The greater the distance, the smaller the attenuation factor, indicating a lower probability that the fire will propagate to the target grid, thus taking into account the impact of distance on fire propagation.

[0055] The directional propagation coefficient is obtained by weighted fusion of the wind direction consistency factor and the distance attenuation factor, which is obtained by multiplying the wind direction consistency factor and the distance attenuation factor. Through weighted fusion, the influence of two important factors, wind direction and distance, on fire propagation is comprehensively considered, so that the directional propagation coefficient can more comprehensively and accurately reflect the characteristics and probability of fire propagation.

[0056] This invention provides a fire early warning system for ancient building complexes based on an internet platform. The system includes: The smoke detection module acquires visible light images of the current monitoring grid within the ancient building complex. It uses a pre-defined target detection network to detect smoke areas in the visible light images and generates a confidence score for smoke detection. Based on this confidence score, the scene type is determined. If the scene type is an interference scene, a registration operation is triggered on the original infrared image; otherwise, a fire warning is issued directly based on the infrared image. The target detection network quickly detects smoke areas in the visible light images, and the confidence score is used to determine the scene type, avoiding misjudgments caused by interference scenes and improving the accuracy of smoke detection. The mechanism of triggering the registration process or issuing a direct warning allows for flexible selection of processing methods based on the actual situation, improving the efficiency of fire warnings for the ancient building complex.

[0057] The image registration module is used to respond to commands that trigger registration operations on the original infrared image. It acquires the original infrared image of the current monitoring grid in the ancient building complex, and performs registration operations on the original infrared image based on the visible light image to generate a registered infrared image. The feature extractor extracts the infrared feature map of the registered infrared image and the visible light feature map of the visible light image, respectively. An attention weight map is generated based on the infrared and visible light feature maps, and preliminary fusion features are generated by combining the infrared and visible light feature maps. The feature extractor is characterized as a feature extractor composed of a two-stream convolutional neural network. Image registration aligns the infrared and visible light images in space, providing a foundation for subsequent feature fusion. The infrared and visible light feature maps extracted by the feature extractor contain unique information from the two images. The attention weight map can highlight important feature locations, enabling the preliminary fusion features to more effectively combine the advantageous information of the two images and improve the ability to identify fire features.

[0058] The fire early warning module performs dimensionality reduction and compression on the preliminary fused features to obtain low-dimensional feature vectors. These low-dimensional feature vectors are then input into a classifier to obtain the fire occurrence probability of the current monitoring grid. Based on the fire occurrence probability, an early warning scheme is selected, which includes at least triggering a level 1 alarm signal and a level 2 alarm signal. Dimensionality reduction and compression reduce the amount of data and computational complexity. The classifier accurately determines the fire occurrence probability based on the low-dimensional feature vectors, providing a basis for selecting an appropriate early warning scheme. Different levels of alarm signals enable timely implementation of corresponding measures according to the degree of fire risk, improving the timeliness and effectiveness of fire early warning.

[0059] The ancient building complex early warning linkage module is used to respond to the command that triggers the first-level alarm signal, mark the current monitoring grid as the risk source grid, calculate the directional propagation coefficient of adjacent target grids relative to the risk source grid, update the fire occurrence probability of adjacent target grids based on the directional propagation coefficient, and adjust the early warning scheme; marking the risk source grid clarifies the possible location of the fire; the directional propagation coefficient takes into account the directional factor of fire propagation, making the update of the fire occurrence probability of adjacent target grids more accurate; adjusting the early warning scheme according to the updated probability can more comprehensively assess the fire risk of the ancient building complex, expand the early warning range in a timely manner, and take more effective prevention and control measures.

[0060] Example 2: Based on Example 1, further analysis is conducted on the incense interference scenario within the interference scenario. An attention weight map is generated based on infrared and visible light feature maps, followed by: like Figure 4 The flowchart shown illustrates the attention weight adjustment process. Only when the current scene is determined to be an interfering scene, infrared thermal radiation characteristics of the corresponding non-edge regions in the visible light image are acquired, and it is determined whether the infrared thermal radiation characteristics reach the flame temperature threshold. Infrared thermal radiation characteristics represent the property of an object to radiate infrared rays outward due to its own temperature. Objects at different temperatures have different infrared thermal radiation characteristics, which can be obtained by measuring relevant data through infrared sensors. By determining whether the scene is an interfering scene, unnecessary complex processing of non-interfering scenes is avoided. Acquiring and judging infrared thermal radiation characteristics separately for non-edge regions helps to accurately identify whether there is a real flame in that region and reduces misjudgments.

[0061] For regions where the infrared thermal radiation characteristics do not reach the flame temperature threshold, the correction factor corresponding to the texture structure features with smoke is set to the first preset attenuation coefficient to reduce the attention weight of the texture structure features with smoke, while maintaining the attention weight of the spectral response features of the flame. The flame temperature threshold represents a preset temperature value; the texture structure features represent the texture pattern and structural information of the object surface in the image; the spectral response features represent the object's response characteristics to different wavelengths of light, and flames have unique spectral response characteristics in the visible light band. Reducing the attention weight of the smoke texture structure features can reduce misjudgments caused by interference factors similar to smoke, highlight the importance of the flame spectral response features in subsequent analysis, and improve the accuracy of fire early warning.

[0062] For regions where the infrared thermal radiation characteristics reach the flame temperature threshold, the correction factor corresponding to the spectral response characteristics of the flame is set as the first preset enhancement coefficient to increase the attention weight of the spectral response characteristics of the flame, while maintaining the attention weight of the texture structure characteristics of the smoke. This enhances the attention weight of the spectral response characteristics of the flame, thereby focusing more on the flames that may exist in the region, improving the ability to identify real fires, while retaining the texture structure characteristics of the smoke, providing more basis for comprehensive judgment.

[0063] For edge regions in visible light images, a joint judgment is made by combining the infrared thermal radiation characteristics of the corresponding regions to determine whether the infrared thermal radiation characteristics reach the flame temperature threshold. Edge regions contain rich object shape and structural information. By combining the infrared thermal radiation characteristics for joint judgment, it is possible to more comprehensively and accurately determine whether there is a flame in the region, reducing the error caused by judgment based on a single feature.

[0064] For areas where the infrared thermal radiation features do not reach the flame temperature threshold, the correction factor corresponding to the texture structure features with smoke is set to the second preset attenuation coefficient to reduce the attention weight of the texture structure features with smoke while maintaining the attention weight of the spectral response features of the flame. Similar to non-edge areas, the attention weight of the smoke texture structure features in edge areas is reduced to reduce the influence of interference factors, highlight the spectral response features of the flame, and improve the accuracy of fire early warning.

[0065] For regions where the infrared thermal radiation characteristics reach the flame temperature threshold, the correction factor corresponding to the flame spectral response characteristics is set as the second preset enhancement coefficient to increase the attention weight of the flame spectral response characteristics while maintaining the attention weight of the smoke texture structure characteristics. Similar to non-edge regions, the attention weight of the flame spectral response characteristics in edge regions is enhanced, thereby more sensitively detecting flames in edge regions and improving the ability to identify real fires, while retaining smoke texture structure characteristics to assist in judgment.

[0066] The first preset attenuation coefficient, the second preset attenuation coefficient, the first preset enhancement coefficient, and the second preset enhancement coefficient represent pre-set specific values ​​used to adjust attention weights, and they increase sequentially to adjust the attention weights of different features under different conditions; the first preset attenuation coefficient and the second preset attenuation coefficient are less than 1, and the first preset enhancement coefficient and the second preset enhancement coefficient are greater than 1.

[0067] Example 3: Based on Example 1, Example 3 can be executed in both interference scenarios and normal scenarios to prevent water vapor interference to infrared images.

[0068] The registration operation is performed on the raw infrared image, which also includes the following steps: By using humidity sensors and other devices to measure and obtain the current ambient humidity value in real time, it can be determined whether the current ambient humidity value is greater than a preset humidity threshold. The preset humidity threshold is a humidity value set in advance. It can accurately determine whether there are water vapor factors in the current environment that interfere with infrared images, providing a basis for selecting image processing parameters according to different environmental conditions, ensuring that the image processing solution can adapt to different environmental conditions, and improving the accuracy of fire early warning.

[0069] If so, the environmental interference mode is determined to be water vapor interference mode. The grid size of the contrast-limited adaptive histogram equalization algorithm is set to the first preset grid size, and the histogram cropping limit is set to the first limiting range. The contrast-limited adaptive histogram equalization algorithm improves the contrast and visual effect of the image by dividing the image into multiple small regions and performing histogram equalization in each small region while limiting the contrast to avoid excessive noise enhancement. The grid size represents the size of the small regions into which the image is divided when using the contrast-limited adaptive histogram equalization algorithm. The histogram cropping limit represents the range of amplitude set when cropping the histogram to limit the contrast. Otherwise, the environmental interference mode is determined to be normal mode. The grid size of the contrast-limited adaptive histogram equalization algorithm is set to the second preset grid size, and the histogram cropping limit is set to the second limiting range.

[0070] Using a smaller first preset grid size allows the contrast-limited adaptive histogram equalization algorithm to process local image details more precisely. This is because water vapor interference causes complex local contrast variations, and finer processing helps to better enhance image features. A larger first limiting range prevents local noise caused by water vapor interference from being excessively amplified, effectively suppressing noise while enhancing image contrast. In normal mode, a larger second preset grid size improves the processing efficiency of the contrast-limited adaptive histogram equalization algorithm, while also achieving better enhancement effects for images with relatively uniform overall contrast. A smaller second limiting range allows for stricter control of contrast under normal conditions, avoiding over-enhancement that could lead to loss of detail or unnatural visual effects. This makes image processing more closely reflect real-world conditions and improves image quality.

[0071] The registered infrared image is subjected to contrast-limited adaptive histogram equalization using adjusted grid size and histogram clipping. Bilinear interpolation is then used to eliminate block artifacts. During image scaling, bilinear interpolation calculates the gray value of a new pixel based on the gray values ​​of surrounding pixels, thereby eliminating block artifacts caused by image segmentation and making the image smoother and more natural. Block artifacts refer to the discontinuous and unnatural visual effects that occur at the boundaries between blocks in image processing due to segmentation (such as segmentation enhancement or compression), manifesting as obvious boundary marks or texture abrupt changes.

[0072] The first preset grid size is smaller than the second preset grid size, and the first limiting range is larger than the second limiting range.

[0073] Example 4: Based on Example 1, an infrared feature map of the registered infrared image and a visible light feature map of the visible light image are extracted using a feature extractor, followed by: The registered infrared and visible light images are input into the feature extractor, which outputs infrared and visible light feature maps, respectively. The feature extractor transforms the original image data into more representative and discriminative feature maps. These feature maps contain key fire-related information in the images, such as the characteristics of high-temperature areas, the shape and texture of objects, etc., providing a more effective data foundation for subsequent fire early warning analysis.

[0074] Based on a pre-defined set of coordinates for high-temperature regions and background regions, a semantic consistency loss function is calculated between the infrared feature map and the visible light feature map. This semantic consistency loss function is then weighted and fused with the cross-entropy loss function for fire classification to obtain the first total loss function. This first total loss function is derived by introducing pre-defined weight coefficients to balance the importance of the two loss tasks during model training. The weight coefficients are multiplied by the semantic consistency loss function to obtain a weighted alignment loss. This weighted alignment loss is then added to the fire classification cross-entropy loss function, and the sum of these two sums is defined as the first total loss function. In subsequent backpropagation training, this first total loss function is used as the final optimization target, and adjustments are made to the feature extractor and... The parameters of the classifier cause the value of the first total loss function to continuously decrease, thereby achieving the dual effect of accurate fire classification and semantic alignment of modal features. The semantic consistency loss function is a function used to measure the semantic consistency of different data (such as infrared feature maps and visible light feature maps in this application). It evaluates the semantic similarity of the two feature maps by calculating the Euclidean distance between the feature vectors of the high-temperature region and the background region. The smaller the distance, the higher the semantic consistency. In the fire classification task, the cross-entropy loss function is used to evaluate the accuracy of the model in predicting whether an image belongs to the fire category. The smaller the loss value, the more accurate the prediction. In the feature vector space, the Euclidean distance can reflect the similarity between two feature vectors. The closer the distance, the more similar they are.

[0075] The semantic consistency loss function is calculated as follows: Feature vectors corresponding to the coordinate set of high-temperature regions in the infrared feature map are extracted as the first high-temperature feature vector set, and feature vectors corresponding to the coordinate set of high-temperature regions in the visible light feature map are extracted as the second high-temperature feature vector set. The Euclidean distance between each pair of feature vectors in the first and second high-temperature feature vector sets is calculated. Feature vectors corresponding to the coordinate set of the background region in the infrared feature map are extracted as the first background feature vector set, and feature vectors corresponding to the coordinate set of the background region in the visible light feature map are extracted as the second background feature vector set. The Euclidean distance between each pair of feature vectors in the first and second background feature vector sets is calculated. The average of the Euclidean distances between each pair of feature vectors in the first and second high-temperature feature vector sets and the average of the Euclidean distances between each pair of feature vectors in the first and second background feature vector sets are summed to obtain the semantic consistency loss function. The first total loss function after weighted fusion comprehensively considers both semantic consistency and fire classification accuracy. During model training, by minimizing the first total loss function, the model can not only accurately classify fires in images but also ensure semantic consistency between the infrared and visible light feature maps, thereby improving the performance and reliability of the entire fire early warning system.

[0076] The semantic consistency loss function can measure the semantic consistency between infrared feature maps and visible light feature maps in high-temperature and background regions. By calculating the Euclidean distance between feature vectors, the similarity between the two feature maps in corresponding regions can be quantified, ensuring that the two feature maps are consistent at the semantic level. This helps to improve the accuracy of fire early warning and avoid misjudgments caused by inconsistencies between the two image features.

[0077] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0078] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system, device, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0079] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0080] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A fire early warning method for ancient building complexes based on an internet platform, characterized in that, The method includes: S1. Obtain the visible light image of the current monitoring grid in the ancient building complex. Use the preset target detection network to detect the smoke area in the visible light image and generate the confidence score of the smoke detection in the visible light image. Determine the scene type based on the generated confidence score. If the scene type is a interference scene, trigger the registration operation on the original infrared image. Otherwise, directly issue a fire warning based on the infrared image. S2, in response to the instruction to trigger the registration operation of the original infrared image, acquire the original infrared image of the current monitoring grid in the ancient building complex, perform the registration operation on the original infrared image based on the visible light image, generate the registered infrared image, use a feature extractor to extract the infrared feature map of the registered infrared image and the visible light feature map of the visible light image respectively, generate an attention weight map based on the infrared feature map and the visible light feature map, and combine the infrared feature map and the visible light feature map to generate preliminary fusion features. The feature extractor is characterized as a feature extractor composed of a two-stream convolutional neural network. S3, perform dimensionality reduction and compression on the preliminary fusion features to obtain a low-dimensional feature vector, input the low-dimensional feature vector into the classifier to obtain the fire occurrence probability of the current monitoring grid, select an early warning scheme based on the fire occurrence probability, the early warning scheme includes at least triggering a first-level alarm signal and triggering a second-level alarm signal; S4, in response to the instruction to trigger a Level 1 alarm signal, marks the current monitoring grid as a risk source grid, calculates the directional propagation coefficient of adjacent target grids relative to the risk source grid, updates the fire occurrence probability of adjacent target grids based on the directional propagation coefficient, and adjusts the early warning scheme.

2. The fire early warning method for ancient building complexes based on an internet platform as described in claim 1, characterized in that, The specific process for determining the scene type based on the generated confidence score is as follows: The confidence score of smoke detection in visible light images is compared with a preset smoke detection threshold. If the confidence score of smoke detection in the visible light image is greater than the preset smoke detection threshold, the current scene is determined to be an interference scene, and a registration operation is triggered on the original infrared image. Otherwise, if the current scene is determined to be a non-interference scene, fire warning will be issued directly based on the infrared image; The fire early warning based on infrared images specifically refers to: Infrared feature maps are extracted from the original infrared image, and statistical and texture features are extracted based on the infrared feature maps to construct an infrared feature vector. The statistical features include at least the highest temperature value and the average temperature value, and the texture features include at least the local binary pattern histogram and area of ​​the infrared image. The infrared feature vector is input into the infrared classifier, and the flame probability is output. The flame probability represents the probability of determining whether a flame exists in the current infrared image. If the calculated fire probability is greater than the preset infrared alarm threshold, a fire is directly determined to have occurred in the current monitoring grid, triggering a level one alarm signal; otherwise, it is determined that no fire has occurred in the current monitoring grid, and monitoring continues.

3. The fire early warning method for ancient building complexes based on an internet platform as described in claim 1, characterized in that, The registration operation on the original infrared image specifically includes: The gradient components of the original infrared image in the horizontal and vertical directions are calculated using a preset gradient operator, and the gradient magnitude is obtained based on the gradient components in the horizontal and vertical directions to generate a gradient magnitude map that reflects the temperature change region. Calculate the mean and standard deviation of the gradient magnitude map, and obtain a dynamic segmentation threshold based on the mean and standard deviation of the gradient magnitude map and a preset adjustment coefficient. The dynamic segmentation threshold represents the result obtained by coupling the product of the preset adjustment coefficient and the standard deviation of the gradient magnitude map with the mean of the gradient magnitude map. The gradient magnitude map is binarized based on the dynamic segmentation threshold to generate an edge mask; The region with a pixel value of 1 in the edge mask is marked as the edge region. A preset edge enhancement coefficient is introduced to weight the feature response values ​​of the pixels in the edge region during the feature detection process. Infrared feature points and corresponding infrared feature descriptors are re-extracted based on the weighted feature response values ​​in the preset scale space. The region with a pixel value of 0 in the edge mask is marked as a non-edge region. The feature response value is calculated for the pixels in the non-edge region. Pixels with feature response values ​​greater than a preset contrast threshold are selected and corresponding infrared feature descriptors are generated. The extracted infrared feature descriptors are matched with the feature descriptors of the visible light image. Mismatched point pairs are eliminated using a random consistency algorithm. The homography matrix is ​​calculated, and the original infrared image is transformed based on the homography matrix to achieve image registration.

4. The fire early warning method for ancient building complexes based on an internet platform as described in claim 1, characterized in that, The generation of the attention weight map based on the infrared feature map and the visible light feature map specifically includes: The infrared feature map is mapped to a query vector, where the infrared feature map represents thermal radiation information; The visible light feature map is mapped into a key vector and a value vector, wherein the visible light feature map characterizes the spectral response features of the flame and the texture structure features of the smoke; Calculate the similarity between the query vector and the key vector to generate an initial attention weight map; The initial attention weight map is optimized by redundancy removal and complementation mechanisms to obtain the attention weight map. The redundancy removal mechanism is used to reduce the weight of regions in the infrared feature map below a preset temperature threshold and regions in the visible light feature map below a preset feature variance threshold in the attention weight map, so as to suppress redundant interference from background noise. The complementary mechanism is used to increase the weight of regions in the visible light feature map that contain feature information lacking in the infrared feature map in the attention weight map. The feature information lacking in the infrared feature map includes the texture structure features of smoke or the spectral response features of flames. The value vectors of the visible light feature map are weighted and summed using the attention weight map to obtain the weighted visible light features. The weighted visible light features and infrared feature maps are subjected to residual connection and layer normalization to generate preliminary fused features.

5. A fire early warning method for ancient building complexes based on an internet platform as described in claim 1, characterized in that, The process of generating an attention weight map based on infrared and visible light feature maps further includes: Only when the current scene is determined to be an interference scene, for the non-edge area in the visible light image, the infrared thermal radiation characteristics of the corresponding area are obtained, and it is determined whether the infrared thermal radiation characteristics reach the flame temperature threshold. For regions where the infrared thermal radiation features do not reach the flame temperature threshold, the correction factor corresponding to the texture structure features with smoke is set to the first preset attenuation coefficient to reduce the attention weight of the texture structure features with smoke while maintaining the attention weight of the spectral response features of the flame. For regions where the infrared thermal radiation features reach the flame temperature threshold, the correction factor corresponding to the spectral response features of the flame is set to the first preset enhancement coefficient to increase the attention weight of the spectral response features of the flame, while maintaining the attention weight of the texture structure features of the smoke. For edge regions in visible light images, a joint judgment is made by combining the infrared thermal radiation characteristics of the corresponding regions to determine whether the infrared thermal radiation characteristics reach the flame temperature threshold. For regions where the infrared thermal radiation features do not reach the flame temperature threshold, the correction factor corresponding to the texture structure features with smoke is set to the second preset attenuation coefficient to reduce the attention weight of the texture structure features with smoke while maintaining the attention weight of the spectral response features of the flame. For regions where the infrared thermal radiation features reach the flame temperature threshold, the correction factor corresponding to the spectral response features of the flame is set to the second preset enhancement coefficient to increase the attention weight of the spectral response features of the flame, while maintaining the attention weight of the texture structure features of the smoke. The first preset attenuation coefficient, the second preset attenuation coefficient, the first preset enhancement coefficient, and the second preset enhancement coefficient increase sequentially.

6. A fire early warning method for ancient building complexes based on an internet platform as described in claim 1, characterized in that, The fire probability-based early warning scheme specifically includes: If the probability of a fire is greater than the first fire probability threshold, a level one alarm signal is triggered. If the probability of a fire occurs is between the second fire probability threshold and the first fire probability threshold, a level two warning signal is triggered. If the probability of a fire is less than the second fire probability threshold, the alarm signal will not be triggered. The first fire probability threshold is greater than the second fire probability threshold; The specific method for obtaining the probability of fire occurrence is as follows: The low-dimensional feature vector is input into a preset classifier to obtain the probability of fire occurrence in the current monitoring grid. The low-dimensional feature vector is obtained specifically as follows: The initial fusion features are input into the bottleneck fully connected layer, which outputs a mean vector and a variance vector. The bottleneck fully connected layer is used to encode the initial fusion features and output statistical parameters representing the feature distribution. The bottleneck fully connected layer is configured to linearly transform and compress the dimension of the input initial fusion features from a first dimension to a second dimension, where the second dimension is smaller than the first dimension, in order to extract fire features from the initial fusion features. By using the reparameterized sampling method, sampling is performed based on the mean vector and variance vector to obtain a compressed low-dimensional feature vector.

7. A fire early warning method for ancient building complexes based on an internet platform as described in claim 1, characterized in that, The method of updating the fire occurrence probability of adjacent target grids based on the directional propagation coefficient specifically includes: Obtain the fire occurrence probability of adjacent target grids and record it as the base fire probability. Introduce a directional propagation coefficient to correct the base fire probability of the target grids in order to update the fire occurrence probability of adjacent target grids. If the updated fire probability is not less than the preset second fire probability threshold, the target grid is determined to be in a fire warning state and an alarm message is generated; otherwise, the target grid is determined to be in a normal state and monitoring continues. The directional propagation coefficient is obtained specifically as follows: Obtain current wind direction and wind speed data; Calculate the cosine of the angle between the direction of the line connecting the risk source grid and the target grid and the current wind direction, and use it as the wind direction consistency factor; Calculate the distance between the risk source grid and the target grid, and calculate the distance decay factor based on the preset decay constant using the exponential decay function; The directional propagation coefficient is obtained by weighting and fusing the wind direction consistency factor and the distance attenuation factor.

8. A fire early warning method for ancient building complexes based on an internet platform as described in claim 1, characterized in that, Prior to performing the registration operation on the original infrared image, the process also includes: Get the current ambient humidity value and determine whether the current ambient humidity value is greater than the preset humidity threshold; If so, the environmental interference mode is determined to be water vapor interference mode, the grid size of the contrast-adaptive histogram equalization algorithm is set to the first preset grid size, and the histogram clipping limit is set to the first limit range. Otherwise, the environmental interference mode is determined to be the normal mode, the grid size of the contrast-adaptive histogram equalization algorithm is set to the second preset grid size, and the histogram clipping limit is set to the second limiting range. The registered infrared image is subjected to contrast-limited adaptive histogram equalization using the adjusted grid size and histogram cropping limit, and bilinear interpolation algorithm is used to eliminate block artifacts. The first preset grid size is smaller than the second preset grid size, and the first limiting range is larger than the second limiting range.

9. A fire early warning method for ancient building complexes based on an internet platform as described in claim 1, characterized in that, The process of extracting infrared feature maps from the registered infrared image and visible light feature maps from the visible light image using a feature extractor, followed by: The registered infrared and visible light images are input into the feature extractor, which outputs infrared feature maps and visible light feature maps, respectively. Based on the preset coordinate set of high-temperature region and background region, the semantic consistency loss function between infrared feature map and visible light feature map is calculated, and the semantic consistency loss function is weighted and fused with the cross-entropy loss function of fire classification task to obtain the first total loss function. The semantic consistency loss function is specifically calculated as follows: Extract the feature vectors of the coordinate set of the corresponding high-temperature region in the infrared feature map as the first high-temperature feature vector set, and the feature vectors of the coordinate set of the corresponding high-temperature region in the visible light feature map as the second high-temperature feature vector set, and calculate the Euclidean distance between the first high-temperature feature vector set and the second high-temperature feature vector set. Extract the feature vectors of the corresponding background region coordinate set in the infrared feature map as the first background feature vector set, and the feature vectors of the corresponding background region coordinate set in the visible light feature map as the second background feature vector set, and calculate the Euclidean distance between the first background feature vector set and the second background feature vector set. The semantic consistency loss function is obtained based on the Euclidean distance between the first high-temperature feature vector set and the second high-temperature feature vector set, and the Euclidean distance between the first background feature vector set and the second background feature vector set.

10. A fire early warning system for ancient building complexes based on an internet platform, characterized in that, The system includes: The smoke detection module is used to acquire the visible light image of the current monitoring grid in the ancient building complex. It uses a preset target detection network to detect the smoke area in the visible light image and generate a confidence score for the smoke detection in the visible light image. Based on the generated confidence score, the scene type is determined. If the scene type is a disturbance scene, the registration operation on the original infrared image is triggered. Otherwise, a fire warning is directly issued based on the infrared image. The image registration module is used to respond to the instruction that triggers the registration operation on the original infrared image, acquire the original infrared image of the current monitoring grid in the ancient building complex, perform the registration operation on the original infrared image with the visible light image as a reference, generate the registered infrared image, and use a feature extractor to extract the infrared feature map of the registered infrared image and the visible light feature map of the visible light image respectively. An attention weight map is generated based on the infrared feature map and the visible light feature map, and a preliminary fusion feature is generated by combining the infrared feature map and the visible light feature map. The feature extractor is characterized as a feature extractor composed of a two-stream convolutional neural network. The fire early warning module is used to perform dimensionality reduction and compression processing on the preliminary fused features to obtain a low-dimensional feature vector. The low-dimensional feature vector is input into the classifier to obtain the fire occurrence probability of the current monitoring grid. Based on the fire occurrence probability, an early warning scheme is selected. The early warning scheme includes at least triggering a first-level alarm signal and triggering a second-level alarm signal. The ancient building complex early warning linkage module is used to respond to the command that triggers the first-level alarm signal, mark the current monitoring grid as the risk source grid, calculate the directional propagation coefficient of the adjacent target grid relative to the risk source grid, update the fire occurrence probability of the adjacent target grid based on the directional propagation coefficient, and adjust the early warning scheme.

Images