Fire monitoring method and system based on high tower camera

Abstract

The invention discloses a fire monitoring method and system based on a high tower camera. According to the system, an existing communication signal tower or a high tower is reused, a camera is deployed for real-time video acquisition, and a recognition model containing an improved backbone network is used for fire target detection; wherein strip convolution and strip pooling operations are introduced into the backbone network, the slender morphological characteristics of smoke and flames are effectively captured, and the feature significance is enhanced in combination with a polarization self-attention module. When a suspected fire behavior is identified, the unmanned aerial vehicle carrying the visible light and infrared dual-light camera is automatically dispatched to go to the scene for secondary confirmation and accurate positioning; according to the invention, large-range, low-cost and high-precision early fire monitoring and early warning are realized.

Description

Technical Field

[0001] This invention relates to the field of fire monitoring and early warning technology, and in particular to a fire monitoring method and system based on a high-tower camera. Background Technology

[0002] Fires, especially forest and grassland fires, are highly destructive and difficult-to-control natural disasters. Their typical characteristics include suddenness, unpredictable spread, and potential for loss of control under favorable weather conditions such as strong winds and drought. Once a fire reaches a significant scale, it not only destroys vast areas of forest resources and grassland ecosystems, leading to long-term environmental problems such as sharp declines in biodiversity and soil erosion, but it can also directly threaten the lives and property of residents in forest and pastoral areas, and even affect neighboring towns and important facilities, posing a serious challenge to the national economy and public safety. Therefore, achieving early detection, precise location, and rapid warning of fires is crucial to minimizing fire losses and remains a technical challenge that continues to be addressed globally.

[0003] Currently, fire monitoring technology systems applied to large-scale field environments mainly rely on the following traditional methods, but each has obvious limitations: First, manual patrols are the most basic and traditional method. This relies on forest rangers or inspectors conducting visual observations on foot or by vehicle. While direct, this method is extremely inefficient, severely limited in coverage, and subject to terrain, weather, and day / night conditions. In the vast and sparsely populated forests and grasslands, achieving all-weather, comprehensive patrols is virtually impossible. Small fires or smoldering fires are difficult to detect in time, leading to missed opportunities for early firefighting.

[0004] Secondly, satellite remote sensing technology provides a macroscopic monitoring perspective. Sensors on polar-orbiting and geostationary satellites can periodically scan large areas or even the entire globe. The main advantage of this technology is its wide coverage. However, its inherent limitations restrict its application in early warning systems. First, its real-time performance is poor; satellite revisit cycles are long, ranging from several hours to tens of hours, failing to meet the needs for minute-level or even second-level responses to fire incidents. Second, its spatial resolution is insufficient; for small fire spots or early fires with indistinct smoke characteristics, its identification capability is limited, easily leading to missed or false alarms. Third, it is greatly affected by meteorological factors such as clouds and aerosols; its monitoring effectiveness is significantly reduced in adverse weather conditions.

[0005] Third, ground-based sensor networks constitute the Internet of Things (IoT) by deploying a large number of optical sensor nodes for temperature, humidity, smoke concentration, and flames in the monitored area. These nodes can collect local environmental parameters and transmit them to the monitoring center via wireless networks. Theoretically, this technology can achieve high-precision local fire detection. However, its bottleneck lies in the fact that to achieve effective coverage, a massive number of sensor nodes need to be deployed over a wide area, resulting in high infrastructure construction costs and a huge workload and high cost for subsequent maintenance (such as battery replacement and equipment fault diagnosis). At the same time, the sensors are susceptible to interference from environmental dust, insects, wild animals, etc., which can produce false alarms, posing a serious challenge to their reliability and stability.

[0006] Fourth, constructing dedicated fire monitoring towers is another solution. These towers are typically equipped with high-definition cameras and sensors, offering a wide field of view. However, building such towers also faces challenges such as complex project approval processes, huge initial investments, long construction periods, and limited site selection, making it difficult to quickly and cost-effectively deploy them on a large scale to form an effective coverage network.

[0007] In summary, existing technologies have shortcomings in real-time performance, coverage, construction and maintenance costs, and the reliability of early warning, creating a significant technological gap. Nationwide, numerous high-tower structures of various types have already been constructed, such as power transmission towers, communication signal base station towers, and watchtowers. These infrastructures possess natural advantages such as large numbers, wide geographical distribution, prominent height, and broad field of vision. However, the potential of these high-quality hardware resources in the field of fire monitoring has not been fully explored and systematically utilized. Therefore, there is an urgent need for a new fire monitoring solution that can effectively integrate existing resources and overcome the aforementioned technological shortcomings. Summary of the Invention

[0008] This invention provides a fire monitoring method and system based on a tower camera, which solves the problem that the potential of existing high-quality hardware resources in the field of fire monitoring has not been fully explored and systematically utilized, and realizes large-scale, low-cost, and high-precision early fire monitoring and warning.

[0009] In a first aspect, the present invention provides a fire monitoring method based on a tower camera, the method comprising: Acquire real-time video streams and preprocess the video streams to obtain fire candidate areas; The fire candidate region is input into a pre-trained recognition network model to obtain the predicted region, category, and confidence score of the fire target. The recognition network model includes a backbone network, a feature fusion layer, and a detection head. The backbone network extracts features from the fire candidate region to obtain a multi-level feature map. The feature fusion layer fuses features from the multi-level feature map to obtain an enhanced feature map. The detection head outputs the predicted bounding box, category label, and confidence score of the fire target based on the enhanced feature map. When the category label is flame or smoke, and the corresponding confidence level exceeds a preset threshold, the predicted area is reconfirmed using a drone to obtain verified fire information.

[0010] In conjunction with the first aspect, in one possible implementation, the preprocessing of the video stream to obtain fire candidate regions includes: The video stream is converted from the RGB color space to the YCbCr and HSV color spaces to separate luminance and color features, resulting in the converted video stream. Histogram equalization is performed on the luminance channel of the converted video stream to enhance contrast, resulting in the processed luminance channel. In the processed luminance channel, the motion region is extracted by the inter-frame difference method, and the pixel region whose motion amplitude exceeds the preset threshold is identified as the fire candidate region.

[0011] In conjunction with the first aspect, in one possible implementation, the backbone network includes: The input convolutional layer consists of a first-level strip convolutional module, a first downsampling layer, a second-level strip convolutional module, a second downsampling layer, a third-level strip convolutional module, a third downsampling layer, a fourth-level strip convolutional module, a spatial pyramid pooling module, and an attention module. The input convolutional layer is used to perform preliminary feature extraction on the fire candidate region to obtain an initial feature map; The first-level strip convolution module is used to extract slender features from the initial feature map to obtain primary slender morphological features; The first downsampling layer is used to downsample the primary elongated morphological features to obtain a first downsampled feature map; The second-level strip convolution module is used to extract the elongated features of the middle layer from the first downsampled feature map to obtain the elongated morphological features of the middle layer. The second downsampling layer is used to downsample the intermediate elongated morphological features to obtain a second downsampled feature map; The third-level strip convolution module is used to perform deep slender feature extraction on the second downsampled feature map to obtain three-level slender morphological features. The third downsampling layer is used to downsample the three-level elongated morphological features to obtain a third downsampling feature map; The fourth-level strip convolution module is used to perform the final long feature extraction on the third downsampled feature map to obtain the final slender morphological features; The spatial pyramid pooling module is used to fuse multi-scale receptive field information from the primary slender morphological features, the intermediate slender morphological features, the tertiary slender morphological features, and the final slender morphological features, and output a fused feature map. The attention module is used to calculate the feature weights of different spatial locations in the fused feature map through a polarization self-attention mechanism, and perform weighted summation to obtain a multi-level feature map.

[0012] In conjunction with the first aspect, in one possible implementation, the first-level strip convolution module includes, in sequence, a first convolutional layer, a segmentation layer, multiple parallel strip building blocks, and a feature concatenation layer; The first convolutional layer is used to perform convolution processing on the initial feature map and output the first feature map; The segmentation layer is used to segment the first feature map into multiple parallel branches to obtain multiple segmented feature maps. The plurality of cascaded strip building blocks are used to perform strip pooling operations on the segmented feature map to obtain an output feature map; The feature splicing layer is used to splice the first feature map with the output feature map to obtain a primary slender morphological feature.

[0013] In conjunction with the first aspect, in one possible implementation, the strip building block comprises, in sequence, a first 1x1 convolutional layer, a strip pooling layer, a second 1x1 convolutional layer, and a residual summing unit; The first 1x1 convolutional layer is used to perform dimensionality reduction processing on the input feature map and output the first dimensionality-reduced feature map. The strip pooling layer is used to perform horizontal strip pooling and vertical strip pooling on the first dimensionality reduction feature map, and add the pooling results in the two directions to output a fused pooled feature map. The second 1x1 convolutional layer is used to perform dimensionality upscaling on the fused pooling feature map, restoring its channel count to the same as the input feature map of the strip building block, and outputting a second feature map; The residual addition unit is used to add the second feature map to the input feature map of the strip building block element by element, and output the final output feature map of the strip building block.

[0014] In conjunction with the first aspect, in one possible implementation, the attention module includes parallel channel attention branches and spatial attention branches, as well as a feature weighted fusion unit; The channel attention branch is used to perform global average pooling on the fused feature map and output a channel statistical vector; the channel statistical vector is input into a multilayer perceptron to calculate the channel attention weights; the calculated channel attention weights are multiplied channel by channel with the input fused feature map to output channel enhanced features; The spatial attention branch is used to perform horizontal and vertical strip pooling on the fused feature map, and then concatenate and convolve the pooled features to calculate the spatial attention weights; the spatial attention weights are multiplied pixel by pixel by the fused feature map to output the spatial enhancement features. The feature weighted fusion unit is used to add and fuse the channel enhancement features and the spatial enhancement features; generate the final comprehensive attention weight map through the Sigmoid activation function; and perform a weighted multiplication of the comprehensive attention weight map with the initially input fused feature map to output a multi-level feature map.

[0015] In conjunction with the first aspect, in one possible implementation, the loss function of the recognition network model is expressed as: ; in, Indicates the first weighting coefficient; This represents the second weighting coefficient; Indicates the third weighting coefficient; Indicates classification loss; Indicates the target confidence loss; This represents the bounding box regression loss; This represents the loss of the recognition network model.

[0016] In conjunction with the first aspect, in one possible implementation, when the category label is flame or smoke and the corresponding confidence level exceeds a preset threshold, a drone is used to perform secondary confirmation of the predicted area to obtain verified fire information, including: Based on the predicted bounding box coordinates, category labels, and confidence scores output by the recognition network model, and combined with the geographical coordinates of the deployed cameras, the orientation and distance of the fire candidate area relative to the cameras are calculated. Based on orientation and distance, as well as the category label and confidence level, the flight path and confirmation mission strategy of the UAV are planned; wherein, if the category label is smoke and the confidence level is low, a wide-area cruise path is planned; if the category label is flame and the confidence level is high, a direct path is planned and the priority of infrared camera acquisition is increased. The drone is controlled to fly along the flight path to the target area and acquire dual-modal image data collected by the infrared camera; The dual-modal image data is fused and analyzed to distinguish between real fires and non-fire interference sources. After confirming the fire, the precise coordinates of the fire point and environmental information are sent to the backend in conjunction with the high-precision positioning data of the UAV.

[0017] Secondly, the present invention provides a fire monitoring system based on a high-tower camera, the system comprising: The data acquisition module is used to acquire real-time video streams and preprocess the video streams to obtain fire candidate areas; The prediction module is used to input the fire candidate region into a pre-trained recognition network model to obtain the predicted region, category, and confidence score of the fire target. The recognition network model includes a backbone network, a feature fusion layer, and a detection head. The backbone network is used to extract features from the fire candidate region to obtain a multi-level feature map. The feature fusion layer is used to fuse features from the multi-level feature map to obtain an enhanced feature map. The detection head is used to output the predicted bounding box, category label, and confidence score of the fire target based on the enhanced feature map. The verification module is used to perform secondary confirmation of the predicted area using a drone when the category label is flame or smoke and the corresponding confidence level exceeds a preset threshold, so as to obtain the verified fire information.

[0018] One or more technical solutions provided in this invention have at least the following technical effects or advantages: This invention obtains fire candidate regions by acquiring and preprocessing real-time video streams. The beneficial effect of this step is that preprocessing effectively reduces noise and irrelevant background information in the video stream, highlighting areas that may contain fire, thus providing clearer input data for subsequent identification stages, improving the overall system's response speed and accuracy, and reducing computational burden. The fire candidate regions are then input into a pre-trained identification network model, which includes a backbone network, a feature fusion layer, and a detection head. The backbone network is responsible for multi-level feature extraction of the fire candidate regions, ensuring the capture of key features of flames or smoke at different scales; the feature fusion layer integrates these multi-level feature maps, enhancing the model's adaptability to complex scenes and improving the richness of feature representation; the detection head outputs the predicted bounding box, category label, and confidence score of the fire target based on the enhanced feature maps. The beneficial effect of this step is that, through deep learning and feature fusion techniques, the model can more accurately identify and locate fire targets, reducing false positives and false negatives. Furthermore, the introduction of confidence scores provides a quantifiable evaluation standard, making the detection results more reliable. When the category label output by the identification network model is flame or smoke, and the corresponding confidence level exceeds a preset threshold, a drone is used to perform secondary confirmation of the predicted area, obtaining verified fire information. The benefit of this step is that on-site verification by drones can further eliminate false alarms, ensuring the authenticity and accuracy of fire information. Drones can provide close-range visual data or sensor readings, enhancing the system's decision-making credibility, thereby supporting timely and effective emergency response and avoiding resource waste. Attached Figure Description

[0019] Figure 1 A flowchart illustrating the steps of a fire monitoring method based on a tower camera, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the attention module structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the network structure of the recognition network model provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the first-level strip convolutional module network structure provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] In a first aspect, the present invention provides a fire monitoring method based on a high-tower camera, see [link to relevant documentation]. Figure 1 The method includes the following steps S101 to S103.

[0022] S101: Acquire the real-time video stream and preprocess the video stream to obtain fire candidate areas; Specifically, in step S101, the video stream is preprocessed to obtain fire candidate areas, including the following steps S1011 to S1013.

[0023] S1011 converts the video stream from the RGB color space to the YCbCr and HSV color spaces to separate luminance and color features, resulting in the converted video stream. S1012, Histogram equalization is performed on the luminance channel in the converted video stream to enhance contrast, and the processed luminance channel is obtained. S1013, in the processed luminance channel, the motion region is extracted by the inter-frame difference method, and the pixel region whose motion amplitude exceeds the preset threshold is determined as the fire candidate region.

[0024] For example, the camera continuously captures images of the surrounding environment and transmits them as a video stream. The sequence of video images captured by the camera is denoted as... ,in, Represents pixel coordinates, Indicates a time frame.

[0025] Converting the acquired raw image from RGB space to YCbCr and HSV space utilizes the advantages of separating luminance and color features: ; In HSV space, the flame region typically exhibits high saturation (S) and high brightness (V).

[0026] Then, histogram equalization is performed on the luminance channel Y to enhance the contrast between dark and bright areas: ; in, Let be the cumulative distribution function. The total number of pixels in the image. The grayscale level is denoted by .

[0027] Flames and smoke exhibit dynamic characteristics; this invention extracts motion regions through inter-frame difference analysis. ; when At that time, pixels are considered to belong to the moving region, where, This is a preset threshold. This allows us to extract candidate areas for the fire zone.

[0028] S102, the fire candidate region is input into the pre-trained recognition network model to obtain the predicted region, category, and confidence score of the fire target; wherein, the recognition network model includes: a backbone network, a feature fusion layer, and a detection head; the backbone network is used to extract features from the fire candidate region to obtain a multi-level feature map; the feature fusion layer is used to fuse features from the multi-level feature map to obtain an enhanced feature map; the detection head is used to output the predicted bounding box, category label, and confidence score of the fire target based on the enhanced feature map; Specifically, in step S102, the input convolutional layer, the first-level strip convolutional module, the first downsampling layer, the second-level strip convolutional module, the second downsampling layer, the third-level strip convolutional module, the third downsampling layer, the fourth-level strip convolutional module, the spatial pyramid pooling module, and the attention module are included.

[0029] The input convolutional layer is used to perform preliminary feature extraction on the fire candidate region to obtain an initial feature map. The first-level strip convolutional module is used to extract elongated features from the initial feature map to obtain primary elongated morphological features. The first downsampling layer is used to downsample the primary elongated morphological features to obtain a first downsampled feature map. The second-level strip convolutional module is used to extract intermediate elongated features from the first downsampled feature map to obtain intermediate elongated morphological features. The second downsampling layer is used to downsample the intermediate elongated morphological features to obtain a second downsampled feature map. The third-level strip convolutional module is used to extract deeper elongated features from the second downsampled feature map. The process involves three levels of elongated morphological features: a third downsampling layer to downsample these features, resulting in a third downsampled feature map; a fourth-level strip convolution module to extract the final long features from the third downsampled feature map, yielding the final elongated morphological features; a spatial pyramid pooling module to fuse multi-scale receptive field information from the primary, intermediate, tertiary, and final elongated morphological features, outputting a fused feature map; and an attention module to calculate feature weights at different spatial locations in the fused feature map using a polarized self-attention mechanism, and then performing a weighted summation to obtain a multi-level feature map.

[0030] Here, the first-level strip convolution module includes, in sequence: a first convolutional layer, a segmentation layer, multiple parallel strip building blocks, and a feature concatenation layer.

[0031] The first convolutional layer is used to perform convolution processing on the initial feature map and output the first feature map; the segmentation layer is used to segment the first feature map into multiple parallel branches to obtain multiple segmentation feature maps; multiple cascaded strip building blocks are used to perform strip pooling operations on the segmentation feature maps to obtain the output feature map; the feature concatenation layer is used to concatenate the first feature map and the output feature map to obtain the primary slender morphological features.

[0032] Here, the strip building block includes, in sequence, a first 1x1 convolutional layer, a strip pooling layer, a second 1x1 convolutional layer, and a residual summing unit.

[0033] The first 1x1 convolutional layer performs dimensionality reduction on the input feature map and outputs a first dimensionality-reduced feature map. The strip pooling layer performs horizontal and vertical strip pooling on the first dimensionality-reduced feature map and adds the pooling results in both directions to output a fused pooled feature map. The second 1x1 convolutional layer performs dimensionality increase on the fused pooled feature map, restoring its channel count to the same as the input feature map of the strip building block, and outputs a second feature map. The residual addition unit adds the second feature map element-wise to the input feature map of the strip building block and outputs the final output feature map of the strip building block.

[0034] Specifically, in step S102, the attention module includes parallel channel attention branches and spatial attention branches, as well as a feature weighted fusion unit; The channel attention branch is used to perform global average pooling on the fused feature map and output channel statistical vectors. The channel statistical vectors are input into the multilayer perceptron to calculate the channel attention weights. The calculated channel attention weights are multiplied channel by channel with the input fused feature map to output channel-enhanced features. The spatial attention branch is used to perform horizontal and vertical strip pooling on the fused feature map, and then concatenate and convolve the pooled features to calculate the spatial attention weights. The spatial attention weights are then multiplied pixel by pixel with the fused feature map to output the spatial enhancement features. The feature weighted fusion unit is used to add and fuse channel enhancement features and spatial enhancement features; generate the final comprehensive attention weight map through the Sigmoid activation function; and multiply the comprehensive attention weight map with the initial input fused feature map in a weighted manner to output a multi-level feature map.

[0035] For example, in the intelligent identification stage of fire detection, this invention proposes a fire target detection method based on an improved YOLOv11. This method introduces a StripPooling module into the YOLOv11 network structure and improves the loss function, thereby enhancing the model's accuracy and robustness in detecting flames and smoke in complex scenarios.

[0036] The YOLOv11 network consists of an input layer, a backbone, a feature fusion layer, and a head. This invention embeds strip pooling modules into some convolutional layers of the backbone and neck. By introducing global contextual information through strip pooling, the model can capture dependencies between long-distance regions in both the horizontal and vertical directions, thereby improving its ability to detect slender, spreading targets such as flames and smoke.

[0037] See Figure 2 This is a schematic diagram of the specific structural hierarchy of the attention module. It is a high-level attention module used at the end of the backbone network (after SPPF) to purify features before they are fed into the neck.

[0038] See Figure 4 The first-level strip convolution module (c3k2_Strip) is a building block with two configurations. This invention replaces Bottleneck in the prior art with StripBlock. StripBlock is a structured unit that uses 1*k and k*1 strip convolution operations, replacing the 3*3 convolution operation of Bottleneck. It is a powerful tool for capturing features of slender targets, such as smoke columns.

[0039] exist Figure 3 In YOLOv11, c3k2 in the backbone network is replaced with c3k2_Strip, which is used to build the model's neck and head for subsequent feature fusion and segmentation tasks. Here, multiple strip convolutional modules have the same structure.

[0040] See Figure 3 The input is a long-distance real-time image or video stream captured by a tower camera. The image first enters the model's backbone network for feature extraction. Its core improvement lies in the introduction of strip building blocks into the backbone network, instead of using conventional square convolutions. These strip building blocks employ asymmetric, large strip convolution kernels such as 1*k and k*1, specifically designed to capture the vertical, elongated shape of early fire smoke columns from a distant tower view. This design addresses the limitations of conventional 3*3 square convolution kernels, such as limited receptive field, difficulty in perceiving the complete smoke column shape, and easy confusion with clouds and fog. After passing through this series of strip convolution modules and being fused by the SPPF module using multi-scale receptive fields, the feature stream enters the attention module (C2PSA).

[0041] See Figure 3Conv: Represents a convolutional layer used for initial transformation or feature extraction of the input features. PSABlock: Represents a Polarized Self-Attention Block. Each PSABlock corresponds to the core unit of the attention module described in the text, containing parallel channel attention branches and spatial attention branches. ... and N: Represent multiple PSABlocks repeated in series (N times), used to progressively refine features and enhance the network's ability to capture key information. Concat: Represents a concatenation layer used to concatenate the output features of multiple PSABlocks along the channel dimension. Conv: Represents another convolutional layer used to fuse and reduce the dimensionality of the concatenated features, outputting the final feature map.

[0042] Conv (first convolutional layer): This convolutional layer is the entry point for the module, used to receive input features (e.g., fused feature maps passed from other parts of the backbone network). It performs convolution operations to extract preliminary features from the input, preparing for subsequent attention processing.

[0043] PSABlock (Polarization Self-Attention Block): Each PSABlock corresponds to the attention module described in the text, including parallel channel attention branches and spatial attention branches, as well as a feature weighted fusion unit. Specifically: Channel Attention Branch: Performs global average pooling on the input feature map to generate channel statistical vectors; then calculates channel attention weights using a multilayer perceptron (MLP); finally, multiplies the weights with the input feature map channel by channel to output channel-enhanced features. Spatial Attention Branch: Performs horizontal and vertical strip pooling on the input feature map to capture long-range spatial dependencies; concatenates the pooling results and fuses them through convolution to calculate spatial attention weights; finally, multiplies the weights with the input feature map pixel by pixel to output spatial-enhanced features.

[0044] Feature weighted fusion unit: Adds channel enhancement features and spatial enhancement features, and then generates a comprehensive attention weight map through the sigmoid activation function; finally, multiplies the weight map with the original input feature map in a weighted manner to output the refined feature map.

[0045] The presence of multiple PSABlocks concatenated in the image (N in total) signifies that the features undergo multiple attention refinements. Each PSABlock further enhances fire-related features (such as the elongated shape of flames or smoke) and suppresses background interference. This concatenated structure allows the network to capture and optimize features at different levels.

[0046] The Concat layer is where the output feature maps of all PSABlocks are passed to and concatenated along the channel dimension. This step integrates features extracted from multiple attention stages, forming a richer feature representation and preserving the diverse information extracted by different PSABlocks.

[0047] Conv (the last convolutional layer): The concatenated feature maps pass through this final convolutional layer for feature fusion and dimensionality reduction. This convolutional layer maps the concatenated high-dimensional features to the desired output dimension, generating the final multi-level feature map. This output feature map can then be used for downstream tasks, such as the detection and classification of fire targets.

[0048] C2PSA utilizes an efficient polarization self-attention mechanism to refine and focus the features extracted by the backbone network: it automatically amplifies weak signals related to fire and smoke while suppressing complex noise interference from the background of mountains, forests, and sky. These highly refined features are then fed into the neck network for multi-scale feature fusion. Finally, the model's output is a pixel-level predicted bounding box, class label, and confidence score for the fire target. This output format is crucial, as it not only enables accurate fire location but also allows for the quantitative analysis of the fire's spread speed and coverage through real-time calculation of the mask area.

[0049] Here, the input convolutional layer, Figure 3 The initial part of multiple Convs: First, through the input convolutional layer, the first Conv performs preliminary feature extraction on the fire candidate region to obtain the initial feature map.

[0050] The first-level strip convolution module, the first C3k2_Strip in the figure: This module extracts elongated features from the initial feature map to obtain primary elongated morphological features.

[0051] The downsampling layer, Conv after the first C3k2_Strip in the figure: performs a downsampling operation on the primary elongated morphological features to obtain a downsampled feature map, thereby achieving scale compression and receptive field expansion of the feature map.

[0052] The second-level strip convolution module, Figure 3 The subsequent C3k2_Strip processes perform multiple rounds of deep, elongated feature extraction on the downsampled feature map, gradually enhancing the expressive power of elongated morphological features.

[0053] Spatial pyramid pooling module, Figure 3 SPPF: Integrates multi-scale receptive field information of slender morphological features to output fused features containing rich scale information.

[0054] Attention module Figure 3C2PSA: Through the polarization self-attention mechanism, dynamic weights are assigned to features at different spatial locations in the fused feature map and the weighted sum is performed to obtain a multi-level feature map, thereby improving the discriminative power of the features.

[0055] The loss function of the recognition network model in step S102 is expressed as: ; in, Indicates the first weighting coefficient; This represents the second weighting coefficient; Indicates the third weighting coefficient; Indicates classification loss; Indicates the target confidence loss; This represents the bounding box regression loss; This represents the loss of the recognition network model.

[0056] For example, the bounding box regression loss uses the SIOU loss: ; in, This represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. This represents the length of the diagonal of the smallest bounding rectangle. For aspect ratio consistency, As an angle penalty term, this design can better fit the boundaries of flames and smoke.

[0057] The classification loss uses Focal loss: ; in, To predict the probability of a category, For balance coefficient, This is an adjustment factor used to reduce the weight of easily classified samples and increase attention to difficult-to-classify samples.

[0058] The final comprehensive loss function is: ; in, Represents the total loss function; The classification loss is used to constrain the model's accuracy in class prediction; Target confidence loss is used to determine whether a target exists. The bounding box regression loss is used to optimize the position and shape matching between the predicted box and the ground truth box; , and These are the weighting coefficients for each loss term, used to balance the contribution of different loss terms to the total loss.

[0059] The network model is trained using a large number of fire image datasets and the images are augmented with random scaling, brightness perturbation, rotation, blurring, etc., to enhance the model’s robustness under different lighting, weather and shooting angles.

[0060] S103, when the category label is flame or smoke and the corresponding confidence level exceeds the preset threshold, the predicted area is reconfirmed by using a drone to obtain the verified fire information.

[0061] Specifically, in step S103, when the category label is flame or smoke and the corresponding confidence level exceeds a preset threshold, the predicted area is reconfirmed using a drone to obtain verified fire information, including the following steps S10312 to S1035.

[0062] S1031. Based on the predicted bounding box coordinates, category labels, and confidence scores output by the recognition network model, and combined with the geographical coordinates of the deployed cameras, calculate the orientation and distance of the fire candidate area relative to the cameras. S1032, based on orientation and distance, as well as category label and confidence level, plans the flight path of the UAV and confirms the mission strategy; among them, if the category label is smoke and the confidence level is low, a wide-area cruise path is planned; if the category label is fire and the confidence level is high, a direct path is planned and the priority of infrared camera acquisition is increased. S1033, control the drone to fly along the flight path to the target area and acquire dual-modal image data collected by the infrared camera; S1034 performs fusion analysis on dual-modal image data to distinguish between real fires and non-fire interference sources. After confirming the fire, it combines the high-precision positioning data of the UAV to send precise fire point coordinates and environmental information to the backend.

[0063] For example, to reduce the false alarm rate of cameras and improve the accuracy of fire location, this invention further proposes a secondary confirmation scheme based on drones. When the camera detects a suspected fire and the confidence level exceeds a preset threshold, the system automatically issues a task and dispatches a drone to the suspected area for secondary confirmation using both visible light and infrared modes.

[0064] After preprocessing, the video frames input from the camera are fed into the recognition network model. When a fire target is detected, the system sends the target bounding box coordinates, category label, and confidence score to the backend. The backend, combined with the camera's installation location and geographical coordinates, calculates the approximate direction and location of the fire.

[0065] The drone is equipped with RTK GPS for high-precision positioning. Based on the approximate direction and location transmitted from the backend, the drone plans its route, enabling it to quickly reach the target area and autonomously hover or cruise according to mission instructions. Its onboard visible light camera captures the visual characteristics of flames, while an infrared camera detects the distribution of heat sources. The combination of these two technologies effectively eliminates interference from factors such as changes in ambient light and smoke obstruction in single-modal recognition. Through multimodal image fusion and deep learning discrimination, the system can more accurately distinguish between real fires and non-fire interference sources. After confirming a fire, it sends the precise coordinates of the fire point and surrounding environmental information to the backend, providing reliable support for subsequent firefighting and emergency response.

[0066] Secondly, the present invention provides a fire monitoring system based on a high-tower camera, the system comprising: The data acquisition module is used to acquire real-time video streams and preprocess the video streams to obtain fire candidate areas; The prediction module is used to input fire candidate regions into a pre-trained recognition network model to obtain the predicted region, category, and confidence score of the fire target. The recognition network model includes a backbone network, a feature fusion layer, and a detection head. The backbone network extracts features from the fire candidate regions to obtain multi-level feature maps. The feature fusion layer fuses the multi-level feature maps to obtain an enhanced feature map. The detection head outputs the predicted bounding box, category label, and confidence score of the fire target based on the enhanced feature map. The verification module is used to perform secondary confirmation of the predicted area using a drone when the category label is flame or smoke and the corresponding confidence level exceeds a preset threshold, so as to obtain the verified fire information.

[0067] For example, a high-definition camera and an edge computing server were installed on an existing 50-meter-high communication tower in a forest area. The camera is connected to a remote monitoring center via the tower's built-in network.

[0068] As the camera continued scanning, it captured what appeared to be smoke in the distance. The edge server immediately analyzed the video and, through image enhancement and motion detection, identified a suspicious area that was expanding.

[0069] The system uses a recognition network model to identify suspicious areas. This model is specifically optimized for detecting thin, elongated smoke. The model quickly returned the results: target = smoke, confidence level = 92%.

[0070] Because the confidence level far exceeded the preset 75% alarm threshold, the system automatically initiated emergency procedures. The monitoring center immediately instructed the drone airport deployed at the tower base. The drone, equipped with dual-light cameras, automatically flew to the target area based on the approximate location provided by the system. Upon arrival, the drone confirmed the high-temperature fire point using an infrared thermal imager and captured images of the open flames using a visible light camera. Simultaneously, using high-precision positioning technology, it obtained the precise latitude and longitude coordinates of the fire point. The drone packaged information such as "confirmed fire situation, precise coordinates, and on-site images" and transmitted it back to the monitoring center's large screen in real time, providing valuable time for early firefighting.

[0071] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this invention can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0072] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present invention.

Claims

1. A fire monitoring method based on a high-tower camera, characterized in that, include Acquire real-time video streams and preprocess the video streams to obtain fire candidate areas; The fire candidate region is input into a pre-trained recognition network model to obtain the predicted region, category, and confidence score of the fire target. The recognition network model includes a backbone network, a feature fusion layer, and a detection head. The backbone network extracts features from the fire candidate region to obtain a multi-level feature map. The feature fusion layer fuses features from the multi-level feature map to obtain an enhanced feature map. The detection head outputs the predicted bounding box, category label, and confidence score of the fire target based on the enhanced feature map. When the category label is flame or smoke, and the corresponding confidence level exceeds a preset threshold, the predicted area is reconfirmed using a drone to obtain verified fire information.

2. The fire monitoring method based on a high-tower camera according to claim 1, characterized in that, The preprocessing of the video stream to obtain fire candidate regions includes: The video stream is converted from the RGB color space to the YCbCr and HSV color spaces to separate luminance and color features, resulting in the converted video stream. Histogram equalization is performed on the luminance channel of the converted video stream to enhance contrast, resulting in the processed luminance channel. In the processed luminance channel, the motion region is extracted by the inter-frame difference method, and the pixel region whose motion amplitude exceeds the preset threshold is identified as the fire candidate region.

3. The fire monitoring method based on a high-tower camera according to claim 1, characterized in that, The backbone network includes: an input convolutional layer, a first-level strip convolutional module, a first downsampling layer, a second-level strip convolutional module, a second downsampling layer, a third-level strip convolutional module, a third downsampling layer, a fourth-level strip convolutional module, a spatial pyramid pooling module, and an attention module. The input convolutional layer is used to perform preliminary feature extraction on the fire candidate region to obtain an initial feature map; The first-level strip convolution module is used to extract slender features from the initial feature map to obtain primary slender morphological features; The first downsampling layer is used to downsample the primary elongated morphological features to obtain a first downsampled feature map; The second-level strip convolution module is used to extract the elongated features of the middle layer from the first downsampled feature map to obtain the elongated morphological features of the middle layer. The second downsampling layer is used to downsample the intermediate elongated morphological features to obtain a second downsampled feature map; The third-level strip convolution module is used to perform deep slender feature extraction on the second downsampled feature map to obtain three-level slender morphological features. The third downsampling layer is used to downsample the three-level elongated morphological features to obtain a third downsampling feature map; The fourth-level strip convolution module is used to perform the final long feature extraction on the third downsampled feature map to obtain the final slender morphological features; The spatial pyramid pooling module is used to fuse multi-scale receptive field information from the primary slender morphological features, the intermediate slender morphological features, the tertiary slender morphological features, and the final slender morphological features, and output a fused feature map. The attention module is used to calculate the feature weights of different spatial locations in the fused feature map through a polarization self-attention mechanism, and perform weighted summation to obtain a multi-level feature map.

4. The fire monitoring method based on a high-tower camera according to claim 3, characterized in that, The first-level strip convolution module includes, in sequence: a first convolutional layer, a segmentation layer, multiple parallel strip building blocks, and a feature concatenation layer; The first convolutional layer is used to perform convolution processing on the initial feature map and output the first feature map; The segmentation layer is used to segment the first feature map into multiple parallel branches to obtain multiple segmented feature maps. The plurality of cascaded strip building blocks are used to perform strip pooling operations on the segmented feature map to obtain an output feature map; The feature splicing layer is used to splice the first feature map with the output feature map to obtain a primary slender morphological feature.

5. The fire monitoring method based on a high-tower camera according to claim 4, characterized in that, The strip building block comprises, in sequence: a first 1x1 convolutional layer, a strip pooling layer, a second 1x1 convolutional layer, and a residual summing unit; The first 1x1 convolutional layer is used to perform dimensionality reduction processing on the segmentation feature map and output a first dimensionality-reduced feature map. The strip pooling layer is used to perform horizontal strip pooling and vertical strip pooling on the first dimensionality reduction feature map, and add the pooling results in the two directions to output a fused pooled feature map. The second 1x1 convolutional layer is used to perform dimensionality upscaling on the fused pooling feature map, restoring its channel count to the same as the input feature map of the strip building block, and outputting a second feature map; The residual addition unit is used to add the second feature map to the input feature map of the strip building block element by element, and output the final output feature map of the strip building block.

6. The fire monitoring method based on a high-tower camera according to claim 3, characterized in that, The attention module includes parallel channel attention branches and spatial attention branches, as well as a feature weighted fusion unit; The channel attention branch is used to perform global average pooling on the fused feature map and output a channel statistical vector; the channel statistical vector is then input into a multilayer perceptron to calculate the channel attention weights. The calculated channel attention weights are multiplied channel by channel with the input fusion feature map to output channel-enhanced features; The spatial attention branch is used to perform horizontal strip pooling and vertical strip pooling on the fused feature map, and then concatenate and convolve the pooled features to calculate the spatial attention weights. The spatial attention weights are multiplied pixel-by-pixel with the fused feature map to output spatial enhancement features; The feature weighted fusion unit is used to add and fuse the channel enhancement features and the spatial enhancement features; generate the final comprehensive attention weight map through the Sigmoid activation function; and perform a weighted multiplication of the comprehensive attention weight map with the initially input fused feature map to output a multi-level feature map.

7. The fire monitoring method based on a high-tower camera according to claim 1, characterized in that, The loss function of the recognition network model is expressed as: ； in, Indicates the first weighting coefficient; This represents the second weighting coefficient; This represents the third weighting coefficient; Indicates classification loss; Indicates the target confidence loss; This represents the bounding box regression loss; This represents the loss of the recognition network model.

8. The fire monitoring method based on a high-tower camera according to claim 1, characterized in that, When the category label is flame or smoke, and the corresponding confidence level exceeds a preset threshold, a drone is used to perform secondary confirmation of the predicted area to obtain verified fire information, including: Based on the predicted bounding box coordinates, category labels, and confidence scores output by the recognition network model, and combined with the geographical coordinates of the deployed cameras, the orientation and distance of the fire candidate area relative to the cameras are calculated. Based on orientation and distance, as well as the category label and confidence level, the flight path and confirmation mission strategy of the UAV are planned; wherein, if the category label is smoke and the confidence level is low, a wide-area cruise path is planned; if the category label is flame and the confidence level is high, a direct path is planned and the priority of infrared camera acquisition is increased. The drone is controlled to fly along the flight path to the target area and acquire dual-modal image data collected by the infrared camera; The dual-modal image data is fused and analyzed to distinguish between real fires and non-fire interference sources. After confirming the fire, the precise coordinates of the fire point and environmental information are sent to the backend in conjunction with the high-precision positioning data of the UAV.

9. A fire monitoring system based on a high-tower camera, characterized in that, include: The data acquisition module is used to acquire real-time video streams and preprocess the video streams to obtain fire candidate areas; The prediction module is used to input the fire candidate region into a pre-trained recognition network model to obtain the predicted region, category, and confidence score of the fire target. The recognition network model includes a backbone network, a feature fusion layer, and a detection head. The backbone network is used to extract features from the fire candidate region to obtain a multi-level feature map. The feature fusion layer is used to fuse features from the multi-level feature map to obtain an enhanced feature map. The detection head is used to output the predicted bounding box, category label, and confidence score of the fire target based on the enhanced feature map. The verification module is used to perform secondary confirmation of the predicted area using a drone when the category label is flame or smoke and the corresponding confidence level exceeds a preset threshold, so as to obtain the verified fire information.

Images