Forest fire detection method based on two-channel time series data suitable for edge calculation

Abstract

The invention discloses a forest fire detection method based on two-channel time series data suitable for edge calculation, and the method comprises the steps: obtaining an original RGB image and a corresponding infrared image when a forest fire occurs, carrying out the normalization processing of the infrared image, obtaining a normalized infrared image, carrying out the smoothing processing of the normalized infrared image through Gaussian filtering, and obtaining a final image. Performing smoothing on the infrared image to obtain a smoothed infrared image, performing time domain statistical feature extraction processing on the smoothed infrared image to obtain a reconstructed infrared image, selecting a static light source and removing the static light source to obtain an infrared image without the static light source, and constructing a third image set; and constructing a novel Forest-fire structure model, inputting the third image set into the novel Forest-fire structure model to obtain the position of a fire point, and completing forest fire detection. According to the method, the missing report rate and the false report rate are remarkably reduced, and a better effect is achieved when the method is operated on the edge equipment with limited resources.

Description

Technical Field

[0001] This invention relates to the field of forest fire detection technology, specifically to a forest fire detection method using dual-channel time-series data suitable for edge computing. Background Technology

[0002] Forest fire detection is a technology that utilizes sensors, visual technology, infrared imaging, satellite remote sensing, and other methods to identify and assess the occurrence of forest fires in their early stages. Its goal is to quickly detect fires and issue early warnings through real-time data acquisition and processing (such as changes in smoke, flames, gas concentration, or temperature), thereby protecting forest resources, reducing fire losses, and safeguarding the ecological environment and human safety. With the development of artificial intelligence, edge computing, and the Internet of Things, forest fire detection is gradually evolving towards multimodal data fusion, intelligent algorithm optimization, and global collaboration, providing strong support for intelligent forest management. For example, a forest fire detection system suitable for edge computing can use various types of data collected by sensors to achieve rapid identification of forest fires. The system first collects historical sensor data, including RGB video and infrared video. Then, it uses this data to train, for example, a convolutional neural network model based on CNN, to learn image features and fine-grained location information from the data. Ultimately, the forest fire detection system can quickly identify the occurrence of forest fires, helping to reduce the losses and damage caused by them.

[0003] Currently, a significant issue in forest fire detection is the need for massive amounts of data, including various types of visual data, to enable rapid detection. However, inconsistent data quality leads to a shortage of high-quality data samples, ultimately affecting the accuracy and timeliness of detection models. Three key engineering problems currently exist:

[0004] (1) Forest fires typically have a long duration, potentially lasting tens of hours from the initial spark to the formation of significant smoke and open flames. During this process, traditional equidistant sampling methods are not suitable for the detection and analysis of forest fires. Therefore, in the context of long-term monitoring, how to efficiently process and analyze massive amounts of data has become an urgent engineering and technical challenge.

[0005] (2) The limited detection perspective and insufficient data samples are challenges in forest fire detection, especially for forest fires, which are rare but significant events. This limitation may lead to insufficient training data for the model, thereby affecting the model's generalization performance, detection accuracy and real-time performance, and significantly restricting its effectiveness and reliability in practical applications.

[0006] (3) At night, large light sources in cities surrounding forests are easily misreported as smoldering fires. Such frequent false alarms may weaken the reliability of forest fire detection models, reduce the ability to identify real forest fire signs, and thus affect the efficiency of emergency response. This problem has become another important engineering challenge in the design of fire detection systems.

[0007] Existing research on forest fire detection solutions focuses on continuously improving the speed of forest fire identification, but does not consider the three engineering problems mentioned above. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide a forest fire detection method for dual-channel time-series data suitable for edge computing, which solves three engineering problems faced by existing forest fire detection algorithms: limited data types, how to analyze data under long-term monitoring, and interference from urban light sources.

[0009] To solve the above technical problems, the present invention adopts the following technical solution:

[0010] A method for detecting wildfires using dual-channel time-series data suitable for edge computing includes the following steps:

[0011] S1. Obtain the original RGB image and its corresponding infrared image at the time of the forest fire, construct the original image set, normalize the infrared image of the image set to obtain the normalized infrared image, and construct the first image set.

[0012] S2. The normalized infrared image is smoothed by Gaussian filtering to obtain a smoothed infrared image, and a second image set is constructed.

[0013] S3. Perform temporal statistical feature extraction on the smoothed infrared image to obtain the reconstructed infrared image. Select and remove the static light source to obtain the infrared image after removing the static light source, and construct the third image set.

[0014] S4. Construct a new forest-fire structure model and train the model to obtain the trained new forest-fire structure model. Input the third image set into the trained new forest-fire structure model to obtain the location of the fire point and complete the forest fire detection.

[0015] Furthermore, in step S1, constructing the first image set includes the following:

[0016] S101. Obtain the original RGB images and corresponding infrared images of a forest fire at a set location and angle, captured by a camera of a set type. Represent the original image set as follows:

[0017]

[0018] Where D represents the original image set, This represents the original RGB image at time 1 and viewpoint i. This represents the original RGB image at time 2 and viewpoint i. This represents the original RGB image from the first perspective at time t. This represents the original RGB image from the i-th viewpoint at time t. This represents the infrared image corresponding to the original RGB image at time 1 and viewpoint i. This represents the infrared image corresponding to the original RGB image at time 2 and viewpoint i. This represents the infrared image corresponding to the original RGB image from the first perspective at time t. This represents the infrared image corresponding to the original RGB image at time t and viewpoint i.

[0019] S102. Normalize the pixel values ​​of the infrared image corresponding to the original RGB image to obtain the normalized infrared image. The specific expression is as follows:

[0020]

[0021] in, Let represent the pixel in row a of the infrared image corresponding to the original RGB image at time t and viewpoint i. I represents the pixel in column b of the infrared image corresponding to the original RGB image at time t and viewpoint i. norm () represents the normalized pixel value, and I() represents the pixel value of the infrared image corresponding to the original RGB image. min I represents the minimum value of the infrared image pixels corresponding to the original RGB image. max This represents the maximum value of the infrared image pixels corresponding to the original RGB image.

[0022] The first image set, consisting of the original RGB image and the normalized infrared image, is represented as:

[0023]

[0024] Where D1 represents the first image set, Y t i Let represent the normalized infrared image at time t and viewpoint i.

[0025] Furthermore, in step S2, constructing the second image set includes the following:

[0026] Using a Gaussian kernel of size (2k+1)×(2k+1) with center point (c,d), the normalized infrared image is smoothed to obtain the smoothed infrared image. The specific expression is as follows:

[0027]

[0028] Among them, Output smooth () represents the smoothed pixel value; F represents the pixel in the E-th row of the infrared image after normalization of the i-th viewpoint at time t; t i Let f(m,n) represent the pixel in the F-th column of the normalized infrared image at time t with viewpoint i; Kernel(m,n) represents the Gaussian kernel. m represents the horizontal offset of the Gaussian kernel, n represents the vertical offset of the Gaussian kernel, k represents the size of the convolution kernel, and σ represents the standard deviation of the Gaussian function.

[0029] The second image set, consisting of the original RGB image and the smoothed infrared image, is represented as follows:

[0030]

[0031] Where D2 represents the second image set, This represents the smoothed infrared image at time 1 and viewpoint i. This represents the smoothed infrared image at time 2 and viewpoint i. This represents the smoothed infrared image with the first viewpoint at time t. Let represent the smoothed infrared image at time t and viewpoint i.

[0032] Furthermore, in step S3, constructing the third image set includes the following:

[0033] S301. Using time-domain mean calculation and variance analysis techniques, the smoothed infrared image is processed to extract temporal statistical features, resulting in the reconstructed infrared image. The specific formula is as follows:

[0034]

[0035] in, Let represent the mean value of the pixels at the i-th viewpoint over the time dimension. Let A represent the pixel in the A-th row of the smoothed infrared image at time t with viewpoint i. Let represent the pixel in column B of the smoothed infrared image at time t with viewpoint i, where T represents the total number of frames at time t. σ represents the pixel value of the infrared image smoothed at time t and viewpoint i. i2() represents the time variance of the i-th viewpoint.

[0036] S302. In the reconstructed infrared image, regions with a temporal variance lower than 20% of the overall temporal variance mean are marked as static light sources.

[0037] A fixed threshold is set to extract spatial features from a static light source, resulting in the static light source features. The specific formula is as follows:

[0038]

[0039] Where M() represents a binary mask, T threshold This represents the preset pixel value threshold, A represents the row coordinate of the smoothed infrared image pixel, and B represents the column coordinate of the smoothed infrared image pixel.

[0040] S303. Replace the pixel value of the static light source with the average value of the neighboring pixels, using the following formula:

[0041]

[0042] Among them, I adjusted () represents the adjusted pixel value of the infrared image, e represents the row coordinate of the neighboring pixel, and f represents the column coordinate of the neighboring pixel. This represents the set of neighboring pixels within a 5×5 area around a static light source pixel. This represents the total number of pixels in the neighborhood, and Is() represents the pixel value of the smoothed infrared image.

[0043] S304. Use bilinear interpolation to repair the blank areas generated after replacing the static light source. The weighted combination of the distances of four neighboring pixels is used as the pixel value of the smoothed infrared image to obtain the infrared image after removing the static light source. The specific expression is:

[0044]

[0045] Among them, I reconstructed () represents the pixel value after removing static light source. This represents the weight coefficient of the A-th row of pixels in the smoothed infrared image at time t with viewpoint i. These are the weight coefficients of the B-th column pixels in the smoothed infrared image at time t and viewpoint i. Let A1 represent the pixel in the smoothed infrared image at time t with viewpoint i. This represents the pixel in column B1 of the smoothed infrared image at time t with viewpoint i. Let A2 represent the pixel in the smoothed infrared image at time t with viewpoint i. This represents the pixel in column B2 of the smoothed infrared image at time t with viewpoint i.

[0046] The third image set, consisting of the original RGB image and the infrared image after removing static light sources, is represented as:

[0047]

[0048] Where D3 represents the third image set, Yf1 i This represents the infrared image at time 1 and viewpoint i after eliminating static light sources. Yf represents the infrared image at time 2, viewed from the i-th perspective, after eliminating static light sources. t 1 Yf represents the infrared image at time t from the first perspective after eliminating static light sources. t i This represents the infrared image at time t, viewed from the i-th perspective, after eliminating static light sources.

[0049] Furthermore, in step S4, the location of the fire point is obtained including the following:

[0050] The novel Forest-fire architecture model consists of two improved VGG16 modules, a sparse attention mechanism module, and a long short-term memory network module connected in sequence.

[0051] The improved VGG16 includes three convolutional blocks, each containing a set number of 3x3 convolutional kernels and DSConv2D convolutional layers with a stride of 1.

[0052] The third image set was divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The training set was used to train the novel Forest-fire structure model, resulting in the trained novel Forest-fire structure model.

[0053] The test set is input into the trained novel Forest-fire architecture model, and features are extracted twice using two improved VGG16 methods: once from the original RGB image and once from the infrared image after removing static light sources. The specific formula is as follows:

[0054]

[0055] in, This represents the feature map of the original RGB image at time t and viewpoint i. This represents the feature map of the infrared image at time t after eliminating the static light source from the i-th viewpoint.

[0056] Will and The dual-channel data is fused to generate a new feature map. The specific formula is as follows:

[0057]

[0058] in, Let α represent the new feature map of the i-th viewpoint at time t. The weight, β represents The weights are α+β=1.

[0059] The new feature map is processed by the sparse attention mechanism module, which generates a spatial attention weight matrix through sparse attention in the spatial dimension and generates an importance weight vector for each channel through sparse attention in the channel dimension. The spatial attention weight matrix and the importance weight vector are then fused and integrated to obtain the corresponding feature representation.

[0060] The corresponding feature representations, processed by the Long Short-Term Memory (LSTM) network module, capture the dependencies between time-series data to determine the location of the fire point. The specific formula is as follows:

[0061]

[0062]

[0063] c t =f t ·c t-1 +j t ·tanh(W c x t +U c h t-1 )+b c

[0064] h t =o t ·tanh(c t )

[0065] Among them, f t Let W represent the activation vector of the forget gate at time t, Sg() represent the sigmoid activation function, and W f W j W o W c These represent the forget gate, input gate, output gate, and input kernel of the memory cell, respectively. U f U j U o U c The recurrent kernel, h, represents the forget gate, input gate, output gate, and memory cell. t-1 Let b represent the activation vector of the hidden state at time t-1. f b j b o b cThese represent the biases of the forgetting gate, input gate, output gate, and memory cells, respectively. t Let o represent the activation vector of the input gate at time t. t c represents the activation vector of the output gate at time t. t c represents the activation vector of the memory cell at time t. t-1 Let x represent the activation vector of the memory cell at time t-1, tanh() represent the hyperbolic tangent activation function, and x t Let h represent the input vector. t Let represent the activation vector of the hidden state at time t.

[0066] Furthermore, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the forest fire detection method for dual-channel time-series data suitable for edge computing.

[0067] Furthermore, the present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the described forest fire detection method for dual-channel time-series data suitable for edge computing.

[0068] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0069] (1) Improve detection accuracy: Based on the dual-channel fusion technology of RGB image and infrared image, this invention effectively combines the texture features of visible light image and the thermal radiation features of infrared image. It achieves an mAP50 index of 77.4% on the FIGLib dataset, which is 4.6 percentage points higher than the existing YOLOV9E model, and significantly reduces the false alarm rate and false alarm rate.

[0070] (2) Enhanced time series analysis capability: The present invention adopts an improved Forest-fire structure model, which can capture the dynamic characteristics of fire development and has higher sensitivity to weak signals in the early stage of fire, thus realizing early detection of forest fire.

[0071] (3) Solving the problem of static light source interference: The present invention can effectively identify and filter the interference of fixed heat sources such as urban light sources through the static light source processing module. Experiments show that the false alarm rate in nighttime scenes is reduced from 34.2% to 8.7%, which improves the reliability in complex environments.

[0072] (4) Reduced computational complexity: This invention uses a sparse attention mechanism to replace the traditional self-attention mechanism, which reduces computational complexity while maintaining the model's expressive power, and achieves good results on resource-constrained edge devices. Attached Figure Description

[0073] Figure 1 This is a flowchart illustrating the overall implementation of the present invention.

[0074] Figure 2 This is a structural diagram of the improved VGG16 in the Forest-fire architecture of this invention.

[0075] Figure 3 This is a structural diagram of the novel Forest-fire structural model of the present invention.

[0076] Figure 4 This is a result diagram of an embodiment of the present invention.

[0077] Figure 5 This is a comparison diagram of different methods used for forest fire detection in embodiments of the present invention. Detailed Implementation

[0078] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0079] To achieve the above objectives, this invention proposes a forest fire detection method for dual-channel time-series data suitable for edge computing, such as... Figure 1 As shown, the specific steps are as follows:

[0080] S1. Obtain the original RGB image and its corresponding infrared image at the time of the forest fire, construct the original image set, normalize the infrared images in this image set to obtain normalized infrared images, and construct the first image set; Specific content includes:

[0081] S101. Obtain the FigLib Dataset, which includes raw RGB images and their corresponding infrared images of forest fires taken by different types of cameras at various locations and angles on a remote mountaintop in Southern California, providing comprehensive and diverse information. As of December 2021, the FigLib Dataset consists of 315 fire sequences from 101 cameras at 30 sites between June 2016 and July 2021. Each sequence typically contains images 40 minutes before and 40 minutes after the fire starts, serving as binary smoke / no-smoke labels for each image, spaced approximately 60 seconds apart, with a total of 81 images per forest fire sequence. In total, the dataset contains 24,800 high-resolution images. Fire detection and pre-fire observation were achieved through a camera cluster deployment, with four 90-degree or higher perspectives maintaining consistency over many years, covering 360 degrees of the target forest.

[0082] The original image set, composed of the original images, is represented as:

[0083]

[0084] Where D represents the original image set, This represents the original RGB image at time 1 and viewpoint i. This represents the original RGB image at time 2 and viewpoint i. This represents the original RGB image from the first perspective at time t. This represents the original RGB image from the i-th viewpoint at time t. This represents the infrared image corresponding to the original RGB image at time 1 and viewpoint i. This represents the infrared image corresponding to the original RGB image at time 2 and viewpoint i. This represents the infrared image corresponding to the original RGB image from the first perspective at time t. This represents the infrared image corresponding to the original RGB image at time t and viewpoint i.

[0085] S102. Normalize the pixel values ​​of the infrared image corresponding to the original RGB image, adjust the overall brightness level of the image to ensure that all parts of the infrared image have similar brightness levels, and obtain the normalized infrared image. The specific expression is:

[0086]

[0087] in, Let represent the pixel in row a of the infrared image corresponding to the original RGB image at time t and viewpoint i. I represents the pixel in column b of the infrared image corresponding to the original RGB image at time t and viewpoint i. norm () represents the normalized pixel value, and I() represents the pixel value of the infrared image corresponding to the original RGB image. min I represents the minimum value of the infrared image pixels corresponding to the original RGB image. max This represents the maximum value of the infrared image pixels corresponding to the original RGB image.

[0088] The first image set, consisting of the original RGB image and the normalized infrared image, is represented as:

[0089]

[0090] Where D1 represents the first image set, Y t i Let represent the normalized infrared image at time t and viewpoint i.

[0091] S2. In forest environments, infrared cameras may be affected by changes in weather conditions or equipment vibration. Therefore, Gaussian filtering is used to smooth the normalized infrared images, removing noise and smoothing local data to obtain smoothed infrared images, which are then used to construct a second image set; the specific content is as follows:

[0092] Using a Gaussian kernel of size (2k+1)×(2k+1) with center point (c,d), the normalized infrared image is smoothed to obtain the smoothed infrared image. The specific expression is as follows:

[0093]

[0094] Among them, Output smooth () represents the smoothed pixel value; F represents the pixel in the E-th row of the infrared image after normalization of the i-th viewpoint at time t; t i Let f(m,n) represent the pixel in the F-th column of the normalized infrared image at time t with viewpoint i; Kernel(m,n) represents the Gaussian kernel. m represents the horizontal offset of the Gaussian kernel, n represents the vertical offset of the Gaussian kernel, k represents the size of the convolution kernel, and σ represents the standard deviation of the Gaussian function, which controls the smoothness (a larger σ value will produce a smoother effect).

[0095] In this embodiment, a window size of 7×7 is selected for smoothing.

[0096] The second image set, consisting of the original RGB image and the smoothed infrared image, is represented as follows:

[0097]

[0098] Where D2 represents the second image set, This represents the smoothed infrared image at time 1 and viewpoint i. This represents the smoothed infrared image at time 2 and viewpoint i. This represents the smoothed infrared image with the first viewpoint at time t. Let represent the smoothed infrared image at time t and viewpoint i.

[0099] S3. The smoothed infrared image is processed using mean calculation and variance analysis to extract temporal statistical features, resulting in a reconstructed infrared image. Static light sources are selected and removed, yielding the infrared image after removing the static light source. A third image set is then constructed. The specific content is as follows:

[0100] S301. Using time-domain mean calculation and variance analysis techniques, the smoothed infrared image is processed to extract temporal statistical features, resulting in the reconstructed infrared image. The specific formula is as follows:

[0101]

[0102] in, Let represent the mean value of the pixels at the i-th viewpoint over the time dimension. Let A represent the pixel in the A-th row of the smoothed infrared image at time t with viewpoint i. Let represent the pixel in column B of the smoothed infrared image at time t with viewpoint i, where T represents the total number of frames at time t. σ represents the pixel value of the infrared image smoothed at time t and viewpoint i. i2 () represents the time variance of the i-th viewpoint.

[0103] S302. In the reconstructed infrared image, regions with a temporal variance lower than 20% of the overall temporal variance mean are marked as static light sources.

[0104] A fixed threshold is set to extract spatial features from a static light source, resulting in the static light source features. The specific formula is as follows:

[0105]

[0106] Where M() represents a binary mask, T threshold This represents the preset pixel value threshold, A represents the row coordinate of the smoothed infrared image pixel, and B represents the column coordinate of the smoothed infrared image pixel.

[0107] In this embodiment, T threshold =180.

[0108] S303. The thermal radiation characteristics of static light sources significantly interfere with fire detection systems in infrared images, becoming a major source of false alarms and severely impacting system reliability and detection accuracy. Therefore, the average value of neighboring pixels is used to replace the pixel value of the static light source to eliminate this interference, thereby improving the ability to identify actual forest fires, enhancing the adaptability of the fire detection system in complex environments, and improving emergency response efficiency and detection accuracy. The specific formula is as follows:

[0109]

[0110] Among them, I adjusted () represents the adjusted pixel value of the infrared image, e represents the row coordinate of the neighboring pixel, and f represents the column coordinate of the neighboring pixel. This represents the set of neighboring pixels within a 5×5 area around a static light source pixel. This represents the total number of pixels in the neighborhood, and Is() represents the pixel value of the smoothed infrared image.

[0111] S304. Use bilinear interpolation to repair blank or discontinuous areas created after replacing the static light source. The weighted combination of the distances of four neighboring pixels is used as the pixel value of the smoothed infrared image to obtain the infrared image after removing the static light source, restoring the image's integrity and visual coherence. The specific expression is as follows:

[0112]

[0113] Among them, I reconstructed () represents the pixel value after removing static light source. This represents the weight coefficient of the A-th row of pixels in the smoothed infrared image at time t with viewpoint i. These are the weight coefficients of the B-th column pixels in the smoothed infrared image at time t and viewpoint i. Let A1 represent the pixel in the smoothed infrared image at time t with viewpoint i. This represents the pixel in column B1 of the smoothed infrared image at time t with viewpoint i. Let A2 represent the pixel in the smoothed infrared image at time t with viewpoint i. This represents the pixel in column B2 of the smoothed infrared image at time t with viewpoint i.

[0114] and It reflects the distance between the target pixel and its neighboring pixels; the closer the distance, the greater the weight.

[0115] The third image set, consisting of the original RGB image and the infrared image after removing static light sources, is represented as:

[0116]

[0117] Where D3 represents the third image set, Yf1 i This represents the infrared image at time 1 and viewpoint i after eliminating static light sources. Yf represents the infrared image at time 2, viewed from the i-th perspective, after eliminating static light sources. t 1 Yf represents the infrared image at time t from the first perspective after eliminating static light sources. t i This represents the infrared image at time t, viewed from the i-th perspective, after eliminating static light sources.

[0118] S4. Construct a novel forest-fire structure model and train it to obtain the trained novel forest-fire structure model. Input the third image set into the trained novel forest-fire structure model to obtain the location of fire points and complete the forest fire detection; the specific content is as follows:

[0119] The novel Forest-fire architecture model consists of two improved VGG16 modules, a sparse attention mechanism module, and an LSTM (Long-Short Term Memory) module connected in sequence.

[0120] like Figure 2 As shown, the improved VGG16 includes three convolutional blocks, each containing 16 DSConv2D convolutional layers with 3x3 kernels and a stride of 1. In the improved VGG16, the input is a 224*224*3 three-channel image, which needs to be converted to a 224*224*1 single-channel 110-channel image when processing infrared images.

[0121] The third image set was divided into training, validation, and test sets, with 80% of the data used for training, 10% for validation, and 10% for testing. The novel Forest-fire structure model was trained using the training set, and stochastic gradient descent was used to update the model weights to determine the optimal learning rate. The loss function and optimizer settings were continuously adjusted to obtain the trained novel Forest-fire structure model.

[0122] Among them, the most suitable learning rate l R The expression is:

[0123]

[0124] Among them, l r i represents the initial learning rate. pr Indicates the current iteration number, i max This represents the maximum number of iterations, and power represents the initial momentum.

[0125] In this embodiment, l r =0.001, power=0.75.

[0126] like Figure 3 As shown, the test set is input into the trained novel Forest-fire architecture model. Features are extracted twice using two improved VGG16 methods: once from the original RGB image and once from the infrared image after removing static light sources. The specific formula is as follows:

[0127]

[0128] in, This represents the feature map of the original RGB image at time t and viewpoint i. This represents the feature map of the infrared image at time t after eliminating the static light source from the i-th viewpoint.

[0129] Will and The dual-channel data is fused to generate a new feature map. The specific formula is as follows:

[0130]

[0131] in, Let α represent the new feature map of the i-th viewpoint at time t. The weight, β represents The weights are α = β = 0.5.

[0132] The new feature map is processed by a sparse attention mechanism module. A spatial attention weight matrix is ​​generated through sparse attention in the spatial dimension, which highlights important spatial locations related to fire in the feature map. Importance weight vectors for each channel are generated through sparse attention in the channel dimension, prioritizing the retention of channels with stronger fire feature information. The spatial attention weight matrix and importance weight vectors are fused and integrated to obtain a feature representation enhanced by the attention mechanism. This results in a more discriminative feature representation, effectively improving the model's ability to perceive and detect forest fire features.

[0133] The feature representation, after passing through the LSTM module, captures the dependencies between time series data to obtain the location of the fire point. The specific formula is as follows:

[0134]

[0135] c t =f t ·c t-1 +j t ·tanh(W c x t +U c h t-1 )+b c

[0136] h t =o t ·tanh(c t )

[0137] Among them, f t Let W represent the activation vector of the forget gate at time t, Sg() represent the sigmoid activation function, and W f W j Wo W c These represent the forget gate, input gate, output gate, and input kernel of the memory cell, respectively. U f U j U o U c The recurrent kernel, h, represents the forget gate, input gate, output gate, and memory cell. t-1 Let b represent the activation vector of the hidden state at time t-1. f b j b o b c These represent the biases of the forgetting gate, input gate, output gate, and memory cells, respectively. t Let o represent the activation vector of the input gate at time t. t c represents the activation vector of the output gate at time t. t c represents the activation vector of the memory cell at time t. t-1 Let x represent the activation vector of the memory cell at time t-1, tanh() represent the hyperbolic tangent activation function, and x t Let h represent the input vector. t Let represent the activation vector of the hidden state at time t.

[0138] The novel forest-fire structural model can effectively combine the spatiotemporal features of RGB and infrared images, while taking into account multi-view information, to achieve accurate detection of forest fires, especially in the early stages of forest fires, with high detection sensitivity and reliability.

[0139] In practical applications, the system can also generate alarm signals and location information based on the detection results, and transmit them to the monitoring center in real time through edge computing devices to achieve early warning and rapid response to forest fires.

[0140] Figure 4 The top left corner is a red heatmap showing the distribution of fire instances, indicating that the selected dataset contains a large number of fire samples. Figure 4 The upper right corner is a set of overlapping rectangles with red borders, presenting a center-focused pattern, representing the bounding box of fire detection, which shows the distribution of the area of ​​interest of the method proposed in this invention. Figure 4 The bottom left corner shows the spatial distribution of the fire points. The density variation of the blue dots indicates the frequency of fire points appearing at different locations, with darker colors indicating higher frequencies. Figure 4 The bottom right corner shows the size distribution of the fire detection frames. It can be seen that most of the detection frames are concentrated in the smaller size area (0.1-0.2), indicating that the detection area of ​​most fire points is small.

[0141] To verify the advantages of the method proposed in this invention, the proposed method, Faster-RCNN (Faster Region-based Convolutional Neural Network), YOLOv5L (You Only Look Once version 5 Large), YOLOv8L (You Only Look Once version 8 Large), and YOLOv9E (You Only Look Once version 9 Enhanced) were selected to detect forest fires. The results are shown in Table 1.

[0142] Table 1 Results of different methods used to detect forest fires

[0143] method mAP50 (%) Number of parameters (M) computational load Faster-RCNN 63.7 48.28 1533.85 YOLOV5L 67.3 46.50 105.11 YOLOV8L 69.8 43.70 121.00 YOLOV9E 72.8 57.30 238.60 The method proposed in this invention 77.4 61.30 259.30

[0144] In Table 1, mAP50 represents the average accuracy value when the cross-union threshold is 50%. The higher the value of mAP50, the better the detection effect, indicating that the method proposed in this invention has higher accuracy, better effect, and is more suitable for application in complex scenarios.

[0145] To provide a more intuitive view of the detection results, the results of forest fire detection using the method proposed in this invention, Faster-RCNN, YOLOV5L, YOLOV8L, and YOLOV9E are presented in images.

[0146] from Figure 5 As can be seen, Faster-RCNN can identify fire areas, but its detection accuracy decreases under certain ambient light conditions (such as low light or long distance). YOLOv5L's bounding boxes are relatively accurate, but its robustness is insufficient in complex backgrounds or nighttime environments. YOLOv8L's detection performance under different lighting conditions is improved compared to YOLOv5L, but it still has some missed detections, especially in distant fire areas. YOLOv9E is relatively accurate in detecting fire areas, especially in clear daytime images, where the bounding box accuracy is high, but its detection performance decreases in poor lighting nighttime environments. The method proposed in this invention maintains high detection accuracy in various environments (including nighttime and long distance), especially for fire points in complex environments. Therefore, the method proposed in this invention performs better than other traditional methods in complex backgrounds and nighttime environments, providing higher detection accuracy, especially with significant advantages in long distance and low light conditions.

[0147] This invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. It should be noted that when the processor executes the computer program, it corresponds to the specific steps of the method provided in this invention, possessing the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in this invention.

[0148] This invention also proposes a computer-readable storage medium storing a computer program. It should be noted that when the computer program is executed by a processor, it corresponds to the specific steps of the method provided in this invention, possessing the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in this invention.

[0149] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting forest fires using dual-channel time-series data suitable for edge computing, characterized in that, include: S1. Obtain the original RGB image and its corresponding infrared image at the time of the forest fire, construct the original image set, normalize the infrared image of the image set to obtain the normalized infrared image, and construct the first image set. S2. Apply Gaussian filtering to smooth the normalized infrared image to obtain a smoothed infrared image, and construct a second image set. S3. Perform temporal statistical feature extraction on the smoothed infrared image to obtain the reconstructed infrared image. Select and remove the static light source to obtain the infrared image after removing the static light source, and construct the third image set. S4. Construct a new forest-fire structure model and train the model to obtain the trained new forest-fire structure model. Input the third image set into the trained new forest-fire structure model to obtain the location of the fire point and complete the forest fire detection.

2. The forest fire detection method for dual-channel time-series data suitable for edge computing according to claim 1, characterized in that, In step S1, constructing the first image set includes the following: S101. Obtain the original RGB images and corresponding infrared images of a forest fire at a set location and angle, captured by a camera of a set type. Represent the original image set as follows: Where D represents the original image set, This represents the original RGB image at time 1 and viewpoint i. This represents the original RGB image at time 2 and viewpoint i. This represents the original RGB image from the first perspective at time t. This represents the original RGB image from the i-th viewpoint at time t. This represents the infrared image corresponding to the original RGB image at time 1 and viewpoint i. This represents the infrared image corresponding to the original RGB image at time 2 and viewpoint i. This represents the infrared image corresponding to the original RGB image from the first perspective at time t. This represents the infrared image corresponding to the original RGB image at time t and viewpoint i. S102. Normalize the pixel values ​​of the infrared image corresponding to the original RGB image to obtain the normalized infrared image. The specific expression is as follows: in, Let represent the pixel in row a of the infrared image corresponding to the original RGB image at time t and viewpoint i. I represents the pixel in column b of the infrared image corresponding to the original RGB image at time t and viewpoint i. norm () represents the normalized pixel value, and I() represents the pixel value of the infrared image corresponding to the original RGB image. min I represents the minimum value of the infrared image pixels corresponding to the original RGB image. max This represents the maximum value of the infrared image pixels corresponding to the original RGB image. The first image set, consisting of the original RGB image and the normalized infrared image, is represented as: Where D1 represents the first image set, Y t i Let represent the normalized infrared image at time t and viewpoint i.

3. The forest fire detection method for dual-channel time-series data suitable for edge computing according to claim 1, characterized in that, In step S2, constructing the second image set includes the following: Using a Gaussian kernel of size (2k+1)×(2k+1) with center point (c,d), the normalized infrared image is smoothed to obtain the smoothed infrared image. The specific expression is as follows: Among them, Output smooth () represents the smoothed pixel value; This represents the pixel in the E-th row of the infrared image after normalization of the i-th viewpoint at time t; Let f(m,n) represent the pixel in the F-th column of the normalized infrared image at time t with viewpoint i; Kernel(m,n) represents the Gaussian kernel. m represents the horizontal offset of the Gaussian kernel, n represents the vertical offset of the Gaussian kernel, k represents the size of the convolution kernel, σ represents the standard deviation of the Gaussian function, and I norm () represents the normalized pixel value; The second image set, consisting of the original RGB image and the smoothed infrared image, is represented as follows: Where D2 represents the second image set, This represents the original RGB image at time 1 and viewpoint i. This represents the original RGB image at time 2 and viewpoint i. This represents the original RGB image from the first perspective at time t. This represents the original RGB image from the i-th viewpoint at time t. This represents the smoothed infrared image at time 1 and viewpoint i. This represents the smoothed infrared image at time 2 and viewpoint i. This represents the smoothed infrared image with the first viewpoint at time t. Let represent the smoothed infrared image at time t and viewpoint i.

4. The forest fire detection method for dual-channel time-series data suitable for edge computing according to claim 1, characterized in that, In step S3, constructing the third image set includes the following: S301. Using time-domain mean calculation and variance analysis techniques, the smoothed infrared image is processed to extract temporal statistical features, resulting in the reconstructed infrared image. The specific formula is as follows: in, Let represent the mean value of the pixels at the i-th viewpoint over the time dimension. Let A represent the pixel in the A-th row of the smoothed infrared image at time t with viewpoint i. Let represent the pixel in column B of the smoothed infrared image at time t with viewpoint i, where T represents the total number of frames at time t. σ represents the pixel value of the infrared image smoothed at time t and viewpoint i. i2 () represents the time variance of the i-th viewpoint; S302. In the reconstructed infrared image, regions with a temporal variance lower than 20% of the overall temporal variance mean are marked as static light sources; A fixed threshold is set to extract spatial features from a static light source, resulting in the static light source features. The specific formula is as follows: Where M() represents a binary mask, T threshold This represents the preset pixel value threshold, where A represents the row coordinate of the smoothed infrared image pixel, and B represents the column coordinate of the smoothed infrared image pixel. S303. Replace the pixel value of the static light source with the average value of the neighboring pixels, using the following formula: Among them, I adjusted () represents the adjusted pixel value of the infrared image, e represents the row coordinate of the neighboring pixel, and f represents the column coordinate of the neighboring pixel. This represents the set of neighboring pixels within a 5×5 area around a static light source pixel. This represents the total number of pixels in the neighborhood, and Is() represents the pixel value of the smoothed infrared image. S304. Use bilinear interpolation to repair the blank areas generated after replacing the static light source. The weighted combination of the distances of four neighboring pixels is used as the pixel value of the smoothed infrared image to obtain the infrared image after removing the static light source. The specific expression is: Among them, I reconstructed () represents the pixel value after removing static light source. This represents the weight coefficient of the A-th row of pixels in the smoothed infrared image at time t with viewpoint i. These are the weight coefficients of the B-th column pixels in the smoothed infrared image at time t and viewpoint i. Let A1 represent the pixel in the smoothed infrared image at time t with viewpoint i. This represents the pixel in column B1 of the smoothed infrared image at time t with viewpoint i. Let A2 represent the pixel in the smoothed infrared image at time t with viewpoint i. This represents the pixel in column B2 of the smoothed infrared image at time t with viewpoint i. The third image set, consisting of the original RGB image and the infrared image after removing static light sources, is represented as: Where D3 represents the third image set, This represents the original RGB image at time 1 and viewpoint i. This represents the original RGB image at time 2 and viewpoint i. This represents the original RGB image from the first perspective at time t. Yf1 represents the original RGB image at time t and viewpoint i. i This represents the infrared image at time 1 and viewpoint i after eliminating static light sources. Yf represents the infrared image at time 2, viewed from the i-th perspective, after eliminating static light sources. t 1 Yf represents the infrared image at time t from the first perspective after eliminating static light sources. t i This represents the infrared image at time t, viewed from the i-th perspective, after eliminating the static light source.

5. The forest fire detection method for dual-channel time-series data suitable for edge computing according to claim 1, characterized in that, In step S4, the location of the fire point is obtained including the following: The novel Forest-fire architecture model consists of two improved VGG16 modules, a sparse attention mechanism module, and a long short-term memory network module connected in sequence. The improved VGG16 includes three convolutional blocks, each of which includes a set number of 3x3 convolutional kernels and DSConv2D convolutional layers with a stride of 1. The third image set was divided into a training set, a validation set, and a test set in a ratio of 8:1:

1. The training set was used to train the new Forest-fire structure model, and the trained new Forest-fire structure model was obtained. The test set is input into the trained novel Forest-fire architecture model, and features are extracted twice using two improved VGG16 methods: once from the original RGB image and once from the infrared image after removing static light sources. The specific formula is as follows: in, This represents the feature map of the original RGB image at time t and viewpoint i. This represents the feature map of the infrared image at time t after eliminating static light sources from the i-th viewpoint. Will and The dual-channel data is fused to generate a new feature map. The specific formula is as follows: in, Let α represent the new feature map of the i-th viewpoint at time t. The weight, β represents The weights; The new feature map is processed by the sparse attention mechanism module, which generates a spatial attention weight matrix through sparse attention in the spatial dimension and generates an importance weight vector for each channel through sparse attention in the channel dimension. The spatial attention weight matrix and the importance weight vector are then fused and integrated to obtain the corresponding feature representation. The corresponding feature representations, processed by the Long Short-Term Memory (LSTM) network module, capture the dependencies between time-series data to determine the location of the fire point. The specific formula is as follows: c t =f t c t-1 +j t ·tanh(W c x t +U c h t-1 )+b c h t =o t ·tanh(c t ) Among them, f t Let Sg() represent the activation vector of the forget gate at time t, and W represent the activation function of the sigmoid function. f W j W o W c These represent the forget gate, input gate, output gate, and input kernel of the memory cell, respectively. U f U j U o U c The recurrent kernel, h, represents the forget gate, input gate, output gate, and memory cell. t-1 Let b represent the activation vector of the hidden state at time t-1. f b j b o b c These represent the biases of the forgetting gate, input gate, output gate, and memory cells, respectively. t Let o represent the activation vector of the input gate at time t. t c represents the activation vector of the output gate at time t. t c represents the activation vector of the memory cell at time t. t-1 Let x represent the activation vector of the memory cell at time t-1, tanh() represent the hyperbolic tangent activation function, and x t Let h represent the input vector. t Let represent the activation vector of the hidden state at time t.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the forest fire detection method for dual-channel time-series data suitable for edge computing as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, The computer program, when run by the processor, executes the forest fire detection method for dual-channel time-series data suitable for edge computing as described in any one of claims 1 to 5.

Images