A forest fire smoke detection method and system

Abstract

The application discloses a forest fire smoke detection method and system, and the method comprises the following steps: acquiring a forest image; inputting the forest image into a pre-trained forest smoke detection model to obtain a forest image containing smoke, wherein the forest smoke detection model comprises a long-distance dependence relationship model, a global visual attention model and a thin mist interference automatic elimination model; inputting the forest image containing smoke into a pre-trained forest smoke concentration estimation model; and determining the forest smoke concentration and the early warning measures according to the output of the forest smoke concentration estimation model. The application can quickly, accurately and efficiently monitor the forest fire in time and give a disaster warning.

Description

Technical Field

[0001] This invention belongs to the field of fire detection technology, specifically relating to a method and system for detecting smoke in forest fires. Background Technology

[0002] In real-life fires, smoke is often the first sign of a fire. Identifying smoke is crucial for the early detection, extinguishing, and evacuation of people.

[0003] Traditional fire detectors generally suffer from the following problems: (1) It takes a long time for smoke particles to reach the detector, inevitably causing alarm delays; (2) Due to airflow, smoke stratification, and other reasons, smoke particles may not reach the sensor, resulting in no alarm; (3) The sensor must be installed near the ignition point, and long-term exposure to the air will cause contact with a large amount of dust, leading to sensor failure. In summary, traditional sensors are not suitable for high spaces, outdoor environments, strong airflow, or dusty conditions, especially for detecting forest fire smoke.

[0004] Furthermore, smoke concentration provides crucial information for applications such as fire simulation, personnel evacuation, and automated firefighting robots. Knowing the distribution of the fire smoke concentration field allows rescue personnel to determine the optimal evacuation route and timing. However, existing forest fire detection and early warning models struggle to eliminate interference from non-fire smoke elements such as fog, leading to inaccurate, untimely fire detection, and even erroneous early warnings. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for detecting forest fire smoke, which can quickly, accurately and efficiently monitor forest fires and provide early warning of disasters.

[0006] This invention provides the following technical solution:

[0007] Firstly, a method for detecting forest fire smoke is provided, including:

[0008] Acquire forest images;

[0009] The forest image is input into a pre-trained forest smoke detection model to obtain a forest image containing smoke. The forest smoke detection model includes a long-distance dependency model, a global visual attention model, and an automatic fog interference removal model.

[0010] Input forest images containing smoke into a pre-trained forest smoke concentration estimation model;

[0011] Based on the output of the forest smoke concentration estimation model, the forest smoke concentration and early warning measures are determined.

[0012] Furthermore, the method for constructing the long-distance dependency model includes:

[0013] Pyramid pooling is performed on forest images containing smoke to obtain a series of encoded feature maps at different scales. Each scale of encoded feature map is serialized into blocks, and positional encoded features are generated for each block. Then, the block features and positional encoded features are fused to obtain fused features.

[0014] By using fused features from the same scale as Query, Key, and Value features in the Transformer self-attention mechanism network structure, a self-attention mechanism within the same scale is achieved, resulting in self-attention features within the same scale.

[0015] By using fused features from different scales as Query, Key, and Value features in the Transformer self-attention mechanism network structure, a cross-attention mechanism at different scales is achieved, resulting in cross-attention features at different scales.

[0016] By fusing self-attention features at the same scale and cross-attention features at different scales, the data is decoded and restored into spatial feature maps at different scales.

[0017] Furthermore, the long-distance dependency model uses a non-local network to capture long-distance dependencies, with the basic formula as follows:

[0018]

[0019] In the formula, y i For the response at the corresponding position, f(x) i ,x j Let x be two distinct feature vectors. i and x j Related functions, g(x) j C(x) is the enhancement operation, and C(x) is the response factor. This is a global response.

[0020] Furthermore, the method for constructing the global visual attention model includes:

[0021] A global category embedding attention module is used to perform image convolution, image normalization, image activation, and image pooling on forest images containing smoke to obtain a category ratio feature map.

[0022] Using the smoke pixel ratio prior generation module, the attention network structure embedded with positive and negative sample priors and the smoke pixel ratio supervision method are used to supervise the positive and negative sample ratio map generated by the class ratio feature and the real label, and a more complete class ratio feature map is obtained.

[0023] The improved class ratio feature map and the input feature map are weighted and fused using a global category embedding attention module to obtain a fused sample feature map.

[0024] Based on the fused sample feature map, a smoke appearance attention mechanism in the channel, horizontal and vertical directions is designed using a spatial channel hybrid attention module, and smoke morphology feature maps are generated through different attention mechanisms.

[0025] An adaptive fusion method and a multi-head supervision method are used to fuse smoke morphology feature maps generated by different attention mechanisms to establish a global visual attention model.

[0026] Furthermore, the method for building the automatic fog interference elimination model includes:

[0027] Calculate the minimum color channel I of a hazy image m1 (x), the formula is:

[0028]

[0029] In the formula, Ι c (x) represents any color channel of the hazy image I;

[0030] Sort the pixel values ​​of local image blocks in the smallest color channel of the hazy image in ascending order to obtain the framework of channel minimum value sorting;

[0031] Based on the framework of channel minimization sorting, and taking into account the characteristics of forest fog, an automatic fog interference elimination model is established by using a fog defogging prior method.

[0032] Furthermore, the forest smoke detection model is obtained by combining and improving the Transformer self-attention model based on a long-distance dependency model, a global visual attention model, and an automatic fog interference elimination model;

[0033] The Transformer self-attention model's self-attention mechanism network structure uses Query vector Q, Key vector K, and Value vector V to calculate their interrelationships, and then uses the softmax function for activation. The specific calculation method is as follows:

[0034]

[0035] In the formula, d k Let K be the dimension of vector K, and T be the transpose of the code vector or matrix.

[0036] Furthermore, the forest smoke detection model is trained using a forest fire smoke image database, which is obtained through a combination of manual annotation, computer simulation, and artificial intelligence.

[0037] Furthermore, the method for constructing the forest smoke concentration estimation model includes:

[0038] A visual attention mechanism and a long-distance dependency model network structure are established. Channel connection, point-by-point residual addition, and channel weighted fusion methods are used to embed them into a unified encoding and decoding network framework in a multi-scale manner to obtain feature signals of the same scale.

[0039] A scale-direction feature fusion module is added between the feature signals at different scales. Finally, all scale attention features and encoding / decoding output features are fused to obtain a very robust smoke concentration feature representation.

[0040] Furthermore, the forest smoke concentration estimation model employs a composite loss function during training, testing, and validation. This composite loss function includes an image-level smoke classification cross-entropy loss function, a pixel-level dense classification cross-entropy loss function, and a pixel-level regression smoothing L1 loss function.

[0041] The formula for the cross-entropy loss function is:

[0042]

[0043] In the formula, i is the total number of samples, and x i For the corresponding sample variable values, p(x) i Let q(x) be the true distribution of the sample. i ) represents the distribution predicted by the model;

[0044] The formula for the smoothed L1 loss function is:

[0045]

[0046] In the formula, x is the sample variable.

[0047] Secondly, a forest fire smoke detection system is provided, including:

[0048] The data acquisition module is used to acquire forest images;

[0049] The image filtering module is used to input the forest image into a pre-trained forest smoke detection model to obtain a forest image containing smoke. The forest smoke detection model includes a long-distance dependency model, a global visual attention model, and an automatic fog interference removal model.

[0050] The forest smoke concentration detection module is used to input the forest image containing smoke into a pre-trained forest smoke concentration estimation model, and determine the forest smoke concentration and early warning measures based on the output of the forest smoke concentration estimation model.

[0051] Compared with the prior art, the beneficial effects of the present invention are:

[0052] (1) This invention inputs forest images into a pre-trained forest smoke detection model to obtain forest images containing smoke. The forest smoke detection model includes a long-distance dependency model, a global visual attention model, and an automatic fog interference removal model. The long-distance dependency model and the global visual attention model can uncover the common appearance features of smoke and extract the best combination of smoke features, which can significantly improve the smoke recognition rate. The automatic fog interference removal model can effectively reduce the influence of interfering targets such as clouds and fog on smoke recognition and improve the recognition accuracy of forest fires.

[0053] (2) The present invention inputs forest images containing smoke into a pre-trained forest smoke concentration estimation model. Based on the output of the forest smoke concentration estimation model, the forest smoke concentration and early warning measures can be determined so as to quickly, accurately and efficiently monitor forest fires and provide early warning of disasters. Attached Figure Description

[0054] Figure 1 This is a flowchart of the forest fire smoke detection method in an embodiment of the present invention;

[0055] Figure 2 This is a long-distance dependency model in the embodiments of the present invention;

[0056] Figure 3 This is the global category information embedding and composite attention network structure in the embodiments of the present invention;

[0057] Figure 4 This is the forest smoke concentration estimation model in the embodiments of the present invention;

[0058] Figure 5 These are schematic diagrams illustrating two real smoke experiment verification methods in embodiments of the present invention;

[0059] Figure 6 Images of a forest fire in Qilin District, Qujing, Yunnan Province, as shown in Embodiment 3 of this invention;

[0060] Figure 7 The images before and after labeling in Embodiment 4 of the present invention are: (a) smoke classification, (b) smoke detection, (c) smoke separation, and (d) smoke concentration estimation. Detailed Implementation

[0061] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0062] Example 1

[0063] like Figure 1 As shown, this embodiment provides a method for detecting forest fire smoke, including:

[0064] Acquire forest images;

[0065] The forest image is input into a pre-trained forest smoke detection model to obtain a forest image containing smoke. The forest smoke detection model includes a long-distance dependency model, a global visual attention model, and an automatic fog interference removal model.

[0066] Input forest images containing smoke into a pre-trained forest smoke concentration estimation model;

[0067] Based on the output of the forest smoke concentration estimation model, the forest smoke concentration and early warning measures are determined.

[0068] Example 2

[0069] This embodiment provides a method for detecting smoke during forest fires, the steps of which are as follows:

[0070] Step 1: Obtain a large number of forest fire smoke images, and use a combination of manual annotation, computer simulation and artificial intelligence to generate high-quality smoke images, thus obtaining a forest fire smoke image library.

[0071] 1.1 Manual annotation method.

[0072] A large number of fire smoke images were obtained from the fire video monitoring system under different times, lighting conditions, and weather conditions. Images containing smoke were manually labeled.

[0073] 1.2 Computer simulation method.

[0074] Computational fluid dynamics simulation theory and volume rendering technology are used to render fluids and generate a highly realistic virtual smoke sample image library that follows physical laws. The macroscopic appearance characteristics of smoke are determined from the virtual smoke sample image library.

[0075] The computational fluid dynamics simulation theory of smoke is based on the Mie scattering principle of smoke particles and the Navier-Stokes equations for fluid dynamics, simulating the motion of smoke particles through numerical calculations. The Mie scattering theory is a rigorous mathematical solution to Maxwell's equations for uniform particles in a homogeneous medium; the specific calculation formula is as follows:

[0076]

[0077]

[0078] in, and These are the extinction and scattering cross sections of a spherical particle, respectively, where Re represents the function with its real part, and a nand b n λ is a complex number, and λ is the wavelength of the reflected light from the background or the light source.

[0079] The scattering coefficient describes the strength of the scattering effect of various scattering elements in the atmosphere on the radiative flux, while the extinction coefficient describes the extinction effect of aerosols. Therefore, the extinction coefficient and scattering coefficient are used to describe the characteristics of smoke images in the atmosphere. Extinction coefficient = absorption coefficient + scattering coefficient, that is:

[0080] γ(λ)=α(λ)+β(λ)

[0081] Due to the different particle size distributions, the overall extinction and scattering coefficients are respectively:

[0082]

[0083]

[0084] Where m is the refractive index of the medium, r is the radius of the particle sphere, and n(r) is the particle spectrum distribution.

[0085] Furthermore, the motion of smoke particles in the air exhibits fluid properties and should conform to the Navier-Stokes equations for viscous Newtonian fluids, as calculated below:

[0086]

[0087] Where ρ is the fluid mass density, v is the drift velocity, ▽ is the divergence operator, p is the pressure, t is the time parameter, I is the identity matrix, τ is the Cauchy stress tensor, and g is the continuum acceleration. It is the outer product operator.

[0088] Based on the Mie scattering principle and the Navier-Stokes equations for fluid dynamics, the motion of smoke particles and their final rendered color and brightness can be simulated through computer image simulation and auxiliary numerical calculations.

[0089] Since the color of pixel x on the image plane is actually the transmitted light formed after the background and ambient reflected light passes through the atmosphere and smoke areas, it is the integral result of the combined scattering of smoke particles and atmospheric aerosols. Under shooting conditions with good air quality and relatively short distances, the scattering by aerosols is almost negligible. Therefore, the background transmitted light of pixel x on the image plane can be calculated using a formula to analyze the scattering characteristics of smoke particles. The specific formula for calculating background transmitted light is as follows:

[0090] i(x)=b(x)(1-α(x))+s(x)α(x)

[0091]

[0092] Where i(x) is the color value of image pixel x, b(x) is the color value reflected by the background point corresponding to image pixel x, τ(v,x) is the attenuation coefficient at the line of sight v, α(x) is the smoke opacity, and s(x) is the color scattered by smoke particles.

[0093] Therefore, by studying the interaction between the macroscopic appearance of smoke and the light scattering mechanism of microscopic particles, fluid simulation, and color synthesis, we can discover and explore effective macroscopic appearance features of smoke.

[0094] 1.3 Artificial Intelligence Approach. Artificial intelligence-based augmentation methods, such as variational autoencoders and generative adversarial networks, are used to generate more high-quality smoke images, further expanding the forest fire smoke image library and comprehensively constructing a smoke image training and testing database.

[0095] Step 2: Establish a forest smoke detection model, which includes a long-range dependency model, a global visual attention model, and an automatic fog interference removal model. The model establishment process consists of five modules: a cross-scale global dependency enhancement module, a global category embedding attention module, a smoke pixel ratio prior generation module, a spatial channel hybrid attention module, and a fog interference removal network module incorporating defogging theory.

[0096] 2.1 Build a long-distance dependency model.

[0097] This model enables features at different scales to fully interact and facilitates the extraction of long-distance dependency features. This embodiment proposes a cross-scale global dependency enhancement module, such as... Figure 2 As shown.

[0098] The long-range dependency model uses a non-local network to capture long-range dependencies. The basic formula is as follows:

[0099]

[0100] In the formula, y i For the response at the corresponding position, f(x) i ,x j Let x be two distinct feature vectors. i and x j Related functions, g(x) j C(x) is the enhancement operation, and C(x) is the response factor. This is a global response.

[0101] The specific method for building a long-distance dependency model is as follows:

[0102] (1) Pyramid pooling is performed on forest images containing smoke to obtain a series of encoded feature maps at different scales. Each scale of encoded feature map is serialized into blocks, and positional encoded features are generated for each block. Then, the block features and positional encoded features are fused to obtain fused features. In this embodiment, blocks are formed in the spatial and channel 3D tensor space. This not only allows for the extraction of spatial long-distance dependent features, but also enables the interaction of spatial and channel information.

[0103] (2) The fused features from the same scale are used as Query, Key and Value features in the Transformer self-attention mechanism network structure to realize the self-attention mechanism within the same scale and obtain the self-attention features within the same scale.

[0104] (3) The fusion features from different scales are used as Query, Key and Value features in the Transformer self-attention mechanism network structure to realize the cross attention mechanism at different scales and obtain cross attention features at different scales.

[0105] (4) The self-attention features within the same scale and the cross-attention features at different scales are fused and decoded to restore spatial feature maps at different scales. In this way, the present invention can not only effectively extract long-distance dependent features of the global appearance of smoke, but also realize cross-scale and cross-channel information interaction.

[0106] 2.2. Build a global visual attention model.

[0107] like Figure 3 As shown, this invention establishes a global visual attention model through global category information embedding and a composite attention network structure.

[0108] (1) Using the global category embedding attention module, image convolution, image normalization, image activation and image pooling are performed on forest images containing smoke to obtain category ratio feature maps and positive and negative sample ratio prior feature maps.

[0109] The image convolution processing uses the following formula:

[0110]

[0111] Where f and g are corresponding elements, specifically f(u,v) is the image matrix, x and y are pixel coordinates, g(xu,yv) is the convolution kernel function, and u and v are constraint variables.

[0112] The image normalization process is specifically formulated as follows:

[0113]

[0114] Where, x i Here, x represents the image pixel value, norm represents the processed value, and min(x) represents the pixel value. i ) represents the minimum pixel value of the image, max(x) i () represents the maximum number of pixels in the image.

[0115] The image activation process uses the sigmoid function as the activation function, and the specific formula is as follows:

[0116]

[0117] Where g(x) is the processing result, and x is the pixel value of the image.

[0118] The image pooling process yields the following result:

[0119]

[0120]

[0121] Where W′ and H′ are the width and height of the image after pooling, respectively, W and H are the width and height of the image before processing, F×F is the size of the convolution kernel, and S is the stride.

[0122] (2) Based on the category ratio feature map obtained in the previous step, the smoke pixel ratio prior generation module is used. The attention network structure embedded with the positive and negative sample prior and the smoke pixel ratio supervision method are used to supervise the category ratio feature map and the positive and negative sample ratio map generated by the real label, so that the overall smoke pixel ratio of the model segmentation is as close as possible, overcoming the problem of positive and negative sample imbalance, and obtaining the improved category ratio feature map.

[0123] In this process, bilinear interpolation was used. The bilinear interpolation formula is as follows:

[0124]

[0125] Where f(x,y) is the pixel value of the point to be found, x and y are the coordinates of the point to be found, and x1, x2, y1, and y2 are the coordinates of four known pixel points, respectively Q 11 (x1,y1),Q 12 (x1,y2),Q 21 (x2,y1),Q 22 (x2,y2), f(Q) i ) represents the pixel value of a known pixel point Qi.

[0126] (3) The global category embedding attention module is used to perform weighted fusion of the improved category ratio feature map and the input feature map in the previous step to obtain the fused sample feature map, thereby realizing the attention mechanism of global ratio information embedding, thus overcoming the problems caused by the imbalance of positive and negative samples and the blurring of smoke boundaries.

[0127] (4) Based on the fused sample feature map obtained in the previous step, a spatial channel hybrid attention module is used to initially design a smoke appearance attention mechanism in the channel, horizontal and vertical directions from a multi-scale perspective, combining traditional squeezing, activation and non-local means methods. In addition, a grid structure is designed to extract features such as local pixel self-similarity and texture smoothness. Through different attention mechanisms, smoke morphology feature maps are generated, effectively extracting the unique rising and fluid morphology of smoke.

[0128] (5) Based on the smoke morphology feature maps obtained in the previous step, an adaptive fusion method is used to fuse the smoke morphology feature maps output by different attention mechanisms using methods such as convolution and point-by-point addition. A multi-head supervision method is also adopted, adding error loss functions at different levels to control the behavior of the entire network and improve the robustness of the fused features. Based on information such as the light scattering characteristics of smoke particles, the light interaction relationship between smoke particles and the background, and the fluid characteristics of smoke, fusion methods such as information entropy, color perception theory, multi-scale, and cross-channel transformation are used to finally establish a global visual attention model.

[0129] 2.3 Establish an automatic fog interference elimination model.

[0130] (1) A haze interference removal network module combining dehazing theory is used. A prior estimation method is proposed by minimizing the channel values ​​of the hazy image I. The minimum color channel I of the hazy image I is... m1 (x) can be calculated using the following formula:

[0131]

[0132] In the formula, Ι c (x) represents any color channel of I.

[0133] (2) Process the smallest color channel I obtained in the previous step in ascending order. m1 Local image patch Ω i The pixel values ​​in the data are sorted to obtain a framework for sorting by channel minimization value. It is found that several existing dehazing priors can be uniformly explained under the framework of sorting by channel minimization value.

[0134] (3) Based on the framework of channel minimization sorting, and taking into account the characteristics of forest fog, an automatic fog interference elimination model is established through existing defogging prior methods to reduce the false alarm rate of the smoke detection model.

[0135] 2.4 Regarding forest smoke detection models.

[0136] This invention first establishes a long-range dependency model. Based on this model, common appearance features of smoke are identified from both global category information classification and long-range dependencies, thus establishing a global visual attention model. Then, based on long-range dependency theory, an automatic fog interference elimination model combining defogging theory is proposed. Finally, a Transformer self-attention model is synthesized and improved based on the above three models; the improved Transformer self-attention model formula is the set of formulas from the above three models. Accordingly, a forest smoke detection model is established using the Transformer self-attention model.

[0137] In attention mechanism theory, the self-attention mechanism of the Transformer self-attention model plays a crucial role. It establishes correlations between multiple related inputs. Multi-head attention structures contain multiple self-attention mechanism modules.

[0138] The self-attention mechanism module is included in the multi-head attention structure. With the help of the multi-head attention mechanism, features and information of the smoke image are extracted more accurately. In the multi-head attention mechanism, the multi-head attention is first subjected to a linear transformation, then input into a scaling dot product attention, and the transformation is repeated m times. The results of the m scaling dot product attention are concatenated, and finally, a linear transformation is performed. The resulting value is used as the result of the multi-head attention.

[0139] The Transformer self-attention model's self-attention mechanism network structure uses Query vector Q, Key vector K, and Value vector V to calculate their interrelationships, and then uses the softmax function for activation. The specific calculation method is as follows:

[0140]

[0141] In the formula, d k Let K be the dimension of vector K, and T be the transpose of the code vector or matrix.

[0142] 2.5 Model Training.

[0143] The forest smoke detection model was trained using a forest fire smoke image database to obtain the trained forest smoke detection model.

[0144] Step 3: Obtain forest images of the monitored area, including forest fire smoke images and forest cloud and fog images.

[0145] Step 4: Input the forest image from Step 3 into the forest smoke detection model trained in Step 2, and compare it with the smoke features obtained from the forest fire smoke image database to obtain forest images containing smoke.

[0146] Step 5: Establish a forest smoke concentration estimation model.

[0147] like Figure 4 As shown, this embodiment proposes a smoke concentration estimation model by employing a global visual attention mechanism, a network embedding method based on a long-range dependency model, and a multi-scale supervision strategy. Specifically, this model is a deep neural network model proposed within the framework of encoding and decoding deep neural networks, combining a visual attention mechanism, a network embedding method based on a long-range dependency model, and a multi-scale supervision strategy. It possesses both mathematical inversion interpretability and the advantage of automatic feature extraction by deep neural networks.

[0148] 5.1 The specific method for constructing the forest smoke concentration estimation model is as follows:

[0149] (1) Establish a network structure for visual attention mechanism and long-distance dependency model, and use channel connection, point-by-point residual addition and channel weighted fusion methods to embed it into a unified encoding and decoding network framework in a multi-scale manner to obtain feature signals of the same scale.

[0150] (2) A scale orientation feature fusion module is added between the feature signals at different scales. Finally, all scale attention features and encoding / decoding output features are fused to obtain a very robust smoke concentration feature expression.

[0151] Special networks such as skip-layer structures, residual structures, and feature fusion modules are added to the smoke concentration estimation model to further enhance the network's feature representation capabilities and establish a deep neural network model oriented towards micro-mechanisms and macro-attention.

[0152] 5.2 Training, testing and validation of the forest smoke concentration estimation model: A composite loss function of binary segmentation dense classification problem and smoke concentration estimation dense regression is adopted. The composite loss function of training normally weighted separately is studied. The penalty rules or mathematical models of the loss function under different scale conditions are dynamically adjusted to explore a suitable concentration estimation network model.

[0153] The composite loss function includes an image-level smoke classification cross-entropy loss function, a pixel-level dense classification cross-entropy loss function, and a pixel-level regression smoothing L1 loss function. Because the loss functions for classification and regression problems have different dimensions, and there are also differences in supervision between image-level and pixel-level methods, designing the overall loss function is quite difficult. Therefore, a dynamic adjustment of the relative importance of the three levels of loss functions is adopted to achieve a training strategy from coarse to fine.

[0154] The formula for the cross-entropy loss function is:

[0155]

[0156] In the formula, i is the total number of samples, and x i For the corresponding sample variable values, p(x) i Let q(x) be the true distribution of the sample. i ) represents the distribution predicted by the model;

[0157] The formula for the smoothed L1 loss function is:

[0158]

[0159] In the formula, x is the sample variable.

[0160] like Figure 5 As shown, two other methods are used to verify the effectiveness and estimation accuracy of the depth algorithm:

[0161] Method 1: The smoke concentration field under controllable conditions is obtained by using the blue screen matting concentration estimation algorithm, and then the accuracy of the algorithm in this project is compared with that under the blue screen matting condition.

[0162] Method 2: First, calibrate the concentration value estimated by the algorithm and the concentration value of a certain point sensor. Then, compare the error between the measured values ​​and the estimated values ​​of other point sensors to obtain the estimation accuracy of the real smoke.

[0163] Step 6: Input the forest image containing smoke into the forest smoke concentration estimation model trained in Step 5 to estimate the concentration.

[0164] Step 7: Determine the forest smoke concentration and early warning measures based on the output of the forest smoke concentration estimation model, and transmit them to the terminal via the network.

[0165] Example 3

[0166] like Figure 6 As shown, a forest fire occurred in Qilin District, Qujing City, Yunnan Province in March 2021, burning 303.76 mu (approximately 20 hectares) of forest land and causing economic losses of 1.885 million yuan. Nearly 3,000 people were involved in the firefighting efforts. In response to the above-mentioned forest fire occurring in the wild, the method proposed in Example 2 was used to provide early warning of the fire and obtain key fire information, as detailed below:

[0167] Step S1: Obtain forest images through methods such as video surveillance of the fire scene, photos taken by citizens' mobile phones, and aerial photography by drones.

[0168] Step S2: Input the forest image into a pre-trained forest smoke detection model and compare it with the smoke features obtained from the forest fire smoke image database to obtain a forest image containing smoke. Through the recognition and processing of the forest smoke detection model, the smoke range at various locations in the fire area is selected on the image, and the smoke on the image is further deepened to facilitate the next step of concentration estimation.

[0169] Step S3: Input the forest image containing smoke into the pre-trained forest smoke concentration estimation model, and output the forest smoke concentration. The model performs real-time analysis on different smoke images after the previous step's enhanced processing and outputs corresponding smoke concentration estimates. Compare the output forest smoke concentration estimates with the actual values ​​measured at the fire scene. If the results are within the error range, the smoke concentration estimation model can be considered to have a certain degree of accuracy.

[0170] Step S4: Assess the forest smoke concentration in different regions and propose early warning measures, such as having firefighting teams composed of personnel from township forest fire prevention agencies extinguish fires in areas with small fires and low smoke concentrations, and having professional firefighters conduct targeted firefighting in a small number of areas with large fires and high smoke concentrations.

[0171] Step S5: Transmit forest smoke concentration and early warning measures information to the terminal network to ensure timely access to information, which can greatly suppress the spread of fire. Simultaneously, when a fire occurs, it can provide more accurate information for fire rescue more quickly.

[0172] Example 4

[0173] This embodiment provides a forest fire smoke detection system, including:

[0174] The data acquisition module is used to acquire forest images;

[0175] The image filtering module is used to input the forest images into a pre-trained forest smoke detection model, obtain forest images containing smoke, and label them, such as... Figure 7 As shown. The forest smoke detection model includes a long-range dependency model, a global visual attention model, and an automatic fog interference removal model;

[0176] The forest smoke concentration detection module is used to input the forest image containing smoke into a pre-trained forest smoke concentration estimation model, and determine the forest smoke concentration and early warning measures based on the output of the forest smoke concentration estimation model.

[0177] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0179] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0180] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0181] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting forest fire smoke, characterized in that, include: Acquire forest images; The forest image is input into a pre-trained forest smoke detection model to obtain a forest image containing smoke. The forest smoke detection model includes a long-distance dependency model, a global visual attention model, and an automatic fog interference removal model. Input forest images containing smoke into a pre-trained forest smoke concentration estimation model; Based on the output of the forest smoke concentration estimation model, determine the forest smoke concentration and early warning measures; The method for constructing the long-distance dependency model includes: Pyramid pooling is performed on forest images containing smoke to obtain a series of encoded feature maps at different scales. Each scale of encoded feature map is serialized into blocks, and positional encoded features are generated for each block. Then, the block features and positional encoded features are fused to obtain fused features. By using fused features from the same scale as Query, Key, and Value features in the Transformer self-attention mechanism network structure, a self-attention mechanism within the same scale is achieved, resulting in self-attention features within the same scale. By using fused features from different scales as Query, Key, and Value features in the Transformer self-attention mechanism network structure, a cross-attention mechanism at different scales is achieved, resulting in cross-attention features at different scales. The self-attention features within the same scale and the cross-attention features at different scales are fused together and decoded to restore spatial feature maps at different scales. The method for constructing the global visual attention model includes: A global category embedding attention module is used to perform image convolution, image normalization, image activation, and image pooling on forest images containing smoke to obtain a category ratio feature map. Using the smoke pixel ratio prior generation module, the attention network structure embedded with positive and negative sample priors and the smoke pixel ratio supervision method are used to supervise the positive and negative sample ratio map generated by the category ratio feature and the real label, and a more complete category ratio feature map is obtained. The improved class ratio feature map and the input feature map are weighted and fused using a global category embedding attention module to obtain a fused sample feature map. Based on the fused sample feature map, a smoke appearance attention mechanism in the channel, horizontal and vertical directions is designed using a spatial channel hybrid attention module, and smoke morphology feature maps are generated through different attention mechanisms. An adaptive fusion method and a multi-head supervision method are used to fuse smoke morphology feature maps generated by different attention mechanisms to establish a global visual attention model; The method for building the automatic fog interference elimination model includes: Calculate the minimum color channel of a hazy image. The formula is: ； In the formula, This represents any color channel of the hazy image I; Sort the pixel values ​​of local image blocks in the smallest color channel of the hazy image in ascending order to obtain the framework of channel minimum value sorting; Based on the framework of channel minimization sorting, and taking into account the characteristics of forest fog, an automatic fog interference elimination model is established by using a fog defogging prior method. The forest smoke detection model is derived by combining and improving upon the long-distance dependency model, the global visual attention model, and the fog interference automatic elimination model, as well as the Transformer self-attention model.

2. The forest fire smoke detection method according to claim 1, characterized in that, The long-range dependency model uses a non-local network to capture long-range dependencies, and the basic formula is as follows: ； In the formula, y i For the response at the corresponding position, f(x) i , x j Let x be two distinct feature vectors. i and x j Related functions, g(x) j C(x) is the enhancement operation, and C(x) is the response factor. This is a global response.

3. The forest fire smoke detection method according to claim 1, characterized in that, The Transformer self-attention model's self-attention mechanism network structure uses Query vector Q, Key vector K, and Value vector V to calculate their interrelationships, and then uses the softmax function for activation. The specific calculation method is as follows: ； In the formula, d k Let K be the dimension of vector K, and T be the transpose of the code vector or matrix.

4. The forest fire smoke detection method according to claim 1, characterized in that, The forest smoke detection model is trained using a forest fire smoke image database, which is obtained through a combination of manual annotation, computer simulation, and artificial intelligence.

5. The forest fire smoke detection method according to claim 1, characterized in that, The method for constructing the forest smoke concentration estimation model includes: A visual attention mechanism and a long-distance dependency model network structure are established. Channel connection, point-by-point residual addition, and channel weighted fusion methods are used to embed them into a unified encoding and decoding network framework in a multi-scale manner to obtain feature signals of the same scale. A scale-direction feature fusion module is added between the feature signals at different scales. Finally, all scale attention features and encoding / decoding output features are fused to obtain a very robust smoke concentration feature representation.

6. The forest fire smoke detection method according to claim 1, characterized in that, The forest smoke concentration estimation model is trained, tested and validated using a composite loss function, which includes an image-level smoke classification cross-entropy loss function, a pixel-level dense classification cross-entropy loss function and a pixel-level regression smoothing L1 loss function. The formula for the cross-entropy loss function is: ； In the formula, i is the total number of samples, and x i For the corresponding sample variable values, p(x) i Let q(x) be the true distribution of the sample. i ) represents the distribution predicted by the model; The formula for the smoothed L1 loss function is: ； In the formula, x is the sample variable.

7. A forest fire smoke detection system, characterized in that, The system for implementing the method according to any one of claims 1 to 6 comprises: The data acquisition module is used to acquire forest images; The image filtering module is used to input the forest image into a pre-trained forest smoke detection model to obtain a forest image containing smoke. The forest smoke detection model includes a long-distance dependency model, a global visual attention model, and an automatic fog interference removal model. The forest smoke concentration detection module is used to input the forest image containing smoke into a pre-trained forest smoke concentration estimation model, and determine the forest smoke concentration and early warning measures based on the output of the forest smoke concentration estimation model.

Images