Forest fire smoke semantic segmentation detection method and system

Abstract

The present application provides a forest fire smoke semantic segmentation detection method and system, which comprises the following steps: synthesizing a data set according to prior knowledge of forest fire smoke and prior knowledge of smoke-like targets in a forest fire background; performing two-stage data enhancement on the data set based on an adversarial autoencoder model; inputting the data set after data enhancement into a forest fire deep semantic segmentation model for pre-training; inputting multiple forest fire smoke samples and smoke whole-image labels corresponding to each forest fire smoke sample into the forest fire deep semantic segmentation model as initial weights of the network, and outputting in two branches; synthesizing new forest fire smoke images according to the outputs of the two branches, and inputting the new forest fire smoke images into the model for semantic segmentation and identification training to obtain a trained forest fire deep semantic segmentation model. The present application can solve the problem that the conventional smoke detection method cannot be applied to complex forest background when detecting forest fire smoke, resulting in high false alarm rate and high false negative rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of forest fire intelligent identification, and particularly relates to a forest fire smoke semantic segmentation detection method and system. BACKGROUND

[0002] Forest fire is a kind of fire that is strong in suddenness, large in destructiveness and difficult to handle and rescue. Timely and accurate discovery of forest fire is the basic guarantee to avoid harm and plays an important role in forest fire early warning and rescue work. Forest fire intelligent detection technology based on machine vision is a kind of non-contact fire detection technology based on machine vision, which is especially suitable for solving the fire detection problem in large space, outdoor and other places. This kind of method has the advantages of fast response, not easy to be affected by environmental factors, wide application and low cost, and has become an important cross-disciplinary research field in fire detection methods.

[0003] Smoke is the most prominent phenomenon of early forest fire, and fire often occurs after smoke produced by smoldering. Sometimes even if there is an open fire, due to the influence of forest topography and environment, such as the fire point in the deep valley, the monitoring tower and the lookout tower cannot directly observe the fire area. Therefore, the research on forest fire smoke intelligent identification based on machine vision has more practical significance. However, the shape and color of smoke change rapidly, and the motion law is difficult to grasp, which brings great challenges to video image smoke detection. These specialities of smoke make the detection methods in the field of non-smoke detection not applicable to smoke detection, on the other hand, the false positive rate, false negative rate and other parameters of the existing smoke detection methods are high.

[0004] In recent years, the introduction of deep semantic segmentation technology has further improved the effect of smoke detection. At present, the forest fire smoke monitoring mostly introduces the conventional smoke detection method, without considering the particularity of forest fire smoke. Although certain achievements have been made, the recognition effect and stability are not ideal, and the false positive rate is high. The main reasons are as follows: (1) many backgrounds and objects have similar color distribution and texture patterns to smoke, such as complex forest environment, variable climate conditions, and clouds, fog, bare ground, dust, etc.; (2) the forest fire image with smoke is generally fuzzy and has low contrast, making it difficult to extract effective features; (3) thin smoke cannot completely block the background image, causing a semi-transparent effect and failing to effectively remove background interference; (4) the visual features such as smoke shape change over time; (5) there are non-smoke interference targets such as occlusion. In addition, data-driven deep learning methods require large sample training data and intensive annotation, and real forest fire smoke samples are difficult to collect and annotate in large quantities. SUMMARY

[0005] Based on this, the purpose of the present application is to propose a forest fire smoke semantic segmentation detection method and system to solve the problem that the conventional smoke detection method cannot be applied to complex forest background when detecting forest fire smoke, resulting in high false alarm rate and high false negative rate.

[0006] The present application proposes a forest fire smoke semantic segmentation detection method, which comprises:

[0007] Establish prior knowledge for guidance, which includes forest fire smoke prior knowledge and smoke-like interference prior knowledge in forest fire background, to synthesize a data set according to the forest fire smoke prior knowledge and the smoke-like target prior knowledge in the forest fire background;

[0008] Based on the two-stage data enhancement of the adversarial autoencoder model on the data set, a data-enhanced data set is obtained, which includes multiple forest fire smoke samples, smoke whole image labels corresponding to each forest fire smoke sample, and multiple smoke-like target samples and smoke whole image labels corresponding to the smoke-like target samples;

[0009] Input multiple forest fire smoke samples and smoke whole image labels corresponding to each forest fire smoke sample, and multiple smoke-like target samples and smoke whole image labels corresponding to the smoke-like target samples into a forest fire deep semantic segmentation model for pre-training, so that the model outputs network initial weights for distinguishing forest fire smoke and smoke-like targets;

[0010] Input multiple forest fire smoke samples and smoke whole image labels corresponding to each forest fire smoke sample as the input of the forest fire deep semantic segmentation model with network initial weights, and divide them into two branches for output, one branch outputs a background image, and the other branch outputs a pure forest fire smoke image and a smoke concentration corresponding to the pure forest fire smoke image;

[0011] Synthesize a new forest fire smoke image according to the output background image, pure forest fire smoke image and smoke concentration corresponding to the pure forest fire smoke image, and input the new forest fire smoke image into the forest fire deep semantic segmentation model for semantic segmentation and recognition training to obtain a trained forest fire deep semantic segmentation model.

[0012] In the preferred embodiment of the present application, the step of establishing prior knowledge for guidance, which includes forest fire smoke prior knowledge and smoke-like interference prior knowledge in forest fire background, to synthesize a data set according to the forest fire smoke prior knowledge and the smoke-like target prior knowledge in the forest fire background comprises:

[0013] The Navier-Stokes equation in the field of computational fluid dynamics is used to describe the free movement of smoke and establish a model;

[0014] After solving the Navier-Stokes equation, the simulated smoke is visualized by volume rendering to generate a plurality of pure forest fire smoke images, so as to construct a pure forest fire smoke image set according to the plurality of pure forest fire smoke images;

[0015] Based on prior knowledge, a real forest image is collected, a complex forest environment background image is selected from an international standard database, and a smoke target image containing a forest fire background is collected, so as to construct a background image set according to the complex forest environment background image, the real forest image and the smoke target image containing the forest fire background.

[0016] In the preferred embodiment of the present application, the prior knowledge for guidance includes forest fire smoke prior knowledge and smoke-like interference prior knowledge in a forest fire background, and the step of synthesizing a data set according to the forest fire smoke prior knowledge and the smoke-like target prior knowledge in the forest fire background further comprises:

[0017] According to the pure forest fire smoke image set and the background image set, a plurality of forest fire smoke images and smoke-like target images are randomly synthesized;

[0018] The step of randomly synthesizing a plurality of forest fire smoke images and smoke-like target images according to the pure forest fire smoke image set and the background image set comprises:

[0019] The forest fire smoke image or the smoke-like target image is synthesized according to the following formula:

[0020] i = b (1 - a) + s a

[0021] Wherein, i represents the synthesized forest fire smoke image or smoke-like target image, b represents a random image in the background image set, a represents the smoke transparency, and s represents a random image in the pure forest fire smoke image set.

[0022] In the preferred embodiment of the present application, the smoke transparency is calculated according to the following formula:

[0023]

[0024] Wherein, d represents the propagation length of light through smoke particles, and tau (t) represents the attenuation coefficient of smoke particles at t.

[0025] In the preferred embodiment of the present application, the data set is subjected to two-stage data enhancement based on the adversarial autoencoder model, to obtain a data-enhanced data set, and the data-enhanced data set comprises a plurality of forest fire smoke samples, smoke whole image labels corresponding to each of the forest fire smoke samples, a plurality of smoke target samples and smoke whole image labels corresponding to each of the smoke target samples.

[0026] The supervised data augmentation of the data set includes: randomly rotating the image to change the orientation of the image content; randomly horizontally or vertically flipping the image; scaling the original image by a random ratio; filtering the image by using a Gaussian kernel of different scales to construct a scale space, and randomly generating sample images with different variable or blurred image contents in the scale space;

[0027] The unsupervised data augmentation of the data set is performed by encoding the images in the input data set by using an encoder, and feeding the obtained low-dimensional features instead of white noise into a generator and a discriminator for adversarial training, so as to finally obtain sample images generated by an adversarial generative network model.

[0028] Another aspect of the present application provides a forest fire smoke semantic segmentation detection system, characterized in that the system comprises:

[0029] A data set synthesis module is configured to establish prior knowledge for guidance, wherein the prior knowledge includes forest fire smoke prior knowledge and forest fire background smoke-like target prior knowledge, so as to synthesize a data set according to the forest fire smoke prior knowledge and the forest fire background smoke-like target prior knowledge.

[0030] A data set enhancement module is configured to perform two-stage data augmentation on the data set based on an adversarial autoencoder model, so as to obtain a data set after data augmentation, wherein the data set after data augmentation includes multiple forest fire smoke samples, smoke whole image labels corresponding to each of the forest fire smoke samples, and multiple smoke-like target samples and smoke whole image labels corresponding to each of the smoke-like target samples.

[0031] A pre-training module is configured to input the multiple forest fire smoke samples, the smoke whole image labels corresponding to each of the forest fire smoke samples, the multiple smoke-like target samples, and the smoke whole image labels corresponding to each of the smoke-like target samples into a forest fire deep semantic segmentation model for pre-training, so as to make the model output network initial weights for distinguishing forest fire smoke and smoke-like targets.

[0032] A semantic segmentation module is configured to input the multiple forest fire smoke samples and the smoke whole image labels corresponding to each of the forest fire smoke samples as inputs of the forest fire deep semantic segmentation model with the network initial weights, and divide the inputs into two branches for output, wherein one branch outputs a background image, and the other branch outputs a pure forest fire smoke image and smoke concentration corresponding to the pure forest fire smoke image.

[0033] A semantic segmentation training module is configured to synthesize new forest fire smoke images according to the output background image, the pure forest fire smoke image, and the smoke concentration corresponding to the pure forest fire smoke image, and input the new forest fire smoke images into the forest fire deep semantic segmentation model for semantic segmentation and identification training, so as to obtain a trained forest fire deep semantic segmentation model.

[0034] In the preferred embodiment of the present application, the data set forming module further comprises:

[0035] a fluid dynamics model building unit for building a model by using Navier-Stokes equation in the field of computational fluid dynamics to describe free movement of smoke;

[0036] a pure forest fire smoke image set building unit for visualizing simulated smoke by using volume rendering after solving the Navier-Stokes equation to generate a plurality of pure forest fire smoke images, and building a pure forest fire smoke image set according to the plurality of pure forest fire smoke images;

[0037] a background image set building unit for collecting real forest images based on prior knowledge, selecting a background image of a complex forest environment from an international standard database, and collecting smoke target images containing a forest fire background, and building a background image set according to the background image of the complex forest environment, the real forest images, and the smoke target images containing the forest fire background.

[0038] In the preferred embodiment of the present application, the data set forming module further comprises:

[0039] an image synthesis unit for synthesizing a plurality of forest fire smoke images and smoke target images randomly according to the pure forest fire smoke image set and the background image set;

[0040] and for synthesizing a forest fire smoke image or a smoke target image according to the following formula:

[0041] i = b(1-α) + sα

[0042] wherein i represents the synthesized forest fire smoke image or smoke target image, b represents a random image in the background image set, α represents the semi-transparency of smoke, and s represents a random image in the pure forest fire smoke image set.

[0043] In the preferred embodiment of the present application, the image synthesis unit further comprises:

[0044] a semi-transparency calculation sub-unit for calculating the semi-transparency of smoke according to the following formula:

[0045]

[0046] wherein d represents the propagation length of light through smoke particles, and τ(t) represents the attenuation coefficient of smoke particles at t.

[0047] In the preferred embodiment of the present application, the data set enhancement module further comprises:

[0048] The supervised data augmentation execution unit is used to perform supervised data augmentation on the dataset, including: randomly rotating the image to change the orientation of the image content; randomly flipping the image horizontally or vertically; scaling the original image by a random ratio; filtering the image with Gaussian kernels of different scales to construct a scale space, and randomly generating sample images with variable image content or varying degrees of blur in the scale space.

[0049] The unsupervised data augmentation execution unit is used to perform unsupervised data augmentation on the dataset. It encodes the images in the input dataset through an encoder, and then feeds the obtained low-dimensional features into the generator and discriminator instead of white noise for adversarial training, ultimately obtaining sample images generated by the adversarial generative network model.

[0050] Compared with the prior art, the present invention has the following advantages:

[0051] 1. This invention uses a simulation synthesis method to establish a large-sample forest fire smoke image dataset, laying the foundation for subsequent research. To ensure the scientific validity of the synthesized data, manifold theory is used for data augmentation, thereby solving the problem of obtaining a large number of real forest fire smoke video images.

[0052] 2. This invention pre-trains the encoding network in the semantic segmentation model to classify smoke-like targets, and reuses the trained model in the semantic segmentation model as an encoder, thereby integrating the ability to remove smoke-like target interference into semantic segmentation, solving the problem that intelligent detection of forest fire smoke is easily interfered with by smoke-like targets such as clouds and fog.

[0053] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0054] Figure 1 This is a flowchart of the forest fire smoke semantic segmentation and detection method in the first embodiment of the present invention;

[0055] Figure 2 A technology roadmap for building large-scale forest fire smoke datasets;

[0056] Figure 3 This is a schematic diagram of forest fire smoke image synthesis based on AAE.

[0057] Figure 4 This is a schematic diagram of shallow convolution features based on modulation filters;

[0058] Figure 5 A schematic diagram of a deep semantic segmentation model for forest fire smoke;

[0059] Figure 6 This is a schematic diagram of the forest fire smoke semantic segmentation and detection system in the second embodiment of the present invention.

[0060] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0061] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0063] Please see Figure 1 The diagram shows a flowchart of a forest fire smoke semantic segmentation and detection method according to the first embodiment of the present invention. The method includes steps S01 to S05, wherein:

[0064] Step S01: Establish prior knowledge for guidance, including prior knowledge of forest fire smoke and prior knowledge of smoke-like interference in the forest fire background, so as to synthesize a dataset based on the prior knowledge of forest fire smoke and prior knowledge of smoke-like targets in the forest fire background;

[0065] It should be noted that the specific process of building the dataset includes:

[0066] As attached Figure 2 As shown in Part a, the prior knowledge used for guidance is first established, including two parts: forest fire smoke prior and smoke-like interference prior in the forest fire background.

[0067] As attached Figure 2 As shown in section b, "Smoke Simulation," the free motion of smoke is described and a model is established based on the Navier-Stokes equations in the field of computational fluid dynamics. After solving the Navier-Stokes equations, volume rendering methods are used with software such as Blender to visualize the simulated smoke and generate a large number of pure forest fire smoke images to construct a pure forest fire smoke image set.

[0068] As attached Figure 2As shown in section b, "Background Selection," on the one hand, background images that highlight complex forest environments are selected from international standard databases; on the other hand, real forest images are extensively collected based on prior knowledge obtained in the previous step, forming a background image set. In particular, images containing smoke-like interference in the forest fire background are selected and collected to construct the background image set.

[0069] A large-scale forest fire smoke composite dataset is formed by randomly synthesizing pure smoke images and forest fire smoke backgrounds based on the pure forest fire smoke image set and the background image set.

[0070] The following formula is used to synthesize forest fire smoke images or smoke-like target images:

[0071] i = b(1-α) + sα

[0072] Where i represents a synthesized forest fire smoke image or smoke-like target image, b represents a random image in the background image set, α represents the smoke's semi-transparency, and s represents a random image in the pure forest fire smoke image set.

[0073] The translucency of smoke, determined by transmission and scattering, is defined as α. The translucency is calculated using the following formula:

[0074]

[0075] Where d represents the propagation length of light through smoke particles, and τ(t) represents the attenuation coefficient of smoke particles at point t.

[0076] Since α physically reflects the smoke concentration value of the image, the scattering intensity s(d) of the smoke particles is further replaced by the product of s and α. Then s represents the result of the combined effect of smoke color and smoke translucency coefficient.

[0077] Step S02: Perform two-stage data augmentation on the dataset based on the adversarial autoencoder model to obtain the data-augmented dataset. The data-augmented dataset includes multiple forest fire smoke samples and smoke map labels corresponding to each forest fire smoke sample, as well as multiple smoke-like target samples and smoke map labels corresponding to each smoke-like target sample.

[0078] It should be noted that during the data augmentation process of the dataset, as shown in the attached document... Figure 2 As shown in section C, specifically:

[0079] The first stage is supervised data augmentation. The image is randomly rotated by a certain angle to change the orientation of the image content; the image is randomly flipped horizontally or vertically; the original image is scaled by a random ratio; the image is filtered using Gaussian kernels of different scales to construct a scale space; and then, sample images with variable content or varying degrees of blur are randomly generated in the scale space.

[0080] The second stage involves unsupervised data augmentation. An adversarial autoencoder model based on a generative adversarial network (GAN) is trained to generate diverse synthetic forest fire smoke images. (See attached image.) Figure 3 As shown, the input image is encoded by encoder E, and the resulting low-dimensional feature Z is used to replace white noise and fed into generator G and discriminator D for adversarial training, finally obtaining the forest fire smoke image (sample image) generated by the model.

[0081] Step S03: Input multiple forest fire smoke samples and smoke map labels corresponding to each forest fire smoke sample, as well as multiple smoke-like target samples and smoke map labels corresponding to each smoke-like target sample, into the forest fire deep semantic segmentation model for pre-training, so that the model outputs the initial network weights for distinguishing forest fire smoke and smoke-like targets.

[0082] It should be noted that the pre-training stage, which first enhances the model's robustness against common smoke-like targets in forest fires such as clouds and fog, is as follows: Figure 5 As shown in the diagram, after fully labeling the smoke in the synthetic database, it is fed into the ResNet model along with smoke-like targets for transfer learning. The trained model has already encoded information for distinguishing between smoke and smoke-like targets, and the encoded part in the recognition model is then reused as the encoded part in the smoke concentration estimation stage.

[0083] Furthermore, since deep features in deep models are mostly semantic information, this method only performs shallow feature learning based on modulated convolutional filters on shallow features. (See attached image) Figure 4 As shown, the controllable filter is combined with the shallow features learned in the deep network model using a method similar to element-wise multiplication to obtain the modulated filter. Then, the modulated filter is used to convolve the input feature map to generate the output feature map, thus eliminating the need to reset the weights.

[0084] Step S04: Take multiple forest fire smoke samples and the smoke image labels corresponding to each forest fire smoke sample as input to the forest fire deep semantic segmentation model to obtain the initial weights of the network, and output them in two branches. One branch outputs the background image, and the other branch outputs the pure forest fire smoke image and the smoke concentration corresponding to the pure forest fire smoke image.

[0085] In this step, as shown in the appendix Figure 5As shown, the two decoding branches are the forest fire background decoding branch and the pure forest fire smoke decoding branch. The outputs of the two branches are the background image, the pure forest fire smoke, and the alpha channel (smoke concentration), respectively. In order to minimize the noise interference of the synthesized image, the outputs of the two branches are further reconstructed into a forest fire image through convolution during the training phase.

[0086] Step S05: Synthesize a new forest fire smoke image based on the output background image, the pure forest fire smoke image, and the smoke concentration corresponding to the pure forest fire smoke image. Input the new forest fire smoke image into the forest fire deep semantic segmentation model for semantic segmentation and recognition training to obtain the trained forest fire deep semantic segmentation model.

[0087] Understandably, by continuously inputting new forest fire smoke images into the model for training until the obtained loss function value meets the training requirements, a trained forest fire deep semantic segmentation model is obtained. This trained forest fire deep semantic segmentation model can be applied to complex forest backgrounds and has high detection accuracy.

[0088] In summary, this paper proposes a semantic segmentation detection method for forest fire smoke, addressing the problem that conventional smoke detection methods are inapplicable to complex forest backgrounds, resulting in high false positive and false negative rates. Specifically targeting the inherent characteristics of forest fire smoke, this paper further simulates forest fire smoke using a fluid dynamics model, collects real complex forest images, and synthesizes a large-scale forest fire dataset, solving the problems of insufficient large-scale samples and difficulty in labeling that plague forest fire smoke detection. Based on this, a two-stage data augmentation approach is adopted, focusing on unsupervised data augmentation based on AAE (Automatic Image Augmentation). This learns the low-dimensional latent manifold distribution of high-dimensional synthetic forest fire smoke images, further approximating and reconstructing the manifold of real forest fire images. This allows for the generation of realistic forest fire smoke images based on given low-dimensional white noise, overcoming the limitations of synthetic images and further enhancing sample diversity. By using modulated convolutional filters to learn shallow features, since controllable filters do not need to participate in the backpropagation learning of the network, a reasonable modulation method can further improve the deep model's ability to model geometric deformations without increasing network parameters and training costs. The encoding network in the semantic segmentation model is pre-trained for smoke-like target classification. The trained model is then reused in the semantic segmentation model as an encoder, thereby integrating the ability to remove smoke-like target interference into semantic segmentation. This solves the problem that intelligent detection of forest fire smoke is easily interfered with by smoke-like targets such as clouds and fog.

[0089] Please see Figure 6 The diagram shows a schematic representation of the forest fire smoke semantic segmentation and detection system in the second embodiment of the present invention. The system includes:

[0090] The dataset synthesis module 10 is used to establish prior knowledge for guidance, the prior knowledge including prior knowledge of forest fire smoke and prior knowledge of smoke-like interference in the forest fire background, so as to synthesize a dataset based on the prior knowledge of forest fire smoke and prior knowledge of smoke-like targets in the forest fire background.

[0091] Furthermore, the dataset synthesis module 10 also includes:

[0092] The fluid dynamics model building unit is used to describe the free motion of smoke and build a model based on the Navier-Stokes equations in the field of computational fluid dynamics.

[0093] A pure forest fire smoke image set construction unit is used to visualize and simulate smoke using volume rendering after solving the Navier-Stokes equations, in order to generate multiple pure forest fire smoke images and construct a pure forest fire smoke image set based on these multiple images.

[0094] The background image set construction unit is used to collect real forest images based on prior knowledge, select background images of complex forest environments from international standard databases, and collect smoke-like target images with forest fire backgrounds, so as to construct a background image set based on the background images of complex forest environments, the real forest images, and smoke-like target images with forest fire backgrounds.

[0095] An image synthesis unit is used to randomly synthesize multiple forest fire smoke images and smoke-like target images based on the pure forest fire smoke image set and the background image set;

[0096] And it is used to synthesize forest fire smoke images or smoke-like target images according to the following formula:

[0097] i = b(1-α) + sα

[0098] Where i represents a synthesized forest fire smoke image or smoke-like target image, b represents a random image in the background image set, α represents the smoke's semi-transparency, and s represents a random image in the pure forest fire smoke image set.

[0099] Furthermore, the image synthesis unit also includes:

[0100] The semi-transparency calculation subunit is used to calculate the semi-transparency of the smoke according to the following formula:

[0101]

[0102] Where d represents the propagation length of light through smoke particles, and τ(t) represents the attenuation coefficient of smoke particles at point t.

[0103] The dataset augmentation module 20 is used to perform two-stage data augmentation on the dataset based on an adversarial autoencoder model to obtain a data-augmented dataset. The data-augmented dataset includes multiple forest fire smoke samples and smoke map labels corresponding to each forest fire smoke sample, as well as multiple smoke-like target samples and smoke map labels corresponding to each smoke-like target sample.

[0104] Furthermore, the dataset augmentation module 20 also includes:

[0105] A semantic segmentation and detection system for forest fire smoke, characterized in that the dataset augmentation module further includes:

[0106] The supervised data augmentation execution unit is used to perform supervised data augmentation on the dataset, including: randomly rotating the image to change the orientation of the image content; randomly flipping the image horizontally or vertically; scaling the original image by a random ratio; filtering the image with Gaussian kernels of different scales to construct a scale space, and randomly generating sample images with variable image content or varying degrees of blur in the scale space.

[0107] The unsupervised data augmentation execution unit is used to perform unsupervised data augmentation on the dataset. It encodes the images in the input dataset through an encoder, and then feeds the obtained low-dimensional features into the generator and discriminator instead of white noise for adversarial training, ultimately obtaining sample images generated by the adversarial generative network model.

[0108] The pre-training module 30 is used to input multiple forest fire smoke samples and smoke map labels corresponding to each forest fire smoke sample, as well as multiple smoke-like target samples and smoke map labels corresponding to each smoke-like target sample, into the forest fire deep semantic segmentation model for pre-training, so that the model outputs the initial network weights for distinguishing forest fire smoke and smoke-like targets.

[0109] The semantic segmentation module 40 is used to take multiple forest fire smoke samples and the smoke image labels corresponding to each forest fire smoke sample as input to the forest fire deep semantic segmentation model to obtain the initial weights of the network, and output them in two branches. One branch outputs the background image, and the other branch outputs the pure forest fire smoke image and the smoke concentration corresponding to the pure forest fire smoke image.

[0110] The semantic segmentation training module 50 is used to synthesize a new forest fire smoke image based on the output background image, the pure forest fire smoke image and the smoke concentration corresponding to the pure forest fire smoke image, and input the new forest fire smoke image into the forest fire deep semantic segmentation model for semantic segmentation and recognition training, so as to obtain the trained forest fire deep semantic segmentation model.

[0111] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0112] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A semantic segmentation and detection method for forest fire smoke, characterized in that, The method includes: Prior knowledge for guidance is established, including prior knowledge of forest fire smoke and prior knowledge of smoke-like interference in the forest fire background, to synthesize a dataset based on the prior knowledge of forest fire smoke and prior knowledge of smoke-like targets in the forest fire background, specifically including: The free motion of the smoke is described and modeled using the Navier-Stokes equations from the field of computational fluid dynamics. After solving the Navier-Stokes equations, the smoke simulation is visualized using volume rendering to generate multiple pure forest fire smoke images, and a pure forest fire smoke image set is constructed based on these multiple pure forest fire smoke images. Based on prior knowledge, real forest images are collected, and background images of complex forest environments are selected from international standard databases. Smoke-like target images with forest fire backgrounds are also collected to construct a background image set based on the background images of the complex forest environments, the real forest images, and the smoke-like target images with forest fire backgrounds. Multiple forest fire smoke images and smoke-like target images are randomly synthesized based on the pure forest fire smoke image set and the background image set; The step of randomly synthesizing multiple forest fire smoke images and smoke-like target images based on the pure forest fire smoke image set and the background image set includes: The following formula is used to synthesize forest fire smoke images or smoke-like target images: Where i represents a synthesized forest fire smoke image or smoke-like target image, b represents a random image in the background image set, α represents the semi-transparency of the smoke, and s represents a random image in the pure forest fire smoke image set. The dataset is augmented in two stages based on an adversarial autoencoder model to obtain an augmented dataset. The augmented dataset includes multiple forest fire smoke samples and smoke map labels corresponding to each forest fire smoke sample, as well as multiple smoke-like target samples and smoke map labels corresponding to each smoke-like target sample. Multiple forest fire smoke samples and smoke map labels corresponding to each forest fire smoke sample, as well as multiple smoke-like target samples and smoke map labels corresponding to each smoke-like target sample, are input into the forest fire deep semantic segmentation model for pre-training, so that the model outputs the initial network weights for distinguishing forest fire smoke and smoke-like targets. Multiple forest fire smoke samples and the smoke image labels corresponding to each forest fire smoke sample are used as inputs to obtain the initial weights of the forest fire deep semantic segmentation model. The output is divided into two branches, one branch outputs the background image and the other branch outputs the pure forest fire smoke image and the smoke concentration corresponding to the pure forest fire smoke image. A new forest fire smoke image is synthesized based on the output background image, the pure forest fire smoke image, and the smoke concentration corresponding to the pure forest fire smoke image. The new forest fire smoke image is then input into the forest fire deep semantic segmentation model for semantic segmentation and recognition training, resulting in the trained forest fire deep semantic segmentation model.

2. The forest fire smoke semantic segmentation and detection method according to claim 1, characterized in that, The translucency of the smoke is calculated using the following formula: Where d represents the distance light travels through smoke particles. This represents the attenuation coefficient of smoke particles at point t.

3. The forest fire smoke semantic segmentation and detection method according to claim 1, characterized in that, The step of performing two-stage data augmentation on the dataset based on an adversarial autoencoder model to obtain a data-augmented dataset, wherein the data-augmented dataset includes multiple forest fire smoke samples and smoke map labels corresponding to each forest fire smoke sample, and multiple smoke-like target samples and smoke map labels corresponding to each smoke-like target sample, includes: Supervised data augmentation of the dataset includes: randomly rotating images to change the orientation of image content; randomly flipping images horizontally or vertically; scaling the original images by random ratios; filtering images with Gaussian kernels of different scales to construct a scale space, and randomly generating sample images with variable content or varying degrees of blur in the scale space. Unsupervised data augmentation is performed on the dataset by encoding the images in the input dataset through an encoder, and the resulting low-dimensional features are used to replace white noise and fed into the generator and discriminator for adversarial training, ultimately obtaining sample images generated by the adversarial generative network model.

4. A semantic segmentation and detection system for forest fire smoke, characterized in that, The system includes: A dataset synthesis module is used to establish prior knowledge for guidance, the prior knowledge including prior knowledge of forest fire smoke and prior knowledge of smoke-like interference in the forest fire background, so as to synthesize a dataset based on the prior knowledge of forest fire smoke and prior knowledge of smoke-like targets in the forest fire background; The dataset synthesis module includes: The fluid dynamics model building unit is used to describe the free motion of smoke and build a model based on the Navier-Stokes equations in the field of computational fluid dynamics. A pure forest fire smoke image set construction unit is used to visualize and simulate smoke using volume rendering after solving the Navier-Stokes equations, in order to generate multiple pure forest fire smoke images and construct a pure forest fire smoke image set based on these multiple images. The background image set construction unit is used to collect real forest images based on prior knowledge, select background images of complex forest environments from international standard databases, and collect smoke-like target images with forest fire backgrounds, so as to construct a background image set based on the background images of complex forest environments, the real forest images, and smoke-like target images with forest fire backgrounds. An image synthesis unit is used to randomly synthesize multiple forest fire smoke images and smoke-like target images based on the pure forest fire smoke image set and the background image set; And it is used to synthesize forest fire smoke images or smoke-like target images according to the following formula: Where i represents a synthesized forest fire smoke image or smoke-like target image, b represents a random image in the background image set, α represents the semi-transparency of the smoke, and s represents a random image in the pure forest fire smoke image set. The dataset augmentation module is used to perform two-stage data augmentation on the dataset based on an adversarial autoencoder model to obtain a data-augmented dataset. The data-augmented dataset includes multiple forest fire smoke samples and smoke map labels corresponding to each forest fire smoke sample, as well as multiple smoke-like target samples and smoke map labels corresponding to each smoke-like target sample. The pre-training module is used to input multiple forest fire smoke samples and smoke map labels corresponding to each forest fire smoke sample, as well as multiple smoke-like target samples and smoke map labels corresponding to each smoke-like target sample, into the forest fire deep semantic segmentation model for pre-training, so that the model outputs the initial network weights for distinguishing forest fire smoke and smoke-like targets. The semantic segmentation module is used to take multiple forest fire smoke samples and the smoke image labels corresponding to each forest fire smoke sample as input to the forest fire deep semantic segmentation model to obtain the initial weights of the network, and outputs them in two branches. One branch outputs the background image, and the other branch outputs the pure forest fire smoke image and the smoke concentration corresponding to the pure forest fire smoke image. The semantic segmentation training module is used to synthesize a new forest fire smoke image based on the output background image, the pure forest fire smoke image, and the smoke concentration corresponding to the pure forest fire smoke image. The new forest fire smoke image is then input into the forest fire deep semantic segmentation model for semantic segmentation and recognition training to obtain the trained forest fire deep semantic segmentation model.

5. The forest fire smoke semantic segmentation and detection system according to claim 4, characterized in that, The image synthesis unit further includes: The semi-transparency calculation subunit is used to calculate the semi-transparency of the smoke according to the following formula: Where d represents the distance light travels through smoke particles. This represents the attenuation coefficient of smoke particles at point t.

6. The forest fire smoke semantic segmentation and detection system according to claim 4, characterized in that, The dataset enhancement module also includes: The supervised data augmentation execution unit is used to perform supervised data augmentation on the dataset, including: randomly rotating the image to change the orientation of the image content; randomly flipping the image horizontally or vertically; scaling the original image by a random ratio; filtering the image with Gaussian kernels of different scales to construct a scale space, and randomly generating sample images with variable image content or varying degrees of blur in the scale space. The unsupervised data augmentation execution unit is used to perform unsupervised data augmentation on the dataset. It encodes the images in the input dataset through an encoder, and then feeds the obtained low-dimensional features into the generator and discriminator instead of white noise for adversarial training, ultimately obtaining sample images generated by the adversarial generative network model.

Images