A forest fire image detection method based on generative adversarial network

Abstract

The present invention discloses a forest fire image detection method based on a generative adversarial network, and the steps are as follows: step 1: obtain forest fire monitoring images to form a data set; step 2: extract the image feature matrix of each image in the data set; step 3: construct the dictionary and classifier parameters in the discriminator and generator through deep dictionary learning; step 4: use the image feature matrix as the input of the discriminator, obtain the dictionary and sparse matrix of each layer in the discriminator; obtain the dictionary and generated image of each layer in the generator; step 5: according to the objective function, repeat step 4 until the dictionary, sparse matrix and generated image are stable; step 6: use the stabilized generative adversarial network to detect forest fire. The present invention constructs the dictionary and classifier parameters in the discriminator and generator of the generative adversarial network through deep dictionary learning, and the constructed generative adversarial network can more effectively identify anomalies, making anomaly detection more accurate.

Description

Technical Field

[0001] The invention relates to the field of image detection technology, and in particular to a forest fire image detection method based on a generative adversarial network. Background Art

[0002] The suddenness and destructiveness of forest fires pose a huge threat to the ecological environment, human life and property safety, especially in rugged mountainous areas, where transportation and communication are difficult, which poses a huge challenge to firefighting. Therefore, forest fires have always been a global hot issue, and the problems of prevention and control and firefighting need to be solved urgently. China has a vast territory and a wide variety of forest resources. Therefore, using computers to automatically detect forest fires and strengthen the comprehensive management capabilities of forest fires is of great significance to protecting my country's rich forest resources.

[0003] The forest fire detection system can be regarded as an anomaly detection problem. Most of the current forest fire detection systems use convolutional neural networks or random forests. In practical applications, it is difficult to quickly extract accurate features of images. Therefore, how to quickly and accurately extract image features and efficiently detect forest fires is an urgent problem to be solved.

[0004] In the existing technology, generative adversarial networks (GANs) are often used for image generation, and research has also shown their potential in image recognition. GAN networks mainly expand training data sets or enhance supervised learning models by generating realistic images, but the generated images may have problems with edge details and authenticity, and may not fully reflect the distribution of the original data, which may reduce the effect of data enhancement or introduce unrealistic samples, making it difficult to improve the final recognition effect of the image. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a forest fireworks image detection method based on a generative adversarial network to solve the technical problems of low detection speed and accuracy of forest fireworks and fires in the prior art.

[0006] The present invention provides a forest fire image detection method based on a generative adversarial network, comprising the following steps:

[0007] Step 1: Obtain forest fire monitoring images to form a data set, and convert the images in the data set into grayscale images;

[0008] Step 2: Extract the image feature matrix of each image in the dataset;

[0009] Step 3: Construct a generative adversarial network, in which the dictionary and classifier parameters in the discriminator and generator are constructed through deep dictionary learning;

[0010] Step 4: Use the image feature matrix as the input of the discriminator to obtain the dictionary and sparse matrix of each layer in the discriminator;

[0011] Given a random noise and a label in the generator, get the dictionary and generated image of each layer in the generator;

[0012] Step 5: According to the objective function of the discriminator and the objective function of the generator, step 4 is repeated until the dictionary and sparse matrix in the discriminator and the dictionary and generated image in the generator are stable;

[0013] Step 6: Use the stabilized generative adversarial network to detect forest fires.

[0014] Furthermore, in step 2, the image feature matrix of each image in the data set is extracted by using the SIFT method.

[0015] Furthermore, in step 4, obtaining the dictionary and sparse matrix of each layer in the discriminator specifically includes: a method for obtaining the dictionary and sparse matrix of non-last layers and a method for obtaining the dictionary and sparse matrix of the last layer, wherein:

[0016] The method to obtain the dictionary and sparse matrix of non-last layer is:

[0017] By solving the formula:

[0018]

[0019] Get each layer dictionary;

[0020] By solving the formula:

[0021]

[0022] Get the sparse matrix of each layer;

[0023] The method to obtain the dictionary and sparse matrix of the last layer is:

[0024] By solving the formula:

[0025]

[0026] Get the dictionary of the last layer;

[0027] By solving the formula:

[0028]

[0029] Get the sparse matrix of the last layer;

[0030] In the formula, Z n-1 is the sparse coding of the n-1th layer; D nis the dictionary at level n; Z n is the sparse coding of the nth layer; λ1 is the regularization parameter; P n is the nth layer local adapter; W is the weight matrix of the linear classifier to be optimized; Y is the label of the sample.

[0031] Furthermore, the process of obtaining the dictionary and sparse matrix of the last layer also includes:

[0032] By solving the formula:

[0033]

[0034] Get the parameters of the discriminator;

[0035] Where W is the weight matrix of the linear classifier to be optimized; Z n is the sparse coding of the nth layer; Y is the label of the sample; λ c is the regularization parameter.

[0036] Furthermore, in step 4, the specific method of obtaining the dictionary of each layer in the generator and generating the image is:

[0037] Step A41: performing an iterative process of acquiring the dictionary and generated image of the current layer layer by layer starting from the last layer to the previous layer, wherein the generated image acquired at the current layer is used as the sparse code of the previous layer;

[0038] Step A42: input the generated image of the first layer into the discriminator, obtain the sparse code, and use the sparse code as the sparse code in the last layer in the next iteration process;

[0039] Step A43: Execute the process of step A41-step A42 several times, and take the dictionary and generated image of each layer finally generated by step A41 as the result.

[0040] Furthermore, by solving the formula:

[0041]

[0042] Get the dictionary of the current layer;

[0043] By solving the formula:

[0044]

[0045] Get the generated image of the current layer;

[0046] In the formula, is the input of the i-th layer; B i is the dictionary of the i-th layer of the generator; G i is the output of the i-th layer of the generator; is the i-th layer local adapter.

[0047] Furthermore, in step 5, the objective function of the discriminator is:

[0048]

[0049] st:X i =Z i-1

[0050] In the formula, X i is the i-th layer input of the discriminator; D i is the dictionary of the i-th level; Z i is the sparse code of the i-th layer; λ1 and λ c is the regularization parameter; P i is the i-th layer local adapter.

[0051] Furthermore, in step 5, the objective function of the generator is:

[0052]

[0053] In the formula, is the input of the i-th layer; B i is the dictionary of the i-th layer of the generator; λ1, λ2 and λ c is the regularization parameter; G i is the output of the i-th layer of the generator; is the local adapter at layer i; X i is the i-th layer input of the discriminator; D i is the dictionary of the i-th level; Z i is the sparse code of the i-th layer; P i is the i-th layer local adapter; Y is the label of the sample.

[0054] Beneficial effects of the present invention:

[0055] 1. The present invention pre-processes each training image using SIFT feature extraction, learns the dictionary and sparse coding of the feature matrix, and the obtained feature vector can more accurately describe the characteristics of the image.

[0056] 2. The present invention constructs the dictionary and classifier parameters in the discriminator and generator of the generative adversarial network through deep dictionary learning. By combining the reconstruction error and the discriminator score, the constructed generative adversarial network can more effectively identify anomalies. This combination utilizes the advantages of the two components of GAN, making anomaly detection more accurate.

[0057] 3. The forest fire image detection method based on generative adversarial network only needs to calculate the sparse relationship between training samples, does not require pre-testing of hyperparameters, has strong generalization ability, and can be better combined with actual detection situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0059] Figure 1 It is a schematic diagram of a flow chart of a specific embodiment of the present invention;

[0060] Figure 2 It is a schematic diagram of the result of the SIFT feature extraction method in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0062] The present invention is further illustrated below in conjunction with specific embodiments. Those skilled in the art should understand that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention, and modifications to various equivalent forms of the present invention fall within the scope defined by the appended claims of this application.

[0063] like Figure 1 As shown, the present invention provides a forest fire image detection method based on a generative adversarial network. The training image is converted into a grayscale image, and after feature extraction is performed using the SIFT method, the feature vector is used as the input of the discriminator in the generative adversarial network, the dictionary and sparse coding of the training image are calculated, and a random noise z and a label Y are given as the input of the generator, and a picture that is highly similar to the training set is generated to continuously improve the discriminator's ability to obtain the dictionary and sparse coding of the training image. Finally, the difference between the test image and the image generated by the generator and the feature difference extracted after the generated fake picture and the test picture pass through the discriminator are calculated. Since the difference score obtained in the test of the image with forest fire situation will be different from the difference score in the retest of the normal forest image, forest fire monitoring can be achieved. The generative adversarial network-based method adopted by the present invention can more accurately describe the characteristics of the image, so as to improve the accuracy of judging whether there are fireworks in the forest, and at the same time has a strong generalization ability, so that anomaly detection is more accurate. The specific steps are as follows:

[0064] Step S1: Based on the forest fire video monitoring system, collect the training data set and the test data set by manual collection or automatic collection by the existing monitoring system;

[0065] Convert the images in the training dataset into grayscale images of size 5677×128;

[0066] Step S2: Use the batch training method for the images in the data set. First, pre-process each image using SIFT feature extraction to obtain the image feature matrix X i , where X i ∈R d×N , d is the number of feature points extracted, N is the dimension of each feature point, and the extraction results are as follows Figure 2 As shown;

[0067] Step S3: constructing a generative adversarial network, wherein the dictionary and classifier parameters in the discriminator and the generator are constructed through deep dictionary learning;

[0068] Step 4: Use the image feature matrix as the input of the discriminator to obtain the dictionary and sparse matrix of each layer in the discriminator;

[0069] Taking three-layer dictionary learning as an example, the dictionary and sparse coding of each layer are initialized to a matrix of all 0s.

[0070] For the first-level dictionary D 1 and sparse coding Z 1 , through the formula:

[0071]

[0072] Calculate and optimize dictionary D 1 and sparse coding Z 1 ;

[0073] The specific analytical solution process is as follows:

[0074] According to the formula:

[0075]

[0076] For the feature matrix X 1 Each column in Calculate separately

[0077]

[0078] In the formula, make but:

[0079]

[0080] According to the formula:

[0081]

[0082] In the formula, X 1 ∈R d×N , For the feature matrix X 1 Each column in have

[0083]

[0084] According to D 1 =(V 1 ) T (U 1 ) -1 The first-level dictionary D can be calculated 1 , where

[0085]

[0086] For the second-level dictionary D 2 and sparse coding Z 2 , through the formula:

[0087]

[0088]

[0089] Calculate and optimize dictionary D 2 and sparse coding Z 2 , the analytical solution process is as follows: 1 and sparse coding Z 1 Analytical solution process.

[0090] For the third-level dictionary D 3 and sparse coding Z 3 , through the following formula:

[0091]

[0092] Calculate and optimize dictionary D 3 , sparse coding Z 3 , dictionary D 3 The analytical solution process of the dictionary D 1 Analytical solution process.

[0093] Sparse coding Z 3 The analytical solution process is as follows:

[0094]

[0095] In the formula,

[0096] make Then there is

[0097] For the discriminator parameter W, according to the formula:

[0098]

[0099] It can be calculated that W = (Z 3 (Z 3 ) T +λ c ) -1 Z 3 Y,

[0100] In the formula, Z n-1 is the sparse coding of the n-1th layer; D n is the dictionary at level n; Z n is the sparse coding of the nth layer; λ1 is the regularization parameter, which is used to balance the trade-off between the data fitting term and the regularization term in the optimization objective; P n is the local adapter at layer n, calculated as in yes and The Euclidean distance between them, j∈[1,d n ] n is the number of dictionary elements in the nth layer, σ is used to adjust the weight decay rate of the local adapter; W is the weight matrix of the linear classifier to be optimized; Z n is the sparse coding of the nth layer; Y is the label of the sample; λ c is the regularization parameter, which is used to balance the trade-off between the data fitting term and the regularization term in the optimization objective.

[0101] By calculating the dictionary and sparse coding of the training samples, the dictionary and sparse coding of the normal forest image can be well obtained.

[0102] Given a random noise and a label in the generator, get the dictionary and generated image of each layer in the generator;

[0103] Taking three-layer dictionary learning as an example, given a random noise and the label Y corresponding to the training sample,

[0104] The third layer solves the following two problems:

[0105]

[0106] Get dictionary B respectively 3 , generate image G 3 , the analytical solution process is as follows:

[0107] For random noise Each column in Calculate separately

[0108]

[0109] In the formula,

[0110] make Then there is

[0111] For random noise Each column in According to (B 3 ) -1 =(V 3 ) T (U 3 ) -1 Computable third-layer dictionary B 3 ,

[0112] in,

[0113]

[0114] G 3 Sparse coding as the second layer

[0115] The second layer is solved by:

[0116]

[0117] Get dictionary B respectively 2 , generate image G 2 .

[0118] Then G 2 Sparse coding as the first layer

[0119] The first layer is solved by:

[0120]

[0121] Get dictionary B respectively 1 , generate image G 1 ,

[0122] In the formula, is the input of the i-th layer; B i is the dictionary of the i-th layer of the generator; λ2 is the regularization parameter, which is used to balance the trade-off between the data fitting term and the regularization term in the optimization objective; G i is the output of the i-th layer of the generator; is the local adapter at layer i, calculated as in yes and The Euclidean distance between them, j∈[1,d i ] i is the number of dictionary elements in the i-th layer, σ is used to adjust the weight decay speed of the local adapter; X i is the i-th layer input of the discriminator; D i is the dictionary of the i-th level; Z i is the sparse code of the i-th layer; λ1 and λ c is the regularization parameter, which is used to balance the trade-off between the data fitting term and the regularization term in the optimization objective; P i is the local adapter at layer i, calculated as in yes and The Euclidean distance between them, j∈[1,d i ] i is the number of dictionary elements in the i-th layer, σ is used to adjust the weight decay rate of the local adapter; W is the weight matrix of the linear classifier to be optimized; Y is the label of the sample.

[0123] G 1 As input, it is put into the discriminator, and the discriminator is used to discriminate the method of generating data, and the calculation is Then The input is given to the generator, and the above process is repeated n times. The dictionary and sparse coding of each layer generated in the last time are taken as the result. The discriminator is used to discriminate the generated data to generate pictures that are highly similar to the training set, thereby improving recognition.

[0124] Step 5: According to the objective function of the discriminator and the objective function of the generator, step 4 is repeated until the dictionary and sparse matrix in the discriminator and the dictionary and generated image in the generator are stable;

[0125] The discriminator uses the objective function:

[0126]

[0127] st:X i =Z i-1

[0128] Get the dictionary D of each layer i and the sparse matrix Z i , and optimized using the back-propagation algorithm;

[0129] The objective function of the generator is:

[0130]

[0131] Optimizing the generator G i and B i ,

[0132] is the input of the i-th layer; B i is the dictionary of the i-th layer of the generator; λ1, λ2 and λ c is the regularization parameter, which is used to balance the trade-off between the data fitting term and the regularization term in the optimization objective; G i is the output of the i-th layer of the generator; is the local adapter at layer i, calculated as in yes and The Euclidean distance between them, j∈[1,d i ] i is the number of dictionary elements in the i-th layer, σ is used to adjust the weight decay speed of the local adapter; after multiple iterative calculations, the generated image G is used as the input of the discriminator, X i is the i-th layer input of the discriminator; D i is the dictionary of the i-th level; Z i is the sparse code of the i-th layer; P i is the local adapter at layer i, calculated as in yes and The Euclidean distance between them, j∈[1,d i ] i is the number of dictionary elements in the i-th layer, σ is used to adjust the weight decay rate of the local adapter; W is the weight matrix of the linear classifier to be optimized; Y is the label of the sample;

[0133] Step 6: Use the stabilized generative adversarial network to detect forest fires.

[0134] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A forest fire image detection method based on generative adversarial network, characterized in that: The steps include: Step 1: Obtain forest fire monitoring images to form a data set, and convert the images in the data set into grayscale images; Step 2: Extract the image feature matrix of each image in the dataset; Step 3: Construct a generative adversarial network, in which the dictionary and classifier parameters in the discriminator and generator are constructed through deep dictionary learning; Step 4: Use the image feature matrix as the input of the discriminator to obtain the dictionary and sparse matrix of each layer in the discriminator; Given a random noise and a label in the generator, get the dictionary and generated image of each layer in the generator; Step 5: According to the objective function of the discriminator and the objective function of the generator, step 4 is repeated until the dictionary and sparse matrix in the discriminator and the dictionary and generated image in the generator are stable; Step 6: Use the stabilized generative adversarial network to detect forest fires.

2. The forest fire image detection method based on generative adversarial network according to claim 1, characterized in that: In step 2, the image feature matrix of each image in the data set is extracted by using the SIFT method.

3. The forest fire image detection method based on generative adversarial network according to claim 1, characterized in that: In step 4, obtaining the dictionary and sparse matrix of each layer in the discriminator specifically includes: a method for obtaining the dictionary and sparse matrix of non-last layers and a method for obtaining the dictionary and sparse matrix of the last layer, wherein: The method to obtain the dictionary and sparse matrix of non-last layer is: By solving the formula: Get each layer dictionary; By solving the formula: Get the sparse matrix of each layer; Where, X i is the i-th layer input of the discriminator; D i is the dictionary of the i-th level; Z i is the sparse code of the i-th layer; P i is the local adapter at layer i; The method to obtain the dictionary and sparse matrix of the last layer is: By solving the formula: Get the dictionary of the last layer; By solving the formula: Get the sparse matrix of the last layer; In the formula, Z n-1 is the sparse coding of the n-1th layer; D n is the dictionary at level n; Z n is the sparse coding of the nth layer; λ1 is the regularization parameter; P n is the nth layer local adapter; W is the weight matrix of the linear classifier to be optimized; Y is the label of the sample.

4. The forest fire image detection method based on generative adversarial network according to claim 3, characterized in that: The process of obtaining the dictionary and sparse matrix of the last layer also includes: By solving the formula: Get the parameters of the discriminator; Where W is the weight matrix of the linear classifier to be optimized; Z n is the sparse coding of the nth layer; Y is the label of the sample; λ c is the regularization parameter.

5. The forest fire image detection method based on generative adversarial network according to claim 1, characterized in that: In step 4, the specific method of obtaining the dictionary of each layer in the generator and generating the image is: Step A41: performing an iterative process of acquiring the dictionary and generated image of the current layer layer by layer starting from the last layer to the previous layer, wherein the generated image acquired at the current layer is used as the sparse code of the previous layer; Step A42: input the generated image of the first layer into the discriminator, obtain the sparse code, and use the sparse code as the sparse code in the last layer in the next iteration process; Step A43: Execute the process of step A41-step A42 several times, and take the dictionary and generated image of each layer finally generated by step A41 as the result.

6. The forest fire image detection method based on generative adversarial network according to claim 5, characterized in that: By solving the formula: Get the dictionary of the current layer; By solving the formula: Get the generated image of the current layer; In the formula, is the input of the i-th layer; B i is the dictionary of the i-th layer of the generator; G i is the output of the i-th layer of the generator; is the local adapter of the i-th layer; λ2 is the regularization parameter.

7. The forest fire image detection method based on generative adversarial network according to claim 1, characterized in that: In step 5, the objective function of the discriminator is: s.t.X i =Z i-1 Where, X i is the i-th layer input of the discriminator; D i is the dictionary of the i-th level; Z i is the sparse code of the i-th layer; λ1 and λ c is the regularization parameter; P i is the local adapter at layer i; Z n is the sparse coding of the nth layer; Y is the label of the sample; W is the weight matrix of the linear classifier to be optimized.

8. The forest fire image detection method based on generative adversarial network according to claim 1 or 7, characterized in that: In step 5, the objective function of the generator is: In the formula, is the input of the i-th layer; B i is the dictionary of the i-th layer of the generator; λ1, λ2 and λ c is the regularization parameter; G i is the output of the i-th layer of the generator; is the local adapter at layer i; X i is the i-th layer input of the discriminator; D i is the dictionary of the i-th level; Z i is the sparse code of the i-th layer; P i is the i-th layer local adapter; Y is the label of the sample; W is the weight matrix of the linear classifier to be optimized; Z n is the sparse coding of the nth layer.

Images