High-precision flame area extraction method based on multi-feature extraction of forest fire scene

Abstract

The present invention provides a high-precision flame region extraction method based on multi-feature extraction of forest fires, and the scheme includes the acquisition of actual fire images and conventional flame data sets, preprocessing of actual fire images, construction of actual fire data sets, design of multiple feature extraction projects, model training and optimization, and extraction of flame regions. The present invention divides image pixels into multiple cluster blocks through a clustering block algorithm, extracts image features using feature engineering, and labels fire and non-fire at the same time, introduces CNN convolutional neural network training, and optimizes the training model for flame region extraction using Adam optimizer, and finally realizes high-precision extraction of flame regions. The flame region extraction method integrates multiple features. Compared with conventional flame region extraction methods, the algorithm calculation complexity of this method is low, the speed of flame region extraction is fast, and the flame region extraction is guaranteed to be highly accurate, providing accurate fire situation for firefighters who rescue fires in complex fire environments.

Description

Technical Field

[0001] The present invention belongs to the interdisciplinary technical field of fire safety technology, image segmentation technology, and computer vision recognition technology, and is specifically a high-precision flame area extraction method based on multi-feature extraction of forest fire scenes, and relates to a method for identifying flames and extracting flame areas through multiple feature conditions in forest fire scenes. Background Art

[0002] Forest fires are extremely harmful to the natural environment. Due to the complexity and high risk of fires, it is difficult for firefighters to put out fires. Therefore, the extraction of flame areas, understanding the actual fire situation through flame areas, and judging the flame trend are of great significance for fire fighting. On the one hand, the fire scene is complex. If large equipment is carried, it may affect the efficiency of fire fighting and may threaten the lives of firefighters. Therefore, it is more appropriate to use portable experimental equipment with flame area extraction algorithm engineering. On the other hand, firefighters' naked eye observation of the fire scene is often limited by the field of vision. They are also threatened by the uncertainty of the flame spread speed and cannot accurately judge the fire situation, which puts them in danger of the fire scene. It is necessary to find a way to accurately obtain the actual situation of the fire scene and understand the fire situation.

[0003] At present, the research on flame area extraction is mainly concentrated on fire scene simulation, with emphasis on flame early warning and fire scene reproduction. However, there are few related patents on the identification, positioning and extraction of flame areas. The simulation of fire scenes is still very different from the real fire scenes. Summary of the invention

[0004] In view of the defects and shortcomings of the prior art, the purpose of the present invention is to provide a high-precision flame area extraction method based on multi-feature extraction of forest fire scenes, which uses computer vision and clustering technology to identify and extract the flame area in the fire scene, so as to assist firefighters in obtaining the actual fire situation, making correct judgments on the forest fire situation, warning of fire information, and making further decisions while ensuring life safety.

[0005] The scheme includes the acquisition of actual fire scene images and conventional flame data sets, preprocessing of actual fire scene images, construction of actual fire scene data sets, design of multiple feature extraction projects, model training and optimization, and extraction of flame areas. The present invention divides image pixels into multiple cluster blocks through a clustering block algorithm, extracts image features using feature engineering, and labels fire and non-fire at the same time, introduces CNN convolutional neural network training, and optimizes the training model for flame area extraction with the Adam optimizer, and finally realizes high-precision extraction of flame areas. The flame area extraction method combines multiple features. Compared with the conventional flame area extraction method, the algorithm calculation complexity of this method is low, the speed of flame area extraction is fast, and the flame area extraction is guaranteed to be highly accurate, providing accurate fire scene conditions for firefighters who rescue fires in complex fire environments, improving the firefighting efficiency of firefighters, and also providing a certain guarantee for the life safety of firefighters.

[0006] The method of the present invention can identify and extract the flame area with high precision in a real fire scene, obtain a larger field of view and more accurate field of view observation, and make early warnings to ensure the safety of firefighters. The present invention can be used in drone monitoring systems, fire monitoring systems, portable fire rescue equipment, and fire safety robots that assist in fire fighting.

[0007] The technical solution specifically adopted by the present invention to solve the technical problem is:

[0008] A high-precision flame area extraction method based on multi-feature extraction of forest fire scene, characterized in that it comprises the following steps:

[0009] Step S1: Obtain flame data set and actual fire scene image;

[0010] Step S2: preprocessing the actual fire scene images, constructing the processed actual fire scene images to obtain an actual fire scene dataset, and dividing the actual fire scene dataset and the conventional flame dataset into a training set, a test set, and a validation set;

[0011] Step S3: Calculate the ratio of the cumulative area of ​​the flame region to the cumulative area of ​​the corresponding cluster block, and use it to adjust the number K of cluster blocks according to the ratio;

[0012] Step S4: extracting image features by feature engineering, and judging whether there is fire in the image cluster block according to the area occupied by the flame in the cluster block, and labeling the cluster as having fire or not having fire;

[0013] Step S5: the labels and the obtained feature vectors are input into the CNN convolutional neural network, and training is performed based on the training set to obtain a flame area extraction model, and the model parameters are adjusted based on the validation set;

[0014] Step S6: Use the trained flame region extraction model to extract the flame region. Here, the effect of the flame region extraction can be verified by using a test set first, and then applied to the actual fire scene data.

[0015] Furthermore, in step S1, the adopted data sets include the flame data set Corsican data set and the actual fire scene data set obtained based on the UAV system.

[0016] Furthermore, in step S2, all actual fire scene data sets are used as test sets, and the public data set Corsican data set is divided into data sets in the ratio of training set: test set: validation set = 6:2:2.

[0017] Furthermore, in step S3, the number of cluster blocks K in the clustering block algorithm is set, and the ratio of the cumulative area of ​​the flame region to the cumulative area of ​​the corresponding cluster block is calculated. The K value is selected according to the size of the ratio, and the selection rule is: if it is greater than the threshold value α, the image pixels in the data set are divided into K cluster blocks according to the distance between each cluster center through the clustering block algorithm; otherwise, the number of cluster blocks K is adjusted until the threshold value α is met, so that the clustering block is more uniform, so as to improve the flame area extraction effect.

[0018] Furthermore, in step S4, color space, shape and texture features are combined, where three color spaces, RGB, YCbCr and HSV, are selected, including R, G, B, Y, Cb, Cr, H, S, V, 9 color features, two components of circularity and rectangularity, and four feature vectors of contrast, correlation, entropy and energy, a total of 15-dimensional vector feature engineering is performed to extract image features.

[0019] Further, in step S4, the cluster block is judged whether there is fire by judging the condition, and the cluster block is labeled, and the label corresponds to the image feature vector, so that the extracted feature vector is associated with the fire;

[0020] Among them, whether it is a flame is judged by the proportion of the flame area to the cluster block area; the judgment conditions are: more than 2 / 3 is judged as fire; less than 1 / 3 is judged as non-fire, and between 1 / 3 and 23 is judged as unpredictable; the cluster blocks are labeled, with fire marked as 1, no fire marked as 0, and unpredictable marked as -1; the image cluster blocks are classified into fire and no fire, and the labels and the 15-dimensional feature vector form a 16-dimensional feature vector.

[0021] Furthermore, in step S5, a clustering block algorithm is first used to perform clustering block processing, and then a feature vector is obtained through feature engineering. The obtained feature vector and the label form a 16-dimensional feature vector and are put into a CNN convolutional neural network for training. After several convolutional layers and pooling layers, the feature dimension is reduced. Finally, after two fully connected layers and a sigmoid function, the Adam optimizer is used to optimize the training to obtain a flame area extraction model for extracting the flame area of ​​the image.

[0022] Compared with the prior art, the data set of the present invention and its preferred solution is sourced from real fire scene images captured by drone high-speed cameras, which shows that this method has practical benefits for real fire scenes. The feature extraction part of the image data set uses multiple features such as color space, shape and texture, and extracts a total of 15-dimensional feature vectors. The number of features is large, and the flame recognition effect is more scientific and reliable. The method can automatically select the number of cluster blocks K value, and adjust the number of cluster blocks by setting a specified threshold by calculating the ratio of the cumulative area of ​​the flame area to the cumulative area of ​​the cluster blocks with fire. If the threshold is met, the clustering block algorithm is used to cluster and block the image pixels. If it is not met, the number of cluster blocks is adjusted, so that the clustering block is more uniform, and the flame extraction effect is also optimized. The algorithm has low computational complexity and is relatively easy to implement. The flame area extraction speed is fast, which is conducive to firefighters to accurately obtain the actual situation of the fire scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0024] Figure 1 is an overall flow chart of an embodiment of the present invention;

[0025] Figure 2 It is a partial image of the actual fire scene data set of the embodiment of the present invention;

[0026] Figure 3 It is a partial image of the Corsian dataset of an embodiment of the present invention;

[0027] Figure 4 It is a structural diagram of a CNN convolutional neural network according to an embodiment of the present invention;

[0028] Figure 5 This is a clustering and blocking effect diagram of an embodiment of the present invention;

[0029] Figure 6 It is a flame area extraction diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail as follows:

[0031] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0032] like Figure 1-Figure 6 As shown, the present invention provides a high-precision flame area extraction method based on forest fire multi-feature extraction, including the steps of acquiring actual fire scene images and conventional flame data sets, preprocessing actual fire scene images, constructing actual fire scene data sets, designing multiple feature extraction projects, model training and optimization, and extracting flame areas, specifically including:

[0033] Step S1: The requirements for obtaining the actual fire scene images are as follows: obtain conventional flame and actual fire scene images. Conventional flame images mainly come from the public dataset Corsican dataset, and actual fire scene images come from the image capture of the high-speed camera of the UAV system, including visible light and infrared images. The public dataset Corsican dataset includes various images and image sequences, with visible light and near-infrared images, including 500 visible spectrum images and corresponding labeling information, 100 pairs of visible light and near-infrared images, and five multimodal sequences with visible light and near-infrared pairs.

[0034] Step S2: Preprocess the actual fire scene images, mainly including image registration and image enhancement, to obtain the actual fire scene dataset. All the actual fire scene datasets obtained are used as test sets, and the public dataset Corsican dataset is divided into the dataset in the ratio of training set: test set: validation set = 6:2:2.

[0035] Step S3: Calculate the ratio of the cumulative area of ​​the flame region to the cumulative area of ​​the corresponding cluster block, set a threshold, and automatically adjust the number of cluster blocks K value according to whether the threshold is met. If it is greater than the threshold, the cluster block algorithm is used to divide the image pixels into K cluster blocks. If it is less than the threshold, the number of cluster blocks K value is reset, and the size of the cluster blocks is adjusted to adjust the effect of cluster block.

[0036] As a preferred method, the slic algorithm is used for the clustering block algorithm. The core idea is to convert the visible light image into a five-dimensional feature vector in the CIELAB color space and rectangular coordinate system. The distance metric is constructed by the five-dimensional feature vector to achieve local clustering of image pixels. The clustering block algorithm divides the image pixels into K cluster blocks of approximately uniform size to reduce the complexity of the image. The distance metric combines the color distance and the spatial distance. The mathematical expressions of the color distance and the spatial distance are as follows:

[0037]

[0038]

[0039]

[0040] Among them, d c Indicates the color distance, d s It is represented as spatial distance and D′ is represented as distance metric.

[0041] Step S4: Extract image features using feature engineering, and determine whether there is fire in the image cluster block based on the area of ​​the flame in the cluster block, and label the cluster as having fire or not.

[0042] In this embodiment, feature engineering mainly extracts the features of image color space, shape and texture. The color space selects three color spaces: RGB, YCbCr and HSV, and obtains 9 color features: R, G, B, Y, Cb, Cr, H, S, V, which are represented by the color feature vector C = (R, G, B, Y, Cb, Cr, H, S, V); the shape feature calculates the two components of circularity Circ and rectangularity Rect, which are represented by the shape feature vector G = (Circ, Rect); the texture feature uses the gray level co-occurrence matrix (GLCM) to obtain four feature vectors: contrast Cont, correlation Corr, energy Ener and entropy value Entr. The extracted features constitute a 15-dimensional feature vector, V = (R, G, B, Y, Cb, Cr, H, S, V, Circ, Rect, Cont, Corr, Ener, Entr).

[0043] The elements in the gray level co-occurrence matrix represent the joint distribution of the gray levels of two pixels with a certain spatial position relationship, and its definition is:

[0044] P(i,j|d,θ)

[0045] Where d is the spatial distance, θ is the direction, i is the number of rows, and j is the number of columns. The probability of gray level j (column) appearing with i as the starting point (row) (normalizing the frequency, i.e. dividing by the sum of all frequencies) constitutes the gray level co-occurrence matrix.

[0046] The mathematical expressions for extracting contrast Cont, correlation Corr, energy Ener and entropy Entr based on the gray level co-occurrence matrix (GLCM) are as follows:

[0047]

[0048]

[0049]

[0050]

[0051] where μ x , μ y is the mean value of the pixel (x, y), σ x , σ y is the variance of the pixel (x,y).

[0052] The main method of labeling cluster blocks is to determine whether there is fire in the image cluster block based on the area occupied by the flame. If there is fire, it is marked as 1, if there is no fire, it is marked as 0, and if it cannot be determined, it is marked as -1. The proportion of the flame area to the cluster block area is used to determine whether it is a cluster block with fire. The judgment conditions are: if it exceeds 2 / 3, it is judged to be fire; if it is less than 1 / 3, it is judged to be no fire; if it is between 1 / 3 and 2 / 3, it is judged to be impossible to determine. The mathematical expression of the label is as follows:

[0053]

[0054]

[0055] Among them, R sp is the area of ​​the current cluster block, R GT is the area covered by the ground truth, R sp_e is the area of ​​the incorrectly segmented superpixel region, and e is the minimum error blocking ratio.

[0056] Step S5: The label and feature vector V are input into the CNN convolutional neural network together, and the feature dimension is reduced through several convolutional layers and maximum pooling. Finally, the cluster blocks are classified through two fully connected layers and sigmoid function to obtain the flame area extraction model. The model optimizer uses the Adam optimizer. Experimental comparison shows that the Adam optimizer has better optimization effect than the SGD optimizer. The loss function uses the binary cross entropy loss function, and the mathematical expression of the loss function is as follows:

[0057]

[0058] where y i represents the true value, Represents the predicted value.

[0059] Step S6: Put the test set image into the flame region extraction model obtained after training. In this embodiment, the flame region in the image is extracted, and the accuracy rate of flame region extraction for the dataset image reaches 96.58%.

[0060] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0061] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0062] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0064] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

[0065] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive other various forms of high-precision flame area extraction methods based on multi-feature extraction of forest fire scenes under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of this invention should be covered by this patent.

Claims

1. A high-precision flame area extraction method based on multi-feature extraction of forest fire scenes, characterized in that: The following steps are involved: Step S1: Obtain flame data set and actual fire scene image; Step S2: preprocessing the actual fire scene images, constructing the processed actual fire scene images to obtain an actual fire scene dataset, and dividing the actual fire scene dataset and the conventional flame dataset into a training set, a test set, and a validation set; Step S3: Calculate the ratio of the cumulative area of ​​the flame region to the cumulative area of ​​the corresponding cluster block, and use it to adjust the number K of cluster blocks according to the ratio; Step S4: extract image features using feature engineering, and determine whether the image cluster block has fire according to the area occupied by the flame in the cluster block, and label the cluster as having fire or not having fire; Step S5: the labels and the obtained feature vectors are input into the CNN convolutional neural network, and training is performed based on the training set to obtain a flame area extraction model, and the model parameters are adjusted based on the validation set; Step S6: extracting the flame region using the trained flame region extraction model; In step S3, the number of cluster blocks K in the clustering block algorithm is set, and the ratio of the cumulative area of ​​the flame region to the cumulative area of ​​the corresponding cluster block is calculated, and then the K value is adjusted according to the size of the ratio. The selection rule is: if it is greater than the threshold α, the image pixels in the data set are divided into K cluster blocks according to the distance between the cluster centers through the clustering block algorithm; otherwise, the number of cluster blocks K is adjusted until the threshold α is met, so that the clustering block is more uniform, so as to improve the flame area extraction effect; In step S4, the color space, shape and texture features are combined, including 9 color features of R, G, B, Y, Cb, Cr, H, S, V, two components of circularity and rectangularity, and four feature vectors of contrast, correlation, entropy and energy, a total of 15-dimensional vector feature engineering is performed to extract image features; In step S4, the cluster block is judged whether there is fire by judging the condition, and the cluster block is labeled, and the label corresponds to the image feature vector, so that the extracted feature vector is associated with the fire; Among them, whether it is a flame is judged by the proportion of the flame area to the cluster block area; the judgment conditions are: more than 2 / 3 is judged as fire; less than 1 / 3 is judged as non-fire, and between 1 / 3 and 23 is judged as unpredictable; the cluster blocks are labeled, with fire marked as 1, no fire marked as 0, and unpredictable marked as -1; the image cluster blocks are classified into fire and no fire, and the labels and the 15-dimensional feature vector form a 16-dimensional feature vector.

2. The high-precision flame area extraction method based on forest fire scene multi-feature extraction according to claim 1 is characterized in that: In step S1, the adopted data sets include the flame data set Corsican data set and the actual fire scene data set obtained based on the UAV system.

3. The high-precision flame area extraction method based on forest fire scene multi-feature extraction according to claim 1 is characterized in that: In step S2, all actual fire scene datasets are used as test sets, and the public dataset Corsican dataset is divided into the dataset in the ratio of training set: test set: validation set = 6:2:

2.

4. The high-precision flame area extraction method based on forest fire scene multi-feature extraction according to claim 1 is characterized in that: In step S5, clustering and blocking processing is first performed using a clustering and blocking algorithm, and then a feature vector is obtained through feature engineering. The obtained feature vector and the label form a 16-dimensional feature vector and are put into a CNN convolutional neural network for training. After several convolutional layers and pooling layers, the feature dimension is reduced. Finally, after two fully connected layers and a sigmoid function, the Adam optimizer is used to optimize the training to obtain a flame area extraction model for extracting the flame area of ​​the image.

Images