An evaluation method for image-based fire detection algorithms based on image complexity

Abstract

The present invention discloses an image-based fire detection algorithm evaluation method based on image complexity. An image complexity measurement algorithm for fire detection is used to determine the difficulty of identifying a fire from an image. The image response time is set to calculate the true value of the image complexity. Then, an Inception Resnet‑v2 deep convolutional neural network is used to establish a complexity evaluation model for predicting the complexity of a fire image. Finally, evaluation results of different fire detection algorithms are obtained according to the evaluation index and image complexity. The method can accurately evaluate the detection level that can be achieved by the fire detection algorithm under different complexity conditions, and the evaluation conclusion has more practical guidance for engineering applications.

Description

Technical Field

[0001] The invention belongs to the technical field of fire detection, and in particular relates to an image-based fire detection algorithm evaluation method based on image complexity. Background Art

[0002] As a new type of detection technology, image-based fire detection technology has become a hot topic in research and application in recent years. It is a process that uses image data as the detection object and uses certain algorithms to analyze the image content to determine whether there is a fire or a risk of fire in the image. It can effectively make up for the shortcomings of traditional fire detection technology in detecting fires in large-space buildings and new complex buildings, and has the advantages of early fire identification, high accuracy, small engineering workload, and flexible system installation. Among them, the detection algorithm is the core of image-based fire detection technology, and the performance of the algorithm directly determines the performance of the image-based fire detector to a certain extent.

[0003] In the prior art, standard experimental evaluation methods and image data evaluation methods are usually used to evaluate fire detection. For the standard experimental evaluation method, since the image signal contains more complex content, the fire images and interference events that the detection algorithm needs to distinguish in actual application scenarios are much more complex than those in the standard experiment. Therefore, it can be said that the experimental evaluation method used in the current domestic and foreign image-based fire detector standards does not fully consider the characteristics of the technology that uses image data as the analysis object. Therefore, the degree of differentiation between the advantages and disadvantages of the detection algorithm is not strong enough, and it is not suitable for the evaluation of image-based fire detection algorithms.

[0004] For the image data evaluation method, since the difficulty of the algorithm in identifying fires from different images varies, when different image data are used to evaluate the same algorithm, the index values ​​obtained may be quite different. At present, due to the lack of standard fire image data sets and evaluation methods that consider the characteristics of fire image detection at home and abroad, the test values ​​of detection rate or false alarm rate are not credible, and it is difficult to find the bottleneck and optimization direction of the algorithm. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an image-based fire detection algorithm evaluation method based on image complexity.

[0006] The specific plan is as follows:

[0007] An image-based fire detection algorithm evaluation method based on image complexity includes the following steps:

[0008] Step S1): Obtain fire image data and pre-process the fire image data;

[0009] Step S2): using the preprocessed fire image data to generate the true value of image complexity and establish a fire image data set containing the true value of image complexity;

[0010] Step S3): Based on the convolutional neural network, an image complexity evaluation model is established;

[0011] Step S4): using a fire image dataset containing true values ​​of image complexity to train the image complexity assessment model;

[0012] Step S5): Acquire test image data, and input the test image data into the image complexity evaluation model to obtain the complexity of each test image;

[0013] Step S6): setting the evaluation index and the image complexity level, and selecting the test image data set according to the set evaluation index and the image complexity level;

[0014] Step S7): Use different fire detection algorithms to perform fire detection on each image in the test image data set, and obtain evaluation results of different fire detection algorithms based on the detection results.

[0015] The preprocessing of the fire image data in step S1) includes the following steps:

[0016] Step T1): Collect the “human response time” of each fire image in the fire image dataset, where the “human response time” is the time it takes for a person to see the image for the first time and recognize that there is fire, flame or smoke in the image;

[0017] Step T2): Eliminate unqualified “human response times” and take the average of the “human response times” of each image as the detection response time of the fire image;

[0018] The method for generating the true value of image complexity in step S2) is to combine the detection response times of all the collected fire images and perform normalization processing to obtain the true value of the complexity of each fire image. The normalization processing formula is:

[0019]

[0020] in, is the true value of image complexity, is the detection response time of the image, is the minimum value of the detection response time of the image in the collected fire image data, It is the maximum value of the detection response time in the collected fire image data.

[0021] In step S3), the image complexity evaluation model is an Inception Resent-v2 deep convolutional neural network, and the Inception Resent-v2 deep convolutional neural network includes an Inception Resent-v2 front-end feature extraction network and a back-end complexity generation network, and the back-end complexity generation network includes a fully connected layer and a Sigmoid function.

[0022] In step S4), the training of the complexity evaluation model includes the following steps:

[0023] Step M1): Load the network weights of the pre-trained convolutional neural network classification model into the front-end feature extraction network;

[0024] Step M2): Select 50% of the images in the fire image dataset containing the true value of image complexity as the development set;

[0025] Step M3): Input the fire image in the development set into the front-end feature extraction network, and the front-end feature extraction network extracts and saves the bottleneck features;

[0026] Step M4): The backend complexity generation network is trained according to the complexity truth value and bottleneck features of the fire image data.

[0027] In step S6), the evaluation index refers to an index used to evaluate the performance of the image-based fire detection algorithm, and the evaluation index includes a performance index and a cost index.

[0028] In step S7), the steps of obtaining the evaluation results of different fire detection algorithms are:

[0029] Step N1): Use different fire detection algorithms to detect the test image data set, and calculate the evaluation index value of the detection algorithm according to the detection results;

[0030] Step N2): Under the same image complexity, the evaluation index values ​​of different fire detection algorithms are compared, and under different image complexities, the evaluation index values ​​of the same fire detection algorithm are compared.

[0031] The present invention discloses an image-based fire detection algorithm evaluation method based on image complexity. The image complexity measurement method of fire detection is used to determine the difficulty of identifying a fire from an image. At the same time, the detection response time is set to calculate the true value of the image complexity. Then, an Inception Resnet-v2 deep convolutional neural network is used to establish a complexity evaluation model for predicting the complexity of a fire image. Finally, the evaluation results of different fire detection algorithms are obtained according to the evaluation index and the image complexity. The detection level that can be achieved by the fire detection algorithm under different complexity conditions can be accurately evaluated, and the evaluation conclusion is more practical and instructive for engineering applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of the present invention.

[0033] Figure 2 It is a schematic diagram of the image complexity evaluation model.

[0034] Figure 3 Evaluation indicator system.

[0035] Figure 4 This is a table of six test subsets divided by complexity level of the test image dataset.

[0036] Figure 5 It is a table of detection performance of image-based fire detection algorithm at different image complexity levels. DETAILED DESCRIPTION

[0037] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the implementation of the present invention, not all of the implementations. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] For those skilled in the art, image complexity refers to the difficulty of the algorithm in detecting a fire from an image when it is affected by factors such as ambient light, interference objects similar to fire sources, occlusion, resolution, signal-to-noise ratio, etc. These influencing factors are manifested as different image features in the image, that is, image complexity can be considered as any combination of image features that affect the performance of the detection algorithm.

[0039] An image-based fire detection algorithm evaluation method based on image complexity is proposed. First, the true value of image complexity is defined, that is, the basis for assigning scores to image complexity is found. Then, the Inception Resnet-v2 deep convolutional neural network is used as the basic architecture of the complexity evaluation model to establish an image complexity evaluation model. The generated true value of image complexity is used to train the image complexity evaluation model. Finally, different image fire algorithms are evaluated based on image complexity to obtain the detection performance that different algorithms can achieve under the set image complexity level.

[0040] The evaluation method of the image-based fire detection algorithm is as follows:

[0041] like Figure 1 As shown, an image-based fire detection algorithm evaluation method based on image complexity includes the following steps:

[0042] Step S1): Obtain fire image data and pre-process the fire image data;

[0043] The preprocessing of the fire image data comprises the following steps:

[0044] Step T1): Collect the “human response time” of each fire image in the fire image dataset, where the “human response time” is the time it takes for a person to see the image for the first time and recognize that there is fire, flame or smoke in the image;

[0045] Step T2): Eliminate unqualified “human response times” and take the average of the “human response times” of each image as the detection response time of the fire image;

[0046] In this embodiment, the fire image data includes 13,400 fire images and 15,780 non-fire images, and a WeChat platform is used to develop a small program for collecting "human response time" of images;

[0047] The image "Human Response Time" collection applet mainly follows the following protocol to collect data:

[0048] (a) Each subject was asked the following question: “Does the image you are about to see contain {fire, flames, smoke}?”

[0049] (b) showing the test image to the person being tested;

[0050] (c) Record the time it takes the subject to answer “yes” or “no” as the “human response time” for that image.

[0051] In order to ensure that the recorded image response time is true and reliable, if an image appears multiple times, the tested person may remember the content of the image, thereby shortening the response time. Therefore, the image response time collection procedure requires that each image can only be tested once by the same person; at the same time, the method for eliminating unqualified "human response time" is: eliminate all data with a response time greater than 20 seconds, eliminate the image response time of the tested person with a judgment accuracy rate less than 90%, eliminate the response time of the tested person with an average response time greater than 10s and less than 10 tests, and eliminate the image response time of all tested persons who participated in the test less than three times, because the results of the testers who participated too few times are not credible; after data preprocessing, this embodiment finally collected 87,540 test results of 535 tested persons and 29,180 images for the next step of processing.

[0052] After eliminating all unqualified data, ensure that more than three "human response times" are collected for each image. The average of all "human response times" for each image is taken as the detection response time of the image.

[0053] Step S2): using the pre-processed fire image data to generate the true value of image complexity;

[0054] The method for generating the true value of image complexity is to combine the detection response times of all collected fire images and perform normalization processing to obtain the true value of the complexity of each fire image. The normalization processing formula is:

[0055]

[0056] in, is the true value of image complexity, is the detection response time of the image, is the minimum detection response time of the image in the collected fire image data, is the maximum value of the detection response time of the image in the collected fire image data. Step S3): Based on the convolutional neural network, an image complexity evaluation model is established;

[0057] like Figure 2 As shown, the image complexity evaluation model is an Inception Resent-v2 deep convolutional neural network, and the Inception Resent-v2 deep convolutional neural network includes an Inception Resent-v2 front-end feature extraction network 1 and a back-end complexity generation network 2, and the back-end complexity generation network 2 includes a fully connected layer 3 and a Sigmoid function 4.

[0058] In this embodiment, when building the model, the feature extraction network of the Inception Resnet-v2 network front end is retained as the detection condition extraction network, and the complexity generation network at the back end is composed of a fully connected layer and a Sigmoid function, that is, the Softmax function at the back end of the original network is replaced by the Sigmoid function.

[0059] Step S4): using fire image data containing true values ​​of image complexity to train the image complexity assessment model;

[0060] The training of the complexity assessment model includes the following steps:

[0061] Step M1): Load the network weights of the pre-trained convolutional neural network classification model into the front-end feature extraction network;

[0062] Step M2): Select 50% of the image data in the fire image dataset (including the true value of image complexity) as the development set, and 50% of the data as the test set;

[0063] Step M3): Input the fire images in the development set into the front-end feature extraction network, and the front-end feature extraction network extracts and saves the bottleneck features; extracting the bottleneck features is conducive to reducing and increasing the feature dimension, and also reduces the number of parameters, thereby reducing the amount of calculation, and after the dimensionality reduction, data training and feature extraction can be more effectively and intuitively performed.

[0064] Step M4): The backend complexity generation network is trained according to the complexity truth value and bottleneck features of the fire image data.

[0065] Step S5): Acquire test image data, and input the test image data into the image complexity evaluation model to obtain the complexity of each test image;

[0066] Step S6): According to the evaluation target, set the evaluation index and image complexity level, and select the test image data set according to the set evaluation index and image complexity level, such as Figure 3 As shown, the evaluation index refers to an index used to evaluate the performance of the image-based fire detection algorithm, including a performance index and a cost index;

[0067] In this embodiment, two correctness indicators of the algorithm, such as fire detection rate and false alarm rate, are selected to evaluate five different image-based fire detection algorithms.

[0068] Step S7): Use different fire detection algorithms to perform fire detection on each image in the test image data set, and obtain evaluation results of different fire detection algorithms based on the detection results.

[0069] The steps of evaluating the results of the different fire detection algorithms are:

[0070] Step N1): Use different fire detection algorithms to detect the test image data set, and calculate the evaluation index value of the detection algorithm according to the detection results;

[0071] Step N2): Under the same image complexity, compare the evaluation index values ​​of different fire detection algorithms, and under different image complexities, compare the evaluation index values ​​of the same fire detection algorithm.

[0072] In this embodiment, five different image-based fire detection algorithms are evaluated. For ease of description, the five different image-based fire detection algorithms are represented by Celik algorithm, Chen algorithm, Rong algorithm, Yan algorithm and Khan algorithm, respectively. Among them, the first four algorithms are algorithms based on manually provided fire image features, and the Khan algorithm is an algorithm based on the GoogleNet deep convolutional neural network to automatically extract fire image features. The test image data set generated in step M2 is represented by Testset.

[0073] At the same time, in order to investigate the detection performance of the image-based fire detection algorithm at different image complexity levels, the image complexity level is divided into six intervals, namely [0,0.1), [0.1,0.2), [0.2,0.3), [0.3,0.4), [0.4,0.6), [0.6,1]. Since the fire image data is relatively concentrated in the low image complexity data, the low complexity level is subdivided when dividing the image complexity level. The final result is as follows Figure 4 shown.

[0074] The fire images in the TestSet are passed through the image complexity evaluation model to predict the corresponding image complexity level, and the fire images in the TestSet are divided into six test subsets according to the obtained image complexity level. Finally, the images in the TestSet are input into the detection algorithm respectively to estimate the detection rate and false alarm rate of the algorithm at different complexity levels. The estimation results are shown in Figure 5 shown.

[0075] By analyzing Figure 4It can be seen that the Khan algorithm based on automatic machine extraction of image features can maintain a relatively high detection rate and a low false alarm rate at various levels of image complexity because it extracts high-level image features for fire detection. The Yan algorithm based on manual extraction of fire image features uses low-level color features, and uses the mid-level flash frequency features by using the cumulative difference method. Therefore, when the image complexity is less than 0.4, the detection rate and false alarm rate are better than the other three algorithms. However, since the Yan algorithm only uses mid-level features, when the image complexity is greater than 0.4, the detection rate drops significantly and the false alarm rate is also high. Therefore, this algorithm cannot cope with complex application scenarios. The Celik algorithm, which only extracts low-level color features, can only maintain a certain detection rate when the complexity is less than 0.2.

[0076] Through evaluation, it was found that the detection algorithm based on deep convolutional neural network can automatically extract high-level image features and is more suitable for fire detection in complex scenes, such as shopping malls, restaurants, town streets, etc. Moreover, the evaluation found that the detection performance of the algorithm varies greatly under different image complexity levels. The detection rate decreases with the increase of image complexity, and the false alarm rate increases with the increase of complexity. Therefore, it is very important to evaluate the performance of the algorithm under different image complexity conditions and deeply analyze the degree to which the algorithm performance is affected by the change of complexity.

[0077] The present invention discloses an image-based fire detection algorithm evaluation method based on image complexity. An image complexity measurement algorithm for fire detection is used to determine the difficulty of identifying a fire from an image. The detection response time of the image is set to calculate the true value of the image complexity. Then, an Inception Resnet-v2 deep convolutional neural network is used to establish a complexity evaluation model for predicting the complexity of a fire image. Finally, evaluation results of different fire detection algorithms are obtained according to the evaluation index and image complexity. The method can accurately evaluate the detection level that can be achieved by the fire detection algorithm under conditions of different complexities, and the evaluation conclusion has more practical guidance for engineering applications.

[0078] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A method for evaluating an image-based fire detection algorithm based on image complexity, characterized in that: The following steps are included: Step S1): Obtain fire image data and pre-process the fire image data; Step S2): using the preprocessed fire image data to generate the true value of image complexity and establish a fire image data set containing the true value of image complexity; Step S3): Based on the convolutional neural network, an image complexity evaluation model is established; Step S4): using a fire image dataset containing true values ​​of image complexity to train the image complexity assessment model; Step S5): Acquire test image data, and input the test image data into the image complexity evaluation model to obtain the complexity of each test image; Step S6): setting the evaluation index and the image complexity level, and selecting the test image data set according to the set evaluation index and the image complexity level; Step S7): using different fire detection algorithms to perform fire detection on each image in the test image data set, and obtaining evaluation results of different fire detection algorithms based on the detection results; The preprocessing of the fire image data in step S1) includes the following steps: Step T1): Collect the "human response time" of each fire image in the fire image dataset, where the "human response time" is the time it takes for a person to see the image for the first time and recognize that there is fire, flame or smoke in the image; Step T2): Eliminate unqualified "human response times" and take the average of the "human response times" of each image as the detection response time of the fire image.

2. The image-based fire detection algorithm evaluation method based on image complexity according to claim 1, characterized in that: The method for generating the true value of image complexity in step S2) is to combine the detection response times of all the collected fire images and perform normalization processing to obtain the true value of the complexity of each fire image. The normalization processing formula is: in, is the true value of image complexity, is the detection response time of the image, is the minimum value of the detection response time of the image in the collected fire image data, It is the maximum value of the detection response time in the collected fire image data.

3. The image-based fire detection algorithm evaluation method based on image complexity according to claim 1, characterized in that: In step S3), the image complexity evaluation model is an Inception Resent-v2 deep convolutional neural network, and the Inception Resent-v2 deep convolutional neural network includes an Inception Resent-v2 front-end feature extraction network and a back-end complexity generation network, and the back-end complexity generation network includes a fully connected layer and a Sigmoid function.

4. The image-based fire detection algorithm evaluation method based on image complexity according to claim 3 is characterized in that: In step S4), the training of the complexity evaluation model includes the following steps: Step M1): Load the network weights of the pre-trained convolutional neural network classification model into the front-end feature extraction network; Step M2): Select 50% of the images in the fire image dataset containing the true value of image complexity as the development set; Step M3): Input the fire image in the development set into the front-end feature extraction network, and the front-end feature extraction network extracts and saves the bottleneck features; Step M4): The backend complexity generation network is trained according to the complexity truth value and bottleneck features of the fire image data.

5. The image-based fire detection algorithm evaluation method based on image complexity according to claim 1, characterized in that: In step S6), the evaluation index refers to an index used to evaluate the performance of the image-based fire detection algorithm, and the evaluation index includes a performance index and a cost index.

6. The image-based fire detection algorithm evaluation method based on image complexity according to claim 5, characterized in that: In step S7), the steps of obtaining the evaluation results of different fire detection algorithms are: Step N1): Use different fire detection algorithms to detect the test image data set, and calculate the evaluation index value of the detection algorithm according to the detection results; Step N2): Under the same image complexity, the evaluation index values ​​of different fire detection algorithms are compared, and under different image complexities, the evaluation index values ​​of the same fire detection algorithm are compared.

Images