A method for predicting a parking lot fire

Abstract

The present application belongs to the technical field of fire prediction, and relates to a parking lot fire prediction method. The method positions a fire area through class activation mapping based on semantic features, and uses a cross-generation feature aggregation module to aggregate iterative output results based on network intermediate layer multi-scale features and results output during a training process. Meanwhile, the method fuses an original RGB input picture based on an initial rough segmentation result, and inputs the picture into a classification network for secondary fine segmentation, which can reduce a false positive rate, is not affected by a surrounding environment, has low data requirements, and can only be used in the field of parking lot fire detection, and is also applicable to other fields with insufficient data sets and requiring a weak supervision mode to improve precision.

Description

Technical Field

[0001] This invention belongs to the field of fire prediction technology, and relates to a parking lot fire prediction method, particularly a parking lot fire prediction method based on weakly supervised cross-iterational feature relationship mutual learning. Background Technology

[0002] This invention belongs to the field of fire prediction technology, and relates to a parking lot fire prediction method, particularly a parking lot fire prediction method based on weakly supervised cross-iterational feature relationship mutual learning.

[0003] With economic development and social progress, the number of vehicles is gradually increasing, which puts forward higher demands on large parking lots. Among them, the installation of parking lot safety facilities is particularly important. For example, if an open fire occurs in a large parking lot and is not extinguished in time, the large amount of fuel and flammable materials in the vehicles will instantly ignite the entire parking lot, which will cause great loss of life and property.

[0004] Currently, sensors can be used to predict fires based on data indicators such as temperature and smoke at the time of ignition. However, these physical devices are highly dependent on environmental conditions, often producing false alarms, and sometimes only triggering an alarm when a specified threshold is reached. This is insufficient for timely fire detection and response. The emergence of fire identification algorithms based on machine learning has provided a more reliable basis for predicting fires using only physical devices. In particular, the rapid development of computer vision has led to the widespread adoption of deep learning-based fire early warning algorithms with very high accuracy. Existing fire early warning algorithms often rely on large amounts of data to train highly accurate prediction models. However, this data-driven approach has a significant drawback: the generalization performance of the network model is very limited. When the model is applied to new environments, the accuracy drops drastically, threatening people's lives and property. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and design a parking lot fire prediction method. Based on weakly supervised cross-iterational feature relationship mutual learning, this method predicts parking lot fires and solves the problems of unreliability in current fire detection methods that rely on physical sensor components, as well as the problems of visual flame detection methods that require a large amount of data for training, have poor model generalization, and can only predict the location of the flame but cannot predict the shape of the flame.

[0006] To achieve the above objectives, the specific process of predicting parking lot fires according to the present invention is as follows:

[0007] (1) Collect images of parking lot fires and images of parking lots without fires, and label them as images with fires and images without fires as category labels. At the same time, divide the images with fires and images without fires into three sub-datasets: training set, validation set and test set.

[0008] (2) The category information is transformed into a category activation map of location information by using the category activation mapping method. The location of the fire is located by using the category label of whether a fire has occurred. The location of the fire in the parking lot is coarsely located to obtain semantic information.

[0009] (3) Fire area fine localization based on multi-scale feature aggregation: The image after coarse localization in step (2) is input into the deep learning network, and the features output by the output side of the deep learning network are used as the multi-scale information of the flame.

[0010] (4) Output the iterative results of step (3) based on the deep learning network training process, as well as the intermediate iterative output information, and use them as a weighted sum of common fire information;

[0011] (5) The semantic information, multi-scale information and intermediate iterative output information are fused together, and the coarse localization information is divided into foreground flame information and background information by the segmentation algorithm CRF. The foreground region is then applied to the input fire detection image to form a partially missing fire detection image. After secondary classification, the undiscovered fire area is further discovered, and finally the accurate segmentation area of ​​the parking lot fire is obtained.

[0012] (6) Based on the precise segmentation area of ​​the parking lot fire obtained in step (5), an end-to-end parking lot fire segmentation network is trained by transfer learning to finely segment the fire area in the parking lot.

[0013] (7) The fire classification network is trained by using the constructed training dataset of fire and non-fire images, and predicts whether a fire has occurred. The error is backpropagated to train the fire and non-fire prediction network. At the same time, the fire location segmentation network is trained by using the output of the fire prediction network.

[0014] (8) Load the trained segmentation network model into the segmentation network and output the parking lot flame segmentation results.

[0015] This invention generates coarse localization of parking lot fire locations as semantic information based on the class activation mapping mechanism of a classification network. Each convolutional block of the network generates multi-scale information, and each iteration during network training generates semantic class activation mapping information of the fire. Therefore, the complementary advantages of the three can be fully utilized. Based on the above fusion results, the original network input image is ablated by "erosion" and then re-input into the fire classification network. Combined with the CRF segmentation algorithm, the residual fire area can be refined. Based on the above refined results, the flame segmentation network is trained. The segmentation network can run independently of the classification network without any annotation information.

[0016] Compared with the prior art, the beneficial effects of the present invention are:

[0017] This invention is based on a weakly supervised fire classification network. It locates fire areas through category activation mapping based on semantic features, and aggregates and iteratively outputs results based on multi-scale features of the intermediate layers of the network and the results continuously output during training using a cross-generational feature aggregation module. At the same time, the original RGB input image is ablated based on the initial coarse segmentation result and then input into the classification network for secondary fine segmentation. This reduces the false alarm rate, is unaffected by the surrounding environment, and has low data requirements. It can be used only in the field of parking lot fire detection, but it is also applicable to other fields where datasets are scarce and weak supervision is needed to improve accuracy. Attached Figure Description

[0018] Figure 1 This is a structural diagram of the cross-iteration feature relationship mutual learning module in the fire prediction network structure framework described in this invention.

[0019] Figure 2 The fire prediction network structure diagram described in this invention.

[0020] Figure 3 This is a flowchart illustrating the workflow of the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the scope of the invention is not limited in any way.

[0022] Example:

[0023] This embodiment uses, as follows: Figure 1 The network structure shown and as Figure 2 The process shown enables parking lot fire prediction, and specifically includes the following steps:

[0024] (1) Constructing a fire detection dataset:

[0025] Collect images of parking lot fires and images of parking lots without fires, and divide the images into two categories: those with fires (class 1) and those without fires (class 0). At the same time, divide the datasets with and without fires into three subsets: training set, validation set, and test set.

[0026] (2) Coarse localization of fire under weak supervision based on semantic feature aggregation

[0027] Currently, the parking lot dataset only has labels indicating whether a fire has occurred (0 and 1), with no other labels. Therefore, it is necessary to utilize the limited category labels to locate the fire's location. Class activation mapping is used to transform the category information into a category activation map of the location information. This method will then locate the fire location L (where L is the fire area) within the parking lot. Specifically:

[0028] Image I is input into a deep learning network, which outputs semantic layer F4 features. Then, aggregated semantic features are obtained based on the CAM mechanism. Its detailed definition is as follows:

[0029]

[0030] Where Cov represents convolution operation, softmax represents normalization operation, and FC represents fully connected layer. The feature layers represent numbers from 1 to n, and GAP represents the global pooling layer.

[0031] (3) Precise fire area localization based on multi-scale feature aggregation

[0032] While category activation mapping can provide a general location information L of a fire, this information is very coarse and even fails in complex situations like parking lots, especially when vehicles are blocking the way. Therefore, further fine-tuning is needed. Deep learning networks (VggNet) can output side features (Conv2->4) as multi-scale information to enhance the perception of multi-scale objects. Flames also have many shapes and sizes; therefore, multi-scale information is also effective in fire detection. The network's output features... As multi-scale information of the flame, it is obtained through downsampling and upsampling operations ( Aggregate multi-scale features:

[0033]

[0034] Where Cov represents the convolution operation and Con represents the concat connection operation;

[0035] (4) Feature aggregation across iterative feature aggregation

[0036] During the training process of a deep learning network (VggNet), intermediate outputs (MiddleLevel) are continuously generated. These MiddleLevels are often discarded because they contain a lot of background noise. However, as intermediate results of each network attempt, they contain positive gain information for correct predictions and negative gain information for each incorrect prediction. For weakly supervised segmentation, their common information needs to be enhanced. Therefore, data flow and positive gain enhancement are required between each iteration. Thus, the iterative results from the training process are output as a weighted sum of common fire information. The detailed definition of cross-iteration feature aggregation is as follows:

[0037] ,

[0038]

[0039] Where SF represents softmax normalization and RE represents the reshape operation. represents the matrix cross product, N represents the total number of iterations N, and t represents the t-th iteration;

[0040] (5) Fire zone segmentation based on region ablation algorithm

[0041] After obtaining the above parking lot fire location information, the semantic information ( ) and multi-scale information ( ) and intermediate iteration output information ( The initial segmentation result Mask is obtained by fusing the data. Then, the CRF segmentation algorithm is used to divide the coarse localization information into foreground flame information and background information. Although the segmented flame results can occupy most of the flame area, there are still some areas that are not included in the current flame area. These areas are still useful for predicting the presence of a fire. Therefore, the foreground area is applied to the input fire detection image I to form a partially missing fire detection image. After secondary classification, it can continue to discover the undiscovered fire areas.

[0042]

[0043] in, Represents a classification network. for Output results Represents the input to the classification network. Representative use Ablation procedure I;

[0044] (6) Training a fire prediction network based on fire segmentation zones

[0045] The classification network described above can obtain the precise segmentation region of a parking lot fire. Then, based on the precise segmentation region, an end-to-end parking lot fire segmentation network (SegmentationNet) can be trained. Both networks have the same goal: to finely segment the fire area in the parking lot. The learning process is similar to transfer learning, which transfers the refined knowledge from the classification network to the segmentation network. However, the predicted segmentation network can work independently without the classification network, thus greatly improving the speed of fire segmentation testing.

[0046] (7) Parking Lot Fire Prediction Network Training

[0047] The fire image training dataset was constructed and used as the fire classification network. The input is used to predict whether a fire will occur, and the error is backpropagated to train the network.

[0048] The refined results are used as pseudo-labels to train the segmentation network SegmentationNet, and the error is backpropagated to achieve the purpose of training the segmentation network.

[0049] (8) Parking lot fire prediction network test

[0050] The trained segmentation network model SegmentationNet is loaded into the segmentation network, and the parking lot fire segmentation results are output. Firefighting plans are then formulated based on the fire segmentation results.

[0051] This invention is based on a weakly supervised fire classification network. It locates fire areas through category activation mapping based on semantic features, and aggregates and iterates the output results based on multi-scale features of the intermediate layers of the network and the results continuously output during training using a cross-generational feature aggregation module. At the same time, it ablates the original RGB input image based on the initial coarse segmentation result and inputs it into the classification network for secondary fine segmentation.

[0052] It should be noted that the networks, algorithms, and processes not described in detail herein are all common technologies in the field. The purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the claims.

Claims

1. A method for predicting parking lot fires, characterized in that, The specific process is as follows: (1) Collect images of parking lot fires and images of parking lots without fires, and label them as images with fires and images without fires as category labels. At the same time, divide the images with fires and images without fires into three sub-datasets: training set, validation set and test set. (2) The category information is transformed into a category activation map of the location information by using the category label of whether a fire has occurred. The location of the fire is located by using the category label of whether a fire has occurred. The location of the fire in the parking lot is coarsely located to obtain semantic information. Specifically, the image I is input into the deep learning network and the semantic layer F4 feature is output. Then, the aggregated semantic feature is obtained based on the CAM mechanism. Its detailed definition is as follows: Where Cov represents convolution, softmax represents normalization, and FC represents fully connected layer. The feature layers represent numbers from 1 to n, and GAP represents the global pooling layer. (3) Fine localization of fire area based on multi-scale feature aggregation: The image after coarse localization in step (2) is input into the deep learning network, and the features output by the deep learning network are used as multi-scale information of the flame; the network output features are used as the fire area. As multi-scale information of the flame, it is obtained through downsampling and upsampling operations. Aggregating multi-scale features yields multi-scale aggregated features: Where Cov represents the convolution operation and Con represents the concat connection operation; (4) Output the iterative results of step (3) based on the deep learning network training process as a weighted sum of common fire information; cross-iteration feature aggregation is obtained, and the detailed definition of cross-iteration feature aggregation is as follows: ， ， Where SF represents softmax normalization and RE represents the reshape operation. represents the matrix cross product, N represents the total number of iterations N, and t represents the t-th iteration; (5) Aggregate semantic features Multi-scale aggregation features and cross-iteration aggregation features The coarse localization information is fused and the CRF segmentation algorithm is used to divide it into foreground flame information and background information. The foreground region is then applied to the input fire detection image to form a partially missing fire detection image. After secondary classification, the undiscovered fire areas are further discovered, and finally the accurate segmentation region of the parking lot fire is obtained. (6) Based on the precise segmentation area of ​​the parking lot fire obtained in step (5), an end-to-end parking lot fire segmentation network is trained by transfer learning to finely segment the fire area in the parking lot. (7) The fire classification network is trained by using the constructed training dataset of fire and non-fire images, and predicts whether a fire has occurred. The error is backpropagated to train the fire and non-fire prediction network. At the same time, the fire location segmentation network is trained by using the output of the fire prediction network. (8) Load the trained segmentation network model into the segmentation network and output the parking lot flame segmentation results.

Images