Fire hazard identification method based on lightweight neural network

Abstract

A fire hazard identification method based on a lightweight neural network comprises the following steps: collecting monitoring video frames through a camera, after 3 * 3 convolution and maximum pooling initialization, extracting features by using a ShuffleNetV2 network comprising three groups of basic units and down-sampling units (the number is 3, 7 and 3), and carrying out transfer learning in combination with an ImageNet pre-training model so as to reduce the weight change influence. And the one-dimensional convolutional layer filter is sequenced and clipped by using an Lnorm, and the network complexity is optimized. And designing an early warning logic based on threshold determination, when a certain frame is detected to be a fire, taking 100 subsequent 99 frames, calculating a local mean value and an adaptive threshold value, and determining and processing continuous non-fire frame conditions according to a fire frame proportion. According to the method, the model is lightened, the detection real-time performance is improved, false alarm and missing alarm are avoided, and the early warning robustness is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning, and specifically relates to a method for fire hazard identification based on lightweight neural networks. Background Technology

[0002] In the power industry, manual inspection is the traditional and most commonly used method for identifying fire hazards. Inspectors regularly conduct on-site checks, observing for oil stains on equipment surfaces and oil accumulation in transformer foundations to determine if oil leaks have occurred. The advantage of this method is its simplicity and intuitiveness, requiring no complex equipment.

[0003] Manual inspections can be influenced by the inspector's experience, attention, and subjective judgment, leading to omissions or misjudgments. Furthermore, the intervals between regular inspections may cause hazards to go undetected, especially in high-risk areas. Manual inspections may not cover all potential risks, particularly in hidden or hard-to-reach areas, and record-keeping and data analysis may be insufficiently systematic, making it difficult to track the development of hazards.

[0004] Sensor detection primarily uses indicators such as temperature, light intensity, and specific gases to detect the presence of fire characteristics in an environment. When the data for these indicators exceeds a set threshold, a fire alarm is triggered. Generally, sensor-based detection methods are low-cost and easy to install.

[0005] High-quality infrared imaging equipment is expensive, potentially increasing operating costs. Some materials may be insensitive to infrared radiation, making detection difficult. Furthermore, sensors are limited by distance, resulting in a limited detection range and limitations in large-scale locations and complex environments. They are also susceptible to interference from steam and dust, leading to false alarms. Additionally, this method is not conducive to determining liability in fire accidents or analyzing the causes of fires.

[0006] A search revealed relevant patent document CN116108884A: This patent describes an improved fuzzy neural network fire detection method based on a genetic algorithm. This method establishes an improved fuzzy neural network fire detection model based on genetic algorithms, using temperature, carbon monoxide concentration, and smoke concentration data detected at the fire scene. It can detect whether a fire has occurred in a timely and accurate manner, and has good versatility. However, the equipment depends on environmental conditions, and the cost of using a multi-sensor system is high. CN118762330A: This patent describes a method and system for indoor fire detection and automatic fire extinguishing based on deep learning. It uses an improved YOLO V8 deep learning model to achieve flame detection, and combines ECA attention mechanism and modifies the CIOU loss function in the model to WIOU, etc., to improve the accuracy of flame detection by the deep learning model. However, the detection speed is slow, the equipment requirements are high, and it can only function in specific scenarios.

[0007] Based on the above analysis, fires are particularly frequent at night and in severe weather conditions with extremely low visibility. Early detection of fires and rapid warning are crucial for protecting personal life and property. Facing scenarios where fire risks exist even before smoke has formed, while simultaneously overcoming the effects of extreme weather, has become a new challenge for fire detection technology. With the development of deep learning, the focus of fire detection has shifted from shallow, localized features of flames to deep, global features. Some researchers have applied convolutional neural network architectures to fire detection, improving feature classification accuracy; however, the large number of parameter inputs leads to excessively long computation times, making it impossible to guarantee real-time detection. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide a fire hazard identification method based on a lightweight neural network. The method uses the entire video frame as input for binary flame detection and adopts a filter pruning strategy to reduce the complexity of the network model. In order to eliminate the influence of false detection results on the early warning results, an early warning logic based on threshold determination is proposed to avoid false alarms and missed alarms.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A fire hazard identification method based on lightweight neural networks, comprising the following steps: S1. For the surveillance video frames captured by the camera, first use a 3×3 convolutional layer to extract basic features, and then use a 3×3 max pooling layer to reduce the dimensionality. S2. A feature extraction module is constructed using ShuffleNetV2. The feature extraction module includes three basic units and downsampling units. Deep features are extracted through Channel Split, convolution, concat and channel shuffle operations. Then, binary classification prediction is completed through pointwise convolution, global pooling and fully connected layers. S3. Use the ImageNet pre-trained ShuffleNetV2 model, replace the fully connected layer with a binary classification linear layer, train the new layer first and freeze other modules, then unfreeze and train the last convolutional layer to improve the model's detection accuracy and generalization ability, and avoid overfitting. S4. For the last convolutional layer of ShuffleNetV2, calculate and sort the L1 norm of each filter, prune the filters with small norms to reduce network complexity, improve detection efficiency while ensuring accuracy, and achieve model lightweighting. S5. When a frame is detected as Fire, take the next 99 frames, calculate the local mean and adaptive threshold, and determine the fire based on the comparison between the proportion of Fire frames and the threshold. At the same time, handle the case of consecutive Non-Fire frames to reduce false alarms and missed alarms and improve the robustness of the early warning.

[0010] Preferably, the sub-step of S1 is: S1.1. Use a 3×3 convolutional layer to perform preliminary processing on the video frames of the monitoring site captured by the camera and extract basic features; S1.2. Use a 3×3 max pooling layer to reduce the dimensionality of the basic features after convolution, thereby reducing the amount of data and preparing for subsequent processing.

[0011] Preferably, the sub-step of S2 is as follows: S2.1, Establishing the network structure: S2.1.1 Construct three sets of combined structures of ShuffleNetV2 basic units and downsampling units, each set containing one downsampling unit, with the number of basic units being 3, 7, and 3 respectively; S2.1.2 First, perform a Channel Split operation on the input to divide the input channel into two parts; perform an identity mapping on the left branch, and perform three consecutive convolutions on the right branch in sequence, with the input and output channels being the same; after performing a concat operation on the two branches, perform a channel shuffle operation to achieve sharing of basic feature information; S2.1.3 Downsampling Unit Processing Flow: No Channel Split operation is performed; both branches use downsampling with stride=2. After processing the two branches, concat and channel shuffle operations are performed. S2.2 Constructing the prediction and output module: S2.2.1. Utilize pointwise convolutional layers to further process the basic features to obtain deeper features; S2.2.2 Aggregate deep features through a global pooling layer; S2.2.3. Use a fully connected layer to complete binary classification prediction, which includes prediction of flames and non-flames.

[0012] Preferably, the sub-step of S3 is as follows: S3.1 Obtain the pre-trained ShuffleNetV2 model on the ImageNet dataset; S3.2 Replace the fully connected layers of the pre-trained model with linear layers for binary classification; S3.3 First, train the new linear layer, and at the same time freeze the rest of the model to ensure that the model is initially adapted to the fire detection task. S3.4 Unfreeze and train the last convolutional layer of the pre-trained model to further optimize model performance, avoid overfitting, and improve generalization ability.

[0013] Preferably, the sub-step of S4 is as follows: S4.1 Determine the pruning target: Prune the last one-dimensional convolutional layer (1×1Conv) in the parameter set of ShuffleNetV2; S4.2 Filter Evaluation and Pruning: S4.2.1 For each filter Calculate its norm ,in This represents the number of input channels of the filter; S4.2.2 Sort Sᵢ in ascending order, select the first m smaller values ​​of the filters for pruning, reduce network complexity and improve detection efficiency.

[0014] Preferably, the sub-step of S5 is as follows: S5.1 Detection Frame Definition: When a frame is detected as a Fire, that frame and the subsequent 99 frames (a total of 100 frames) are defined as P0, P1, ..., P... 99 ; S5.2 Local Mean Calculation: For each position i, calculate the local mean: (1); ω is the size of the local mean window, Nᵢ is the number of valid data points in the window, and if i+j>99, it is not included in the summation; S5.3 Adaptive Threshold Calculation: Calculate the adaptive threshold: (2); It is an element in the data sequence, if If the summation is not complete, then that element is not included in the summation. S5.4 Fire Detection: Calculate the proportion T of Fire frames in 100 frames. If T>β, determine that there is a fire W=1; if T≤β, determine that there is no fire W=0. (3); S5.5 Handling of false results: If the detection result of consecutive ω frames within the current 100 frames is Non-Fire, then it is determined that there is no fire (W=0, V=0), and the newly detected Fire frame is used as the new starting frame for re-early warning judgment (V is a one-dimensional vector of length ω, which records the detection results of consecutive ω frames).

[0015] Preferably, in S2.1.2, the three consecutive convolutions include two 1×1 convolutions and one depthwise separable convolution.

[0016] Preferably, in S5.5, to address the issue of false results during detection leading to misjudgment, it is stipulated that if there are consecutive frames (i.e., window size) within the current 100 frames with a detection result of Non-Fire, then it is determined that no fire has occurred; frames that are re-detected as Fire are defined as new, and the warning judgment is re-performed; the warning logic is shown in equation (3), where For the early warning results; It is a length of A one-dimensional vector: , indicating continuity Frame detection results.

[0017] A fire hazard identification system based on a lightweight neural network is provided, which employs the aforementioned fire hazard identification method based on a lightweight neural network. A computer device, comprising: One or more processors, on which one or more programs are stored; When one or more programs are executed by one or more processors, they are used to implement a lightweight neural network-based fire hazard identification method as described in any one of claims 1-8.

[0018] The present invention can achieve the following beneficial effects: 1. By combining the ShuffleNetV2 network, the network architecture is optimized by reducing the number of filters in the one-dimensional convolutional layers, thereby reducing network complexity, improving computational efficiency, and enabling deployment on more lightweight platforms.

[0019] 2. The feature extraction module of the network was not pruned during the optimization process. Through multi-level feature extraction, it achieved better generalization ability and ensured the feature extraction performance of the model, thus improving the real-time performance compared to traditional early warning methods.

[0020] 3. To address the issue of false detection results, a threshold-based early warning logic is proposed, which eliminates false alarms and missed alarms caused by false detection results and improves the robustness of the early warning system.

[0021] 4. Through experimental data, the optimal number of filter trims was obtained, achieving model compression without a significant decrease in accuracy. Compared with existing CNN models, this invention has a speed advantage.

[0022] 5. Compared to VGG16 and ResNet50, ShuffleNetV2 employs depthwise separable convolutions, resulting in a more streamlined network architecture. In the ImageNet classification task, ShuffleNetV2 outperforms VGG16 and ResNet50. Therefore, choosing ShuffleNetV2 as the backbone network for further optimization offers significant advantages.

[0023] 6. To address the issue of false results leading to misjudgments during testing, a threshold-based early warning logic is adopted to effectively filter out false results and reduce false alarms. Attached Figure Description

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a schematic diagram of the basic unit structure of ShuffleNetV2 in this invention; Figure 2 This is a schematic diagram of the ShuffleNetV2 downsampling unit structure of the present invention; Figure 3 This is a diagram of the overall architecture of the fire hazard identification network based on ShuffleNetV2 in this invention. Detailed Implementation

[0025] Preferred solutions include Figures 1 to 3 As shown, a fire hazard identification method based on lightweight neural networks is proposed. This invention utilizes deep learning networks to perform real-time detection of fire hazards using video. The specific method is as follows: 1. Input initialization: The input initialization module consists of a 3×3 convolutional layer and a 3×3 max pooling layer. The input data is video frames captured by the camera from the monitored scene.

[0026] 2. Feature extraction: The feature extraction module consists of three sets of ShuffleNetV2 basic units and downsampling units. See [link to network structure] for details. Figure 1 , Figure 2 Each group contains only one downsampling unit, with the number of basic units being 3, 7, and 3 respectively; the prediction and output module has one pointwise convolutional layer, one global pooling layer, and one fully connected layer. The network structure is shown in [link to network structure]. Figure 3 .

[0027] Among them: Appendix Figure 1This section demonstrates the basic unit structure of ShuffleNetV2, which is initiated by a Channel Split operation, dividing the input channels into two parts, which then enter the left and right branches respectively. The left branch performs an identity mapping, while the right branch sequentially undergoes two 1×1 convolutions and one depthwise separable convolution. The number of input and output channels remains consistent across all convolutional layers, adhering to the design principle of minimizing memory access cost (MAC) by ensuring identical input and output channels. The outputs of the two branches are concatenated and then subjected to a channel shuffle operation, achieving full integration of information between channels, enhancing feature representation capabilities, and providing higher-quality features for subsequent network layers.

[0028] Figure 2 The downsampling unit presented here is used for spatial downsampling and differs from the basic unit. This unit does not have a channel split operation; both branches use a stride=2 downsampling method, resulting in a halved feature map space size and a doubling of the number of channels. The operation flow is as follows: input data simultaneously enters both branches, each completing its downsampling process. The results from the two branches are then concatenated, and finally, a channel shuffle operation is performed. This structural design reduces the feature map resolution while increasing channel-dimensional information, meeting the network's need to extract features at different scales. This is of great significance for capturing fire-related features of different sizes in fire hazard identification.

[0029] Figure 3 This paper describes the overall structure of a fire hazard identification network built on ShuffleNetV2. The input consists of surveillance video frames. Basic features are initially extracted using 3×3 convolutional layers, followed by dimensionality reduction using 3×3 max pooling layers. Next, the data flows into the main feature extraction module, which comprises three groups of ShuffleNetV2 basic units and downsampling units. Each group contains one downsampling unit, with the number of basic units being 3, 7, and 3 respectively. After feature extraction, the data is sequentially passed through pointwise convolutional layers, global pooling layers, and fully connected layers to achieve binary classification prediction of flame / non-flame. The network also employs transfer learning and filter pruning optimization strategies to improve performance and efficiency. The overall structure is compact and efficient, making it suitable for real-time fire hazard detection tasks.

[0030] 2.1 Basic Unit: The basic unit first performs a Channel Split operation on the input, which essentially divides the input channel into two parts. Next, it performs an identity mapping on the left branch, while the right branch contains three consecutive convolutions with the same number of input and output channels. The number of input and output channels in the convolutional layers within each branch is consistent, including two 1×1 convolutions and one depthwise separable convolution. Then, it performs a concat operation on the two branches, and finally a channel shuffle operation. By further dividing each grouped channel into several sub-channels and mixing the different sub-channels within different groups, it ensures that feature information can be shared between different groups. The network structure is shown in Figure (1). 2.2 Downsampling Unit: The downsampling unit does not perform a channel split operation; both branches have downsampling with stride=2, as shown in Figure (2). Finally, the two branches are concatenated together to perform a channel shuffle operation.

[0031] 3. Transfer learning: To ensure the network's detection accuracy, a pre-trained ShuffleNetV2 model on the ImageNet dataset was used for transfer learning: the fully connected layers of the pre-trained model were replaced with linear layers mapping binary classification (Figure 3, FC). The new linear layers were trained first, while the remaining modules were frozen; then, the last convolutional layer of the pre-trained model was unfrozen and trained. This reduced the impact of weight changes on detection performance, avoided overfitting during training, and allowed for increased model training time to achieve better generalization ability.

[0032] 4. Filter trimming: To reduce network complexity and improve detection efficiency, a strategy is employed to prune and optimize the one-dimensional convolutional layers (as shown in Figure 3, 1×1 Conv), because the parameters of the ShuffleNetV2 network are concentrated in the last convolutional layer. For a filter, all weight values ​​are... The norm is used as an evaluation metric for the filter; filters with small norms are ranked and pruned. Pruning of one-dimensional convolutional layers is also performed. The process of each filter is as follows: 1) For each filter Calculate its norm ,in This represents the number of input channels of the filter; 2) To Sort by size from smallest to largest, select the top... Cut each one.

[0033] 5. Early warning logic: Based on the network output, when a frame is detected as a Fire, that frame and the subsequent 99 frames, totaling 100 frames, are defined as... According to the formula as follows: ; Set adaptive threshold The formula is as follows: ; in It is the window size for the local mean. It is the number of valid data points actually existing within the window. It is an element in the data sequence, if If the element is not specified, it is not included in the summation. The determination of whether a fire has occurred is based on the comparison between the percentage T of fire frames out of 100 frames and a threshold, using the following formula: ; To address the issue of false results during detection leading to misjudgment, it is stipulated that if there are consecutive frames (i.e., the window size) within the current 100 frames that have a non-fire detection result, then it is determined that no fire has occurred; frames that are re-detected as fire are defined as new, and the warning judgment is re-performed. The warning logic is shown in equation (3), where For the early warning results, It is a length of a one-dimensional vector , indicating continuity Frame detection results.

[0034] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A fire hazard identification method based on a lightweight neural network, characterized in that The method comprises the following steps: S1, the monitoring video frame collected by the camera is first extracted by a 3*3 convolution layer, and then dimensionality reduction is performed by a 3*3 maximum pooling layer; S2, a feature extraction module is constructed by using ShuffleNetV2, the feature extraction module comprises three groups of basic units and a down-sampling unit, deep layer features are extracted through channel split, convolution, concat and channel shuffle operations, and then binary classification prediction is completed through point-by-point convolution, global pooling and a full connection layer; S3, the ShuffleNetV2 model pre-trained by using ImageNet is adopted, a binary classification linear layer is replaced for the full connection layer, the new layer is trained and other modules are frozen, then the last convolution layer is unfrozen and trained, the detection precision and generalization ability of the model are improved, and overfitting is avoided; S4, for the last one-dimensional convolution layer of the ShuffleNetV2, the L1 norm of each filter is calculated and sorted, the filters with small norm are pruned, the network complexity is reduced, the detection efficiency is improved under the premise of ensuring the precision, and the model is lightened; S5, when a frame is detected as fire, 99 subsequent frames are taken, the local mean and the adaptive threshold are calculated, the fire is determined according to the comparison result of the fire frame proportion and the threshold, and the continuous non-fire frame is processed, so that the false alarm and the missed alarm are reduced, and the early warning robustness is improved.

2. The fire hazard identification method based on a lightweight neural network according to claim 1, characterized in that: The sub-steps of S1 are: S1.1, a 3*3 convolution layer is used to preliminarily process the monitoring scene video frame collected by the camera, and basic features are extracted; S1.2, a 3*3 maximum pooling layer is used to reduce the dimension of the basic features processed by convolution, and the data amount is reduced to prepare for subsequent processing.

3. The fire hazard identification method based on a lightweight neural network according to claim 1, characterized in that: The sub-steps of S2 are: S2.1, the network structure is built: S2.1.1, a combined structure of three groups of ShuffleNetV2 basic units and down-sampling units is constructed, each group comprises one down-sampling unit, and the number of basic units is 3, 7 and 3 respectively; S2.1.2, the input is first subjected to a Channel Split operation, the input channels are divided into two parts; the left branch is subjected to an identity mapping, and the right branch is subjected to three consecutive convolutions with the same input and output channels; after the concat operation of the two branches, the channel shuffle operation is performed to realize the sharing of basic feature information; S2.1.3, the processing flow of the down-sampling unit: no Channel Split operation is performed, and the two branches are subjected to down-sampling with a stride of 2; after the processing of the two branches, the concat and channel shuffle operations are performed; S2.2, the prediction and output module is constructed: S2.2.1, a point-by-point convolution layer is used to further process the basic features to obtain deep layer features; S2.2.2, the deep layer features are aggregated by a global pooling layer; S2.2.3, a full connection layer is used to complete binary classification prediction, and the binary classification includes flame and non-flame prediction.

4. The fire hazard identification method based on a lightweight neural network according to claim 1, characterized in that: The sub-steps of S3 are: S3.1, obtain a pre-trained model of ShuffleNetV2 on the ImageNet dataset; S3.2, replace the fully connected layer of the pre-trained model with a linear layer for binary classification; S3.3, first train the new linear layer while freezing the remaining modules of the model, ensuring that the model is initially adapted to the fire detection task; S3.4, unfreeze and train the last convolutional layer of the pre-trained model, further optimize the model performance, avoid overfitting, and improve the generalization ability.

5. The fire hazard identification method based on the lightweight neural network according to claim 1, characterized in that: The sub-steps of S4 are: S4.1, determine the pruning object: the last one-dimensional convolutional layer in the parameter set of ShuffleNetV2 is pruned; S4.2, filter evaluation and pruning: S4.2.1, for each filter , compute its norm , where is the number of input channels of the filter; S4.2.2, sort Sᵢ from small to large, select the top m smaller values of the filter for pruning, reduce the network complexity, and improve the detection efficiency.

6. The fire hazard identification method based on a lightweight neural network according to claim 1, characterized in that: The sub-steps of S5 are: S5.1, Detect frame definition: When a frame is detected as Fire, define the frame and the following 99 frames as P0, P1,..., P99. 99 ; S5.2, local mean calculation: for each position i, calculate the local mean: （1）； ω is the size of the local mean window, Nᵢ is the number of valid data points in the window, if i+j>99, it is not included in the summation; S5.3, adaptive threshold calculation: calculate the adaptive threshold: （2）； It is an element in the data sequence, if If the summation is not complete, then that element is not included in the summation. S5.4, fire determination: calculate the proportion T of Fire frames in 100 frames, if T>β, determine that there is fire W=1; if T≤β, determine that there is no fire W=0; （3）； S5.5, false result processing: if there are consecutive ω frames of Non-Fire detection results in the current 100 frames, it is determined that there is no fire, and the newly detected Fire frame is taken as the new starting frame, and the warning judgment is re-performed, where V is a one-dimensional vector with length ω, recording the detection results of consecutive ω frames.

7. The fire hazard identification method based on a lightweight neural network according to claim 1, characterized in that: In S2.1.2, the three consecutive convolutions include two 1×1 convolutions and one depth separable convolution.

8. The fire hazard identification method based on a lightweight neural network according to claim 1, characterized in that: In S5.5, in order to solve the problem that false results will lead to misjudgment, it is stipulated that if there are continuous frames with Non-Fire detection results in the current 100 frames, it is determined that no fire occurs; the frame that is re-detected as Fire is defined as new, and the pre-warning judgment is re-performed; the pre-warning logic is shown in formula (3), wherein is the pre-warning result; is a one-dimensional vector with a length of ; and represents the detection results of continuous frames. 9.A fire hazard identification system based on a lightweight neural network, characterized in that: A fire hazard identification method based on a lightweight neural network according to any one of claims 1-8 is adopted.

10. A computer device, comprising: It includes: One or more processors, the processor has one or more programs stored thereon; When one or more programs are executed by the one or more processors, a fire hazard identification method based on a lightweight neural network according to any one of claims 1-8 is implemented.

Images