Lightweight fire detection method based on two-stage pruning and adaptive quantification

Abstract

The invention discloses a lightweight fire detection method based on two-stage pruning and adaptive quantization, and belongs to the field of fire detection machine learning application. According to the method, a non-lightweight convolutional neural network model is constructed according to a fire detection task, a convolutional layer in an original model is replaced by using a fusible branch convolutional block, and pruning from a channel level and a hierarchy is realized through sparsification of a channel level importance factor and a hierarchy importance factor during model training; and fusing two branches of the fusible branch convolution into a common convolution layer, and further performing quantization processing on the pruned model through an adaptive outlier percentage parameter to obtain an optimal quantization model. According to the method, pruning is carried out on the channel level and the hierarchy at the same time, quantization is carried out in a self-adaptive mode according to the characteristics of each layer of the model, and light weight of the model is effectively achieved; by lightening the fire detection model, the problems of high hardware cost and low operation efficiency of the existing fire detection model are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning application in fire detection, and in particular relates to a lightweight fire detection method based on double-stage pruning and adaptive quantization. Background Art

[0002] In areas with high incidence of fire, such as new energy vehicles, photovoltaic power generation and flammable material processing plants, fire detection devices are generally used to monitor fires. Traditional fire monitoring technologies are mainly contact fire detectors, such as carbon monoxide sensors, carbon dioxide sensors and smoke sensors, but contact fire detectors are difficult to apply to outdoor open spaces. Therefore, existing methods generally obtain images of the detection area in real time. The convolutional neural network model is used to detect and identify video images in real time to achieve monitoring of areas with high incidence of fire.

[0003] Existing fire detection convolutional neural network models can effectively detect and identify fires. However, since convolutional neural network models usually have a large number of parameters and a complex structure, they face problems such as high resource consumption and low operating efficiency in practical applications. The above problems are mainly solved by pruning and quantization. Pruning reduces the complexity of the model by removing unimportant weights or neurons in the model, while quantization reduces the storage and computing requirements of the model by reducing the number of bits representing weights and activation values. Both methods have their own unique advantages.

[0004] Currently, most pruning methods are channel-level pruning. Channel-level pruning has low granularity and high flexibility, which can effectively reduce the number of model parameters, but cannot reduce the number of model layers. The number of layers has a great positive correlation with the inference time during hardware deployment. Generally speaking, the more layers, the more time it takes. The layer-level pruning method can effectively reduce the number of layers and significantly reduce the inference time, but it lacks the flexibility of channel-level pruning. Existing pruning methods cannot take into account the characteristics of significantly reducing the inference time and flexibility.

[0005] Quantization methods can effectively reduce the size of the model, but the accuracy of the model will decrease to a certain extent after quantization, resulting in poor model performance. An important reason for the decrease in model accuracy is the existence of outliers. Figure 1 The corresponding relationship between the floating point and quantized values ​​of the eight-bit symmetrical quantization is shown in the figure. The maximum floating point value deviates greatly from the overall floating point value, that is, the quantization blank interval is too large, resulting in a more limited ability to represent the information that can be used by the quantized value, which ultimately greatly reduces the model accuracy. Values ​​like the maximum floating point value in the figure are usually defined as outliers. Existing quantization methods lead to the problem of low model accuracy.

[0006] The existing lightweight methods cannot take into account the characteristics of high flexibility, high efficiency and high precision, resulting in an inability to effectively solve the problems of high hardware cost and low operating efficiency of fire detection models. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a lightweight fire detection method based on two-level pruning and adaptive quantization in response to the above-mentioned deficiencies in the prior art. The method improves the flexibility of model lightweighting, improves the accuracy of the lightweight model, reduces the scale of the lightweight model, and reduces the time consumption of lightweight model reasoning.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0009] A lightweight fire detection method based on double-stage pruning and adaptive quantization comprises the following steps:

[0010] Step 1: construct a non-lightweight convolutional neural network model according to the fire detection task to be implemented, wherein the convolutional neural network model includes an input layer, a convolutional layer, a pooling layer and an output layer, wherein the input layer receives known fire image data and passes it to the convolutional layer, wherein the convolutional layer outputs a corresponding feature map according to the input feature map, the pooling layer downsamples the feature map output by the convolutional layer, and outputs the corresponding feature map, and the output layer makes a final recognition result according to the feature maps output by the convolutional layer and the pooling layer; the convolutional neural network includes a plurality of convolutional layers; all convolutional layers in the convolutional neural network model use fusible branch convolution blocks, wherein the fusible branch convolution blocks include trunk branches and shortcut branches, and the outputs of the trunk branches and shortcut branches are fused as the final output of the fusible branch convolution block;

[0011] The input feature map in the trunk branch is sequentially passed through the first convolutional layer and the batch normalization layer to output the feature map, the feature map of each channel is linearly multiplied with the channel-level importance factor of the corresponding channel to obtain the feature map of each channel, and then linearly multiplied with the hierarchical importance factor to complete the output of the first feature map; the input feature map in the shortcut branch is first subjected to the feature map matching operation, and then the feature map of each channel is linearly multiplied with the channel-level importance factor of the corresponding channel, and then linearly multiplied with the adaptive information control factor to complete the output of the second feature map;

[0012] Step 2: Perform L1 sparsification training on the convolutional neural network model to make the layer-level importance factor and channel-level importance factor approach zero. After training, unimportant layers and channels are pruned, and the unpruned fusible branch convolution blocks are fused into ordinary convolution layers. The fused convolutional neural network model is called a pruned model; an unimportant layer refers to a layer whose layer-level importance factor is less than 0.0001, and an unimportant channel refers to a channel-level importance factor of a channel less than 0.0001;

[0013] Step 3, quantize the pruned model, and remove weight outliers and activation value outliers from each layer of the network according to the weight outlier percentage parameter and the activation value outlier percentage parameter; the quantized pruned model is called a dynamic quantization model;

[0014] Step 4: Taking the minimum loss value of the feature map output by the dynamic quantization model and the pruning model as the optimization goal, the weight outlier percentage parameter and the activation value outlier percentage parameter are continuously iterated until the final weight outlier percentage parameter and the activation value outlier percentage parameter are obtained;

[0015] Step 5: The pruned model is actually quantized according to the final weight outlier percentage parameter and the activation value outlier percentage parameter. The model after actual quantization is a lightweight fire detection model based on dual-stage pruning and adaptive quantization.

[0016] Step 6: Input the fire image to be detected into the lightweight fire detection model based on two-level pruning and adaptive quantization, and output the corresponding recognition result.

[0017] Furthermore, the feature map matching operation includes a yes / no operation, a 1×1 convolution operation, and an average pooling operation. The number of channels and the size of the input feature map and the output first feature map are judged based on the input feature map and the convolution layer of the trunk branch. When the number of channels of the input feature map and the output first feature map are inconsistent, the feature map matching operation adopts a 1×1 convolution operation. When the size of the input feature map and the output first feature map are inconsistent, the feature map matching operation adopts an average pooling operation. When the number of channels and the size of the input feature map and the output first feature map are consistent, the feature map matching operation adopts a no operation.

[0018] Furthermore, the fused branch convolution block in step 2 has the property of being fused into a common convolution, and its expression is as follows:

[0019] Y=(X*W+b)+(X*W s +b s )·h i ·g i =X*W f +b f

[0020] Among them, * is the convolution operation, · represents linear multiplication, and h i is the channel-level importance factor of the i-th layer, g i is the adaptive information control factor of the shortcut branch of the i-th layer, X and Y are the feature maps of the corresponding fusion branch convolution block input and output, W and b are the weights and biases of the convolution after the main branch convolution layer is fused with the batch normalization layer and the importance factor, W s and bs is the weight and bias of the convolution transformed by the shortcut branch feature map matching operation, W f and b f It is the weight and bias of the ordinary convolution after the fusion of the trunk branch and the shortcut branch.

[0021] Furthermore, when the feature map matching operation adopts a 1×1 convolution operation, is equal to the bias value corresponding to the 1×1 convolution, The expression is as follows:

[0022]

[0023] When the feature map matching operation adopts the average pooling operation, is zero, The expression is as follows:

[0024]

[0025] When the feature map matching operation adopts no operation, is zero, The expression is as follows:

[0026]

[0027] Among them, j is the output channel, q is the input channel, and v represents the convolution kernel size of the convolution layer. and The weights and biases of the j-th output channel and the q-th input channel of the convolution transformed by the shortcut branch feature map matching operation, For Fill the surrounding with 0 to generate a convolution kernel of size v×v. is the weight of the j-th output channel and the q-th input channel of the 1×1 convolution on the shortcut branch. Copy(0) generates a convolution kernel of size v×v by copying 0. To copy Generate a convolution kernel of size v×v. Padding(1) is to fill 0 around the value 1 to generate a convolution kernel of size v×v.

[0028] Furthermore, the initial values ​​of the weight outlier percentage parameter and the activation value outlier percentage parameter are randomly generated by a computer.

[0029] Furthermore, the removal process of step 3 is as follows: obtaining the absolute value of the weight to be quantified, taking the product of the maximum value among the absolute values ​​of the weight to be quantified and the weight outlier percentage parameter as the outlier threshold, and removing the weight to be quantified that is greater than the outlier threshold among the absolute values ​​of the weight to be quantified; obtaining the absolute value of the activation to be quantified, taking the product of the maximum value among the absolute values ​​of the activation to be quantified and the activation outlier percentage parameter as the outlier threshold, and removing the activation to be quantified that is greater than the outlier threshold among the absolute values ​​of the activation to be quantified.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] The present invention combines the advantages of pruning and quantization to make the lightweight model smaller and more accurate. Specifically, the method first constructs a non-lightweight convolutional neural network model based on the fire detection task, and replaces the convolution layer in the original convolutional neural network model with a fusible branch convolution block to build a convolutional neural network model that can support two-level pruning and quantization. The fusible branch convolution block includes a trunk branch and a shortcut branch. During model training, pruning is performed simultaneously at both the channel level and the level by sparsifying the channel-level importance factor and the level importance factor. After pruning is completed, the two branches of the fused branch convolution block are fused into a common convolution layer to obtain the final pruned model; then the pruned model is quantized through the adaptive optimal weight outlier percentage parameters and activation value outlier percentage parameters; this method performs pruning at both the channel level and the layer level, and adaptively quantizes according to the characteristics of each layer of the model, effectively realizing the lightweight of the model, making the lightweight model higher in performance and smaller in scale; by lightweighting the fire detection model, the problems of high hardware cost and low operating efficiency of the existing fire detection model are solved.

[0032] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of the corresponding relationship between floating point and quantization values ​​of eight-bit symmetrical quantization in the prior art;

[0034] Figure 2a It is a structural diagram of a branch convolution block that can be fused when the feature graph matching operation of an embodiment of a lightweight fire detection method based on double-stage pruning and adaptive quantization of the present invention is a no-operation operation;

[0035] Figure 2b It is a schematic diagram of the structure of a branch convolution block that can be fused when the feature map matching operation of the embodiment of the lightweight fire detection method based on double-stage pruning and adaptive quantization of the present invention is a 1×1 convolution kernel;

[0036] Figure 2cIt is a structural diagram of a branch convolution block that can be fused when the feature map matching operation of an embodiment of a lightweight fire detection method based on double-stage pruning and adaptive quantization of the present invention is average pooling;

[0037] Figure 3 A schematic diagram of a non-lightweight model structure of fire detection according to an embodiment of a lightweight fire detection method based on double-stage pruning and adaptive quantization of the present invention;

[0038] Figure 4 It is a schematic diagram of the fusion process of fusible branch convolution blocks in an embodiment of the lightweight fire detection method based on two-level pruning and adaptive quantization of the present invention. DETAILED DESCRIPTION

[0039] Embodiment of a lightweight fire detection method based on dual-level pruning and adaptive quantization:

[0040] The lightweight fire detection method based on two-level pruning and adaptive quantization includes the following steps:

[0041] Step 1: Construct a non-lightweight convolutional neural network model according to the fire detection task to be implemented. The convolutional neural network model includes an input layer, a convolutional layer, a pooling layer and an output layer. The input layer receives known fire image data and passes it to the convolutional layer. The convolutional layer outputs the corresponding feature map according to the input feature map. The pooling layer downsamples the feature map output by the convolutional layer and outputs the corresponding feature map. The output layer makes the final recognition result according to the feature maps output by the convolutional layer and the pooling layer. All convolutional layers in the convolutional neural network model use fusion-branch convolution blocks, which is equivalent to replacing the original convolution in the convolutional neural network model with the fusion-branch convolution block. The fusion-branch convolution block includes a trunk branch and a shortcut branch. The outputs of the trunk branch and the shortcut branch are fused as the final output of the fusion-branch convolution block. The convolutional neural network model can be: R-CNN model, YOLO model or SSD model, etc.

[0042] When the convolutional neural network model is an R-CNN model, the R-CNN model includes a region proposal network, a convolution layer, a pooling layer and a classifier. The region proposal network is used to generate a suggestion window, the RPN network is used to accept the feature map extracted by the convolution layer as input and output the feature map, the pooling layer converts the feature map output by the convolution layer into a feature map of a fixed size, and the classifier performs a specific classification operation according to the above feature map and outputs the final detection result; by replacing the convolution layer with the fusible branch convolution block of the present application, the lightweight operation of the model is achieved.

[0043] When the convolutional neural network model is a YOLO model, the YOLO model includes an input end, a backbone network, a neck network and an output end, wherein the backbone network includes 5 convolutional layers and 3 fully connected layers; similarly, by replacing all 5 convolutional layers with the fusible branch convolution blocks of the present application, lightweight operation of the model is achieved.

[0044] When the convolutional neural network model is an SSD model, the SSD model includes a feature extraction layer, an additional feature layer, a detection layer, an anchor frame and a loss function. The additional feature layer includes a plurality of feature maps to form a feature pyramid for target detection, and improves the detection accuracy by performing target detection at multiple scales; the detection layer sets feature layers of different scales so that the model can predict targets on feature maps of different receptive fields, thereby realizing the detection of targets of different sizes. By replacing all convolutional layers with the fusible branch convolution blocks of the present application, the lightweight operation of the model is realized.

[0045] like Figure 3 As shown, the input feature map in the above trunk branch passes through the first convolution layer and the batch normalization layer in turn to output the feature map. The feature map of each channel is linearly multiplied with the channel-level importance factor of the corresponding channel to obtain the feature map of each channel, and then linearly multiplied with the hierarchical importance factor to complete the output of the first feature map; the input feature map in the above shortcut branch first undergoes the feature map matching operation, and then the feature map of each channel is linearly multiplied with the channel-level importance factor of the corresponding channel, and then linearly multiplied with the adaptive information control factor to complete the output of the second feature map. Figure 3 The fusion process shown is that first, the channel-level and layer-level importance factor product operations are linear multiplications, which can be fused in the batch normalization layer in front of it after model training; then, the batch normalization layer and the discrete convolution layer in front of it can also be linearly fused; therefore, the trunk branch can be finally fused into a convolution layer after the model training is completed; after the trunk branch is fused, the expression of the fusible branch convolution block is as follows:

[0046] Y=X*W+b+h i ·g i ·f(X)

[0047] Among them, X and Y are the feature maps of the input and output of the corresponding fusion branch convolution block, f(X) represents the feature map matching operation on the input X, * is the convolution operation, W and b are the weights and biases of the convolution after the main branch convolution layer is fused with the batch normalization layer and the importance factor, · is the linear multiplication, and g i is the adaptive information control factor of the shortcut branch at the i-th layer, h i is the channel-level importance factor of the i-th layer.

[0048] The shortcut branches are fused into a convolutional layer, and the convolutional layer is fused with the convolutional layer of the trunk branch into a single convolutional layer. The expression of the fused branch convolutional block is as follows:

[0049] Y=X*W f +b f

[0050] Among them, W f and b f The weights and biases of the ordinary convolution after the fusion of the trunk branch and the shortcut branch.

[0051] Use f(·) with a weight W s , bias is b s The convolution of f and b f The expression is as follows:

[0052]

[0053] Where t and u represent the total number of output and input channels respectively; and The weights and biases of the j-th output channel and the q-th input channel of the convolution transformed by the shortcut branch feature map matching operation, and is the weight and bias of the j-th output channel and the q-th input channel of the ordinary convolution after the fusion of the trunk branch and the shortcut branch, W jq and b jq The weights and biases of the j-th output channel and the q-th input channel are convolved after the main branch convolution layer fuses the batch normalization layer and the importance factor.

[0054] Figure 2a-2c Medium, m i is the hierarchical importance factor, which is used to indicate the importance of the i-th layer, h ij is the channel-level importance factor, which is used to indicate the importance of the jth channel in the i-th layer. i is the adaptive information control factor of the shortcut branch, which is used to adaptively control the flow of the shortcut branch information of the i-th layer. All three are trainable adaptive parameters. As shown in Figure 2, the above feature map matching operation includes the presence or absence operation, the 1×1 convolution operation and the average pooling operation. Figure 4 Where f(·) represents the feature map matching operation, which determines whether the number of channels and size of the input feature map and the output first feature map are consistent based on the input feature map and the convolution layer of the trunk branch;

[0055] like Figure 2b As shown in , when the number of channels of the input feature map and the first output feature map is inconsistent, the feature map matching operation adopts a 1×1 convolution operation. The expression is as follows:

[0056]

[0057] like Figure 2c As shown in , when the size of the input feature map is inconsistent with the size of the first output feature map, the feature map matching operation adopts the average pooling operation. The expression is as follows:

[0058]

[0059] like Figure 2a As shown in , when the number of channels and size of the input feature map and the output first feature map are consistent, the feature map matching operation adopts no operation. The expression is as follows:

[0060]

[0061] Among them, Padding is to get a v×v convolution kernel by filling 0 around the operand value, and Copy is to get a v×v convolution kernel by copying the operand value. is the weight of the jth output channel and the qth input channel of the 1×1 convolution on the shortcut branch, and v represents the convolution kernel size of the convolution layer. Copy(0) generates a convolution kernel of size v×v by copying 0 (the convolution kernel is a matrix, that is, all values ​​in the matrix are 0). To copy Generate a convolution kernel of size v×v (the values ​​in the convolution kernel matrix are ), For Fill the surrounding with 0 to generate a convolution kernel of size v×v (the value at the center of the convolution kernel matrix is The rest of the values ​​are 0), Padding(1) is to add 0 around the value 1 to generate a convolution kernel of size v×v (the value at the center of the convolution kernel matrix is ​​1, and the rest of the values ​​are 0).

[0062] Above The calculation of is relatively simple. Specifically, when the feature map matching operation on the shortcut branch adopts a 1×1 convolution operation, Equal to the bias value of the corresponding 1×1 convolution; when the feature map matching operation is no operation or average pooling operation, Both are 0.

[0063] In summary, the fusible branch convolution block structure is fused into an ordinary convolution layer through mathematical linear calculation; therefore, the fusible branch convolution block can replace its ordinary convolution layer in the convolutional neural network model without affecting the final structure of the model.

[0064] Step 2: Perform L1 sparsification training on the convolutional neural network model to make the layer-level importance factor and channel-level importance factor tend to zero. After training, unimportant layers and channels are pruned, and the unpruned fused branch convolution blocks are fused into ordinary convolution layers. The fused convolutional neural network model is called a pruned model; unimportant layers refer to layers whose layer-level importance factor is less than 0.0001, and unimportant channels refer to channels whose channel-level importance factor is less than 0.0001. L1 sparsification refers to the use of L1 regularization technology to make many values ​​in the model parameters tend to zero, thereby achieving model sparsification.

[0065] Step 3: quantize the pruned model, and remove weight outliers and activation outliers from each layer of the network according to the weight outlier percentage parameter and the activation outlier percentage parameter; the quantized pruned model is called a dynamic quantization model. The initial values ​​of the weight outlier percentage parameter and the activation outlier percentage parameter are randomly generated by a computer.

[0066] The removal process is as follows: obtaining the absolute value of the weight to be quantified, taking the product of the maximum value among the absolute values ​​of the weight to be quantified and the weight outlier percentage parameter as the outlier threshold, and removing the weight to be quantified that is greater than the outlier threshold among the absolute values ​​of the weight to be quantified; obtaining the absolute value of the activation to be quantified, taking the product of the maximum value among the absolute values ​​of the activation to be quantified and the activation outlier percentage parameter as the outlier threshold, and removing the activation to be quantified that is greater than the outlier threshold among the absolute values ​​of the activation to be quantified.

[0067] Step 4: The optimization goal is to minimize the loss value of the feature map output by the dynamic quantization model and the pruning model. The loss value is determined by the difference between the feature map output by the dynamic quantization model and the pruning model. The weight outlier percentage parameter and the activation value outlier percentage parameter are continuously iterated until the final weight outlier percentage parameter and the activation value outlier percentage parameter are obtained.

[0068] Step 5: The pruned model is actually quantized according to the final weight outlier percentage parameter and activation value outlier percentage parameter. The model after actual quantization is a lightweight fire detection model based on dual-stage pruning and adaptive quantization.

[0069] Step 6: Input the fire image to be detected into the lightweight fire detection model based on two-level pruning and adaptive quantization, and output the corresponding recognition result.

[0070] like Figure 3As shown in the figure, this lightweight method can be applied to the lightweight of fire detection model. First, a convolutional neural network model is built based on the SSD target detection algorithm. Since the SSD algorithm has a high speed advantage when processing detection tasks, in order to meet the high real-time requirements of fire detection, the SSD algorithm is selected here to build a convolutional neural network model for fire detection; of course, other target detection algorithms can also be used to build the convolutional neural network model. The image data used for training uses the public data set of the School of Computer Science and Technology of Harbin Institute of Technology (Shenzhen). The data set contains a total of 2688 flame images and 3273 flame targets, including many complex scenes, covering daytime, nighttime, indoors, outdoors, forests, houses, etc. The model is lightweighted using the above steps 1-5 to obtain the corresponding lightweight fire detection model based on two-level pruning and adaptive quantization. In the lightweight process, after pruning, the layers and channels of the model are reduced. Before quantization, 32-bit floating point numbers are used, and after quantization, they become 8-bit fixed point numbers, and the scale of the corresponding model becomes smaller. The lightweight model is compared with the non-lightweight model and the model lightweighted by the existing technology, mainly comparing the detection accuracy and scale of the models. The specific comparison results are shown in Table 1 below.

[0071] Table 1

[0072]

[0073]

[0074] According to the experimental results in Table 1, two-level pruning can effectively compress the model: compared with separate channel-level pruning and layer-level pruning, the lightweight model obtained by two-level pruning has higher accuracy and smaller model size; adaptive quantization can effectively compress the model: compared with general quantization, the lightweight model obtained by adaptive quantization has higher accuracy; the fire detection model is lightweighted by using the lightweight fire detection method based on two-level pruning and adaptive quantization of the present invention, which can compress the model to less than one-tenth of its original size without significantly reducing the performance.

[0075] The above fire detection model examples show that the lightweight fire detection method based on double-stage pruning and adaptive quantization proposed in the present invention can effectively reduce the model scale while ensuring the model performance, and has outstanding technical advancement and engineering practicality.

[0076] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A lightweight fire detection method based on two-level pruning and adaptive quantization, Features: The method comprises the following steps: Step 1: construct a non-lightweight convolutional neural network model according to the fire detection task to be implemented, wherein the convolutional neural network model includes an input layer, a convolutional layer, a pooling layer and an output layer, wherein the input layer receives known fire image data and passes it to the convolutional layer, the convolutional layer outputs a corresponding feature map according to the input feature map, the pooling layer downsamples the feature map output by the convolutional layer, and outputs the corresponding feature map, and the output layer makes a final recognition result according to the feature maps output by the convolutional layer and the pooling layer; the convolutional neural network includes multiple convolutional layers; All convolutional layers in the convolutional neural network model use fusible branch convolution blocks, which include trunk branches and shortcut branches. The outputs of the trunk branches and shortcut branches are fused as the final output of the fusible branch convolution block. The input feature map in the trunk branch is sequentially passed through the first convolution layer and the batch normalization layer to output the feature map, the feature map of each channel is linearly multiplied with the channel-level importance factor of the corresponding channel to obtain the feature map of each channel, and then linearly multiplied with the level importance factor to complete the output of the first feature map; The input feature map in the shortcut branch is first subjected to a feature map matching operation, and then the feature map of each channel is linearly multiplied with the channel-level importance factor of the corresponding channel, and then linearly multiplied with the adaptive information control factor to complete the output of the second feature map; Step 2: Perform L1 sparsification training on the convolutional neural network model to make the layer-level importance factor and channel-level importance factor approach zero. After training, unimportant layers and channels are pruned, and the unpruned fusible branch convolution blocks are fused into ordinary convolution layers. The fused convolutional neural network model is called a pruned model; an unimportant layer refers to a layer whose layer-level importance factor is less than 0.0001, and an unimportant channel refers to a channel-level importance factor of a channel less than 0.0001; Step 3, quantize the pruned model, and remove weight outliers and activation value outliers from each layer of the network according to the weight outlier percentage parameter and the activation value outlier percentage parameter; the quantized pruned model is called a dynamic quantization model; Step 4: Taking the minimum loss value of the feature map output by the dynamic quantization model and the pruning model as the optimization goal, the weight outlier percentage parameter and the activation value outlier percentage parameter are continuously iterated until the final weight outlier percentage parameter and the activation value outlier percentage parameter are obtained; Step 5: The pruned model is actually quantized according to the final weight outlier percentage parameter and the activation value outlier percentage parameter. The model after actual quantization is a lightweight fire detection model based on dual-stage pruning and adaptive quantization. Step 6: Input the fire image to be detected into the lightweight fire detection model based on two-level pruning and adaptive quantization, and output the corresponding recognition result.

2. The lightweight fire detection method based on double-stage pruning and adaptive quantization according to claim 1, Features: The feature map matching operation includes a yes / no operation, a 1×1 convolution operation, and an average pooling operation. It is determined whether the number of channels and the size of the input feature map and the output first feature map are consistent based on the input feature map and the convolution layer of the trunk branch. When the number of channels of the input feature map and the output first feature map are inconsistent, the feature map matching operation adopts a 1×1 convolution operation. When the size of the input feature map and the output first feature map are inconsistent, the feature map matching operation adopts an average pooling operation. When the number of channels and the size of the input feature map and the output first feature map are consistent, the feature map matching operation adopts a no operation.

3. The lightweight fire detection method based on double-stage pruning and adaptive quantization according to claim 2, Features: The fusible branch convolution block in step 2 has the property of being fused into a normal convolution, and its expression is as follows: Y=(X*W+b)+(X*W s +b s )·h i ·g i =X*W f +b f Among them, * is the convolution operation, · represents linear multiplication, and h i is the channel-level importance factor of the i-th layer, g i is the adaptive information control factor of the shortcut branch of the i-th layer, X and Y are the feature maps of the corresponding fusion branch convolution block input and output, W and b are the weights and biases of the convolution after the main branch convolution layer is fused with the batch normalization layer and the importance factor, W s and b s is the weight and bias of the convolution transformed by the shortcut branch feature map matching operation, W f and b f The weights and biases of the ordinary convolution after the fusion of the trunk branch and the shortcut branch.

4. The lightweight fire detection method based on double-stage pruning and adaptive quantization according to claim 3, Features: When the feature map matching operation adopts a 1×1 convolution operation, is equal to the bias value corresponding to the 1×1 convolution, The expression is as follows: When the feature map matching operation adopts the average pooling operation, is zero, The expression is as follows: When the feature map matching operation adopts no operation, is zero, The expression is as follows: Among them, j is the output channel, q is the input channel, and v represents the convolution kernel size of the convolution layer. and The weights and biases of the j-th output channel and the q-th input channel of the convolution transformed by the shortcut branch feature map matching operation, For Fill the surrounding with 0 to generate a convolution kernel of size v×v. is the weight of the j-th output channel and the q-th input channel of the 1×1 convolution on the shortcut branch. Copy(0) generates a convolution kernel of size v×v by copying 0. To copy Generate a convolution kernel of size v×v. Padding(1) is to fill 0 around the value 1 to generate a convolution kernel of size v×v.

5. The lightweight fire detection method based on double-stage pruning and adaptive quantization according to claim 1, Features: The initial values ​​of the weight outlier percentage parameter and the activation value outlier percentage parameter are randomly generated by a computer.

6. The lightweight fire detection method based on double-stage pruning and adaptive quantization according to claim 1, Features: The removal process of step 3 is as follows: obtaining the absolute value of the weight to be quantified, taking the product of the maximum value among the absolute values ​​of the weight to be quantified and the weight outlier percentage parameter as the outlier threshold, and removing the weight to be quantified that is greater than the outlier threshold among the absolute values ​​of the weight to be quantified; obtaining the absolute value of the activation to be quantified, taking the product of the maximum value among the absolute values ​​of the activation to be quantified and the activation outlier percentage parameter as the outlier threshold, and removing the activation to be quantified that is greater than the outlier threshold among the absolute values ​​of the activation to be quantified.

Images