Post-training pruning method for flame detection

Abstract

The invention relates to a flame identification method of a post-training pruning model based on gradient weighted class activation mapping. The post-training pruning method for flame detection comprises the following steps: acquiring a flame image to be identified; training the post-training pruning model based on the gradient weighted class activation mapping to obtain a trained post-training pruning model of the gradient weighted class activation mapping; inputting a flame image to be recognized into the trained post-training pruning model of the gradient weighting class activation mapping to obtain a single-channel thermodynamic diagram of the flame; screening the single-channel thermodynamic diagrams of the flames, and screening and synthesizing the screened single-channel thermodynamic diagrams to obtain a final thermodynamic diagram of the flames; the post-training pruning method based on gradient weighted class activation mapping can flexibly define and adjust channel importance evaluation standards, and is suitable for different application scenes and requirements.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning model optimization, and relates to a post-training pruning method for flame detection. Background Art

[0002] With the rapid development of deep learning technology, convolutional neural network models have achieved significant performance improvements in tasks such as image classification, object detection, and semantic segmentation. However, these models usually have a large number of parameters and high computational resource consumption, posing great challenges to practical applications, especially the deployment on resource-constrained devices. To reduce the computational complexity and storage requirements of the models, model pruning has become an important optimization method.

[0003] Model pruning, that is, extracting a small model by analyzing a large-sized model. By means of the process of deleting unimportant weights or channels, the reduction of the model's parameter quantity is achieved while maintaining or minimizing the performance loss. This process is of great significance for improving the inference speed of the model, reducing power consumption, and deployment on embedded devices. By pruning the model, redundant parameters can be effectively removed, making the model more efficient and lightweight.

[0004] In the field of flame detection, the application of convolutional neural network models faces huge challenges:

[0005] 1. Insufficient data diversity: The flame sample data is relatively limited and lacks diversity, which makes the generalization ability of the network model poor in complex environments.

[0006] 2. High real-time requirements: Flame detection usually requires making accurate predictions within a short time to take timely measures, which poses high requirements for the inference speed of the model.

[0007] 3. Resource constraints: Flame detection usually needs to process a large amount of image data. When the system is deployed on embedded devices or mobile devices, by splitting a large recognition system into multiple independent small recognition systems, computational resources and storage space can be effectively saved, while maintaining high efficiency and low resource consumption.

[0008] 4. Task specificity: Flame detection tasks in different environments have different characteristics and requirements. If the general model can be optimized and modified for specific tasks, the adaptability and robustness of the model will be greatly improved. Summary of the Invention

[0009] To address some significant limitations of existing model pruning methods in actual flame detection tasks:

[0010] Lack of interpretability: Traditional pruning methods based on weights and feature importance lack an intuitive interpretation mechanism, making it difficult for users to understand the specific reasons for pruning decisions. These methods usually simply prune channels with smaller weights or rely on certain statistical metrics to evaluate feature importance, but the interpretability of these metrics is limited and they cannot provide detailed channel importance information.

[0011] Weak task relevance: Traditional pruning methods have weak task relevance and cannot be specifically modified for a particular task. These methods usually rely on the overall weights or feature importance of the model, rather than the requirements of a specific task.

[0012] Therefore, in the flame detection task, traditional pruning methods may not be able to effectively identify and retain channels that are important for specific flames or environments. The technical solution adopted in the present invention is: a post-training pruning method for flame detection, including the following steps:

[0013] Train a flame detection network model to obtain a trained flame detection network model;

[0014] Use the gradient-weighted class activation mapping method on the trained network model to obtain a single-channel heatmap of the feature layer of interest under the flame detection model;

[0015] Evaluate the importance of each channel in the trained flame detection network model based on the single-channel heatmap;

[0016] Based on the pruning scale, prune the single channel according to the pruning principle while retaining features crucial for flame recognition to obtain a pruned flame detection network model;

[0017] Fine-tune the pruned flame detection network model;

[0018] Input the images in the test set into the fine-tuned flame detection network model to achieve flame recognition.

[0019] Furthermore, the flame detection network model adopts an image classification model or an object detection model.

[0020] Furthermore, the process of using the gradient-weighted class activation mapping method on the trained network model to obtain a single-channel heatmap of the feature layer of interest under the flame detection model is as follows:

[0021] Evaluate the contribution degree by using the mean size of the heatmap: With the mean size of each heatmap, that is, the average intensity of the pixel values in the heatmap, indirectly evaluate the contribution of each channel to flame recognition. If the mean value of the single-channel heatmap generated by a certain channel exceeds the threshold mean value, it indicates that this channel plays a greater role in the model decision, and thus its importance for flame recognition is determined to be relatively high.

[0022] Furthermore, based on the pruning scale, the single channel is pruned according to the pruning principle while retaining the features crucial for flame recognition, obtaining the pruned flame detection network model. The process of fine-tuning the pruned flame detection network model is as follows:

[0023] Calculate the gradient of the output of the selected feature layer in the flame detection network model with respect to the final output, and perform global average pooling to obtain the weight of each channel;

[0024] Multiply the weight of each channel by the corresponding feature map, and perform weighted summation to generate the class activation mapping. According to the generated class activation mapping, identify the feature layers and channels that contribute more to the model output, where the feature layers and channels in the high-value regions are regarded as important features;

[0025] Quantize the gradient information, divide the quantized information into 10 levels, and then delete channels according to the set pruning threshold according to the levels;

[0026] Prune the channels according to the set pruning threshold to obtain a lightweight basic model. After pruning, the lightweight model needs to be fine-tuned, that is, retrained with a smaller learning rate and a shorter training period to ensure the maximum recovery of performance.

[0027] Furthermore, the process of calculating the gradient of the output of the selected feature layer in the network model with respect to the final output, performing global average pooling to obtain the weight of each channel, and multiplying the weight of each channel by the corresponding feature map and performing weighted summation to generate the class activation mapping is as follows:

[0028] First, pass the input image x through the network for forward propagation to obtain the feature map A of the target feature layer k ;

[0029] Subsequently, perform backpropagation layer by layer to calculate the gradient of the feature map A k of the target feature layer with respect to the final output category;

[0030] Then perform global average pooling operation to obtain the weight of each channel

[0031] Then use the calculated weight to perform weighted summation on the feature map A k to obtain the class activation mapping

[0032] Analyze the activation status of the class activation mapping of different feature layers, and the feature layers that contribute more to the model can be screened out. Here, no summation operation is performed, that is, the class activation mapping k of different channels of the k-th feature map A is obtained

[0033] Furthermore, the process of setting the pruning threshold is as follows:

[0034] Unify the numerical orders of magnitude between different feature layers to the same scale, specifically as follows:

[0035] Through the normalization process, normalize all channels in the class activation map L c to the interval [0, 1], and then perform quantization operations. Each level is represented by an integer (from 0 to 9);

[0036] Then, by setting a threshold T, the identification and pruning operations for channels below level T are realized. For each feature map A k , a specific selection process is carried out.

[0037] A post-training pruning method for flame detection provided by the present invention can intuitively display the feature regions that the model focuses on in a specific task by means of the heat map generated by the gradient-weighted class activation mapping technology, thereby improving the interpretability of the pruning process. Through the heat map, it is possible to clearly show the contribution degrees of each channel in the flame detection task and which channels can be safely pruned. At the same time, the importance of channels can be evaluated through the heat map or other task-related metrics to ensure that the performance of the pruned model in a specific task is not affected. Finally, the post-training pruning method based on gradient-weighted class activation mapping proposed by us can flexibly define and adjust the channel importance evaluation criteria to adapt to different application scenarios and requirements. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0039] Figure 1 It is a flowchart of a post-training pruning method based on gradient-weighted class activation mapping of the present invention;

[0040] Figure 2 It is a schematic diagram of class activation mapping and feature extraction;

[0041] Figure 3 It is a single-channel heat map of the model_22 layer trained with the Yolov5_m model for flames in the present invention;

[0042] Figure 4(a) is the original input image in the present invention, and (b) is the heat map of the feature layer for flames generated by the model_22 layer trained using the Yolov5_m model;

[0043] Figure 5 is the class activation mapping statistical chart for flames produced by the model_22 layer trained using the Yolov5_m model in the present invention. Detailed implementation manners

[0044] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the drawings and in combination with the embodiments.

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and in no way constitutes a limitation to the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0046] Figure 1 is the flowchart of a post-training pruning method for flame detection in the present invention;

[0047] A post-training pruning method for flame detection includes the following steps:

[0048] S1: Train a flame detection network model to obtain a trained flame detection network model;

[0049] S2: Use the Gradient-weighted Class Activation Mapping (Grad-CAM) method for the trained network model to obtain a single-channel heat map of the feature layer of interest under the flame detection model;

[0050] S3: Evaluate the importance of each channel in the trained flame detection network model based on the single-channel heat map;

[0051] S4: Based on the pruning scale, prune the single channel according to the pruning principle while retaining the features crucial for flame recognition to obtain a pruned flame detection network model;

[0052] S5: Fine-tune the pruned flame detection network model;

[0053] S6: Input the images in the test set into the fine-tuned flame detection network model to achieve flame recognition.

[0054] Steps S1 / S2 / S3 / S4 / S5 / S6 are executed in sequence;

[0055] The flame detection network model uses an image classification model or an object detection model.

[0056] Through a normal training process, the flame detection network model is brought to a predetermined performance level. Subsequently, this flame detection model is used as the base model for subsequent pruning operations;

[0057] Use the gradient-weighted class activation mapping technique to calculate the class activation mapping of these feature layers. Based on the generated class activation mapping graph, identify the feature layers and channels that contribute more to the prediction results. Among them, the feature layers and channels in the high-value regions are regarded as important features;

[0058] Set a pruning threshold to determine which feature layers or channels have negligible contributions;

[0059] According to the set pruning threshold, prune those feature layers or channels that contribute less to the model. After pruning, fine-tune the model to restore the performance that may be lost due to pruning.

[0060] The process of obtaining the single-channel heat map of the feature layer of interest under the flame detection model by using the gradient-weighted class activation mapping method for the trained network model is as follows:

[0061] Evaluate the contribution degree using the mean value of the heat map: With the mean value of each heat map: that is, the average intensity of the pixel values in the heat map, indirectly evaluate the contribution of each channel to flame recognition. If the mean value of the single-channel heat map generated by a certain channel exceeds the threshold mean value, it indicates that this channel plays a greater role in model decision-making, and thus its importance for flame recognition is determined to be relatively high.

[0062] Based on the pruning scale, prune the single channel according to the pruning principle, while retaining the features crucial for flame recognition, to obtain the pruned flame detection network model. The process of fine-tuning the pruned flame detection network model is as follows:

[0063] Calculate the gradient of the output of the selected feature layer in the flame detection network model with respect to the final output, and perform global average pooling to obtain the weight of each channel;

[0064] Multiply the weight of each channel by the corresponding feature map, and perform weighted summation to generate the class activation mapping graph. Based on the generated class activation mapping, identify the feature layers and channels that contribute more to the model output. Among them, the feature layers and channels in the high-value regions are regarded as important features;

[0065] Quantize the gradient information, divide the quantized information into 10 levels, and then delete channels according to the set pruning threshold by level;

[0066] Prune the channels according to the set pruning threshold to obtain a lightweight basic model. After pruning, the lightweight model needs to be fine-tuned, that is, retrained with a smaller learning rate and a shorter training cycle to ensure the maximum recovery of performance.

[0067] The process of calculating the gradient of the output of the selected feature layer in the computing network model to the final output, performing global average pooling, and obtaining the weight of each channel; multiplying the weight of each channel by the corresponding feature map and summing the weighted values to generate a class activation mapping is as follows:

[0068] First, pass the input image x through the network for forward propagation to obtain the feature map A of the target feature layer k , and its calculation formula is shown in Equation (1), where k represents the kth layer. If multiple feature layers need to be selected, this operation needs to be performed multiple times;

[0069] A k = Conv(W k , x) (1)

[0070] where W k is the convolution kernel weight of the kth layer; then perform backpropagation layer by layer to calculate the gradient of the feature map A k of the target feature layer to the final output category y c , denoted as A k ', and the formula is shown in Equation (2), as follows:

[0071]

[0072] where, is the gradient of the last layer feature map A L to the final output category y c , is the gradient of the current layer feature map A k to the next layer feature map A k+1 ; then perform global average pooling operation to obtain the weight of each channel The formula is shown in (3), as follows:

[0073]

[0074] where j represents the jth channel under this feature layer, c represents the category c, Z is the size of the feature map, that is, the product of the width and height of the feature map, and A k (i, j) represents the value of the feature map at the (i, j) position of the kth layer; then use the calculated weight Perform a weighted sum on the feature map A k to obtain the class activation map The formula is shown in Equation (4).

[0075]

[0076] By analyzing the activation states of the class activation maps of different feature layers the feature layers that contribute more to the model can be screened out. If the summation operation is not performed here, the k-th feature map A k of the class activation maps under different channels can be obtained The formula is shown in Equation (5):

[0077]

[0078] Furthermore, a pruning threshold is set to determine which feature layers or channels have negligible contributions. The process is as follows;

[0079] First, it is necessary to perform quantization processing on the class activation maps of the selected feature layers and divide them into 10 levels. This is because the numerical orders of magnitude between different feature layers may be different and need to be unified to the same scale for comparison and analysis. The formulas are shown in Equation (6) and Equation (7).

[0080] N_L c =(L c -min(L c )) / (max(L c )-min(L c )) (6)

[0081]

[0082] Among them, Equation (6) is the normalization process, which normalizes all channels in the class activation map L c to the interval [0, 1], and then performs quantization operations through Equation (7). Each level is represented by an integer (0 to 9), where the operation represents rounding down. By setting a threshold T, the identification and pruning operations for the channels below level T can be achieved. For each feature map A k the specific selection process is as follows:

[0083]

[0084] T4. After the selection is completed, pruning operations are performed on the network model to retain important channels and feature layers. Here, the important channels and important feature layers refer to the high-value parts after quantization.

[0085] T5. Fine-tune the network model after pruning to restore its performance to the greatest extent possible.

[0086] T6. End the fine-tuning process when the minimum requirements set for the task are met, and obtain the final network model.

[0087] The above operations are performed in the order of T1 / T2 / T3 / T4 / T5 / T6. If you are not satisfied with the pruning rate and accuracy after T5, you can repeat the operations of T2 / T3 / T4 / T5 multiple times.

[0088] Figure 2 It is a schematic diagram of class activation mapping and feature extraction.

[0089] Figure 3 It shows a single-channel heatmap of the flame generated by the model_22 layer trained using the YOLOv5_m model in the present invention. It can be seen from the figure that the contribution degrees of different channels to the final prediction result are different. Specifically, different channels correspond to different features of the flame, such as the flame core, outer flame, ignition point, etc. This indicates that there are significant differences in the roles and importance of each channel in different feature layers.

[0090] Figure 4 (a) is the input image in the present invention, and (b) is the heatmap of the feature layer of the flame generated by the model_22 layer trained using the Yolov5_m model.

[0091] Among them, the heatmap of the feature layer is obtained by superimposing the heatmaps of its channels.

[0092] Figure 5 It is a statistical chart of class activation mapping of the flame generated by the model_22 layer trained using the Yolov5_m model in the present invention. It can be seen from the figure the sparsity of the activation pattern: in most cases, the channels in the feature layer show weak activation states, and only a few channels show significant high activation levels. This sparsity indicates that in the information processing process, the model tends to rely on a limited number of key channels for effective feature extraction.

[0093] Table 1 shows the training results of a post-training pruning method based on gradient-weighted class activation mapping in the present invention. It can be analyzed from Table 1 that as the pruning rate increases, the model size decreases significantly, and within a certain range, the loss of accuracy is not significant.

[0094] Table 1 Training Results of This Method

[0095]

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A post-training pruning method for flame detection, characterized in that, It includes the following steps: Train the flame detection network model to obtain a trained flame detection network model; Use the Gradient-weighted Class Activation Mapping (Grad-CAM) method on the trained network model to obtain a single-channel heatmap of the feature layer of interest under the flame detection model; Evaluate the importance of each channel in the trained flame detection network model based on the single-channel heatmap; Based on the pruning scale, prune the single channel according to the pruning principle while retaining the features crucial for flame recognition to obtain a pruned flame detection network model; Fine-tune the pruned flame detection network model; Input the images in the test set into the fine-tuned flame detection network model to achieve flame recognition.

2. The post-training pruning method for flame detection according to claim 1, wherein, The flame detection network model adopts an image classification model or an object detection model.

3. A post-training pruning method for flame detection according to claim 1, characterized in that The process of using the Gradient-weighted Class Activation Mapping (Grad-CAM) method on the trained network model to obtain a single-channel heatmap of the feature layer of interest under the flame detection model is as follows: Evaluate the contribution degree using the mean value of the heatmap: With the help of the mean value of each heatmap, that is, the average intensity of the pixel values in the heatmap, indirectly evaluate the contribution of each channel to flame recognition. If the mean value of the single-channel heatmap generated by a certain channel exceeds the threshold mean value, it indicates that this channel plays a greater role in model decision-making, and thus determine its higher importance for flame recognition.

4. A post-training pruning method for flame detection according to claim 1, characterized in that, The process of pruning the single channel according to the pruning scale and the pruning principle while retaining the features crucial for flame recognition to obtain a pruned flame detection network model and then fine-tuning the pruned flame detection network model is as follows: Calculate the gradient of the output of the selected feature layer in the flame detection network model with respect to the final output, and perform global average pooling to obtain the weight of each channel; Multiply the weight of each channel by the corresponding feature map, sum the weighted values to generate a class activation mapping (CAM) map, and based on the generated CAM, identify the feature layers and channels that contribute more to the model output, where the feature layers and channels in the high-value regions are regarded as important features; Quantify the gradient information, divide the quantified information into 10 levels, and then delete channels according to the set pruning threshold according to the levels; Prune the channels according to the set pruning threshold to obtain a lightweight basic model. After pruning, the lightweight model needs to be fine-tuned, that is, retrained with a smaller learning rate and a shorter training period to ensure the maximum recovery of performance.

5. A post-training pruning method for flame detection according to claim 1, characterized in that The process of calculating the gradient of the output of the selected feature layer in the network model with respect to the final output, performing global average pooling to obtain the weight of each channel, multiplying the weight of each channel by the corresponding feature map, and summing the weighted values to generate a class activation mapping (CAM) map is as follows: First, the input image x is propagated forward through the network to obtain the feature map A of the target feature layer k ; Subsequently, backpropagation is performed layer by layer to calculate the feature map A of the target feature layer k for the gradient of the final output category; Then perform global average pooling operation to obtain the weights of each channel Then use the calculated weights to perform weighted summation on the feature map A k to obtain the class activation map Analyze the class activation maps of different feature layers By analyzing the activation states, the feature layers that contribute more to the model can be screened out. Here, no summation operation is performed, and the k-th feature map A is obtained k The class activation maps of different channels of 6. The post-training pruning method for flame detection according to claim 1, wherein, The process of setting the pruning threshold is as follows: Unify the numerical orders of magnitude between different feature layers to the same scale, specifically as follows: Through the normalization process, the class activation map L c All channels in are normalized to the interval [0, 1], and then a quantization operation is performed, with each level represented by an integer (0 to 9); By setting a threshold T, the recognition and pruning operations for channels below level T are realized. For each feature map A k , a specific selection process is carried out.

Images