Flame detection method and system in multi-source scene

Abstract

The invention discloses a flame detection method and system in a multi-source scene. The method comprises the following steps: firstly, constructing a flame data set in the multi-source scene; secondly, improving a pyramid feature dynamic extraction module in a PS-Net (Progressive Selection Network) by designing jump connection, and improving the detection capability of the model on a small flame target; secondly, a PS-Net mesoscale perception fusion module is improved by designing a flame feature attention mechanism, and the inhibition capability of the model on similar shape feature interference is enhanced; then, by designing a decoder structure, the influence of similar color feature interference on a detection result is reduced; and finally, flame detection is carried out by using the PS-Net + model after training and testing. According to the flame detection method in the multi-source scene, high-precision detection of the flame target in the multi-source scene is facilitated, and technical support is provided for quickly and accurately finding the flame target in reality and timely solving potential safety hazards caused by flames.

Description

Technical Field

[0001] The present invention relates to a flame detection technology, and in particular to a flame detection method and system in a multi-source scenario. Background Art

[0002] Fire not only brings unnecessary economic losses to society, but also endangers people's lives and health. Since the flames are small and easier to control in the early stages of a fire, how to detect the flames in time is the key to effectively prevent fires and has always been the focus of attention in the fire protection field.

[0003] At present, flame detection mainly relies on temperature-sensing, smoke-sensing and light-sensing sensors. Compared with the first two sensors, the detection efficiency of light-sensing image sensors is higher. However, the background of flames in actual environments is often complex, and the collected flame images are often affected by various interferences, such as daytime light, nighttime light, and similar color backgrounds. Early image detection methods were mainly based on the color and motion features of flames, but the features usually need to rely on manual extraction, and the designed detection methods are usually targeted at a specific environmental background, and the generalization ability and robustness of the methods are poor. In recent years, with the rapid development of computer vision, salient object detection (SOD) has been widely used in image editing, target tracking, medical image processing and other fields because it can improve detection accuracy and efficiency. Convolutional Neural Networks (CNN) have excellent feature learning and feature processing capabilities, especially in processing large-scale complex data sets and complex backgrounds, making it a powerful tool for achieving SOD tasks. Therefore, CNN-based SOD has become a trend in the application of machine vision. Compared with traditional flame detection methods, the CNN-based SOD detection method has obvious advantages in autonomous learning and extraction of flame features, especially for flame targets in simple environments, and can meet the needs of flame target detection to a certain extent. However, as the flame background environment becomes more complex, especially when multi-source interference occurs, the existing flame intelligent detection process still has the shortcomings of flame feature loss and false detection. How to improve the accuracy of flame detection in complex multi-source environments is still a technical difficulty. Summary of the invention

[0004] Aiming at the detection of flames in real scenarios, the present invention provides a flame detection method and system in a multi-source scenario. By improving the model's ability to discern and extract flame features, the model can effectively solve the two major problems of missed detection and false detection of flames in a complex multi-source environment, thereby improving the accuracy of flame detection.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions to achieve it:

[0006] A flame detection method in a multi-source scenario is proposed. First, the PS-Net model is improved and optimized to address the challenges of ground leakage detection and false detection in flame detection tasks. Then, the improved PS-Net+ model is used to achieve high-precision flame detection in a multi-source scenario. The specific steps are as follows:

[0007] (1) Construct a flame dataset in a multi-source scenario;

[0008] (2) The pyramid feature dynamic extraction module in PS-Net is improved by designing skip connections to enhance the model’s ability to detect small flame targets;

[0009] The jump connection fuses the output of the previous dilated convolution in the pyramid feature dynamic extraction module with the module input features by adding features, and then uses the fused features as the input of the next dilated convolution;

[0010] (3) The PS-Net mid-scale perception fusion module is improved by designing a flame feature attention mechanism to enhance the model’s ability to suppress interference from similar shape features;

[0011] The flame feature attention mechanism changes the fusion mode of different levels of features in the scale-aware fusion module from addition to splicing, retains more target features, and uses the Softmax normalization function to constrain the weight of each channel of the feature map after a series of convolutions;

[0012] (4) By designing the decoder structure, the influence of similar color feature interference on the detection results is reduced;

[0013] The decoder structure performs convolution and normalization on the input features, then adds and fuses the input features again, and uses the ReLu activation function for nonlinear mapping;

[0014] (5) Use the trained and tested PS-Net+ model to perform flame detection.

[0015] Furthermore, in step (1), a flame dataset in a multi-source scenario is constructed. For the collected flame image data, the flame dataset is expanded by using the random rotation function RandomRotation, the horizontal flip function Randint, and the random cropping function RandomCrop in the deep learning framework, and the dataset is labeled and divided.

[0016] Furthermore, in step (2), the pyramid feature dynamic extraction module in PS-Net is improved by designing a skip connection to improve the model's detection capability for small flame targets, specifically by the following steps:

[0017] (2-1) By dilating the output f of the previous convolution layer j With the input feature f c The features obtained by adding and fusion are used as the input of the next layer of dilated convolution. The calculation formula is as follows:

[0018]

[0019] Where D 1 (.) represents a dilated convolution operation with a dilation rate of 1, D 2 (.) represents a dilated convolution operation with a dilation rate of 3, D 3 (.) represents a dilated convolution operation with a dilation rate of 5, D 4 (.) indicates a dilated convolution operation with a dilation rate of 7;

[0020] (2-2) The dilated convolution output features obtained using skip connections are fused in a progressive manner using three feature dynamic extraction modules.

[0021] Furthermore, the dilated convolution output features obtained by skip connection are fused in a progressive manner using three feature dynamic extraction modules, specifically using the following steps:

[0022] (2-2-1) For the first feature dynamic extraction module, the feature map f obtained by the dilated convolution with dilation rates of 1 and 3 is 1 、f 2 As input, by extracting global features and local features After fusion, we get the feature f 1 ', the calculation formula is as follows:

[0023]

[0024] Where AvP(.) is the average value in the channel dimension, GAP(.) is the global average pooling operation, Conv(.) is the convolution operation, FC(.) is the fully connected layer, δ(.) is the Sigmoid activation function, and θ(.) is the ReLu activation function;

[0025] (2-2-2) For the second dynamic feature extraction module, the output f of the previous dynamic feature extraction module is 1 ', and the feature map f obtained by dilation convolution with dilation rate 5 3 As input, by extracting global features and local features After fusion, we get the feature f 2 ', the calculation formula is as follows:

[0026]

[0027] (2-2-3) For the third dynamic feature extraction module, the output f of the previous dynamic feature extraction module is 2 ', and the feature map f obtained by dilation convolution with dilation rate 7 4 As input, by extracting global features and local features After fusion, the output feature f of the improved pyramid dynamic feature extraction module is obtained p , the calculation formula is as follows:

[0028]

[0029]

[0030] Furthermore, in step (3), the PS-Net mid-scale perception fusion module is improved by designing a flame feature attention mechanism to enhance the model's ability to suppress interference from similar shape features. Specifically, the following steps are adopted:

[0031] (3-1) Upsample the high-level input feature x to the same size as the low-level feature input feature y, and fuse them through splicing operations to retain more target features to obtain the output feature f u , the calculation formula is as follows:

[0032] f u =Concate(upsample(x),y)

[0033] In the formula, upsample(.) is the upsampling operation, and Concate(.) is the concatenation operation;

[0034] (3-2) Use convolution, normalization and activation function to perform feature f u After integration, the Softmax normalization function is used to constrain the weight of each channel of the feature map. The weight of the flame feature area is increased through model self-learning, thereby highlighting the target feature and obtaining the output feature f t , the calculation formula is as follows:

[0035] f t =Softmax(θ(BN(Conv(f u ))))

[0036] In the formula, BN(.) is the normalization layer;

[0037] (3-3) The feature f t Perform splitting operation along the channel direction to obtain the feature f for high-level fusion A and features f for low-level fusion B After that, it is fused with the input features x and y to obtain a more accurate flame feature fs , the calculation formula is as follows:

[0038] f A 、f B = split(f t )

[0039] f s =f A ×x+f B ×y

[0040] In the formula, split(.) is the channel splitting operation.

[0041] Furthermore, in step (4), the decoder structure is designed to reduce the influence of similar color feature interference on the detection result. s The similar color interference in s After convolution and normalization, the feature f is reused s Add and fuse, and use the ReLu activation function for nonlinear mapping to obtain the feature f d , in order to better highlight the flame characteristics and suppress the interference of similar colors. The calculation formula is as follows:

[0042] f d =θ(BN(Conv(f s ))+f s )

[0043] A flame detection system in a multi-source scenario, comprising:

[0044] A data acquisition unit, used to construct flame data in multi-source scenarios;

[0045] PS-Net+ model building unit, improves the pyramid feature dynamic extraction module in PS-Net by designing jump connections, improves the scale perception fusion module in PS-Net by designing the flame feature attention mechanism, and designs a decoder structure between the high-level features and low-level feature fusion of PS-Net;

[0046] Model training unit, which uses the flame dataset to train and test the PS-Net+ model;

[0047] The detection output unit uses the PS-Net+ model after training and testing for flame detection.

[0048] A computer storage medium stores an executable program, and the executable program is executed by a processor to implement the steps of the flame detection method.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] (1) The present invention designs a skip connection for each dilated convolution in the pyramid feature dynamic extraction module of PS-Net, which can effectively alleviate the influence of the dilated convolution on the small flame detection, make the flame features in the detection results more complete, reduce the probability of missed detection, and improve the detection accuracy of the small flame target in the scene;

[0051] (2) Based on the scale-aware fusion module of PS-Net, the present invention designs a flame feature attention mechanism, which can constrain the weight of each channel to make the model pay more attention to the real flame feature information, thereby effectively distinguishing virtual flames with similar shape features and reducing the model's false detection of flames;

[0052] (3) The present invention fuses high-level features with low-level features of PS-Net by designing a decoder structure, which helps the model to further suppress the interference of similar color features on flame detection during the feature fusion process, making the flame target in the detection result more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The figure is a general flow chart of a flame detection method in a multi-source scenario of the present invention.

[0054] Figure 2 This is a schematic diagram of the structure of the improved pyramid dynamic feature extraction module in the present invention. Figure 2 (a) is the overall structural principle diagram of the improved pyramid dynamic feature extraction module, and (b) is the structural principle diagram of the dynamic feature extraction module in the module.

[0055] Figure 3 This is a test comparison diagram before and after the improved pyramid dynamic feature extraction module is added to the present invention. Figure 3 (a) is the original image, (b) is the test result before adding, and (c) is the test result after adding.

[0056] Figure 4 This is the overall test result diagram of the model after the improved pyramid dynamic feature extraction module is added to the present invention. Figure 4 (a) is the original image, (b) is the label image, and (c) is the overall test result of the model.

[0057] Figure 5 This is a schematic diagram of the improved scale-aware fusion module structure in the present invention.

[0058] Figure 6 This is a test comparison chart before and after adding the improved scale-aware fusion module in the present invention. Figure 6 (a) is the original image, (b) is the test result before adding, and (c) is the test result after adding.

[0059] Figure 7 This is the overall test result of the model after adding the improved scale-aware fusion module in the present invention. Figure 7 (a) is the original image, (b) is the label image, and (c) is the overall test result image of the model.

[0060] Figure 8 This is a schematic diagram of the decoder structure in the present invention.

[0061] Fig. 9 This is a test comparison chart before and after the decoder structure is added to the present invention. Fig. 9 (a) is the original image, (b) is the test result before adding, and (c) is the test result after adding.

[0062] Fig.10 This is the overall test result diagram of the model after adding the decoder structure of the present invention. Fig.10 (a) is the original image, (b) is the label image, and (c) is the overall test result image of the model.

[0063] Fig.11 This is a diagram of the PS-Net+ model designed in the present invention.

[0064] Fig.12 This is a performance comparison chart of the model of the present invention and other models. DETAILED DESCRIPTION

[0065] In order to make the purpose and technical solution of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] like Figure 1 As shown, a flame detection method in a multi-source scenario of the present invention first improves and optimizes the PS-Net model (progressive selection network) to address the challenges of missed detection and false detection in flame detection tasks, and then uses the improved PS-Net+ model to achieve high-precision detection of flames in a multi-source scenario. The specific steps are as follows:

[0067] (1) Construct a flame dataset in a multi-source scenario;

[0068] (2) The pyramid feature dynamic extraction module in PS-Net is improved by designing skip connections to enhance the model’s ability to detect small flame targets;

[0069] (3) The PS-Net mid-scale perception fusion module is improved by designing a flame feature attention mechanism to enhance the model’s ability to suppress interference from similar shape features;

[0070] (4) By designing the decoder structure, the influence of similar color feature interference on the detection results is reduced;

[0071] (5) Use the trained and tested PS-Net+ model to perform flame detection.

[0072] Step 1: Construct a flame dataset in a multi-source scenario.

[0073] In the construction of the flame dataset in a multi-source scenario, image data is collected using an industrial camera, and the specific steps are as follows:

[0074] 1-1) Collect image data containing flame combustion under different scenes, different lighting conditions, different backgrounds, and different fire sources;

[0075] 1-2) Use the random rotation function RandomRotation, the horizontal flip function Randint, and the random cropping function RandomCrop in the deep learning framework to expand the flame dataset;

[0076] 1-3) Label the flame image data to obtain the true label, and then divide the image data into training set and test set according to 6:4.

[0077] Step 2 improves the pyramid feature dynamic extraction module in PS-Net by designing jump connections to enhance the model's ability to detect small flame targets.

[0078] When the pyramid dynamic feature extraction module in the basic PS-Net model uses the dilated convolution operation to extract flame features, the flickering characteristics of the flame often cause small flame targets to appear, which increases the difficulty of target recognition under the flame flickering characteristics due to the loss or blurring of detailed features, and ultimately causes missed detection. Figure 2 As shown, the present invention designs a skip connection for the dilated convolution process of the pyramid dynamic feature extraction module in the model, that is, the output feature after the dilated convolution in the previous step is added to the input feature as the input of the current dilated convolution, so that when the convolution kernel is used to perform a convolution operation on the feature map, the obtained feature map will contain more information while keeping the receptive field unchanged, and the specific steps are as follows:

[0079] (2-1) By dilating the output f of the previous convolution layer j With the input feature f c The features obtained by adding and fusion are used as the input of the next layer of dilated convolution. The calculation formula is as follows:

[0080]

[0081] Where D 1 (.) represents a dilated convolution operation with a dilation rate of 1, D 2 (.) represents a dilated convolution operation with a dilation rate of 3, D 3 (.) represents a dilated convolution operation with a dilation rate of 5, D 4 (.) indicates a dilated convolution operation with a dilation rate of 7;

[0082] (2-2) The dilated convolution output features obtained using skip connections are fused in a progressive manner using three feature dynamic extraction modules.

[0083] The dilated convolution output features obtained by skip connection are fused in a progressive manner using three feature dynamic extraction modules. The specific steps are as follows:

[0084] (2-2-1) For the first feature dynamic extraction module, the feature map f obtained by the dilated convolution with dilation rates of 1 and 3 is 1 、f 2 As input, by extracting global features and local features After fusion, we get the feature f 1 ', the calculation formula is as follows:

[0085]

[0086] Where AvP(.) is the average value in the channel dimension, GAP(.) is the global average pooling operation, Conv(.) is the convolution operation, FC(.) is the fully connected layer, δ(.) is the Sigmoid activation function, and θ(.) is the ReLu activation function.

[0087] (2-2-2) For the second dynamic feature extraction module, the output f of the previous dynamic feature extraction module is 1 ', and the feature map f obtained by dilation convolution with dilation rate 5 3 As input, by extracting global features and local features After fusion, we get the feature f 2 ', the calculation formula is as follows:

[0088]

[0089] (2-2-3) For the third dynamic feature extraction module, the output f of the previous dynamic feature extraction module is 2 ', and the feature map f obtained by dilation convolution with dilation rate 7 4As input, by extracting global features and local features After fusion, the output feature f of the improved pyramid dynamic feature extraction module is obtained p , the calculation formula is as follows:

[0090]

[0091] Compared with the original pyramid feature dynamic extraction module, the improved module reduces the feature loss caused by dilated convolution without introducing additional parameters, and further improves the accuracy of the model in flame target detection. Figure 3 As shown in the figure, the feature maps output by the model pyramid dynamic feature extraction module at different scales are compared before and after the improvement. It can be seen from the figure that compared with the feature map output by the original pyramid feature dynamic extraction module, the improved module better retains the flame feature information at the corresponding scale. Figure 4 As shown in the figure, it can be seen from the overall test result diagram of the improved model that after adding the jump connection, the problem of flame feature information loss has been effectively solved.

[0092] Step 3 improves the PS-Net mid-scale perception fusion module by designing a flame feature attention mechanism to enhance the model's ability to suppress interference from similar shape features.

[0093] In order to reduce the false detection probability of the model in scenes containing false flames, such as Figure 5 As shown, the present invention designs a flame feature attention mechanism in the scale perception fusion module of the PS-Net model, and changes the original addition operation of high-level features and low-level features to a splicing operation, so as to retain information of different scales of the flame, help the model to better handle multi-scale problems, and avoid the mixing and redundancy of information caused by the addition of feature maps. At the same time, in order to make the model pay more attention to or highlight the real flame features and reduce the interference of similar shape features, the normalization operation is completed by using the Softmax function after Conv, BN and ReLu to obtain the probability that each pixel represents the real flame feature, thereby generating a flame feature attention mechanism, the purpose of which is to better balance the weight of the feature map through the normalization operation, and through self-learning, make the model pay more attention to the area that is helpful for the flame detection task and reduce the sensitivity to irrelevant areas, so as to achieve the interference caused by similar shape features further suppressed on the basis of the original scale perception fusion module, which will help reduce the probability of false detection of the flame detection task and improve the detection accuracy. The following steps are specifically adopted:

[0094] (3-1) Upsample the high-level input feature x to the same size as the low-level feature input feature y, and fuse them through splicing operations to retain more target features to obtain the output feature f u , the calculation formula is as follows:

[0095] f u =Concate(upsample(x),y)

[0096] In the formula, upsample(.) is the upsampling operation, and Concate(.) is the concatenation operation.

[0097] (3-2) Use convolution, normalization and activation function to perform feature f u After integration, the Softmax normalization function is used to constrain the weight of each channel of the feature map. The weight of the flame feature area is increased through model self-learning, thereby highlighting the target feature and obtaining the output feature f t , the calculation formula is as follows:

[0098] f t =Softmax(θ(BN(Conv(f u ))))

[0099] In the formula, BN(.) is the normalization layer.

[0100] (3-3) The feature f t Perform splitting operation along the channel direction to obtain the feature f for high-level fusion A and features f for low-level fusion B After that, it is fused with the input features x and y to obtain a more accurate flame feature f s , the calculation formula is as follows:

[0101] f A 、f B = split(f t )

[0102] f s =f A ×x+f B ×y

[0103] In the formula, split(.) is the channel splitting operation.

[0104] like Figure 6 As shown in the figure, the output comparison of the third scale-aware fusion module of the model shows that there are not only real flames in the flame scene image, but also virtual flames generated by mirroring. The test results before the improvement of the scale-aware fusion module clearly include virtual flame targets. The problem of false detection is effectively solved by optimizing and improving the scale-aware fusion module. Figure 7 As shown in the figure, it can be seen from the overall test results of the model that the flame feature attention mechanism has a good inhibitory effect on the virtual flame and reduces the false detection of the model.

[0105] Step 4 reduces the impact of similar color feature interference on the detection results by designing the decoder structure.

[0106] In order to further reduce the impact of similar color interference on detection accuracy, inspired by the residual network, such as Figure 8 As shown in FIG. 1 , the present invention designs a jump connection composed of Convolution, BN, and ReLu as a decoder in the feature fusion process. The jump connection in the decoder can emphasize important feature areas and reduce the model's attention to color interference information, thereby further suppressing interference information. At the same time, by enhancing the pixel value of the feature area, the decoder can better maintain the extracted flame features during the upsampling process. For the scale-aware fusion module output feature f s The similar color interference in s After convolution and normalization, the feature f is reused s Add and fuse, and use the ReLu activation function for nonlinear mapping to obtain the feature f d , in order to better highlight the flame characteristics and suppress the interference of similar colors. The calculation formula is as follows:

[0107] f d =θ(BN(Conv(f s ))+f s )

[0108] like Fig. 9 As shown in the figure, the comparison of the model scale perception module before and after adding the decoder structure shows that when the decoder is not added, the model will recognize part of the red flag as a flame feature, but after adding the decoder structure, the model can better avoid the interference caused by similar color features. Fig.10 As shown in the figure, it can be seen from the overall test results of the model that after adding the decoder structure in the feature fusion process, it also has a good suppression effect on color interference in multi-source scenes.

[0109] Step 5 uses the PS-Net+ model after data set training and testing to perform flame detection.

[0110] like Fig.11 As shown in the figure, the PS-Net model is improved and optimized by designing skip connections, flame feature attention mechanism and decoder structure. After obtaining the PS-Net+ model for flame detection in multi-source scenarios, the loss function is determined to help the model to be better trained. The trained model is tested to verify the accuracy of the model. Finally, the trained model is used for flame detection in multi-source scenarios. The specific steps are as follows:

[0111] (1) Using BCELoss and CELoss loss functions, the model training results are optimized by calculating the loss of model test results and labels. The total loss function calculation formula is as follows:

[0112]

[0113] L = l BCEL +l CEL

[0114] In the formula, l BCEL is the BCELoss loss, l CEL is the CELoss loss, L is the total model loss; W represents the image width pixel; H represents the image height pixel; G is the label; S is the predicted saliency map; G(i, j) is the value of the label pixel (i, j); S(i, j) is the probability that the predicted image pixel (i, j) belongs to the salient area; TP represents that the prediction result of each pixel in the image is a positive category, and the label is also a positive category, that is, correct classification; FN represents that the prediction result of each pixel in the image is a negative category, but the label is a positive category, that is, it is not correctly classified; FP represents that the prediction result of each pixel in the image is a positive category, but the label is a negative category, that is, misclassification; FP+FN represents the difference between the union and intersection of the predicted map and the label; FP+2TP+FN represents the sum of the union and intersection of the predicted map and the label. The above formula balances the focus on misclassification and failure to correctly identify. When there are more FPs, the numerator will increase, resulting in an increase in the loss value, thereby prompting the model to reduce the number of FPs; when there are more FNs, the numerator will also increase, prompting the model to increase its attention to FN to reduce the number of failures to correctly identify.

[0115] (2) According to the obtained PS-Net+ model, the experimental environment and parameter configuration are performed, as shown in Table 1:

[0116] Table 1

[0117]

[0118]

[0119] (3) using the flame data set constructed by the present invention for training and testing;

[0120] (4) Mean absolute error (MAE), mean Em, max Em, adp Em, mean Fm, max Fm, adp Fm and precision-recall (PR) are used as evaluation indicators to verify the accuracy of the model and improve its detection accuracy.

[0121] In order to verify the effectiveness and superiority of the flame detection method in the multi-source scenario in the present invention, the flame dataset constructed in the present invention is used for testing, and the model test results are evaluated by using MAE, mean Em, max Em, adp Em, mean Fm, max Fm, adp Fm and PR indicators. The evaluation results are compared with eleven models including PiCANet, R3Net, PoolNet, BASNet, U2Net, GCPANet, SUCANet, SINet, ICON, SCRN and PS-Net. The comparative test results are shown in Table 2.

[0122] Table 2

[0123]

[0124] It can be concluded from Table 2 that PiCANet mainly introduces a pixel-level contextual attention mechanism. When detecting flames, the model will consider the pixels around each pixel. If there is a high similarity, the area will be identified as the target area, but due to the lack of constraints on interference features, the detected flame saliency map is larger than the saliency map area obtained by other models. At the same time, false flames with similar features in the background are also identified as flame targets, affecting the detection accuracy; R3Net as a whole mainly uses the residual refinement module to achieve target detection. It is more accurate for flame target detection without background interference, but when there is interference from similar features, the residual refinement module of the model can emphasize important features, but it cannot suppress interference information well. At the same time, the extracted features are also subjected to simple upsampling operations, resulting in feature dilution and ultimately leading to information loss, fuzziness and false detection problems in the detection results. In the process of feature fusion, PoolNet only takes simple addition operations for the features extracted from the backbone, the high-level features of the model, and the high-level features extracted from the improved part. Since the features extracted from the backbone are interfered by useless information, simple addition operations will accumulate useless information without interference suppression. At the same time, in the fusion, the focus is on high-level features while ignoring the attention to the details in the low-level features, which will lead to a large loss of edge information of the flame features, and ultimately affect the detection accuracy of the flame. BASNet does not perform further important feature extraction after preliminary feature extraction through the ResNet network, resulting in interference information in the features. At the same time, the subsequent feature fusion method adopts a simple addition operation, which fails to highlight important features and suppress background interference. In the test results, the detection accuracy is ultimately affected by the loss of feature information and the presence of interference information. When obtaining the saliency map, U2Net adds and convolves the saliency maps predicted by the last encoder and five decoders to obtain the final saliency map. This simple addition operation accumulates and amplifies the interference information. In addition, the simple upsampling operation used in the fusion process will not only dilute the flame area with similar color background, but also dilute the flame edge, resulting in feature information loss in the saliency map, affecting the detection accuracy. GCPANet uses a feature interleaving aggregation module to effectively fuse high-level features and low-level features, reducing the impact of detail information loss caused by feature dilution. However, in the feature fusion process, the design of the model does not highlight important features well and consider the interference of background information, resulting in the model's weak anti-interference ability in an environment with similar color and similar shape interference, and low flame detection accuracy. In SUCANet, each layer of features extracted by ResNet-50 is further extracted using a parallel dilated convolution module containing four dilated convolutions. In this process, the impact of dilated convolution on feature information loss is not considered, resulting in serious loss of flame detail features in the saliency map, which ultimately leads to low detection accuracy.SINet first generates a rough feature map, and then iteratively mines edge information by using the method of inverting attention to erase the target, which will cause serious loss of edge information of the flame. This is mainly because the last three layers of the backbone are selected for feature detection in feature extraction, and the last three layers of features originally lack a lot of detailed information about the flame. After the asymmetric convolution and dilated convolution in its texture enhancement module, the feature information will be further lost, which will lead to the failure to capture enough detailed information when mining edge information, and ultimately affect the detection accuracy of the flame. The diverse feature aggregation module in ICON uses different types of convolution operations to extract diverse features, but also introduces redundant information. At the same time, in the flame detection task, the model usually mistakenly identifies similar shape or color interference features as flame targets, and the channel enhancement module used by the ICON model enhances the channels containing important features, and cannot effectively suppress the interference in the channel, so it will enhance the feature channels containing interference information, which will lead to a decrease in detection accuracy. SCRN extracts four level features of two tasks from four residual blocks based on the backbone network ResNet-50, and performs feature fusion operations through a series of cross-adjustment units to achieve target detection. However, in the cross-adjustment unit, due to the problem of different spatial sizes of the extracted features, the model needs to continuously perform upsampling and downsampling operations to achieve feature fusion. After multiple simple ups and downsampling, the feature information is diluted and lost, which ultimately causes the loss of flame detection accuracy. Compared with the other eleven models, the model of the present invention has achieved the best values ​​in MAE, mean Em, max Em, adp Em, mean Fm, max Fm and adp Fm, which is due to the design and application of jump connections, flame feature attention mechanism and decoder structure. The PS-Net+ model obtained through improvement can better identify and extract flame target features, thereby improving flame detection accuracy.

[0125] like Fig.12 As shown in the figure, PR curves of twelve models are given, and each PR curve contains the Precision and Recall corresponding to 256 thresholds. As can be seen from the figure, the PR curve of the model of the present invention is closer to the upper right corner, and can better wrap the PR curves of other algorithms, indicating that the model can avoid false detection and misdetection to the greatest extent, and detect as many accurate flame targets as possible. It further illustrates that the model designed in this paper has better feature extraction and feature recognition capabilities than other models in flame detection tasks containing small flame targets and similar interference features, ensuring the accuracy of flame detection.

[0126] In order to verify the effectiveness of the structural design in this invention, a series of ablation experiments were conducted on the constructed flame dataset. The experiment used PS-Net as the basic model, and added skip connections, flame feature attention mechanism and decoder in turn for comparison. The experimental results are shown in Table 3.

[0127] Table 3

[0128]

[0129]

[0130] In Table 3, PS-Net 1 To add the jump connection to the PS-Net model, PS-Net 2 The PS-Net model with skip connections and flame feature attention mechanism is introduced. It can be seen from Table 3 that the PS-Net+ model (model of the present invention) has been improved in MAE, mean Em and mean Fm indicators, indicating that the improvement measures have effectively improved the model detection ability in small flame targets and similar feature interference scenarios. Since the original PS-Net model is easily affected by flame flickering features, as well as interference from similar color features and similar shape features, the performance of various indicators of the model flame detection is poor. Therefore, in order to effectively alleviate the phenomenon of flame feature loss during network detection caused by flame flickering features, this paper introduces skip connections in the PFDE module of the original model, which reduces the flame feature loss caused by a large expansion rate while ensuring a larger receptive field. Therefore, compared with the original PS-Net model, the PS-Net with skip connections 1 The MAE of the model was reduced by 6.25%, mean Em was increased by 0.48%, and meanF-m was increased by 1.43%. In view of the problem that virtual flame environments with similar shape features may cause model misdetection, a flame feature attention mechanism was added in this paper, and more attention was paid to flame features through network self-learning, thereby improving the model's ability to discriminate virtual flames. The MAE of the improved PS-Net2 model was reduced by 10.00%, mean Em was increased by 0.61%, and mean Fm was increased by 2.33%. In view of the fact that the upsampling operation in feature fusion will amplify color interference information, thereby affecting the final flame detection accuracy, a skip connection based on a residual network was introduced as a decoder in this paper, so that the network model can enhance the flame pixel value of the feature area during the upsampling process, reduce the interference of useless information such as color, and further improve the accuracy of the network model. The MAE of the improved model of the present invention was reduced by 15.00%, mean Em was increased by 0.81%, and mean Fm was increased by 3.18%.

[0131] In order to verify that the VGG16 public network structure adopted in the improved PS-Net+ backbone is still suitable for flame detection tasks, the backbone of the model is replaced with VGG11, VGG13, and VGG19 network structures and trained and tested accordingly. The test results are shown in Table 4.

[0132] Table 4

[0133]

[0134] It can be concluded from Table 4 that the backbone network used in the model of the present invention and the other three VGG backbone networks, after training and testing on the established data set, all evaluation indicators have achieved better values. Compared with VGG11 and VGG13, VGG16 has relatively more layers and parameters, and therefore has a more powerful feature expression capability. In the SOD task, a deeper and more complex network can better capture the saliency information in the image to a certain extent, and this paper mainly completes the single flame target detection in the multi-source scene. The VGG19 network is deeper and more complex than VGG16. Applying it to the PS-Net+ model may cause the performance of the network to decline due to the complexity of the network. Therefore, through comprehensive comparison and performance verification, the VGG16 network is finally selected as the backbone of the PS-Net+ model to complete the flame detection task in the multi-source scene.

[0135] The flame detection method in a multi-source scenario of the present invention helps to achieve high-precision detection of flame targets in a multi-source scenario, and provides technical support for quickly and accurately finding flame targets in reality and promptly resolving safety hazards caused by flames.

[0136] This embodiment also provides a flame detection system in a multi-source scenario, including:

[0137] A data acquisition unit, used to construct flame data in multi-source scenarios;

[0138] PS-Net+ model building unit, improves the pyramid feature dynamic extraction module in PS-Net by designing jump connections, improves the scale perception fusion module in PS-Net by designing the flame feature attention mechanism, and designs a decoder structure between the high-level features and low-level feature fusion of PS-Net;

[0139] Model training unit, which uses the flame dataset to train and test the PS-Net+ model;

[0140] The detection output unit uses the PS-Net+ model after training and testing for flame detection.

[0141] The computer-readable storage medium of the exemplary embodiment of the present invention is introduced below. The embodiment of the present invention provides a computer-readable storage medium, which stores computer-executable instructions, and the executable program is executed by a processor to implement the steps of the flame detection method of various exemplary embodiments of the present invention.

[0142] In addition, the flame detection method of various exemplary embodiments of the present invention can also be implemented as a program product, which includes program code. When the program product runs on a computer storage medium, the program code is used to execute the steps in the flame detection method of various exemplary embodiments of the present invention.

[0143] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0144] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A flame detection method in a multi-source scenario, characterized in that: The steps include: (1) Construct a flame dataset in a multi-source scenario; (2) The pyramid feature dynamic extraction module in PS-Net is improved by designing a jump connection. The designed jump connection is: the output of the previous layer of dilated convolution in the pyramid feature dynamic extraction module is fused with the module input feature by feature addition, and then the fused feature is used as the input of the next layer of dilated convolution; (3) The scale-aware fusion module in PS-Net is improved by designing a flame feature attention mechanism. The flame feature attention mechanism is designed as follows: the fusion mode of different levels of features in the scale-aware fusion module is changed from addition to splicing. After a series of convolutions, the Softmax normalization function is used to constrain the weight of each channel of the feature map. (4) Design a decoder structure between the high-level features and low-level features of PS-Net; (5) Using the flame data set, the PS-Net+ model obtained by improving steps (2), (3), and (4) is trained and tested, and the PS-Net+ model after training and testing is used for flame detection.

2. The flame detection method in a multi-source scenario according to claim 1, characterized in that: The step (1) of constructing a flame dataset in a multi-source scenario specifically includes the following steps: (1-1) Collect flame image data including flame burning under different scenes, different lighting conditions, different backgrounds, and different fire sources; (1-2) For the collected flame image data, random rotation, horizontal flipping, and random cropping are used for data processing to expand the flame dataset, and the dataset is labeled and divided.

3. The flame detection method in a multi-source scenario according to claim 1, characterized in that: In step (2), the pyramid feature dynamic extraction module in PS-Net is improved by designing a skip connection, which specifically includes the following steps: (2-1) By dilating the output f of the previous convolution layer j With the input feature f c The features obtained by adding and fusion are used as the input of the next layer of dilated convolution. The calculation formula is as follows: In the formula, D1(.) represents the dilated convolution operation with a dilation rate of 1, D2(.) represents the dilated convolution operation with a dilation rate of 3, D3(.) represents the dilated convolution operation with a dilation rate of 5, and D4(.) represents the dilated convolution operation with a dilation rate of 7; (2-2) The dilated convolution output features obtained using skip connections are fused in a progressive manner using three feature dynamic extraction modules.

4. The flame detection method in a multi-source scenario according to claim 3, characterized in that: The step (2-2) specifically includes the following steps: (2-2-1) For the first dynamic feature extraction module, the feature maps f1 and f2 obtained by the dilated convolution with dilation rates of 1 and 3 are used as input to extract the global features. and local features After fusion, the feature f1' is obtained, and the calculation formula is as follows: Where AvP(.) is the average value in the channel dimension, GAP(.) is the global average pooling operation, Conv(.) is the convolution operation, FC(.) is the fully connected layer, δ(.) is the Sigmoid activation function, and θ(.) is the ReLu activation function; (2-2-2) For the second dynamic feature extraction module, the output f1' of the previous dynamic feature extraction module and the feature map f3 obtained by the dilated convolution with a dilation rate of 5 are used as input to extract the global features. and local features After fusion, the features are obtained The calculation formula is as follows: (2-2-3) For the third feature dynamic extraction module, the output f2' of the previous feature dynamic extraction module and the feature map f4 obtained by the dilation convolution with a dilation rate of 7 are used as input to extract the global features. and local features After fusion, the output feature f of the improved pyramid dynamic feature extraction module is obtained p , the calculation formula is as follows:

5. The flame detection method in a multi-source scenario according to claim 1, characterized in that: The PS-Net mid-scale perception fusion module is improved by designing a flame feature attention mechanism as described in step (3), which specifically includes the following steps: (3-1) Upsample the high-level input feature x to the same size as the low-level feature input feature y, and fuse them through splicing operations to obtain the output feature f u , the calculation formula is as follows: f u =Concate(upsample(x),y) In the formula, upsample(.) is the upsampling operation, and Concate(.) is the concatenation operation; (3-2) Use convolution, normalization and activation function to perform feature f u After integration, the Softmax normalization function is used to constrain the weight of each channel of the feature map, and the weight of the flame feature area is improved through model self-learning to obtain the output feature f t , the calculation formula is as follows: f t =Softmax(θ(BN(Conv(f u )))) In the formula, BN(.) is the normalization layer; (3-3) The feature f t Perform splitting operation along the channel direction to obtain the feature f for high-level fusion A and features f for low-level fusion B After that, it is fused with the input features x and y to obtain the flame feature f s , the calculation formula is as follows: f A 、f B =split(f t ) f s =f A ×x+f B ×y In the formula, split(.) is the channel splitting operation.

6. The flame detection method in a multi-source scenario according to claim 1, characterized in that: The decoder structure described in step (4) is designed between the high-level features and low-level features of PS-Net, specifically including: for the scale-aware fusion module output feature f s The similar color interference in s After convolution and normalization, the feature f is reused s Add and fuse, and use the ReLu activation function for nonlinear mapping to obtain the feature f d , the calculation formula is as follows: f d =θ(BN(Conv(f s ))+f s ) Where BN(.) is the normalization layer, Conv(.) is the convolution operation, and θ(.) is the ReLu activation function.

7. The flame detection method in a multi-source scenario according to claim 1, characterized in that: The PS-Net+ model is trained using the BCELoss and CELoss loss functions for optimization, and the total loss function is: L＝l BCEL +l CEL In the formula, l BCEL is the BCELoss loss, l CEL is the CELoss loss, L is the total model loss; W represents the pixels in the image width direction; H represents the pixels in the image height direction; G is the label; S is the predicted saliency map; G(i, j) is the value of the label pixel (i, j); S(i, j) is the probability that the pixel (i, j) of the prediction image belongs to the salient area; TP means that the prediction result of each pixel in the image is a positive category, and the label is also a positive category, that is, it is correctly classified; FN means that the prediction result of each pixel in the image is a negative category, but the label is a positive category, that is, it is not correctly classified; FP means that the prediction result of each pixel in the image is a positive category, but the label is a negative category, that is, it is misclassified; FP+FN means the difference between the union and intersection of the prediction image and the label; FP+2TP+FN means the sum of the union and intersection of the prediction image and the label.

8. The flame detection method in a multi-source scenario according to claim 1, characterized in that: The PS-Net+ model uses the VGG16 network as the backbone.

9. A flame detection system in a multi-source scenario for implementing any flame detection method according to any one of claims 1 to 8, characterized in that: include: A data acquisition unit, used to construct flame data in multi-source scenarios; PS-Net+ model building unit, improves the pyramid feature dynamic extraction module in PS-Net by designing jump connections, improves the scale perception fusion module in PS-Net by designing the flame feature attention mechanism, and designs a decoder structure between the high-level features and low-level feature fusion of PS-Net; Model training unit, which uses the flame dataset to train and test the PS-Net+ model; The detection output unit uses the PS-Net+ model after training and testing for flame detection.

10. A computer storage medium, characterized in that: The computer storage medium stores an executable program, and the executable program is executed by a processor to implement the steps of the flame detection method according to any one of claims 1 to 8.

Images