A flame detection method and system under a multi-source scene

Abstract

The application discloses a flame detection method and system in a multi-source scene, first constructs a flame data set in a multi-source scene; secondly, a jump connection is designed to improve a pyramid feature dynamic extraction module in a PS-Net (progressive selection network), so that the detection capability of the model for small flame targets is improved; then, a flame feature attention mechanism is designed to improve a scale perception fusion module in the PS-Net, so that the inhibition capability of the model for similar shape feature interference is strengthened; subsequently, a decoder structure is designed to reduce the influence of similar color feature interference on the detection result; finally, a PS-Net+ model after training and testing is used for flame detection. The flame detection method in the multi-source scene of the application is helpful to realize high-precision detection of flame targets in a multi-source scene, and provides technical support for quickly and accurately finding flame targets in reality and timely solving safety hazards caused by the flame.

Description

TECHNICAL FIELD

[0001] The present application relates to a flame detection technology, in particular to a flame detection method and system in a multi-source scene. BACKGROUND

[0002] Fire not only brings unnecessary economic losses to society, but also endangers people's life and health. Since the fire is small at the initial stage and is relatively easy to control, how to discover the flame in time is the key to effectively prevent the occurrence of fire, which has always been the focus of the fire protection field.

[0003] Current flame detection mainly relies on temperature sensors, smoke sensors and light sensors. Compared with the first two sensors, the detection efficiency of light sensors is higher. However, the background of the flame in the actual environment is often complex, and the collected flame images are often affected by various disturbances, such as daylight, night light, and similar color background. Early image detection methods are mainly based on the color and motion features of the flame, but the features usually need to rely on manual extraction, and the designed detection method is usually for a certain specific environment background, and the generalization ability and robustness of the method are poor. In recent years, with the rapid development of computer vision, salient object detection (SOD) is widely used in image editing, target tracking, medical image processing and other fields because it can improve the detection accuracy and efficiency. Convolutional neural networks (CNN) have excellent feature learning and feature processing capabilities, especially in handling large-scale complex data sets and complex backgrounds, making it a powerful tool for realizing SOD tasks. Therefore, SOD based on CNN has become a trend in the application of machine vision field. Compared with traditional flame detection methods, SOD based on CNN has obvious advantages in autonomous learning and extracting flame features, especially for extracting flame targets in simple environments, which can meet the demand of flame target detection to a certain extent. However, as the flame background environment becomes more complex, especially when multiple sources of interference occur, there are still deficiencies in the loss of flame features and false detection in the existing intelligent flame detection process. How to improve the flame detection accuracy in complex multi-source environment is still a technical difficulty. SUMMARY

[0004] The present application provides a flame detection method and system in a multi-source scene for the detection of flame in real scenes. By improving the recognition and extraction ability of the model to the flame features, the model can effectively solve the two problems of missing detection and false detection of the flame in complex multi-source environment, and improve the flame detection accuracy.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0006] The method for flame detection in a multi-source scene first improves and optimizes the PS-Net model in view of challenges such as missed detection and false detection in the flame detection task, and then uses the improved PS-Net+ model to realize high-precision detection of flames in a multi-source scene. The specific steps are as follows:

[0007] (1) Construct a flame data set in a multi-source scene;

[0008] (2) Improve the pyramid feature dynamic extraction module in PS-Net by designing a skip connection to improve the model's detection ability for small flame targets;

[0009] The skip connection fuses the output of the last layer of the pyramid feature dynamic extraction module with the input features of the module using feature addition, and then uses the fused features as the input of the next layer of dilated convolution;

[0010] (3) Improve the scale perception fusion module in PS-Net by designing a flame feature attention mechanism to enhance the model's ability to suppress similar shape feature interference;

[0011] The flame feature attention mechanism changes the fusion method of features at different levels in the scale perception fusion module from addition to concatenation, retaining more target features. After a series of convolutions, the Softmax normalization function is used to constrain the weight of each channel of the feature map;

[0012] (4) Reduce the impact of similar color feature interference on the detection result by designing a decoder structure;

[0013] The decoder structure uses the input features after convolution and normalization to add and fuse with the input features again, and uses the ReLu activation function for non-linear mapping;

[0014] (5) Use the trained and tested PS-Net+ model to detect flames.

[0015] Further, the step (1) of constructing a flame data set in a multi-source scene uses the random rotation function RandomRotation, the horizontal flip function Randint, and the random cropping function RandomCrop in the deep learning framework to expand the flame data set, and labels and divides the data set.

[0016] Further, step (2) improves the pyramid feature dynamic extraction module in PS-Net by designing a skip connection to improve the model's detection ability for small flame targets, which specifically uses the following steps:

[0017] (2-1) The output f of the last layer of dilated convolution is taken as the input of the next layer of dilated convolution by adding the output of the last layer of dilated convolution j to the input feature f c The feature obtained by using the addition fusion method is taken as the input of the next layer of dilated convolution, and the calculation formula is as follows:

[0018]

[0019] In the formula, D1(.) represents dilated convolution operation with a dilated rate of 1, D2(.) represents dilated convolution operation with a dilated rate of 3, D3(.) represents dilated convolution operation with a dilated rate of 5, and D4(.) represents dilated convolution operation with a dilated rate of 7;

[0020] (2-2) The dilated convolution output features obtained by using the jump connection are fused in a progressive manner by using three feature dynamic extraction modules.

[0021] Further, the dilated convolution output features obtained by using the jump connection are fused in a progressive manner by using three feature dynamic extraction modules, and the following steps are specifically used:

[0022] (2-2-1) For the first feature dynamic extraction module, the feature maps f1 and f2 obtained by dilated convolution with a dilated rate of 1 and 3 are taken as the input, and global features and local features are extracted and fused to obtain the feature f1', and the calculation formula is as follows:

[0023]

[0024] In the formula, AvP(.) is the average value in the channel dimension, GAP(.) is the global average pooling operation, Conv(.) is the convolution operation, FC(.) is the full connection layer, δ(.) is the Sigmoid activation function, and θ(.) is the ReLu activation function;

[0025] (2-2-2) For the second feature dynamic extraction module, the output f1' of the last feature dynamic extraction module is taken as the input together with the feature map f3 obtained by dilated convolution with a dilated rate of 5, and global features and local features are extracted and fused to obtain the feature f2', and the calculation formula is as follows:

[0026]

[0027] (2-2-3) For the third feature dynamic extraction module, the output f2' of the last feature dynamic extraction module is taken as the input together with the feature map f4 obtained by dilated convolution with a dilated rate of 7, and global features and local features After fusion, the output feature f of the improved pyramid dynamic feature extraction module is obtained p , and the calculation formula is as follows:

[0028]

[0029]

[0030] Further, step (3) improves the mesoscale perception fusion module in PS-Net by designing a flame feature attention mechanism to enhance the model's ability to suppress similar shape feature interference. The specific steps are as follows:

[0031] (3-1) Upsample the high-level input feature x to the same size as the low-level input feature y, and fuse them by concatenation to retain more target features, obtaining the output feature f u , and the calculation formula is as follows:

[0032] f u =Concate(upsample(x),y)

[0033] In the formula, upsample(.) is the upsample operation, and Concate(.) is the concatenation operation.

[0034] (3-2) After integrating the feature f u using convolution, normalization and activation function, the weights of each channel of the feature map are constrained using the Softmax normalization function, the weights of the flame feature region are improved through model self-learning, and the target feature is highlighted, obtaining the output feature f t , and the calculation formula is as follows:

[0035] f t =Softmax(θ(BN(Conv(f u ))))

[0036] In the formula, BN(.) is the normalization layer.

[0037] (3-3) Split the feature f t along the channel direction to obtain the feature f A for high-level fusion and the feature f B for low-level fusion, and then fuse with the input features x and y to obtain more accurate flame feature f s , and the calculation formula is as follows:

[0038] f A 、f B =split(f t )

[0039] f s= f A x x + f B x y

[0040] In the formula, split(.) is a channel split operation.

[0041] Further, step (4) is designed to reduce the influence of similar color features on the detection result. For the scale-aware fusion module output feature f s Similar color interference existing in f s After convolution and normalization processing, the feature f s is reused to add fusion, and a ReLu activation function is used for nonlinear mapping to obtain the feature f d , so as to better highlight the flame feature and suppress similar color interference. The calculation formula is as follows:

[0042] f d = theta (BN (Conv (f s ))+f s )

[0043] A flame detection system in a multi-source scene includes:

[0044] A data acquisition unit is configured to construct flame data in a multi-source scene.

[0045] A PS-Net+ model construction unit improves the pyramid feature dynamic extraction module in PS-Net by designing a skip connection, improves the scale-aware fusion module in PS-Net by designing a flame feature attention mechanism, and designs a decoder structure between the fusion of high-level features and low-level features in PS-Net.

[0046] A model training unit trains and tests the PS-Net+ model using a flame data set.

[0047] A detection output unit uses the PS-Net+ model after training and testing to detect flames.

[0048] A computer storage medium stores an executable program, and the executable program is executed by a processor to implement the steps of the flame detection method.

[0049] Compared with the prior art, the present application has the following beneficial effects:

[0050] (1) The present application designs a skip connection in the pyramid feature dynamic extraction module of PS-Net for each dilated convolution, which can better alleviate the influence of dilated convolution on small flame detection, make the flame feature in the detection result more complete, reduce the missed detection probability, and improve the detection accuracy of small flame targets in the scene.

[0051] (2) The flame feature attention mechanism is designed on the basis of the scale perception fusion module of the PS-Net, the attention mechanism can constrain the weight of each channel to make the model pay more attention to the real flame feature information, thereby effectively distinguishing the virtual flame with similar shape features, and reducing the false detection of the model on the flame;

[0052] (3) The decoder structure is designed between the high-level feature and the low-level feature fusion of the PS-Net, which is helpful to further suppress the interference of similar color features on the flame detection in the feature fusion process, so that the flame target in the detection result is more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 It is a total flow chart of a flame detection method in a multi-source scene of the application.

[0054] Figure 2 It is a structure principle diagram of the improved pyramid dynamic feature extraction module in the application, Figure 2 (a) is a general structure principle diagram of the improved pyramid dynamic feature extraction module, and (b) is a structure principle diagram of the dynamic feature extraction module in the module.

[0055] Figure 3 It is a test comparison diagram before and after adding the improved pyramid dynamic feature extraction module in the application, Figure 3 (a) is an original diagram, (b) is a test result diagram before adding, and (c) is a test result diagram after adding.

[0056] Figure 4 It is a whole test result diagram of the model after adding the improved pyramid dynamic feature extraction module in the application, Figure 4 (a) is an original diagram, (b) is a label diagram, and (c) is a whole test result diagram of the model.

[0057] Figure 5 It is a structure principle diagram of the improved scale perception fusion module in the application.

[0058] Figure 6 It is a test comparison diagram before and after adding the improved scale perception fusion module in the application, Figure 6 (a) is an original diagram, (b) is a test result diagram before adding, and (c) is a test result diagram after adding.

[0059] Figure 7 It is a whole test result of the model after adding the improved scale perception fusion module in the application, Figure 7 (a) is an original diagram, (b) is a label diagram, and (c) is a whole test result diagram of the model.

[0060] Figure 8 The decoder structure schematic diagram in the application.

[0061] Figure 9 The test comparison diagram before and after adding the decoder structure in the application, Figure 9 In (a), the original diagram, (b) is the test result diagram before adding, (c) is the test result diagram after adding.

[0062] Figure 10 The overall test result diagram of the model after adding the decoder structure in the application, Figure 10 In (a), the original diagram, (b) is the label diagram, (c) is the overall test result diagram of the model.

[0063] Figure 11 The PS-Net+ model diagram designed in the application.

[0064] Figure 12 The performance comparison diagram of the model in the application and other models. DETAILED DESCRIPTION

[0065] In order to make the purpose and technical scheme of the embodiments of the application clearer, the technical scheme of the embodiments of the application will be described clearly and completely below in combination with the drawings of the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments. Based on the described embodiments of the application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the application.

[0066] As shown in Figure 1 The flame detection method in a multi-source scene of the application first improves and optimizes the PS-Net model (progressive selection network) in view of the challenges such as missed detection and false detection in the flame detection task, and then realizes high-precision detection of flames in a multi-source scene by using the improved PS-Net+ model. The specific steps are as follows:

[0067] (1) Construct a flame data set in a multi-source scene;

[0068] (2) Improve the pyramid feature dynamic extraction module in PS-Net by designing a skip connection to improve the detection ability of the model for small flame targets;

[0069] (3) Improve the scale perception fusion module in PS-Net by designing a flame feature attention mechanism to strengthen the suppression ability of the model for similar shape feature interference;

[0070] (4) Reduce the influence of similar color feature interference on the detection result by designing a decoder structure;

[0071] (5) Using the PS-Net+ model after training and testing to perform flame detection.

[0072] Step 1: Constructing a flame data set in a multi-source scene.

[0073] In the construction of the flame data set in a multi-source scene, the image data is collected using an industrial camera, and the following steps are taken:

[0074] 1-1) Collect image data containing flame burning under different scenes, different lighting conditions, different backgrounds, and different fire sources;

[0075] 1-2) Use the random rotation function RandomRotation, the horizontal flip function Randint, and the random cropping function RandomCrop in the deep learning framework to expand the flame data set;

[0076] 1-3) Label the flame image data to obtain the true label, and then divide the image data into a training set and a test set according to a ratio of 6:4.

[0077] Step 2: Improve the pyramid feature dynamic extraction module in PS-Net by designing a skip connection to improve the detection ability of the model for small flame targets.

[0078] When the pyramid dynamic feature extraction module in the basic PS-Net model uses dilated convolution operation to extract flame features, the flickering features of the flame often cause small flame targets to appear, which easily increases the difficulty of target recognition under flame flickering features due to loss or blurring of detailed features, ultimately causing missed detection. Therefore, as shown in Figure 2 , the invention designs a skip connection for the dilated convolution process of the pyramid dynamic feature extraction module in the model, that is, the output features after the dilated convolution in the previous step are added to the input features to serve as the input of the current dilated convolution. In this way, when using a convolution kernel to perform convolution operation on the feature map, the obtained feature map will contain more information while keeping the receptive field unchanged. The following steps are taken:

[0079] (2-1) Obtain the feature of the next layer dilated convolution by adding the output f j of the previous layer dilated convolution to the input feature f c , and the calculation formula is as follows:

[0080]

[0081] In the formula, D1(.) represents dilated convolution operation with a dilated rate of 1, D2(.) represents dilated convolution operation with a dilated rate of 3, D3(.) represents dilated convolution operation with a dilated rate of 5, and D4(.) represents dilated convolution operation with a dilated rate of 7.

[0082] (2-2) For the dilated convolution output features obtained by using the skip connection, three feature dynamic extraction modules are used to fuse in a progressive manner.

[0083] For the dilated convolution output features obtained by using the skip connection, three feature dynamic extraction modules are used to fuse in a progressive manner, specifically by using the following steps:

[0084] (2-2-1) For the first feature dynamic extraction module, the feature maps f1 and f2 obtained by dilated convolution with dilated rates of 1 and 3 are taken as inputs, global features and local features are extracted and fused to obtain the feature f1', and the calculation formula is as follows:

[0085]

[0086] In the formula, AvP(.) is the average value in the channel dimension, GAP(.) is the global average pooling operation, Conv(.) is the convolution operation, FC(.) is the full connection layer, δ(.) is the Sigmoid activation function, and θ(.) is the ReLu activation function.

[0087] (2-2-2) For the second feature dynamic extraction module, the output f1' of the previous feature dynamic extraction module and the feature map f3 obtained by dilated convolution with a dilated rate of 5 are taken as inputs, global features and local features are extracted and fused to obtain the feature f2', and the calculation formula is as follows:

[0088]

[0089] (2-2-3) For the third feature dynamic extraction module, the output f2' of the previous feature dynamic extraction module and the feature map f4 obtained by dilated convolution with a dilated rate of 7 are taken as inputs, global features and local features are extracted and fused to obtain the output feature f p of the improved pyramid dynamic feature extraction module, and the calculation formula is as follows:

[0090]

[0091] Compared with the original pyramid feature dynamic extraction module, the improved module reduces the feature loss caused by dilated convolution without introducing additional parameters, further improving the precision of the model in flame target detection. For example, Figure 3As shown in the figure, the flame feature information at the corresponding scale is better preserved compared with the feature map output by the original pyramid feature dynamic extraction module. As shown in the figure, Figure 4 As can be seen from the overall test result map of the improved model, after adding the skip connection, the problem of loss of flame feature information is effectively solved.

[0092] Step 3 improves the scale perception fusion module in PS-Net by designing a flame feature attention mechanism to enhance the model's ability to suppress similar shape feature interference.

[0093] In order to reduce the false detection probability of the model in the scene containing false flames, as shown in the figure, Figure 5 As shown in the figure, the flame feature attention mechanism is designed in the scale perception fusion module of the PS-Net model, and the addition operation of the original high-level feature and the low-level feature is changed to a splicing operation, so as to retain the information of different scales of the flame, help the model better process the multi-scale problem, and avoid the mixing and redundancy of the feature map caused by addition. At the same time, in order to make the model pay more attention to or highlight the real flame feature and reduce the interference of similar shape features, the Softmax function is used after Conv, BN and ReLu to complete the normalization operation, so as to obtain the probability of each pixel representing the real flame feature, thereby generating the flame feature attention mechanism. The purpose is to better balance the weight of the feature map through the normalization operation, and through self-learning, the model pays more attention to the area that is helpful to the flame detection task, and reduces the sensitivity to irrelevant areas, so as to further suppress the interference brought by similar shape features on the basis of the original scale perception fusion module. This will help to reduce the false detection probability of the flame detection task and improve the detection accuracy. The specific steps are as follows:

[0094] (3-1) upsample the high-level input feature x to the same size as the low-level feature input feature y, and fuse by splicing operation to retain more target features, to obtain the output feature f u , the calculation formula is as follows:

[0095] f u = Concate(upsample(x), y)

[0096] In the formula, upsample(.) is the upsample operation, and Concate(.) is the splicing operation.

[0097] (3-2) use convolution, normalization and activation function to process the feature f uAfter integration, the Softmax normalization function is used to constrain the weights of each channel of the feature map. Through model self-learning, the weights of the flame feature region are improved, thereby highlighting the target features and obtaining the output feature f. t The calculation formula is as follows:

[0098] f t =Softmax(θ(BN(Conv(f)) u ))))

[0099] In the formula, BN(.) is the normalized layer.

[0100] (3-3) Feature f t A splitting operation is performed along the channel direction to obtain the feature f used for high-level fusion. A and features f used for low-level fusion B Then, it is fused with the input features x and y to obtain a more accurate flame feature f. s The calculation formula is as follows:

[0101] f A f B =split(f t )

[0102] f s =f A ×x+f B ×y

[0103] In the formula, split(.) is the channel splitting operation.

[0104] like Figure 6 As shown in the comparison of the outputs of the third scale-aware fusion module of the model, it can be seen that the flame scene image contains not only real flames but also virtual flames generated by mirroring. The test results before the improvement of the scale-aware fusion module clearly included virtual flame targets. The optimization and improvement of the scale-aware fusion module effectively solved the problem of false detection. Figure 7 As shown in the overall test results, the flame feature attention mechanism has a good suppression effect on virtual flames and reduces the false detection of the model.

[0105] Step 4 involves designing a decoder structure to reduce the impact of interference from similar color features on the detection results.

[0106] To further reduce the impact of similar color interference on detection accuracy, inspired by residual networks, such as Figure 8As shown, the present application designs a skip connection composed of Convolution, BN and ReLu as a decoder in the feature fusion process. The skip connection in the decoder can emphasize important feature areas and reduce the model's attention to color interference information, thereby achieving further suppression of interference information. At the same time, by enhancing the pixel value of the feature area, the decoder can make the extracted flame feature better maintain during the upsampling process. For the scale perception fusion module output feature f s Similar color interference existing in f s After convolution and normalization processing, the feature f s is reused, added and fused, and a ReLu activation function is used for nonlinear mapping to obtain the feature f d , which better highlights the flame feature and suppresses similar color interference, and the calculation formula is as follows:

[0107] f d =θ(BN(Conv(f s ))+f s )

[0108] As Figure 9 shown in the comparison chart before and after adding the decoder structure to the model scale perception module, when the decoder structure is not added, the model will identify part of the red flag as a flame feature, and after the decoder structure is added, the model can better avoid the interference caused by similar color features. As Figure 10 shown in the overall test results of the model, after adding the decoder structure in the feature fusion process, the color interference in the multi-source scene is also well suppressed.

[0109] Step 5 uses the PS-Net+ model after training and testing the dataset to perform flame detection.

[0110] As Figure 11 shown, by designing a skip connection, a flame feature attention mechanism and a decoder structure to improve and optimize the PS-Net model, obtaining the PS-Net+ model for flame detection in a multi-source scene, determining the loss function to help the model train better, testing the trained model to verify the accuracy of the model, and finally using the trained model to detect flames in a multi-source scene, the following steps are used:

[0111] (1) Use BCELoss and CE Loss loss functions to calculate the loss of the model test results and labels, optimize the model training results, and the total loss function calculation formula is as follows:

[0112]

[0113] L=l BCEL +lCEL

[0114] wherein, l BCEL is BCELoss loss, l CEL is CE loss, L is the total loss of the model; W represents the pixel in the width direction of the image; H represents the pixel in the height direction of the image; G is a label; S is a predicted saliency map; G(i,j) is the value of the label pixel (i,j); S(i,j) is the probability that the predicted pixel (i,j) belongs to a saliency region; TP represents that the prediction result of each pixel of the image is a positive class, and the label is also a positive class, that is, correct classification; FN represents that the prediction result of each pixel of the image is a negative class, but the label is a positive class, that is, incorrect classification; FP represents that the prediction result of each pixel of the image is a positive class, but the label is a negative class, that is, incorrect classification; FP+FN represents the difference between the union and intersection of the predicted map and the label; FP+2TP+FN represents the sum of the union and intersection of the predicted map and the label. The above formula balances the attention to incorrect classification and incorrect identification. When FP is large, the numerator will increase, resulting in an increase in the loss value, thereby prompting the model to reduce the number of FP; when FN is large, the numerator will also increase, prompting the model to increase attention to FN to reduce the number of incorrect identifications.

[0115] (2) According to the obtained PS-Net+ model, the experimental environment and parameter configuration are carried out, as shown in Table 1:

[0116] Table 1

[0117]

[0118]

[0119] (3) The flame data set constructed by the present application is used for training and testing;

[0120] (4) The mean absolute error (MAE), mean E-m, max E-m, adp E-m, mean F-m, max F-m, adp F-m and accuracy-recall rate (P-R) are used as evaluation indexes to verify the accuracy of the model and improve the detection precision.

[0121] In order to verify the effectiveness and superiority of the flame detection method in the multi-source scene in the application, the flame dataset constructed in the application is tested, the MAE, mean E-m, max E-m, adp E-m, mean F-m, max F-m, adp F-m and P-R indexes are used to evaluate the model test results, and the evaluation results are compared with eleven kinds of models of PiCANet, R3Net, PoolNet, BASNet, U2Net, GCPANet, SUCANet, SINet, ICON, SCRN and PS-Net, and the comparison test results are shown in Table 2.

[0122] Table 2

[0123]

[0124] From Table 2, it can be concluded that PiCANet mainly introduces a pixel-level context attention mechanism. In flame detection, the model considers the pixels around each pixel. If there is a high degree of similarity, the region will be identified as the target region. However, due to the lack of constraints on interfering features, the detected flame saliency map has a larger area compared to other models. At the same time, false flames with similar features in the background are also identified as flame targets, affecting the detection accuracy. R3Net mainly uses residual refinement modules to achieve target detection. For flame target detection without background interference, it is relatively accurate. However, when there are similar feature interferences, the residual refinement module of the model can emphasize important features but cannot effectively suppress interference information. At the same time, the extracted features also use simple upsampling operations, causing feature dilution and ultimately leading to information loss, blurring, and false detection problems in the detection results. In the feature fusion process of PoolNet, the model simply adds the features extracted by the main backbone, high-level features of the model, and high-level features extracted by the improved part. Due to the presence of useless information interference in the features extracted by the main backbone, simple addition operations without interference suppression will accumulate useless information. In the fusion process, focusing on high-level features while ignoring the attention to detail information in low-level features will cause significant loss of edge information of flame features, ultimately affecting the detection accuracy of flames. BASNet does not further extract important features after preliminary feature extraction using the ResNet network, resulting in interference information in the features. The simple addition operation used in the subsequent feature fusion process cannot effectively highlight important features and suppress background interference. In the test results, the detection accuracy is affected by the presence of feature information loss and interference information. U2Net adds and convolves the saliency maps predicted by the last encoder and five decoders to obtain the final saliency map. This simple addition operation amplifies the interference information. In addition, the simple upsampling operation used in the fusion process not only dilutes the flame regions with similar color backgrounds but also dilutes the flame edges, causing feature information loss in the saliency map and affecting the detection accuracy. GCPANet uses a feature interweaving aggregation module to effectively fuse high-level and low-level features, reducing the impact of detail information loss caused by feature dilution. However, the model design does not effectively highlight important features and consider background information interference during feature fusion, resulting in a slightly inferior anti-interference ability in environments with similar color and shape interferences, and a lower detection accuracy for flames. In SUCANet, the ResNet-50 extracts each layer of features, and a parallel dilated convolution module containing four dilated convolutions is used to further extract features. However, the model does not consider the impact of dilated convolution on feature information loss, resulting in severe loss of flame detail features in the saliency map and ultimately causing low detection accuracy.SINet first generates a rough feature map, and then iteratively excavates edge information by using a method of erasing the target by reversing attention, which causes the edge information of the flame to be lost more seriously. This is mainly because the last three layers of the backbone are selected for feature detection in feature extraction, and the last three layers of features originally lack a large amount of detailed information of the flame. After asymmetric convolution and dilated convolution in the texture enhancement module, the feature information is further lost, which affects the detection accuracy of the flame when excavating edge information and ultimately affects the detection accuracy of the flame. The multi-feature aggregation module in ICON uses different types of convolution operations to extract diverse features, but also introduces redundant information. In the flame detection task, the model usually incorrectly identifies similar shape or color interference features as flame targets. The channel enhancement module used in the ICON model enhances the channels containing important features and cannot effectively suppress the interference in the channel, so the feature channels containing interference information are enhanced, which causes the detection accuracy to decrease. SCRN extracts four levels of features of two tasks from four residual blocks of the backbone network ResNet-50, and realizes target detection by performing feature fusion through a series of cross adjustment units. However, in the cross adjustment unit, the extracted features have different spatial sizes, so the model needs to constantly perform upsampling and downsampling operations to realize feature fusion. After a plurality of simple upsampling and downsampling operations, the feature information is diluted and lost, which ultimately causes the loss of flame detection accuracy. Compared with the other eleven models, the model of the present application achieves the optimal value in MAE, mean E-m, max E-m, adp E-m, mean F-m, max F-m and adp F-m. This is due to the design and application of the skip connection, the flame feature attention mechanism and the decoder structure. The PS-Net+ model obtained by improvement can better distinguish and extract flame target features, thereby improving the flame detection accuracy.

[0125] As shown in Figure 12 , the P-R curves of twelve models are given, each P-R curve contains the Precision and Recall corresponding to 256 threshold values. As can be seen from the figure, the P-R curve of the model of the present application is closer to the upper right corner, and can better wrap the P-R curves of other algorithms, indicating that the model can avoid false positives and false negatives to the greatest extent, and detect as many accurate flame targets as possible. Further, the model designed in this paper has better feature extraction and feature discrimination ability than other models in the flame detection task containing small flame targets and similar interference features, which ensures the accuracy of flame detection.

[0126] To verify the effectiveness of the structural design in this invention, a series of ablation experiments were conducted on the constructed flame dataset. The experiments used PS-Net as the base model, and skip connections, a flame feature attention mechanism, and a decoder were added sequentially for comparison. The experimental results are shown in Table 3.

[0127] Table 3

[0128]

[0129]

[0130] In Table 3, PS-Net 1 To add skip connections to the PS-Net model, PS-Net 2 As shown in Table 3, the PS-Net+ model (the model of this invention) improved in terms of MAE, mean Em, and mean Fm, indicating that the improvement measures effectively enhanced the model's detection capability in scenarios with small flame targets and interference from similar features. Because the original PS-Net model is susceptible to the influence of flame flickering features, as well as interference from similar color and shape features, its flame detection performance across various metrics is poor. Therefore, to effectively alleviate the phenomenon of flame feature loss during network detection caused by flame flickering features, skip connections were introduced into the PFDE module of the original model, reducing flame feature loss caused by a large expansion rate while maintaining a large receptive field. Therefore, compared with the original PS-Net model, the PS-Net+ model with skip connections... 1 The model's MAE was reduced by 6.25%, mean Em was improved by 0.48%, and mean F-m was improved by 1.43%. To address the issue of false detections caused by virtual flame environments with similar shape features, a flame feature attention mechanism was added, and the network self-learning was used to pay more attention to flame features, thereby improving the model's ability to distinguish virtual flames. The improved PS-Net2 model reduced MAE by 10.00%, improved mean Em by 0.61%, and improved mean Fm by 2.33%. To address the issue that upsampling in feature fusion amplifies color interference information, thus affecting the final flame detection accuracy, a residual network-based skip connection was introduced as a decoder. This allows the network model to enhance the flame pixel values ​​in feature regions during upsampling, reducing interference from useless information such as color, thereby further improving the accuracy of the network model. The improved model of this invention reduced MAE by 15.00%, improved mean Em by 0.81%, and improved mean Fm by 3.18%.

[0131] In order to verify that the improved VGG16 public network structure adopted by the PS-Net+ backbone part still has good applicability to the flame detection task, the backbone part of the model is replaced with VGG11, VGG13 and VGG19 network structures respectively and corresponding training and testing are performed, and the test results are shown in Table 4.

[0132] Table 4

[0133]

[0134] As can be seen from Table 4, after training and testing on the established data set, the backbone network adopted by the model of the application and the other three VGG backbone networks all achieve relatively optimal values of evaluation indexes. Compared with VGG11 and VGG13, VGG16 has relatively more layers and parameters, and therefore has stronger feature expression capability. In the SOD task, a deeper and more complex network can better capture the saliency information in an image to a certain extent, and in this paper, the single flame target detection in a multi-source scene is mainly completed, and the VGG19 network is deeper and more complex than the VGG16 network, so that when it is applied to the PS-Net+ model, the performance of the network may be reduced due to the complexity of the network. Therefore, through comprehensive comparison and performance verification, the VGG16 network is finally selected as the backbone part of the PS-Net+ model to complete the flame detection task in a multi-source scene.

[0135] The flame detection method in a multi-source scene of the application helps to realize high-precision detection of a flame target in a multi-source scene, and provides technical support for quickly and accurately finding a flame target in reality and timely solving safety hazards caused by the flame.

[0136] The embodiment also provides a flame detection system in a multi-source scene, which comprises:

[0137] A data acquisition unit is configured to construct flame data in a multi-source scene.

[0138] A PS-Net+ model construction unit is configured to improve a pyramid feature dynamic extraction module in the PS-Net by designing a skip connection, improve a scale perception fusion module in the PS-Net by designing a flame feature attention mechanism, and design a decoder structure between fusion of high-level features and low-level features of the PS-Net.

[0139] A model training unit is configured to train and test the PS-Net+ model by using a flame data set.

[0140] A detection output unit is configured to perform flame detection by using the PS-Net+ model after training and testing.

[0141] The computer readable storage medium of the exemplary embodiments of the present application is described below. The exemplary embodiments of the present application provide a computer readable storage medium storing computer executable instructions, and the executable program is executed by a processor to implement the steps of the flame detection method of the various exemplary embodiments of the present application.

[0142] Furthermore, the flame detection method of the various exemplary embodiments of the present application can also be implemented as a program product including program codes for executing the steps of the flame detection method of the various exemplary embodiments of the present application when the program product is run on a computer storage medium.

[0143] Although the preferred embodiments of the present application have been described, those skilled in the art will be able to make additional changes and modifications to these embodiments once they grasp the basic inventive concept. Therefore, the appended claims are intended to be construed as including all the preferred embodiments and all the changes and modifications falling within the scope of the present application.

[0144] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A flame detection method in a multi-source scenario, characterized in that, Includes the following steps: (1) Construct a flame dataset for multi-source scenarios; (2) Improve the pyramid feature dynamic extraction module in PS-Net by designing skip connections. The designed skip connections are as follows: the output of the previous layer dilated convolution in the pyramid feature dynamic extraction module is fused with the module input features by feature addition, and then the fused features are used as the input of the next layer dilated convolution. (3) Improve the scale-aware fusion module in PS-Net by designing a flame feature attention mechanism. The designed flame feature attention mechanism is as follows: change the fusion method of different levels of features in the scale-aware fusion module from addition to concatenation. After a series of convolutions, use the Softmax normalization function to constrain the weight of each channel of the feature map. (4) Design a decoder structure between the high-level and low-level features of PS-Net; (5) Use the flame dataset to train and test the PS-Net+ model obtained in steps (2), (3), and (4), and use the PS-Net+ model after training and testing to perform flame detection.

2. The flame detection method in a multi-source scenario according to claim 1, characterized in that, The steps in step (1) to construct the flame dataset in a multi-source scenario specifically include the following steps: (1-1) Collect flame image data containing flame combustion under different scenarios, different lighting conditions, different backgrounds, and different fire sources; (1-2) For the collected flame image data, random rotation, horizontal flipping and random cropping are used for data processing to expand the flame dataset, and the dataset is labeled and divided.

3. The flame detection method in a multi-source scenario according to claim 1, characterized in that, Step (2) improves the pyramid feature dynamic extraction module in PS-Net by designing skip connections, specifically including the following steps: (2-1) By dilating the output f of the previous layer of convolution... j With input features f c The features obtained by additive fusion are used as the input to the next layer of dilated convolution, and the calculation formula is as follows: In the formula, D1(.) represents the dilation convolution operation with a dilation rate of 1, D2(.) represents the dilation convolution operation with a dilation rate of 3, D3(.) represents the dilation convolution operation with a dilation rate of 5, and D4(.) represents the dilation convolution operation with a dilation rate of 7. (2-2) The dilated convolution output features obtained by skip connections are fused in a progressive manner using three feature dynamic extraction modules.

4. The flame detection method in a multi-source scenario according to claim 3, characterized in that, Step (2-2) specifically includes the following steps: (2-2-1) For the first feature dynamic extraction module, the feature maps f1 and f2 obtained by dilated convolutions with dilation rates of 1 and 3 are used as input, and global features are extracted. and local features After fusion, feature f1' is obtained, and the calculation formula is as follows: In the formula, AvP(.) is the average value calculated along the channel dimension, GAP(.) is the global average pooling operation, Conv(.) is the convolution operation, FC(.) is the fully connected layer, δ(.) is the Sigmoid activation function, and θ(.) is the ReLU activation function. (2-2-2) For the second feature dynamic extraction module, the feature map f3 obtained by the output f1' of the previous feature dynamic extraction module and the dilation convolution with a dilation rate of 5 is used as input to extract global features. and local features After fusion, features are obtained. The calculation formula is as follows: (2-2-3) For the third feature dynamic extraction module, the output f2' of the previous feature dynamic extraction module is used as input, and the feature map f4 obtained by dilation convolution with dilation rate 7 is used to extract global features. and local features After fusion, the output features f of the improved pyramid dynamic feature extraction module are obtained. p The calculation formula is as follows:

5. The flame detection method in a multi-source scenario according to claim 1, characterized in that, The improvement of the PS-Net mesoscale perception fusion module by designing a flame feature attention mechanism as described in step (3) specifically includes the following steps: (3-1) Upsample the high-level input feature x to the same size as the low-level input feature y, and fuse them through a concatenation operation to obtain the output feature f. u The calculation formula is as follows: f u =Concate(upsample(x),y) In the formula, upsample(.) is the upsampling operation, and Concate(.) is the concatenation operation; (3-2) Using convolution, normalization, and activation functions to process feature f u After integration, the Softmax normalization function is used to constrain the weights of each channel of the feature map. Through model self-learning, the weights of the flame feature region are improved, resulting in the output feature f. t The calculation formula is as follows: f t =Softmax(θ(BN(Conv(f u )))) In the formula, BN(.) is the normalized layer; (3-3) Feature f t A splitting operation is performed along the channel direction to obtain the feature f used for high-level fusion. A and features f used for low-level fusion B Then, it is fused with the input features x and y to obtain the flame feature f. s The calculation formula is as follows: f A 、f B =split(f t ) f s =f A ×x+f B ×y In the formula, split(.) is the channel splitting operation.

6. The flame detection method in a multi-source scenario according to claim 1, characterized in that, The design of the decoder structure between the high-level and low-level features of PS-Net described in step (4) specifically includes: for the scale-aware fusion module output feature f s The presence of similar color interference in the feature f s After convolution and normalization, the feature f is reused. s The features f are obtained by performing addition and fusion, followed by nonlinear mapping using the ReLU activation function. d The calculation formula is as follows: f d =θ(BN(Conv(f s ))+f s ) In the formula, BN(.) is the normalization layer, Conv(.) is the convolution operation, and θ(.) is the ReLU activation function.

7. The flame detection method in a multi-source scenario according to claim 1, characterized in that, The PS-Net+ model is optimized using BCELoss and CELoss loss functions during training. The total loss function is: L＝l BCEL +l CEL In the formula, l BCEL For BCELoss loss, l CEL CELoss is the loss, L is the total model loss; W represents the pixels in the image width direction; H represents the pixels in the image height direction; G is the label; S is the predicted saliency map; G(i,j) is the value of the label pixel (i,j); S(i,j) represents the probability that pixel (i,j) in the predicted image belongs to a salient region; TP indicates that the predicted class of each pixel in the image is positive and the label is also positive, i.e., the classification is correct; FN indicates that the predicted class of each pixel in the image is negative but the label is positive, i.e., the classification is incorrect; FP indicates that the predicted class of each pixel in the image is positive but the label is negative, i.e., the classification is incorrect; FP+FN represents the difference between the union and intersection of the predicted image and the label; FP+2TP+FN represents the sum of the union and intersection of the predicted image and the label.

8. The flame detection method in a multi-source scenario according to claim 1, characterized in that, The PS-Net+ model uses the VGG16 network as its backbone.

9. A flame detection system for multi-source scenarios implementing the flame detection method of any one of claims 1-8, characterized in that, include: The data acquisition unit is used to construct flame data in multi-source scenarios; The PS-Net+ model building unit improves the pyramid feature dynamic extraction module in PS-Net by designing skip connections, improves the scale-aware fusion module in PS-Net by designing a flame feature attention mechanism, and designs a decoder structure between the high-level and low-level feature fusion in PS-Net. The model training unit uses the Flame dataset to train and test the PS-Net+ model; The detection output unit uses the PS-Net+ model after training and testing to perform flame detection.

10. A computer storage medium, characterized in that, The computer storage medium stores an executable program, which is executed by a processor to implement the steps of the flame detection method according to any one of claims 1-8.

Images