A fire detection method and system based on color edge depth perception

Abstract

The present invention discloses a fire detection method and system based on color edge depth perception, comprising: obtaining an image to be identified, and inputting the image to be identified into a fire detection model so that the fire detection model performs feature extraction on the image to be identified, obtains a first color feature map and a first edge feature map of the image to be identified, and performs implicit feature extraction on the first feature map after merging the first color feature map and the first edge feature map, and then outputs a fire detection result based on the implicit feature extraction result. The fire detection model disclosed in the present invention is a two-stage feature extraction architecture, including an explicit feature extraction stage and an implicit feature extraction stage, which realizes feature extraction of image color, edges, spatial details and depth. At the same time, by optimizing the internal architecture of the fire detection model, the model is further simplified, the computational complexity of the model is reduced, and the processing speed and accuracy of the model are improved.

Description

Technical Field

[0001] The present invention relates to the field of fire detection, and in particular to a fire detection method and system based on color edge depth perception. Background Art

[0002] Fires caused by uncontrolled combustion, either temporally or spatially, are often accompanied by severe property damage and personal injury. Statistical analysis of data from the Fire and Rescue Bureau of China's Ministry of Emergency Management shows that rural areas are disproportionately affected by fires, with a higher probability of major fires and a high proportion of on-site deaths. Fires in residential areas are also the leading cause of death from fires, posing significant challenges in preventing these types of fires. Furthermore, industrial and commercial venues, where people gather, production and business facilities are concentrated, electricity, oil, and gas usage is high, and finished products and raw materials are piled up, easily spread and cause casualties. These fires often burn large areas and have significant social impacts. In addition to proactively preventing potential fires in all aspects, the most effective approach is to control fires in their early stages to prevent further spread and serious consequences.

[0003] Fire detection challenges vary in different scenarios, but current fire detection systems are still mostly based on traditional sensors such as smoke sensors, temperature sensors, and infrared sensors. Due to various sensor-based limitations and limitations, these sensor-based fire detection systems struggle to provide effective and immediate fire warnings in a variety of complex environments. However, with the rapid adoption of surveillance cameras, their high coverage and real-time capabilities have led to rapid development of real-time fire monitoring technology based on surveillance video. Fire detection in surveillance camera image data represents a specific type of image classification or target detection task. There are two main approaches to vision-based fire detection: traditional computer vision-based methods and deep learning-based methods. Traditional computer vision-based methods rely on manually extracted features such as color, shape, and texture and basic image processing techniques to achieve fire detection. These methods perform well in simple flame detection but are difficult to generalize to more complex scenarios. Deep learning, a cutting-edge technology in computer vision, extracts high-level, abstract features from data through nonlinear expressions, enabling higher accuracy in classification and detection tasks. Convolutional neural networks (CNNs) have demonstrated superior performance in image classification and detection. Numerous classic CNN architectures have been proposed, including AlexNet, VGGNet, GoogLeNet, ResNet, EfficientNet, and HRNet. In particular, the Efficient Fire Detection Network (EFDNet), proposed by Li et al. in 2020, boasts a 4.8MB model size and achieves accuracies of 95.3% and 81.4% on their FD-dataset and challenging test set, respectively.

[0004] The construction of smart cities and the rapid development of technologies such as 5G, the Internet of Things, edge computing, and cloud computing have laid a solid foundation for the implementation of real-time fire detection and monitoring technology based on surveillance cameras. However, due to the resource and performance limitations of edge monitoring devices, deep learning-based fire monitoring methods face a trade-off between accuracy, model size, and monitoring speed. The currently most effective EFDNet method has a large number of parameters, making it difficult to embed on resource-limited mobile devices for real-time fire monitoring. Furthermore, there is still room for improvement in monitoring speed and accuracy. Summary of the Invention

[0005] The present invention provides a fire detection method and system based on color edge depth perception to solve the technical problems of existing fire detection methods, such as large computational complexity and low monitoring efficiency, which lead to large constructed network models and limited mobile devices embedded in the network models.

[0006] To solve the above technical problems, an embodiment of the present invention provides a fire detection method based on color edge depth perception, comprising:

[0007] Acquire an image to be identified, and input the image to be identified into a fire detection model, so that the fire detection model performs feature extraction on the image to be identified, obtains a first color feature map and a first edge feature map of the image to be identified, performs implicit feature extraction on a first feature map formed by combining the first color feature map and the first edge feature map, and then outputs a fire detection result based on the implicit feature extraction result;

[0008] Among them, the fire detection model is a two-stage feature extraction architecture, which includes an explicit feature extraction stage and an implicit feature extraction stage. The explicit feature extraction stage is used to extract color features and edge features of the input image, and the implicit feature extraction stage is used to merge the input color feature map and edge feature map, and perform implicit feature extraction on the merged image.

[0009] Furthermore, the acquisition of the first color feature map is specifically as follows:

[0010] Extracting initial feature information of each color channel of the image to be identified through a first convolutional layer, exchanging the initial feature information of each color channel using a channel shuffling operation, and then obtaining channel feature information of the image to be identified through a second convolutional layer;

[0011] Matching the size and number of channels of the first color feature map through the third convolutional layer of identity mapping;

[0012] The first color feature map is obtained by combining the channel feature information of the image to be identified and the size and number of channels of the first color feature map.

[0013] Furthermore, the acquisition of the first edge feature map is specifically as follows:

[0014] The fourth convolutional layer calculates the derivatives of the image to be identified in four directions, thereby obtaining gradient images in the four directions to extract the initial edge features of the image to be identified in the corresponding directions, and the fifth convolutional layer further extracts the edge features of the image to be identified; wherein the fourth convolutional layer includes four sub-branch convolutions with fixed convolution kernel weights, corresponding to the four directional sub-branches of the Sobel operator template convolution;

[0015] Matching the size and number of channels of the first edge feature map through the third convolutional layer of identity mapping;

[0016] The first edge feature map is obtained by combining the edge features of the image to be identified and the size and number of channels of the first edge feature map.

[0017] Furthermore, performing implicit feature extraction on a first feature map obtained by merging the first color feature map and the first edge feature map includes:

[0018] Merging the first color feature map and the first edge feature map to obtain a first feature map;

[0019] Through the sixth convolutional layer, the number of channels of the first feature map is exchanged to extract small-scale features, thereby obtaining a first detail feature map corresponding to the first feature map;

[0020] Capturing the local spatial detail features in the large-scale receptive field and the local spatial detail features in the medium-scale receptive field of the first feature map through the seventh convolutional layer and the eighth convolutional layer, respectively, to obtain a second detail feature map and a third detail feature map corresponding to the first feature map;

[0021] Capturing key features in the mid-scale receptive field of the first feature map through the ninth convolutional layer to obtain a fourth detail feature map corresponding to the first feature map;

[0022] The first detail feature map, the second detail feature map, the third detail feature map, and the fourth detail feature map are merged to obtain a second feature map corresponding to the first feature map, and the second feature map is used as the first feature map. The second feature map is iteratively processed, and the second feature map is updated according to the current first feature map during each iterative processing until a preset number of operations is met, and the current second feature map is used as the third feature map corresponding to the first feature map.

[0023] Furthermore, after using the current second feature map as the third feature map corresponding to the first feature map, the method further includes performing depth feature extraction on the third feature map, specifically:

[0024] Extracting initial depth features of the third feature map through a base layer stacked residual network structure to obtain a fourth feature map corresponding to the third feature map;

[0025] According to the size of the third feature map, combined with the average pooling operation of the dimensionality reduction layer, the size of the fourth feature map is adjusted to a preset range to obtain a fifth feature map corresponding to the fourth feature map;

[0026] An efficient channel attention mechanism is adopted to perform a first global average pooling operation on the fifth feature map to obtain a corresponding channel-level feature vector, and according to a preset initial deep feature extraction process, initial deep feature extraction is performed on the fifth feature map to obtain a sixth feature map corresponding to the fifth feature map, and the sixth feature map is used as the third feature map, and the second feature map is iteratively processed. During each iterative processing, the sixth feature map is updated according to the current third feature map until the preset number of processing times is met, and the current sixth feature map is used as the seventh feature map corresponding to the third feature map to complete the deep feature extraction of the third feature map.

[0027] Furthermore, the fire detection result is outputted based on the implicit feature extraction result, specifically:

[0028] Inputting a seventh feature map corresponding to the third feature map into a classification layer, performing a batch normalization operation and a second global average pooling operation on the seventh feature map, and extracting key features of the seventh feature map;

[0029] According to the extraction result, combined with the classification calculation of the Softmax classifier, the classification result of whether there is flame in the image to be identified is used as the fire detection result, and the fire detection result is output.

[0030] As a preferred solution, the activation function used in the two-stage feature extraction architecture is the PReLU activation function.

[0031] In order to solve the same technical problem, the present invention also provides a fire detection system based on color edge depth perception, comprising: an input module, an explicit feature extraction module, an implicit feature extraction module and a result output module;

[0032] The input module is used to obtain an image to be identified and input the image to be identified into the fire detection model; the fire detection model is a two-stage feature extraction architecture, which includes an explicit feature extraction stage and an implicit feature extraction stage. The explicit feature extraction stage is used to extract color features and edge features from the input image, and the implicit feature extraction stage is used to merge the input color feature map and edge feature map and perform implicit feature extraction on the merged image.

[0033] The explicit feature extraction module is used to enable the fire detection model to perform feature extraction on the image to be identified, and obtain a first color feature map and a first edge feature map of the image to be identified;

[0034] The implicit feature extraction module is used to perform implicit feature extraction on the first feature map formed by merging the first color feature map and the first edge feature map to obtain an implicit feature extraction result;

[0035] The result output module is used to output the fire detection result according to the implicit feature extraction result.

[0036] Furthermore, the explicit feature extraction module further includes: a color feature extraction unit, a first merging unit, an edge feature extraction unit and a second merging unit;

[0037] The color feature extraction unit is configured to extract initial feature information of each color channel of the image to be identified through a first convolutional layer, exchange the initial feature information of each color channel using a channel shuffling operation, obtain channel feature information of the image to be identified through a second convolutional layer, and match the size and number of channels of the first color feature map through a third convolutional layer of identity mapping;

[0038] The first merging unit is configured to combine the channel feature information of the image to be identified and the size and number of channels of the first color feature map to obtain the first color feature map;

[0039] The edge feature extraction unit is configured to calculate the derivatives of the image to be identified in four directions through a fourth convolutional layer, thereby obtaining gradient images in the four directions to extract the initial edge features of the image to be identified in the corresponding directions, further extract the edge features of the image to be identified through a fifth convolutional layer, and match the size and number of channels of the first edge feature map through a third convolutional layer of identity mapping; wherein the fourth convolutional layer includes four sub-branch convolutions with fixed convolution kernel weights, respectively corresponding to the four directional sub-branches of the Sobel operator template convolution;

[0040] The second merging unit is configured to combine the edge features of the image to be identified and the size and number of channels of the first edge feature map to obtain the first edge feature map.

[0041] Furthermore, the implicit feature extraction module further includes: a detail feature extraction unit, a first iteration unit, a depth feature extraction unit and a second iteration unit;

[0042] Among them, the detail feature extraction unit is used to merge the first color feature map and the first edge feature map to obtain a first feature map, and through the sixth convolution layer, exchange the number of channels of the first feature map to extract small-scale features to obtain a first detail feature map corresponding to the first feature map, and at the same time, through the seventh convolution layer and the eighth convolution layer, respectively, capture the local spatial detail features in the large-scale receptive field of the first feature map and the local spatial detail features in the medium-scale receptive field to obtain the second detail feature map and the third detail feature map corresponding to the first feature map, and through the ninth convolution layer, capture the key features in the medium-scale receptive field of the first feature map to obtain the fourth detail feature map corresponding to the first feature map;

[0043] The first iteration unit is configured to merge the first detail feature map, the second detail feature map, the third detail feature map, and the fourth detail feature map to obtain a second feature map corresponding to the first feature map, and use the second feature map as the first feature map to iteratively process the second feature map, updating the second feature map according to the current first feature map during each iterative process until a preset number of operations is satisfied, and use the current second feature map as the third feature map corresponding to the first feature map;

[0044] The depth feature extraction unit is configured to extract the initial depth features of the third feature map through the operation of the base layer stacked residual network structure, obtain a fourth feature map corresponding to the third feature map, and adjust the size of the fourth feature map to a preset range based on the size of the third feature map in combination with the average pooling operation of the dimensionality reduction layer, to obtain a fifth feature map corresponding to the fourth feature map;

[0045] The second iteration unit is used to adopt an efficient channel attention mechanism to perform a first global average pooling operation on the fifth feature map to obtain a corresponding channel-level feature vector, and perform initial deep feature extraction on the fifth feature map according to a preset initial deep feature extraction process to obtain a sixth feature map corresponding to the fifth feature map, and use the sixth feature map as the third feature map to iteratively process the second feature map. During each iterative process, the sixth feature map is updated according to the current third feature map until a preset number of processing times is met, and the current sixth feature map is used as the seventh feature map corresponding to the third feature map to complete the deep feature extraction of the third feature map.

[0046] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0047] The present invention provides a fire detection method and system based on color edge depth perception. A two-stage feature extraction architecture is formed by an explicit feature extraction stage and an implicit feature extraction stage to realize explicit visual feature extraction of image color and edges, as well as implicit spatial feature extraction of spatial details and depth. Combined with a classification layer, the final results of explicit and implicit feature extraction are classified, and a classification result of whether there is flame in the input image is output, thereby realizing real-time monitoring of fire.

[0048] Furthermore, the two-stage feature extraction architecture of the present invention simplifies the fire detection model by adjusting the number of base layers in the stacked residual network structure. It also performs channel-dimensional convolution operations between the base layer and the dimensionality reduction layer to recalibrate channel features, thereby reducing the model's computational complexity and improving image processing speed while ensuring the effectiveness of capturing channel features. Furthermore, the use of the PReLU activation function in the two-stage feature extraction architecture prevents the fire detection model network from failing to learn due to gradient death, which occurs when the activation function outputs zero when the input parameter is less than zero. This improves the accuracy of the fire detection model and further enhances its performance, enabling the fire detection model to monitor video fire targets in real time on edge detection devices with limited resources and performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 : A flow chart of a fire detection method based on color edge depth perception provided by an embodiment of the present invention;

[0050] Figure 2 : A schematic diagram of the network architecture of a fire detection method based on color edge depth perception provided by an embodiment of the present invention;

[0051] Figure 3 : A structural diagram of a fire detection system based on color edge depth perception provided by an embodiment of the present invention;

[0052] Figure 4 : A schematic structural diagram of an explicit feature extraction module of a fire detection system based on color edge depth perception provided by an embodiment of the present invention;

[0053] Figure 5 : A structural diagram of an implicit feature extraction module of a fire detection system based on color edge depth perception provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] Example 1:

[0056] Please refer to Figure 1 , a fire detection method based on color edge depth perception provided by an embodiment of the present invention, comprising:

[0057] Step S1: Acquire an image to be identified, and input the image to be identified into a fire detection model, so that the fire detection model performs feature extraction on the image to be identified, and obtains a first color feature map and a first edge feature map of the image to be identified.

[0058] Furthermore, step S1 specifically includes steps S11 to S17, and each step is specifically as follows:

[0059] Step S11: Acquire an image to be identified, and input the image to be identified into a fire detection model.

[0060] Among them, the fire detection model is a two-stage feature extraction architecture, which includes an explicit feature extraction stage and an implicit feature extraction stage. The explicit feature extraction stage is used to extract color features and edge features of the input image, and the implicit feature extraction stage is used to merge the input color feature map and edge feature map, and perform implicit feature extraction on the merged image.

[0061] It should be noted that the two-stage feature extraction architecture uses the PReLU (Parametric Rectified Linear Unit) activation function. While improving the classification accuracy of the fire detection model network, it also avoids the situation where the activation function outputs zero when the input parameter is zero, thereby avoiding the gradient death that makes the fire detection model network unable to learn.

[0062] Step S12: extracting the initial feature information of each color channel of the image to be identified through the first convolutional layer, and using the channel shuffling operation to exchange the initial feature information of each color channel, and then obtaining the channel feature information of the image to be identified through the second convolutional layer.

[0063] As an example, the first convolutional layer can be composed of two 3×3 group convolutional layers, and the second convolutional layer can be composed of one 1×1 pointwise convolution.

[0064] Step S13: Match the size and number of channels of the first color feature map through the third convolutional layer of identity mapping.

[0065] As an example, the third convolutional layer of the identity map can consist of a 1×1 pointwise convolution and a 3×3 depthwise convolution.

[0066] Step S14: combining the channel feature information of the image to be identified and the size and number of channels of the first color feature map to obtain the first color feature map.

[0067] It should be noted that, since color channel values ​​have unique statistical characteristics in different color spaces (such as RGB, YUV, etc.), and considering that the convolution operation is equivalent to feature extraction, steps S12 to S14 are proposed in the explicit feature extraction stage to replace the statistical analysis method of traditional image color analysis, and residual learning is used to highlight the color features of the image to be identified.

[0068] Step S15: Through the fourth convolution layer, the derivatives of the image to be identified in four directions are calculated, and then the gradient images in the four directions are obtained to extract the initial edge features of the image to be identified in the corresponding directions, and the edge features of the image to be identified are further extracted through the fifth convolution layer; wherein, the fourth convolution layer includes four sub-branch convolutions with fixed convolution kernel weights, which correspond to the four directional sub-branches of the Sobel operator template convolution.

[0069] It should be noted that residual learning is also used to highlight the edge features of the image to be identified. The Sobel operator is applied to calculate the derivatives of the image to be identified in four directions, and then the gradient images in the four directions are obtained to extract the initial edge features of the image to be identified in the corresponding directions, that is, to extract the special irregular shape texture information of the flame in the image to be identified.

[0070] As an example, the fourth convolutional layer can be composed of four sub-branch convolutions with fixed convolution kernel weights, corresponding to the four directional branches of the Sobel template convolution, to extract the initial edge features of the image to be identified in the corresponding directions; the fifth convolutional layer can be composed of a 3×3 convolutional layer.

[0071] Step S16: Match the size and number of channels of the first edge feature map through the third convolutional layer of identity mapping.

[0072] It should be noted that the third convolutional layer in step S16 and the third convolutional layer in step S13 can be used interchangeably because both are used to match the size and number of channels of the output feature map, thereby reducing the number of convolutional layers used and further simplifying the fire detection model.

[0073] Step S17: Combining the edge features of the image to be identified with the size and number of channels of the first edge feature map, a first edge feature map is obtained.

[0074] Step S2: performing implicit feature extraction on the first feature map formed by merging the first color feature map and the first edge feature map.

[0075] Further, please refer to Figure 2 , step S2 specifically includes step S21 to step S28, each step is as follows:

[0076] Step S21: merging the first color feature map and the first edge feature map to obtain a first feature map.

[0077] It should be noted that in this embodiment, an efficient general neural network structure is constructed in the implicit feature extraction stage to obtain better detail and depth features, and the first feature map obtained by merging the first color feature map and the first edge feature map is used as the input of the implicit feature extraction stage. The implicit feature extraction stage is divided into a detail feature extraction process and a depth feature extraction process. The detail feature extraction process specifically includes steps S22 to S25, and the depth feature extraction process specifically includes steps S26 to S28.

[0078] Step S22: Through the sixth convolutional layer, the number of channels of the first feature map is exchanged to extract small-scale features, thereby obtaining a first detail feature map corresponding to the first feature map.

[0079] As an example, the sixth convolutional layer can consist of a 1×1 convolution.

[0080] Step S23: Capture the local spatial detail features in the large-scale receptive field and the local spatial detail features in the medium-scale receptive field of the first feature map through the seventh convolutional layer and the eighth convolutional layer, respectively, to obtain the second detail feature map and the third detail feature map corresponding to the first feature map.

[0081] As an example, the seventh convolutional layer can be composed of a 1×1 convolution and a 3×3 convolution, and the eighth convolutional layer can be composed of a 1×1 convolution and a 5×5 convolution.

[0082] Step S24: Through the ninth convolutional layer, the key features in the mid-scale receptive field of the first feature map are captured to obtain a fourth detail feature map corresponding to the first feature map.

[0083] As an example, the ninth convolutional layer can be composed of a 3×3 max pooling layer and a 1×1 convolution.

[0084] Step S25: Merge the first detail feature map, the second detail feature map, the third detail feature map, and the fourth detail feature map to obtain a second feature map corresponding to the first feature map, and use the second feature map as the first feature map, and iteratively process the second feature map. During each iterative process, update the second feature map according to the current first feature map until the preset number of operations is met, and use the current second feature map as the third feature map corresponding to the first feature map.

[0085] It should be noted that the sixth, seventh, eighth, and ninth convolutional layers are used to capture the detailed features in the multiple-scale receptive fields of the first feature map, and finally a second feature map corresponding to the first feature map is obtained. The second feature map is used as the first feature map and iteratively processed. During each iteration, the second feature map is updated according to the current first feature map until a preset number of operations is met. As an example, the preset number of operations is three, so as to further extract the spatial detail information of the first feature map. The number of operations can be adjusted according to actual needs.

[0086] Step S26: extracting the initial depth features of the third feature map through the operation of the base layer stacked residual network structure to obtain a fourth feature map corresponding to the third feature map.

[0087] It should be noted that the number of base layers for extracting the initial deep features of the third feature map by stacking the residual network structure is related to the size of the fire detection model. The number of base layers of EFDNet in the prior art is (6, 8, 12, 24). In comparison, this embodiment reduces it to (4, 6, 8, 12), reducing the size of the fire detection model from 4.8 MB to 3.39 MB.

[0088] Step S27: According to the size of the third feature map, combined with the average pooling operation of the dimensionality reduction layer, the size of the fourth feature map is adjusted to a preset range to obtain a fifth feature map corresponding to the fourth feature map.

[0089] As an example, the average pooling operation is used to reduce the size of the fourth feature map, reduce parameters, and reduce the complexity of the fire detection model.

[0090] Step S28: Using an efficient channel attention mechanism, perform a first global average pooling operation on the fifth feature map to obtain a corresponding channel-level feature vector, and perform initial deep feature extraction on the fifth feature map according to the preset initial deep feature extraction process to obtain a sixth feature map corresponding to the fifth feature map, and use the sixth feature map as the third feature map, and iteratively process the second feature map. During each iterative process, update the sixth feature map according to the current third feature map until the preset number of processing times is met, and use the current sixth feature map as the seventh feature map corresponding to the third feature map to complete the deep feature extraction of the third feature map.

[0091] It should be noted that, between the base layer and the dimensionality reduction layer, a first global average pooling operation is performed to extract global spatial information within each channel. A one-dimensional convolution kernel of size k is then used to perform a channel-wise convolution operation. This avoids dimensionality reduction while enabling local cross-channel interaction, improving the ability to capture channel features and reducing the computational complexity of the fire detection model. The convolution result is then mapped to the range of 0 to 1 using a sigmoid function. A dot product is then performed between the mapped output and the fifth feature map to obtain the sixth feature map corresponding to the fifth feature map.

[0092] At the same time, the PReLU activation function is combined to enhance the performance of the fire detection model, raising its accuracy from 93.8% in the existing technology to 95.1%. This enables the fire detection model to monitor fire targets in real time in video on edge detection devices with limited resources and performance. As an example, the value of k is 3.

[0093] As an example, the preset number of processing times is four to achieve the purpose of deeply extracting the depth features of the second feature map. The number of processing times can be adjusted according to actual needs.

[0094] Step S3: Output the fire detection result based on the implicit feature extraction result.

[0095] In this embodiment, please refer to Figure 2 , step S3 specifically includes step S31 to step S32, each step is as follows:

[0096] Step S31: Input the seventh feature map corresponding to the third feature map into the classification layer, perform batch normalization and a second global average pooling operation on the seventh feature map, and extract key features of the seventh feature map.

[0097] It should be noted that performing a batch normalization operation on the seventh feature map can improve the classification speed of the fire detection model, and extract key features through the second global average pooling operation to reduce parameters.

[0098] Step S32: Based on the extraction result and combined with the classification calculation of the Softmax classifier, the classification result of whether there is flame in the image to be identified is used as the fire detection result, and the fire detection result is output.

[0099] In order to solve the same technical problem, the present invention also provides a fire detection system based on color edge depth perception, see Figure 3 , including: input module 1, explicit feature extraction module 2, implicit feature extraction module 3 and result output module 4;

[0100] The input module 1 is used to obtain an image to be identified and input the image to be identified into the fire detection model. The fire detection model is a two-stage feature extraction architecture, which includes an explicit feature extraction stage and an implicit feature extraction stage. The explicit feature extraction stage is used to extract color features and edge features from the input image. The implicit feature extraction stage is used to merge the input color feature map and edge feature map and perform implicit feature extraction on the merged image.

[0101] The explicit feature extraction module 2 is used to enable the fire detection model to extract features from the image to be identified, and obtain a first color feature map and a first edge feature map of the image to be identified;

[0102] An implicit feature extraction module 3 is configured to perform implicit feature extraction on the first feature map formed by merging the first color feature map and the first edge feature map to obtain an implicit feature extraction result;

[0103] The result output module 4 is used to output the fire detection result according to the implicit feature extraction result.

[0104] Further, please refer to Figure 4 , the explicit feature extraction module 2 further includes: a color feature extraction unit, a first merging unit, an edge feature extraction unit and a second merging unit;

[0105] The color feature extraction unit is configured to extract initial feature information of each color channel of the image to be identified through the first convolutional layer, exchange the initial feature information of each color channel using a channel shuffling operation, obtain channel feature information of the image to be identified through the second convolutional layer, and match the size and number of channels of the first color feature map through the third convolutional layer of identity mapping;

[0106] a first merging unit, configured to combine the channel feature information of the image to be identified and the size and number of channels of the first color feature map to obtain the first color feature map;

[0107] An edge feature extraction unit is configured to calculate the derivatives of the image to be identified in four directions through the fourth convolutional layer, thereby obtaining gradient images in the four directions to extract the initial edge features of the image to be identified in the corresponding directions, further extract the edge features of the image to be identified through the fifth convolutional layer, and match the size and number of channels of the first edge feature map through the third convolutional layer of identity mapping; wherein the fourth convolutional layer includes four sub-branch convolutions with fixed convolution kernel weights, corresponding to the four directional sub-branches of the Sobel operator template convolution;

[0108] The second merging unit is used to combine the edge features of the image to be identified and the size and number of channels of the first edge feature map to obtain the first edge feature map.

[0109] Further, please refer to Figure 5 , the implicit feature extraction module 3 further includes: a detail feature extraction unit, a first iteration unit, a depth feature extraction unit and a second iteration unit;

[0110] Among them, the detail feature extraction unit is used to merge the first color feature map and the first edge feature map to obtain a first feature map, and through the sixth convolution layer, exchange the number of channels of the first feature map to extract small-scale features to obtain a first detail feature map corresponding to the first feature map, and at the same time, through the seventh convolution layer and the eighth convolution layer, respectively, capture the local spatial detail features in the large-scale receptive field of the first feature map and the local spatial detail features in the medium-scale receptive field to obtain the second detail feature map and the third detail feature map corresponding to the first feature map, and through the ninth convolution layer, capture the key features in the medium-scale receptive field of the first feature map to obtain the fourth detail feature map corresponding to the first feature map;

[0111] a first iterative unit, configured to merge the first detail feature map, the second detail feature map, the third detail feature map, and the fourth detail feature map to obtain a second feature map corresponding to the first feature map, and use the second feature map as the first feature map to iteratively process the second feature map, updating the second feature map according to the current first feature map during each iterative process until a preset number of operations is satisfied, and use the current second feature map as the third feature map corresponding to the first feature map;

[0112] A depth feature extraction unit is used to extract the initial depth features of the third feature map through the operation of the base layer stacked residual network structure, obtain a fourth feature map corresponding to the third feature map, and adjust the size of the fourth feature map to a preset range based on the size of the third feature map in combination with the average pooling operation of the dimensionality reduction layer, to obtain a fifth feature map corresponding to the fourth feature map;

[0113] The second iterative unit is used to adopt an efficient channel attention mechanism to perform a first global average pooling operation on the fifth feature map to obtain a corresponding channel-level feature vector, and perform initial deep feature extraction on the fifth feature map according to a preset initial deep feature extraction process to obtain a sixth feature map corresponding to the fifth feature map, and use the sixth feature map as the third feature map to iteratively process the second feature map. During each iterative process, the sixth feature map is updated according to the current third feature map until a preset number of processing times is met, and the current sixth feature map is used as the seventh feature map corresponding to the third feature map to complete the deep feature extraction of the third feature map.

[0114] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0115] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0116] The present invention provides a fire detection method and system based on color edge depth perception. A two-stage feature extraction architecture is formed by an explicit feature extraction stage and an implicit feature extraction stage to realize explicit visual feature extraction of image color and edges, as well as implicit spatial feature extraction of spatial details and depth. Combined with a classification layer, the final results of explicit and implicit feature extraction are classified, and a classification result of whether there is flame in the input image is output, thereby realizing real-time monitoring of fire.

[0117] Furthermore, the two-stage feature extraction architecture of the present invention simplifies the fire detection model by adjusting the number of base layers in the stacked residual network structure. It also performs channel-dimensional convolution operations between the base layer and the dimensionality reduction layer to recalibrate channel features, thereby reducing the model's computational complexity and improving image processing speed while ensuring the effectiveness of capturing channel features. Furthermore, the use of the PReLU activation function in the two-stage feature extraction architecture prevents the fire detection model network from failing to learn due to gradient death, which occurs when the activation function outputs zero when the input parameter is less than zero. This improves the accuracy of the fire detection model and further enhances its performance, enabling the fire detection model to monitor video fire targets in real time on edge detection devices with limited resources and performance.

[0118] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A fire detection method based on color edge depth perception, characterized in that: include: Acquire an image to be identified, and input the image to be identified into a fire detection model, so that the fire detection model performs feature extraction on the image to be identified, obtains a first color feature map and a first edge feature map of the image to be identified, performs implicit feature extraction on a first feature map formed by combining the first color feature map and the first edge feature map, and then outputs a fire detection result based on the implicit feature extraction result; The fire detection model is a two-stage feature extraction architecture, which includes an explicit feature extraction stage and an implicit feature extraction stage. The explicit feature extraction stage is used to extract color features and edge features from the input image, and the implicit feature extraction stage is used to merge the input color feature map and edge feature map and perform implicit feature extraction on the merged image. The performing implicit feature extraction on a first feature map formed by merging the first color feature map and the first edge feature map includes: Merging the first color feature map and the first edge feature map to obtain a first feature map; Through the sixth convolutional layer, the number of channels of the first feature map is exchanged to extract small-scale features, thereby obtaining a first detail feature map corresponding to the first feature map; Capturing the local spatial detail features in the large-scale receptive field and the local spatial detail features in the medium-scale receptive field of the first feature map through the seventh convolutional layer and the eighth convolutional layer, respectively, to obtain a second detail feature map and a third detail feature map corresponding to the first feature map; Capturing key features in the mid-scale receptive field of the first feature map through the ninth convolutional layer to obtain a fourth detail feature map corresponding to the first feature map; Merging the first detail feature map, the second detail feature map, the third detail feature map, and the fourth detail feature map to obtain a second feature map corresponding to the first feature map, and using the second feature map as the first feature map. Iteratively processing the second feature map, updating the second feature map according to the current first feature map during each iterative processing, until a preset number of operations is met, and using the current second feature map as the third feature map corresponding to the first feature map; After using the current second feature map as the third feature map corresponding to the first feature map, the method further includes performing depth feature extraction on the third feature map, specifically: Extracting initial depth features of the third feature map through a base layer stacked residual network structure to obtain a fourth feature map corresponding to the third feature map; According to the size of the third feature map, combined with the average pooling operation of the dimensionality reduction layer, the size of the fourth feature map is adjusted to a preset range to obtain a fifth feature map corresponding to the fourth feature map; Adopting an efficient channel attention mechanism, performing a first global average pooling operation on the fifth feature map to obtain a corresponding channel-level feature vector, and performing initial deep feature extraction on the fifth feature map according to a preset initial deep feature extraction process to obtain a sixth feature map corresponding to the fifth feature map, and using the sixth feature map as the third feature map, iteratively processing the sixth feature map, updating the sixth feature map according to the current third feature map during each iterative processing until a preset number of processing times is met, and using the current sixth feature map as the seventh feature map corresponding to the third feature map, thereby completing the deep feature extraction of the third feature map; The fire detection result is output according to the implicit feature extraction result, specifically: Inputting a seventh feature map corresponding to the third feature map into a classification layer, performing a batch normalization operation and a second global average pooling operation on the seventh feature map, and extracting key features of the seventh feature map; According to the extraction result, combined with the classification calculation of the Softmax classifier, the classification result of whether there is flame in the image to be identified is used as the fire detection result, and the fire detection result is output.

2. The fire detection method based on color edge depth perception according to claim 1, characterized in that: The acquisition of the first color feature map is specifically as follows: Extracting initial feature information of each color channel of the image to be identified through a first convolutional layer, exchanging the initial feature information of each color channel using a channel shuffling operation, and then obtaining channel feature information of the image to be identified through a second convolutional layer; Matching the size and number of channels of the first color feature map through the third convolutional layer of identity mapping; The first color feature map is obtained by combining the channel feature information of the image to be identified and the size and number of channels of the first color feature map.

3. The fire detection method based on color edge depth perception according to claim 1, characterized in that: The first edge feature map is obtained by: The fourth convolutional layer calculates the derivatives of the image to be identified in four directions, thereby obtaining gradient images in the four directions to extract the initial edge features of the image to be identified in the corresponding directions, and the fifth convolutional layer further extracts the edge features of the image to be identified; wherein the fourth convolutional layer includes four sub-branch convolutions with fixed convolution kernel weights, corresponding to the four directional sub-branches of the Sobel operator template convolution; Matching the size and number of channels of the first edge feature map through the third convolutional layer of identity mapping; The first edge feature map is obtained by combining the edge features of the image to be recognized and the size and number of channels of the first edge feature map.

4. The fire detection method based on color edge depth perception according to claim 1, characterized in that: The activation function used in the two-stage feature extraction architecture is the PReLU activation function.

5. A fire detection system based on color edge depth perception, characterized in that: include: Input module, explicit feature extraction module, implicit feature extraction module and result output module; The input module is used to obtain an image to be identified and input the image to be identified into the fire detection model; the fire detection model is a two-stage feature extraction architecture, which includes an explicit feature extraction stage and an implicit feature extraction stage. The explicit feature extraction stage is used to extract color features and edge features from the input image, and the implicit feature extraction stage is used to merge the input color feature map and edge feature map and perform implicit feature extraction on the merged image. The explicit feature extraction module is used to enable the fire detection model to perform feature extraction on the image to be identified, and obtain a first color feature map and a first edge feature map of the image to be identified; The implicit feature extraction module is used to perform implicit feature extraction on the first feature map formed by merging the first color feature map and the first edge feature map to obtain an implicit feature extraction result; The implicit feature extraction module further includes: a detail feature extraction unit, a first iteration unit, a depth feature extraction unit, and a second iteration unit; Among them, the detail feature extraction unit is used to merge the first color feature map and the first edge feature map to obtain a first feature map, and through the sixth convolution layer, exchange the number of channels of the first feature map to extract small-scale features to obtain a first detail feature map corresponding to the first feature map, and at the same time, through the seventh convolution layer and the eighth convolution layer, respectively, capture the local spatial detail features in the large-scale receptive field of the first feature map and the local spatial detail features in the medium-scale receptive field to obtain the second detail feature map and the third detail feature map corresponding to the first feature map, and through the ninth convolution layer, capture the key features in the medium-scale receptive field of the first feature map to obtain the fourth detail feature map corresponding to the first feature map; a first iterative unit, configured to merge the first detail feature map, the second detail feature map, the third detail feature map, and the fourth detail feature map to obtain a second feature map corresponding to the first feature map, and use the second feature map as the first feature map to iteratively process the second feature map, updating the second feature map according to the current first feature map during each iterative process until a preset number of operations is satisfied, and use the current second feature map as the third feature map corresponding to the first feature map; A depth feature extraction unit is used to extract the initial depth features of the third feature map through the operation of the base layer stacked residual network structure, obtain a fourth feature map corresponding to the third feature map, and adjust the size of the fourth feature map to a preset range based on the size of the third feature map in combination with the average pooling operation of the dimensionality reduction layer, to obtain a fifth feature map corresponding to the fourth feature map; A second iterative unit is configured to adopt an efficient channel attention mechanism to perform a first global average pooling operation on the fifth feature map to obtain a corresponding channel-level feature vector, and perform initial deep feature extraction on the fifth feature map according to a preset initial deep feature extraction process to obtain a sixth feature map corresponding to the fifth feature map, and use the sixth feature map as the third feature map to iteratively process the second feature map, updating the sixth feature map according to the current third feature map during each iterative process until a preset number of processing times is met, and using the current sixth feature map as the seventh feature map corresponding to the third feature map to complete the deep feature extraction of the third feature map; The result output module is used to output the fire detection result according to the implicit feature extraction result; According to the implicit feature extraction results, the fire detection results are output, specifically: Inputting a seventh feature map corresponding to the third feature map into a classification layer, performing a batch normalization operation and a second global average pooling operation on the seventh feature map, and extracting key features of the seventh feature map; According to the extraction result, combined with the classification calculation of the Softmax classifier, the classification result of whether there is flame in the image to be identified is used as the fire detection result, and the fire detection result is output.

6. A fire detection system based on color edge depth perception as claimed in claim 5, characterized in that: The explicit feature extraction module further includes: a color feature extraction unit, a first merging unit, an edge feature extraction unit and a second merging unit; The color feature extraction unit is configured to extract initial feature information of each color channel of the image to be identified through a first convolutional layer, exchange the initial feature information of each color channel using a channel shuffling operation, obtain channel feature information of the image to be identified through a second convolutional layer, and match the size and number of channels of the first color feature map through a third convolutional layer of identity mapping; The first merging unit is configured to combine the channel feature information of the image to be identified and the size and number of channels of the first color feature map to obtain the first color feature map; The edge feature extraction unit is configured to calculate the derivatives of the image to be identified in four directions through a fourth convolutional layer, thereby obtaining gradient images in the four directions to extract the initial edge features of the image to be identified in the corresponding directions, further extract the edge features of the image to be identified through a fifth convolutional layer, and match the size and number of channels of the first edge feature map through a third convolutional layer of identity mapping; wherein the fourth convolutional layer includes four sub-branch convolutions with fixed convolution kernel weights, respectively corresponding to the four directional sub-branches of the Sobel operator template convolution; The second merging unit is configured to combine the edge features of the image to be identified and the size and number of channels of the first edge feature map to obtain the first edge feature map.

Images