Lightweight smoke fire identification method and device based on multi-scale fusion

Abstract

The invention discloses a multi-scale fusion-based lightweight smoke and fire behavior identification method and device, and the method comprises the steps: constructing a smoke and fire behavior identification model based on improved YOLOv7-Tiny, and carrying out the training of the model, and obtaining a trained smoke and fire behavior identification model; according to the smog and fire behavior identification model, on the basis of a YOLOv7-Tiny network, an ELAN-TINY module in the YOLOv7-Tiny network is replaced by an ELAN-TPC module, and on the basis of the ELAN-TINY module, a convolution module with the convolution kernel size of 3 * 3 is replaced by a partial convolution module by the ELAN-TPC module; a feature fusion network in the YOLOv7-Tiny network is replaced by a multi-scale fusion network; a convolution module with the convolution kernel size of 3 * 3 in an output network in the YOLOv7-Tiny network is replaced with a parameter-free attention module and a convolution module with the convolution kernel size of 1 * 1; and obtaining a to-be-detected image, inputting the to-be-detected image into the trained smoke and fire behavior recognition model, and outputting a smoke and fire behavior recognition result which comprises bounding box coordinates, target confidence and category probability. According to the invention, the accuracy and real-time performance of smoke fire detection can be improved.

Description

Technical Field

[0001] This invention relates to the field of target recognition, specifically to a lightweight smoke and fire identification method and apparatus based on multi-scale fusion. Background Technology

[0002] The global fire situation is becoming increasingly severe, posing a serious threat to human life and property. Due to the suddenness and rapid spread of fires, real-time monitoring and early warning are crucial to mitigating fire losses. Current mainstream smoke and fire detection systems primarily rely on temperature, humidity, and carbon monoxide concentration sensors. While these sensors are low-cost and easy to deploy, they suffer from high false alarm rates, response delays, and significant limitations in application scenarios.

[0003] To overcome the technical bottlenecks of traditional fire detection systems, visual detection methods based on surveillance video have become the mainstream solution. These systems decompose smoke and fire conditions into multimodal visual features, encompassing brightness distribution, texture patterns, and chromatographic characteristics. Celik et al. combined fire color modeling with dynamic motion detection to improve the reliability of detection in complex scenes; Chen et al. utilized the dynamic irregularities of flames across the RGB-HSV color space to construct a pixel-level classification model. However, these methods have limited nonlinear representation capabilities and weak scene adaptability, requiring a certain degree of generalization ability.

[0004] With the development of deep learning technology, computer vision has received significant attention in the field of smoke and fire detection. Deep learning-based object detection methods are divided into single-stage (YOLO, SSD) and two-stage (R-CNN, Faster R-CNN, etc.). Hou Yicheng et al. introduced a channel attention mechanism in Faster R-CNN to enhance the detection accuracy of smoke and fire through feature channels and reduce interference. Cai Hangyu et al. integrated a learnable center module and a Res2Net module to enhance the ability to read information from smoke and fire edge features. Yan Xun et al. optimized the adaptive anchor box operation, reducing the model's dependence on the scene to some extent. While these methods have made up for the shortcomings of traditional methods to some extent, they have weak multi-scale fusion capabilities for smoke and fire, poor real-time performance, and are not suitable for edge deployment.

[0005] Computer vision-based smoke and fire detection technology has shown great potential in the field of public safety, but it still faces technical bottlenecks in practical applications. On the one hand, the large number of model parameters and high computational complexity make it difficult to deploy on resource-constrained edge computing devices; on the other hand, the diverse nature of smoke and fire targets hinders model learning in specific scenarios. Summary of the Invention

[0006] The purpose of this application is to propose a lightweight smoke and fire detection method and device based on multi-scale fusion to address the aforementioned technical problems, aiming to improve the accuracy and real-time performance of smoke and fire detection.

[0007] In a first aspect, the present invention provides a lightweight smoke and fire identification method based on multi-scale fusion, comprising the following steps:

[0008] A smoke and fire recognition model based on the improved YOLOv7-Tiny network was constructed and trained to obtain the trained smoke and fire recognition model. The smoke and fire recognition model was developed by replacing the ELAN-TINY module in the YOLOv7-Tiny network with the ELAN-TPC module. The ELAN-TPC module, in turn, replaced the 3×3 convolutional kernels with partial convolutional modules. The feature fusion network in the YOLOv7-Tiny network was replaced with a multi-scale fusion network. Finally, the 3×3 convolutional kernels in the output network of the YOLOv7-Tiny network were replaced with a parameterless attention module and a 1×1 convolutional kernel.

[0009] The system acquires the image to be detected and inputs it into a trained smoke and fire recognition model, outputting smoke and fire recognition results, including bounding box coordinates, target confidence, and class probability.

[0010] Preferably, the ELAN-TPC module contains two partially convolutional modules connected in sequence; the input features of the partially convolutional modules are... and binary mask The calculation process of some convolutional modules is as follows:

[0011] ;

[0012] ;

[0013] Where W represents the weights of the convolution kernel, b is the bias term, ⊙ represents element-wise multiplication, and Ω is the current convolution window. The total number of pixels covered by the convolution kernel. Coordinates in the current convolution window The number of effective pixels within. This represents the output features of a portion of the convolutional module. This represents the updated binary mask;

[0014] The output features and updated binary mask of the first partial convolutional module are used as the input features and binary mask of the second partial convolutional module.

[0015] As a preferred option, the multi-scale fusion network uses the output features of the four ELAN-TPC modules in the feature extraction network as multi-scale input features, based on the feature fusion network in the YOLOv7-Tiny network, and achieves deep fusion of cross-level features through a bidirectional closed-loop path.

[0016] Preferably, the bidirectional closed-loop path includes a top-down propagation path and a bottom-up propagation path. In the top-down propagation path, the second upsampling module in the feature fusion network of the YOLOv7-Tiny network is replaced with a transposed convolutional layer with a kernel size of 2×2. A convolutional layer with a kernel size of 1×1 is added between the second concatenation layer and the second ELAN-TPC module in the multi-scale fusion network. The third concatenation layer in the bottom-up propagation path is replaced with a concatenation layer using BiFPN_Add2. The first feature extraction network... The output features of the first ELAN-TPC module are sequentially passed through a 1×1 convolutional layer and a 3×3 convolutional layer in the multi-scale fusion network to obtain the first feature. The output features of the second ELAN-TPC module in the feature extraction network are passed through a 1×1 convolutional layer in the multi-scale fusion network to obtain the second feature. The first feature, the second feature, and the output features of the transposed convolutional layer are input into the second concatenation layer for concatenation, and then pass through a 1×1 convolutional layer before entering the bottom-up propagation path.

[0017] As a preferred method, the calculation process of the parameterless attention module is as follows:

[0018] The statistics of the input features of the parameterless attention module in the spatial dimension are calculated independently for each channel. The statistics include the mean μ and the variance σ².

[0019] The energy value of the input feature of the parameterless attention module at each position is constructed based on statistics, as shown in the following formula:

[0020] ;

[0021] Among them, e t x represents the energy value of the input feature at position t for the parameterless attention module. t Let ϵ be the value of the input feature of the parameterless attention module at position t, and let ϵ be a minimal constant.

[0022] The energy values ​​are inversely mapped to attention weights using the sigmoid function, as shown in the following formula:

[0023] ;

[0024] Among them, w t Let be the attention weights of the input features of the parameterless attention module at position t. Represents the sigmoid function;

[0025] The input features of the parameterless attention module are enhanced element-wise by attention weights to obtain the output features of the parameterless attention module, as shown in the following formula:

[0026] ;

[0027] in, This represents the value of the output feature of the parameterless attention module at position t.

[0028] Preferably, the convolutional module with a kernel size of 1×1 includes a convolutional layer with a kernel size of 1×1, a BN layer, and a LeakyReLU activation function layer connected in sequence.

[0029] Secondly, the present invention provides a lightweight smoke and fire detection device based on multi-scale fusion, comprising:

[0030] The model building module is configured to build and train a smoke and fire recognition model based on the improved YOLOv7-Tiny network, resulting in a trained smoke and fire recognition model. The smoke and fire recognition model is based on the YOLOv7-Tiny network, but with the ELAN-TINY module replaced by the ELAN-TPC module. The ELAN-TPC module, in turn, replaces the 3×3 convolutional kernels with partial convolutional modules. The feature fusion network in the YOLOv7-Tiny network is replaced by a multi-scale fusion network. Finally, the 3×3 convolutional kernels in the output network of the YOLOv7-Tiny network are replaced with parameterless attention modules and 1×1 convolutional kernels.

[0031] The prediction module is configured to acquire the image to be detected and input it into the trained smoke and fire recognition model, and output the smoke and fire recognition result, which includes the bounding box coordinates, target confidence and class probability.

[0032] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0033] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0034] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) The lightweight smoke and fire identification method based on multi-scale fusion proposed in this invention integrates a parameterless attention module, a lightweight ELAN-TPC module and a multi-scale fusion network Bi-FusFPN into the YOLOv7-Tiny network, which can improve the feature fusion capability of smoke and fire video at different scales, reduce the network computation, and effectively improve the efficiency of smoke and fire identification and early warning in video scenes.

[0037] (2) The lightweight smoke and fire identification method based on multi-scale fusion proposed in this invention combines the extremely high computational efficiency and good memory utilization of the partial convolution (PConv) module to design the ELAN-TPC module, thereby achieving network lightweighting and improving the model's efficiency in smoke and fire detection; and combined with the multi-scale fusion network, by processing the features of smoke at different scales, it integrates the global and multi-scale spatial information of smoke, reducing the model's false negative and false positive rates; and further integrates the parameterless attention (SimAM) module to enhance the feature loss of the detection head and improve the overall detection accuracy of the model.

[0038] (3) The lightweight smoke and fire detection method based on multi-scale fusion proposed in this invention aims to improve the accuracy and real-time performance of smoke and fire detection from the perspective of network lightweighting and multi-scale fusion. While quantizing the network, it enhances the network's ability to perceive and fuse multi-scale information. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating a lightweight smoke and fire identification method based on multi-scale fusion, as an embodiment of this application.

[0041] Figure 2 This is a schematic diagram of the structure of a smoke and fire identification model for a lightweight smoke and fire identification method based on multi-scale fusion, as an embodiment of this application.

[0042] Figure 3This is a schematic diagram of the structure of the ELAN-TPC module, which is a lightweight smoke and fire identification method based on multi-scale fusion, according to an embodiment of this application.

[0043] Figure 4 This is a schematic diagram of the structure of a multi-scale fusion network for a lightweight smoke and fire identification method based on multi-scale fusion, as an embodiment of this application.

[0044] Figure 5 This is a schematic diagram of a lightweight smoke and fire detection device based on multi-scale fusion, as an embodiment of this application.

[0045] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0047] Figure 1 The illustration shows a lightweight smoke and fire identification method based on multi-scale fusion provided by an embodiment of this application, including the following steps:

[0048] S1. Construct and train a smoke and fire recognition model based on the improved YOLOv7-Tiny network to obtain the trained smoke and fire recognition model. The smoke and fire recognition model is based on the YOLOv7-Tiny network, with the ELAN-TINY module replaced by the ELAN-TPC module. The ELAN-TPC module is based on the ELAN-TINY module, with the 3×3 convolutional kernel module replaced by a partial convolutional module. The feature fusion network in the YOLOv7-Tiny network is replaced by a multi-scale fusion network. The 3×3 convolutional kernel module in the output network of the YOLOv7-Tiny network is replaced by a parameterless attention module and a 1×1 convolutional kernel module.

[0049] For details, please refer to Figure 2The embodiments of this application construct a smoke and fire recognition model based on the improved YOLOv7-Tiny. The smoke and fire recognition model includes three parts: a feature extraction network, a multi-scale fusion network, and an output network. The image to be detected is input into the smoke and fire recognition model. Through the feature extraction network and the multi-scale fusion network, fused features of three sizes are obtained. In the output network, localization and classification convolution operations are performed based on the fused features, thereby locating and classifying the output.

[0050] The embodiments of this application have made relevant improvements to the YOLOv7-Tiny network, resulting in an improved YOLOv7-Tiny network as a smoke and fire identification model. The specific improvements are as follows:

[0051] (1) The 3×3 convolutional kernel in the ELAN-Tiny module of the original YOLOv7-Tiny network has redundant parameters, introducing additional computational overhead. To make full use of computational resources, embodiments of this application introduce a partial convolution (PConv) module, proposing a lightweight ELAN-TPC module, such as... Figure 3 As shown, while ensuring detection accuracy, the timeliness of the model is greatly improved;

[0052] (2) The original YOLOv7-Tiny network's bidirectional fusion mechanism PANet, which combines top-down and bottom-up approaches, cannot process multi-scale information equally, leading to the loss of information across multiple levels. This application proposes a multi-scale fusion network (Bi-FusFPN), such as... Figure 3 As shown, this multi-scale fusion network combines BiFusion Neck (parallel fusion, lightweight convolution, and learnable weights) and BiFPN (weighted fusion, simplified structure) to make up for the shortcomings of PANet and enhance the network's ability to integrate multi-dimensional information.

[0053] (3) The output network in the original YOLOv7-Tiny network directly outputs the fused features through the convolution module, which may result in information loss. To ensure the output effect of features at different levels, a parameterless attention (SimAM) module is added before the convolution, which does not introduce additional computation and reduces information loss.

[0054] In a specific embodiment, the ELAN-TPC module includes two partially convolutional modules connected in sequence; the input features of the partially convolutional modules... and binary mask The calculation process of some convolutional modules is as follows:

[0055] ;

[0056] ;

[0057] Where W represents the weights of the convolution kernel, b is the bias term, ⊙ represents element-wise multiplication, and Ω is the current convolution window. The total number of pixels covered by the convolution kernel. Coordinates in the current convolution window The number of effective pixels within. This represents the output features of a portion of the convolutional module. This represents the updated binary mask;

[0058] The output features and updated binary mask of the first partial convolutional module are used as the input features and binary mask of the second partial convolutional module.

[0059] For details, please refer to Figure 2 The feature extraction network proposed in the embodiments of this application includes four ELAN-TPC modules. The input of the ELAN-TPC modules mainly comes from convolutional modules with a kernel size of 1×1 or max pooling operation (MP). In the improved YOLOv7-Tiny network, all convolutional modules use convolutional modules with a kernel size of 1×1, and there are no convolutional modules with a kernel size of 3×3. The convolutional module with a kernel size of 1×1 includes a convolutional layer with a kernel size of 1×1, a BN layer, and a LeakyReLU activation function layer connected in sequence. (See reference...) Figure 3 The ELAN-TPC module learns features from the input through four channels and then concatenates these learned features into a concatenation layer. Finally, a 1×1 convolutional module adjusts the number of output channels to match subsequent networks. The first and second channels are directly connected to the concatenation layer via 1×1 convolutional modules for better global feature learning. The third channel is concatenated with the concatenation layer after passing through a 1×1 convolutional module and a PConv module. The fourth channel is concatenated with the concatenation layer after passing through a 1×1 convolutional module and two PConv modules. This ELAN-TPC module processes the same input features through different channels, integrating them into relatively comprehensive and rich information for the next layer to learn and iterate. The PConv module's computational cost is only one-sixteenth that of a regular convolution, effectively utilizing the device's computing power and maintaining high computational speed.

[0060] The input features X and binary masks M marking valid regions (mask value of 1 corresponds to valid pixels, mask value of 0 corresponds to invalid pixels) of the Pconv module are input into the Pconv module. Based on the binary mask indication, the weighted sum of valid pixels (regions with mask value of 1) within the convolutional kernel's coverage area is calculated, while invalid pixels (mask value of 0) are forced to zero to avoid interference. The output of the Pconv module consists of two parts: first, the updated features, i.e., the output features of the Pconv module, where invalid edge regions are initially generated with reasonable prediction values ​​based on neighboring valid pixels, and valid region information is preserved and passed on; second, the updated binary mask, where any output position only needs at least one original valid pixel within the corresponding convolutional window, and a mask value of 0 typically transforms to close to 1, representing a transition from unknown to known. As the network depth increases, subsequent Pconv modules will treat "predicted regions" with mask values ​​greater than 0 from the previous Pconv module as new valid information sources, thereby driving convolutional computation and mask updates. In the embodiments of this application, the mask of the output position is updated to 1 (valid) only when all pixels within the convolution kernel window are valid; otherwise, it is 0 (invalid), and the boundary is automatically processed to 0.

[0061] The Pconv module replaces the 3×3 convolutional module in the ELAN-Tiny module. Following a 1×1 convolutional module, the PConv module is connected. Through a dual strategy of channel dimensionality reduction and partial channel processing, it minimizes the loss of feature representation ability while maximizing the reduction of model parameter complexity and number, requiring only the processing of 1 / 4 of the channel information. Furthermore, to ensure comprehensive information while maintaining lightweight design, the ELAN-TPC module retains the connection mechanism of the ELAN-Tiny module, concatenating the information from the remaining three channels and then outputting it after passing it through a 1×1 convolutional module to accommodate the next channel feature iteration.

[0062] Leveraging the advantages of the ELAN-TPC module, embodiments of this application tested its placement in different locations within the YOLOv7-Tiny network. Specifically, ELAN-TPC-backbone indicates that the ELAN-TPC module replaced the ELAN-Tiny module in the feature extraction network; ELAN-TPC-neck indicates that the ELAN-TPC module replaced the ELAN-Tiny module in the feature fusion network; and ELAN-TPC-all indicates that both the feature extraction and feature fusion networks had their ELAN-Tiny modules replaced with ELAN-TPC modules. After comparing the computational cost (FLOPs), parameter count (Parameters), and frame rate (FPS) of the YOLOv7-Tiny network, the final solution adopted was to replace all ELAN-Tiny modules in the original YOLOv7-Tiny network with ELAN-TPC modules.

[0063] Table 1 Comparison results of ELAN-TPC modules in different locations

[0064]

[0065] In a specific embodiment, the multi-scale fusion network uses the output features of four ELAN-TPC modules in the feature extraction network as multi-scale input features, based on the feature fusion network in the YOLOv7-Tiny network, and achieves cross-level feature deep fusion through a bidirectional closed-loop path.

[0066] In a specific embodiment, the bidirectional closed-loop path includes a top-down propagation path and a bottom-up propagation path. In the top-down propagation path, the second upsampling module in the feature fusion network of the YOLOv7-Tiny network is replaced with a transposed convolutional layer with a kernel size of 2×2. A convolutional layer with a kernel size of 1×1 is added between the second concatenation layer and the second ELAN-TPC module in the multi-scale fusion network. The third concatenation layer in the bottom-up propagation path is replaced with a concatenation layer using BiFPN_Add2. The feature extraction network... The output features of the first ELAN-TPC module pass through a convolutional layer with a kernel size of 1×1 and a convolutional layer with a kernel size of 3×3 in the multi-scale fusion network to obtain the first feature. The output features of the second ELAN-TPC module in the feature extraction network pass through a convolutional layer with a kernel size of 1×1 in the multi-scale fusion network to obtain the second feature. The first feature, the second feature, and the output features of the transposed convolutional layer are input into the second concatenation layer for concatenation. Then, they pass through a convolutional layer with a kernel size of 1×1 and enter the bottom-up propagation path.

[0067] Specifically, the YOLOv7-Tiny network employs a top-down and bottom-up bidirectional fusion mechanism in its feature fusion network. PANet equally weights features at each level, resulting in severe loss of shallow features and redundant network parameters. This application's implementation combines the advantages of BiFusion Neck's multi-scale information processing with BiFPN's structured feature selection and learnable dynamic weights, proposing a multi-scale fusion network (Bi-FusFPN), such as... Figure 4 As shown.

[0068] The processing flow of this multi-scale fusion network begins with multi-scale input features (such as P3, P4, P5, and P6, which are the output features of the first, second, third, and fourth ELAN-TPC modules in the feature extraction network, respectively). It achieves deep feature fusion across layers through a bidirectional closed-loop path: First, a top-down propagation path is executed, progressively upsampling deep features (such as P6) and weightedly fusing them with mid-level features (P5 or P4), ultimately injecting high-resolution shallow features (P3). Then, a bottom-up propagation is initiated, downsampling the enhanced shallow features, fusing them a second time with semantically corrected mid-level features, and then back-injecting deep features (P5 or P6). In the third concatenation layer, the BiFPN_Add2 concatenation operation is used to reduce the number of network parameters and computational cost. During this process, each fusion node dynamically learns the weights of the input features, enabling the network to adaptively strengthen key layer information. This process can be iterated multiple times, and the final output features have both high-resolution details and strong semantic representation, which significantly improves the ability to perceive targets of different scales in detection tasks.

[0069] In this multi-scale fusion network, besides replacing the third concatenation layer of the YOLOv7-Tiny network's feature fusion network with a concatenation layer using BiFPN_Add2, the output features of the first ELAN-TPC module in the feature extraction network must pass through two convolutional layers before being input into the second concatenation layer: a convolutional layer with a kernel size of 1×1 and a stride of 1, and a convolutional layer with a kernel size of 3×3 and a stride of 2. Similarly, the output features of the second ELAN-TPC module in the feature extraction network must pass through another convolutional layer with a kernel size of 1×1 and a stride of 1 before being input into the second concatenation layer. Furthermore, an additional convolutional layer with a kernel size of 1×1 and a stride of 1 is added between the second concatenation layer and the second ELAN-TPC module of the multi-scale fusion network. The remaining structure and connections are consistent with the feature fusion network of the YOLOv7-Tiny network. Replacing the other three concatenation layers of the YOLOv7-Tiny network's feature fusion network would significantly increase the model's computational cost. To reduce the loss of shallow information during training and better output effective information, it is best to replace only the third concatenation layer of the YOLOv7-Tiny network's feature fusion network with a concatenation layer using BiFPN_Add2. This layer connects the first half of the shallow information content and is located near the detection head. Finally, the second, third, and fourth ELAN-TPC modules of the multi-scale fusion network output three fused features at different scales.

[0070] In a specific embodiment, the calculation process of the parameterless attention module is as follows:

[0071] The statistics of the input features of the parameterless attention module in the spatial dimension are calculated independently for each channel. The statistics include the mean μ and the variance σ².

[0072] The energy value of the input feature of the parameterless attention module at each position is constructed based on statistics, as shown in the following formula:

[0073] ;

[0074] Among them, e t x represents the energy value of the input feature at position t for the parameterless attention module. t Let ϵ be the value of the input feature of the parameterless attention module at position t, and let ϵ be a minimal constant.

[0075] The energy values ​​are inversely mapped to attention weights using the sigmoid function, as shown in the following formula:

[0076] ;

[0077] Among them, w t Let be the attention weights of the input features of the parameterless attention module at position t. Represents the sigmoid function;

[0078] The input features of the parameterless attention module are enhanced element-wise by attention weights to obtain the output features of the parameterless attention module, as shown in the following formula:

[0079] ;

[0080] in, This represents the value of the output feature of the parameterless attention module at position t.

[0081] Specifically, the fused features at three different scales output by the multi-scale fusion network are input into the corresponding detection heads in the output network, which then output three-dimensional tensors corresponding to large, medium, and small sizes. The 3×3 convolutional modules in the detection heads of the YOLOv7-Tiny network's output network are replaced with 1×1 convolutional modules, and a parameter-free attention (SimAM) module is added before the 1×1 convolutional modules. This reduces feature loss while maintaining certain parameters, thereby improving detection accuracy. The SimAM module is a parameter-free attention mechanism that evaluates the importance of each neuron through an energy function. The energy function calculates the energy value for each position in the input features of the SimAM module; lower energy values ​​indicate greater importance. The energy values ​​are then converted into attention weights through a non-linear transformation; lower energy values ​​result in higher attention weights. Multiplying the input features of the SimAM module by the attention weights yields the enhanced features, i.e., the output features of the SimAM module.

[0082] In the output network, a three-scale collaborative prediction mechanism is employed. Fusion features at different resolutions (typically 80×80, 40×40, and 20×20) are input into the SimAM module. First, the mean μ and variance σ² of the spatial dimension are independently calculated channel by channel. Based on these statistics, an energy function is constructed to calculate the difference between each position on the input feature and the overall distribution. Lower energy values ​​indicate more significant neuron activation and greater information importance. Next, the energy values ​​are mapped back to attention weights using the sigmoid function, automatically assigning high attention weights to low-energy regions. Finally, element-wise enhancement is performed on the original input features, dynamically strengthening key feature regions and suppressing noise without requiring any learnable parameters, outputting the enhanced features. Each feature map has three pre-defined anchor boxes for each grid point. Each detector head uses a lightweight 1×1 convolutional module to directly output the prediction tensor, whose channel dimension includes bounding box coordinates (center offset, width and height scaling values), target confidence (after sigmoid activation), and class probability (using sigmoid multi-label classification). The entire process is computationally efficient while maintaining spatial structural integrity.

[0083] After constructing the smoke and fire recognition model described above, the embodiments of this application use a custom dataset Firesmoke_DataSet and perform relevant data preprocessing operations in accordance with the YOLOv7-Tiny standard, including Mosaic, MixUp, HSV color space perturbation, etc., to construct training data. After training the smoke and fire recognition model with the training data, the trained smoke and fire recognition model is obtained.

[0084] S2: Acquire the image to be detected and input it into the trained smoke and fire recognition model, and output the smoke and fire recognition result, which includes the bounding box coordinates, target confidence and category probability.

[0085] Specifically, the trained smoke and fire recognition model is deployed, and the image to be detected is input into the trained smoke and fire recognition model. Feature extraction and feature fusion are performed by improving the feature extraction network and multi-scale fusion network in the YOLOv7-Tiny network, respectively. Furthermore, the output network outputs the smoke and fire recognition result corresponding to the image to be detected. The smoke and fire recognition result includes the bounding box coordinates of the identified smoke and fire target, the target confidence score, and the corresponding smoke and fire category probability.

[0086] Further reference Figure 5 As an implementation of the methods shown in the above figures, this application provides an embodiment of a lightweight smoke and fire detection device based on multi-scale fusion. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0087] This application provides a lightweight smoke and fire detection device based on multi-scale fusion, comprising:

[0088] Model building module 1 is configured to build and train a smoke and fire recognition model based on the improved YOLOv7-Tiny network to obtain the trained smoke and fire recognition model. The smoke and fire recognition model is based on the YOLOv7-Tiny network, but with the ELAN-TINY module replaced by the ELAN-TPC module. The ELAN-TPC module, in turn, replaces the 3×3 convolutional kernels with partial convolutional modules. The feature fusion network in the YOLOv7-Tiny network is replaced by a multi-scale fusion network. The 3×3 convolutional kernels in the output network of the YOLOv7-Tiny network are replaced with a parameterless attention module and a 1×1 convolutional kernel.

[0089] Prediction module 2 is configured to acquire the image to be detected and input it into the trained smoke and fire recognition model, and output the smoke and fire recognition result, which includes the bounding box coordinates, target confidence and class probability.

[0090] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 6As shown, the electronic device of this embodiment includes a processor 601 and a memory 602; wherein the memory 602 is used to store computer execution instructions; and the processor 601 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0091] Alternatively, the memory 602 can be either standalone or integrated with the processor 601.

[0092] When the memory 602 is set up independently, the electronic device also includes a bus 603 for connecting the memory 602 and the processor 601.

[0093] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by processor 601, implement the above method.

[0094] This invention also provides a computer program product, including a computer program that, when executed by a processor 601, implements the above-described method.

[0095] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0096] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0097] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0098] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 601 to execute some steps of the methods of the various embodiments of this application.

[0099] It should be understood that the processor 601 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 601 can be any conventional processor 601. The steps of the method disclosed in this invention can be directly manifested as the hardware processor 601 executing the steps, or as a combination of hardware and software modules within the processor 601 executing the steps.

[0100] The memory 602 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0101] Bus 603 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 603 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 603 in the accompanying drawings of this application is not limited to only one bus 603 or one type of bus 603.

[0102] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0103] An exemplary storage medium is coupled to a processor 601, enabling the processor 601 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor 601. The processor 601 and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor 601 and the storage medium can exist as discrete components in an electronic device or a host device.

[0104] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight smoke and fire detection method based on multi-scale fusion, characterized in that, Includes the following steps: A smoke and fire detection model based on an improved YOLOv7-Tiny network was constructed and trained to obtain the trained smoke and fire detection model. The model was developed by replacing the ELAN-TINY module in the YOLOv7-Tiny network with an ELAN-TPC module. The ELAN-TPC module, in turn, replaced the 3×3 convolutional kernel module with a partial convolutional module. The feature fusion network in the YOLOv7-Tiny network was replaced with a multi-scale fusion network. Finally, the 3×3 convolutional kernel module in the output network of the YOLOv7-Tiny network was replaced with a parameterless attention module and a 1×1 convolutional kernel module. The image to be detected is acquired and input into the trained smoke and fire recognition model, and the smoke and fire recognition result is output, which includes bounding box coordinates, target confidence and category probability.

2. The lightweight smoke and fire identification method based on multi-scale fusion according to claim 1, characterized in that, The ELAN-TPC module contains two partially convolutional modules connected in sequence; the input features of the partially convolutional modules... and binary mask The calculation process of the aforementioned convolutional module is as follows: ； ； Where W represents the weights of the convolution kernel, b is the bias term, ⊙ represents element-wise multiplication, and Ω is the current convolution window. The total number of pixels covered by the convolution kernel. Coordinates in the current convolution window The number of effective pixels within. This represents the output features of a portion of the convolutional module. This represents the updated binary mask; The output features and updated binary mask of the first partial convolutional module are used as the input features and binary mask of the second partial convolutional module.

3. The lightweight smoke and fire identification method based on multi-scale fusion according to claim 1, characterized in that, The multi-scale fusion network, based on the feature fusion network in the YOLOv7-Tiny network, uses the output features of four ELAN-TPC modules in the feature extraction network as multi-scale input features, and achieves cross-level feature deep fusion through a bidirectional closed-loop path.

4. The lightweight smoke and fire identification method based on multi-scale fusion according to claim 3, characterized in that, The bidirectional closed-loop path includes a top-down propagation path and a bottom-up propagation path. In the top-down propagation path, the second upsampling module in the feature fusion network of the YOLOv7-Tiny network is replaced with a transposed convolutional layer with a kernel size of 2×2. A convolutional layer with a kernel size of 1×1 is added between the second concatenation layer and the second ELAN-TPC module in the multi-scale fusion network. The third concatenation layer in the bottom-up propagation path is replaced with a concatenation layer using BiFPN_Add2. The first ELAN-TPC module in the feature extraction network... The output features of the LAN-TPC module are sequentially passed through a convolutional layer with a kernel size of 1×1 and a convolutional layer with a kernel size of 3×3 in the multi-scale fusion network to obtain the first feature. The output features of the second ELAN-TPC module in the feature extraction network are passed through a convolutional layer with a kernel size of 1×1 in the multi-scale fusion network to obtain the second feature. The first feature, the second feature, and the output features of the transposed convolutional layer are input into the second splicing layer for splicing operation, and then pass through a convolutional layer with a kernel size of 1×1 before entering the bottom-up propagation path.

5. The lightweight smoke and fire identification method based on multi-scale fusion according to claim 1, characterized in that, The calculation process of the parameterless attention module is as follows: The statistics of the input features of the parameterless attention module in the spatial dimension are calculated independently for each channel. The statistics include the mean μ and the variance σ². Based on the aforementioned statistics, the energy value of the input feature of the parameterless attention module at each position is constructed, as shown in the following formula: ； Among them, e t x is the energy value of the input feature of the parameterless attention module at position t. t Let ϵ be the value of the input feature of the parameterless attention module at position t, where ϵ is a minimal constant; The energy values ​​are inversely mapped to attention weights using the sigmoid function, as shown in the following formula: ； Among them, w t The attention weights of the input features of the parameterless attention module at position t. Represents the sigmoid function; The input features of the parameterless attention module are enhanced element-wise using the attention weights to obtain the output features of the parameterless attention module, as shown in the following formula: ； in, This represents the value of the output feature of the parameterless attention module at position t.

6. The lightweight smoke and fire identification method based on multi-scale fusion according to claim 1, characterized in that, The convolutional module with a kernel size of 1×1 includes a convolutional layer with a kernel size of 1×1, a BN layer, and a LeakyReLU activation function layer connected in sequence.

7. A lightweight smoke and fire detection device based on multi-scale fusion, characterized in that, include: The model building module is configured to build and train a smoke and fire recognition model based on an improved YOLOv7-Tiny network to obtain a trained smoke and fire recognition model. The smoke and fire recognition model is based on the YOLOv7-Tiny network, but with the ELAN-TINY module replaced by the ELAN-TPC module. The ELAN-TPC module, in turn, replaces the 3×3 convolutional modules with partial convolutional modules. The feature fusion network in the YOLOv7-Tiny network is replaced by a multi-scale fusion network. The 3×3 convolutional modules in the output network of the YOLOv7-Tiny network are replaced with parameterless attention modules and 1×1 convolutional modules. The prediction module is configured to acquire the image to be detected and input it into the trained smoke and fire recognition model, and output the smoke and fire recognition result, which includes bounding box coordinates, target confidence and category probability.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images