Efficient fire smoke detection method based on lightweight convolution and cross channel feature fusion

Abstract

The invention discloses an efficient fire smoke detection method based on lightweight convolution and cross channel feature fusion. The method belongs to the field of deep learning in computer vision, and comprises the following steps: capturing a video signal by using a camera, segmenting the video signal into independent images, and carrying out image preprocessing; and detecting the images by using a trained fire detection model, and displaying a detection result. According to the method, through lightweight convolution fitting and cross channel feature fusion, the detection precision and the real-time performance of the model are improved; on the basis of a YOLOv11 model, a lightweight convolution structure is adopted to reduce the calculation overhead, and meanwhile, the distinguishing capability of fire smoke features is enhanced through a cross channel feature fusion mechanism, so that the detection robustness and precision of the model in a complex scene are further improved. The improvement provides a new technical path for a fire early warning system.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent medical treatment and relates to an efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion. Background Art

[0002] At present, fire detection technology has received extensive attention as an important research direction in the field of security prevention. Traditional fire detection technology is usually based on sensor data or image processing, and issues early warnings by identifying fire-related features such as temperature, smoke, and flames; however, sensor technology is often limited by the environment. For example, smoke sensors are sensitive to wind speed and humidity and cannot detect fires in a timely and accurate manner; and traditional image-based methods, such as background difference and edge detection, have weak ability to distinguish between changes in lighting, smoke, and other interferences (such as dust and fog) in complex scenes, and have certain false alarm and missed alarm problems.

[0003] At present, the rapid development of deep learning technology, especially convolutional neural networks (CNNs), has provided new solutions for fire detection. The YOLO (You Only Look Once) series of models is a representative example. It can process images in real time through an end-to-end target detection framework and has high detection accuracy. The YOLO model has been widely used in target detection tasks, such as traffic monitoring, security monitoring and other scenarios, and has achieved remarkable results in fire detection tasks.

[0004] However, the existing YOLO model still has certain limitations in feature extraction and model lightweighting in fire detection. First, the YOLO model has limited ability to capture and distinguish smoke and flame features in complex environments in fire detection tasks. Fire smoke is easily confused with interference factors such as conventional fog, shadows or dust, resulting in reduced detection accuracy. In addition, the existing YOLO model is difficult to run in real time on resource-constrained devices (such as drones and embedded systems) due to its complex network structure, large number of parameters and high computational overhead.

[0005] In recent years, most research has focused on achieving higher detection accuracy, but has ignored the problems of large model parameters and computational complexity, high requirements for operating equipment, and difficulty in real-time operation on resource-constrained devices (such as drones and embedded systems). Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to propose an efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion. The lightweight convolutional network improves the operation efficiency of the network by reducing the number of parameters and computational complexity, making it more suitable for real-time fire detection tasks; and the cross-channel feature fusion (Cross-Channel Feature Fusion, CCFM) enhances the model's characterization ability of fire smoke characteristics and improves the detection accuracy by fusing features from different channels; the present invention improves the detection performance to a certain extent, and further improves the lightweight degree of the model and the robustness of feature extraction while ensuring the detection accuracy. This improvement provides a new technical path for the fire warning system.

[0007] The technical solution of the present invention is: the efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion described in the present invention uses a camera to capture video signals, divides them into separate images, and performs image preprocessing; uses a trained fire detection model to detect these images and display the detection results; it specifically includes the following steps:

[0008] Step 1: Collect smoke signals and image data of relevant monitoring scenes, use cameras to receive video stream signals, and capture recorded videos frame by frame;

[0009] Use the annotation tool CVAT to label the images in the dataset and automatically generate the corresponding TXT format annotation files;

[0010] Step 2: Use random scaling, image flipping, and Mosaic data image enhancement techniques to expand the training set and validation set samples;

[0011] The labeled image samples are divided into training set, validation set and test set according to the proportion. The data folder is Datasets, which includes two subfolders, namely Images and Labels. Images stores the training images, and Labels stores the xml files corresponding to the training images. The training set image path is saved in train.txt, and the test set image path is saved in test.txt, which stores the relative paths of the training images and labels.

[0012] Among them, the training set contains 8420 pictures, and the validation set and test set each contain 1915 pictures;

[0013] Step 3: Build an improved YOLOv11 model;

[0014] Build a YOLOv11 network model. The YOLOv11 network includes a feature extraction network, a feature fusion network, and a detection head. The improvements are as follows:

[0015] (1): A new convolutional module DPGConv is designed in the feature extraction network of the original YOLOv11 algorithm to better extract fire smoke features and reduce the number of model parameters;

[0016] The DPGConv convolution module consists of two depthwise convolutions and pointwise convolutions, a GroupNorm and a SiLU activation function;

[0017] DPGConv convolution uses two depthwise convolutions and pointwise convolutions to reduce the number of parameters and computations while maintaining the expressiveness of the model;

[0018] The outputs of the two convolutions are concatenated, normalized using GroupNorm, and finally the feature map is output through the SiLU activation function;

[0019] In depth convolution, each input channel is convolved with a separate filter (kernel); each input channel generates a corresponding output channel;

[0020] Point-wise convolution is a 1×1 convolution operation that convolves all channels of the input at each position;

[0021] Point-by-point convolution can be seen as a convolution operation performed on the channel dimension of the input data without involving spatial information. It is used to linearly combine the feature maps of each channel generated by deep convolution, further reducing the amount of calculation and the number of parameters by reducing the dimension of the input channel, while improving the efficiency and speed of the model.

[0022] While maintaining relatively high performance, significantly reduce the size and computational requirements of the model, using GroupNorm for normalization;

[0023] Compared to BatchNorm, GroupNorm divides the input into multiple groups and normalizes within each group; this normalization method maintains performance when the batch size changes and is more effective for cases with a small number of channels;

[0024] (2): Based on (1), we further improve the model and design a new deep residual spatial self-attention module DRSA (Deep Residual Spatial Attention) module. The DRSA module combines deep convolution and spatial attention mechanisms. It performs spatial transformation on the input feature map by combining horizontal and vertical self-attention mechanisms. The module is integrated into the improved CSP structure C3k2. C3k2 itself integrates multi-scale features through the CSP structure. After combining with DRSA, it can more effectively integrate features of different scales, so that the model can maintain high performance when detecting small smoke or long-distance smoke.

[0025] The DRSA module uses a depthwise separable convolution, the DSCN structure in DSCNPair, which effectively reduces the number of parameters by decomposing the standard convolution into channel-by-channel convolution and 1×1 convolution;

[0026] This structure not only reduces the computational complexity, but also helps to reduce the storage space of the model;

[0027] (3): A new attention mechanism LWSM (LightWeight SimAM) is designed in the C2PSA module in the feature extraction network to replace the original self-attention module to form a new module C2PS_LWSM;

[0028] In the C2PSA module, PSABlock introduces the LWSM attention mechanism. LWSM is a lightweight spatial attention module. SimAM is improved to make it more lightweight. Sigmoid in SimAM is replaced with ReLU because ReLU is simpler to calculate and usually performs better in deep learning. At the same time, some unnecessary intermediate variables are removed, and only necessary calculations are retained.

[0029] This makes the model faster during inference while taking up less memory and computing resources. LWSM is specifically designed to capture the unevenness of smoke distribution in the spatial dimension. Since smoke is usually diffuse, LWSM can help the model focus more accurately on these key tobacco features scattered in different areas. By focusing on feature enhancement at different locations, the spatial changes of smoke can be detected, especially the weaker smoke features of fires.

[0030] (4): In the feature fusion module of YOLOv11, the cross-channel feature fusion module CCFM and the deep residual spatial self-attention module DRSA (Deep Residual Spatial Attention) in RT-DETR are integrated into the neck part of the YOLOv11 model to ensure the accuracy of fire smoke feature extraction and target detection while reducing the number of parameters and computational cost;

[0031] In the neck structure of YOLOv11, there are usually multiple feature fusion layers, which mainly realize the fusion of features of different resolutions through operations such as convolution, upsampling, and Concat;

[0032] The CCFM module replaces some traditional feature fusion layers, allowing the model to efficiently utilize cross-channel information during the fusion process;

[0033] The use of CCFM module also optimizes the overall parameter quantity of the model, so that the fused model can reduce the demand for computing resources while maintaining efficient feature extraction;

[0034] At the same time, the C3k2 module processed by DRSA is inserted into the fusion layer, so that the fused feature map not only has rich cross-channel information, but also further improves the recognition of the target through the spatial attention mechanism;

[0035] By combining the CCFM module and C3k2_DRSA module of RT-DETR, the network's ability to identify fire smoke is enhanced; this not only improves the detection accuracy of the model, but also reduces the parameters and computational overhead;

[0036] (5): During model training, a loss function Powerful-SIoU that combines Powerful-IoU and SIoU is proposed; the Powerful-SIoU loss function uses the power and shape information introduced by Powerful-IoU to enhance the structure of the overlapping area while retaining SIoU's consideration of the box size and ratio;

[0037] It can consider information such as intersection, union, shape, and size at the same time to obtain a more comprehensive loss calculation; Powerful-SIoU can be widely used in target detection models, especially when dealing with targets of different sizes, shapes, and complex backgrounds; By combining the advantages of Powerful-IoU and SIoU, Powerful-SIoU provides more accurate bounding box regression, thereby improving the overall detection performance of the model;

[0038] Step 4: Train the improved YOLOv11 model;

[0039] When collecting the data set used in the model training process, it is necessary to collect more high-quality fire smoke images, smoke images in different environments, different time periods, different concentrations, and different fire conditions; and use different data sets to train the model and evaluate the model's performance.

[0040] The beneficial effects of the present invention are as follows: the improved YOLOv11 model in the present invention significantly enhances the extraction and detection capabilities of fire smoke features in complex environments by introducing a new convolution module (DPGConv) and a deep residual spatial self-attention module (DRSA); in particular, by combining a lightweight spatial attention module LWSM, the model can more accurately identify diffuse and weakened smoke features, reduce the confusion between smoke and conventional fog, shadows or dust, and improve detection accuracy; at the same time, the application of the deep separable convolution (DSCNPair) and DPGConv convolution modules greatly reduces the number of parameters and calculations of the model, while maintaining detection performance, reducing computational complexity; this makes The improved YOLOv11 model can run in real time on resource-constrained devices. In addition, the combination of the DRSA module and the C3k2 structure significantly improves the model's ability to process multi-scale features, making it perform better in long-distance or small smoke detection, and effectively improving its adaptability in actual fire scenes. Through the Powerful-SIoU loss function, the model has also been enhanced in the accuracy of bounding box regression, especially when dealing with targets of different sizes, shapes and complex backgrounds, it can more accurately locate the fire smoke area. This is because the loss function combines the advantages of Powerful-IoU and SIoU, and can fully consider factors such as the intersection, shape, and size of the target.

[0041] The design of the LWSM lightweight module not only improves the reasoning speed of the model, but also reduces the occupation of memory and computing resources, thereby improving resource utilization efficiency; thanks to this improvement, the model can still achieve efficient reasoning and detection on low-computing power devices; the present invention greatly improves the accuracy, efficiency and resource adaptability of fire smoke detection by introducing more advanced feature extraction, attention mechanism and feature fusion strategy, and is suitable for deployment in resource-constrained embedded devices and drone systems for fire monitoring and other scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is an operation flow chart of the present invention;

[0043] Figure 2 It is a diagram of the overall structure of the model in an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of the DPGConv convolution module in an embodiment of the present invention;

[0045] Figure 4 is a schematic diagram of a deep residual space self-attention module DRSA in an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of a C2PS_LWSM module in an embodiment of the present invention;

[0047] Figure 6 It is a schematic diagram of the CCFM_DRSA module in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The specific technical scheme of the present invention is further described in detail below with reference to specific examples.

[0049] As shown in the figure, this embodiment provides an efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion;

[0050] First, the smoke signal is captured using a camera and segmented into individual images;

[0051] Next, the collected fire smoke images are data enhanced and preprocessed to provide the best input for the subsequent fire smoke detection model; the trained fire smoke detection model is used to detect flames and smoke in these images, and the detection results are annotated on the images;

[0052] Specifically, the fire detection model is as follows: YOLOv11n is used as the basic model, and YOLOv11 includes a feature extraction network and a feature fusion network; the feature extraction network adopts a lighter DPGConv module, and designs a new deep residual spatial self-attention module DRSA (Deep Residual Spatial Attention) module, and a new attention mechanism LWSM (LightweightSimAM) is designed in the C2PSA module; in the feature fusion module of YOLOv11, the cross-channel feature fusion module CCFM and the deep residual spatial self-attention module DRSA (Deep Residual Spatial Attention) module in RT-DETR are integrated to ensure the accuracy of fire smoke feature extraction and target detection while reducing the number of parameters and computational costs; Figure 1 As shown, the method specifically comprises the following steps:

[0053] Step 1: Collect smoke signals and image data for relevant monitoring scenes;

[0054] Use the camera to receive the video stream signal and capture the recorded video frame by frame; use the annotation tool CVAT to label the images in the dataset and automatically generate the corresponding TXT format annotation file;

[0055] Step 2: Perform data enhancement and preprocessing on the collected fire smoke images;

[0056] Random scaling, image flipping, and Mosaic data augmentation techniques are used to expand the training and validation set samples;

[0057] The labeled image samples are divided into training set, validation set and test set according to the proportion. The data folder is Datasets, which includes two subfolders, namely Images and Labels. Images stores the training images, and Labels stores the xml files corresponding to the training images. The training set image path is saved in train.txt, and the test set image path is saved in test.txt, which stores the relative paths of the training images and labels.

[0058] Among them, the training set contains 8420 pictures, and the validation set and test set each contain 1915 pictures;

[0059] Step 3: Build an improved YOLOv11 model;

[0060] Build the YOLOv11 network model. The YOLOv11 network includes feature extraction network, feature fusion network, and detection head. The improved model is as follows Figure 2 The improvements are as follows:

[0061] (1): A new convolution module DPGConv is designed in the feature extraction network of the original YOLOv11 algorithm to better extract fire smoke features and reduce the number of model parameters; the DPGConv convolution module consists of two depthwise convolutions and pointwise convolutions, a GroupNorm and SiLU activation function; Figure 3As shown in the figure, DPGConv convolution uses two deep convolutions and point-by-point convolutions to reduce the number of parameters and calculations while maintaining the expressiveness of the model; the outputs of the two convolutions are concatenated, normalized using GroupNorm, and finally the feature map is output after the SiLU activation function; in deep convolution, each input channel is convolved with a separate filter (kernel); each input channel generates a corresponding output channel; point-by-point convolution is a 1×1 convolution operation that convolves all channels of the input at each position; point-by-point convolution can be regarded as a convolution operation performed on the channel dimension of the input data without involving spatial information; it is used to linearly combine the feature maps of each channel generated by the deep convolution, further reducing the amount of calculation and the number of parameters by reducing the dimension of the input channel, while improving the efficiency and speed of the model; while maintaining relatively high performance, it significantly reduces the size and computational requirements of the model; normalization is performed using GroupNorm; compared to BatchNorm, GroupNorm divides the input into multiple groups and normalizes within each group; this normalization method maintains performance when the batch size changes, and is more effective for cases with a small number of channels;

[0062] The specific operations are as follows:

[0063] Depthwise convolution is a special convolution operation in which the convolution kernel operates independently on each input channel instead of performing convolution across channels like standard convolution. The benefit of this is that the amount of computation and parameters is greatly reduced.

[0064] For the input feature map

[0065] Where H and W are the height and width of the feature map, respectively, and C is the number of input channels;

[0066] The operation of depth convolution is expressed as:

[0067] Y (c) =X (c) *K (c) (1)

[0068] Where Y (c) Represents the feature map of the output channel, X (c) represents the cth channel of the input, K (c) Represents the corresponding depth convolution kernel; the size of the depth convolution kernel is usually k×k, and each convolution kernel only acts on a single channel;

[0069] The output dimension after depth convolution is

[0070] Point-by-point convolution is a 1×1 convolution operation, which performs a linear combination of the output feature maps of the depth convolution in the channel dimension, thereby reducing the channel dimension and improving the feature expression.

[0071] Point-wise convolution is expressed as:

[0072] Z=W*Y (2)

[0073] In the formula, represents the output of point-wise convolution, Represents the weight matrix of point-by-point convolution, C ′ Indicates the number of output channels;

[0074] The output dimension after point-by-point convolution becomes

[0075] In the DPGConv module, two depthwise convolution and pointwise convolution operations are performed, and an output feature map is obtained after each convolution; then the results of the two convolution outputs are concatenated in the channel dimension:

[0076] Z concat =Concat(Z1,Z2) (3)

[0077] In the formula, Z1 and Z2 represent the output of two convolutions, and the dimension after splicing is

[0078] After concatenation, the output is normalized. Compared with BatchNorm, GroupNorm is more stable on small batches of data. It divides the feature channels into several groups and normalizes them within the groups.

[0079] The calculation formula of GroupNorm is as follows:

[0080]

[0081] In the formula, represents the i-th value of the g-th group of input features, μ g and Respectively represent the mean and variance of the group, ∈ represents a small constant used to prevent division by zero errors;

[0082] The benefit of GroupNorm is that it does not depend on the batch size and performs better when there are fewer channels;

[0083] The normalized output is transformed nonlinearly using the SiLU activation function (Sigmoid-Weighted Linear Unit):

[0084] SiLU(x)=x·σ(x) (5)

[0085] Where σ(x) represents the Sigmoid function, defined as SiLU combines the linearity of ReLU and the smoothness of Sigmoid, which can accelerate convergence and improve model performance.

[0086] (2): Based on (1), the model is further improved. A new deep residual spatial self-attention module DRSA (Deep Residual Spatial Attention) is designed on the backbone network of YOLOv11. The DRSA module combines deep convolution and spatial attention mechanisms. It performs spatial transformation on the input feature map by combining horizontal and vertical self-attention mechanisms. The module is integrated into the improved CSP structure C3k2. C3k2 itself integrates multi-scale features through the CSP structure. After combining with DRSA, features of different scales can be more effectively integrated, so that the model can maintain high performance when detecting small smoke or long-distance smoke.

[0087] The DRSA module uses a depthwise separable convolution, the DSCN structure in DSCNPair, which effectively reduces the number of parameters by decomposing the standard convolution into channel-by-channel convolution and 1×1 convolution. This structure not only reduces the computational complexity, but also helps to reduce the storage space of the model. Figure 4 As shown in the figure, before starting the processing, the clone() method is used to save the feature x of the input feature map; the input feature undergoes a 1×1 convolution, adjusts the number of channels and performs a linear transformation; then the GELU activation function is applied to increase the nonlinear expression capability; the input feature x is reduced in dimension and nonlinearly activated by linear projection:

[0088] x′=GELU(W1x) (6)

[0089] In the formula, W1 represents the weight matrix, GELU represents the activation function;

[0090] Then, through the DSCN structure in DSCNPair, deep convolution and spatial transformation are realized, and the spatial self-attention mechanism is performed to obtain the weighted feature x″:

[0091] x″=Spatial Self-Attention(x′) (7)

[0092] After another 1×1 convolution, the feature map is further processed; finally, a residual connection is used to improve the deep feature extraction capability, and the processed feature map is added element by element to the originally saved input shortcut to retain the original features:

[0093] y=W2x″+x (8)

[0094] Where W2 represents another linear projection weight matrix, and y represents the final output of the module;

[0095] (3): A new attention mechanism LWSM (LightWeight SimAM) is designed in the C2PSA module in the feature extraction network to replace the original self-attention module to form a new module C2PS_LWSM; Figure 5 As shown in the figure, SimAM is improved to make it lighter. Sigmoid in SimAM is replaced with ReLU, because ReLU is simpler to calculate and usually performs better in deep learning. At the same time, some unnecessary intermediate variables are removed, and only necessary calculations are retained. This makes the model faster during inference and occupies less memory and computing resources.

[0096] First, the image captured by the fire monitoring camera is input into the neural network. After a 1×1 convolution operation, the number of channels of the feature map is adjusted while maintaining efficient calculation and retaining the main information in the image.

[0097] The input feature map is then processed by multiple PSABlock modules. PSABlock introduces the LWSM attention mechanism, which is a lightweight spatial attention module specifically designed to capture the uneven distribution of smoke in the spatial dimension. Since smoke is usually diffuse, LWSM can help the model focus more accurately on these key tobacco features scattered in different areas. By focusing on feature enhancement at different locations, the spatial changes of smoke can be detected, especially the weaker smoke features of fires.

[0098] The specific steps of LWSM are as follows:

[0099] Assume that the shape of the input feature is C×H×W, that is, there are C channels and the spatial dimension of each channel is H×W;

[0100] In order to measure the overall distribution of each channel feature, the features are first statistically operated to calculate the mean and variance;

[0101] Mean calculation: For the input feature X of the cth channel c , and its mean is:

[0102]

[0103] In the formula, μ c Represents the average value of all pixels on the cth channel;

[0104] Variance calculation: The variance of the cth channel is:

[0105]

[0106] The variance measures the spread of the eigenvalues ​​relative to the mean;

[0107] Next, the sparse representation of each channel is calculated based on the deviation of the feature from the mean and the variance; for each pixel X c (i,j), the sparsity is expressed as:

[0108]

[0109] In the formula, ∈ represents a very small positive number to prevent division by 0;

[0110] This formula measures the degree of deviation of each pixel from the mean and normalizes it by the variance; the larger the variance, the more dramatic the change in features in the channel, and the smaller the impact of the deviation on the final result;

[0111] In the original SimAM, Sigmoid is used to normalize the sparse representation so that the attention value is between [0,1];

[0112] However, Sigmoid has a high computational overhead, so it can be replaced by the ReLU function; ReLU is a simple threshold activation function defined as follows:

[0113] A c (i,j)=ReLU(S c (i,j))=max(0,S c (i,j)) (12)

[0114] This means that only when the sparsity S c When (i,j) is greater than 0, the activation value A c (i,j) will be non-zero, further enhancing important features and sparse unimportant features;

[0115] Compared with Sigmoid, ReLU requires less computation because it does not involve complex exponential operations, but only simple comparison operations.

[0116] Finally, use the generated attention weight A c (i, j) to adjust the original input features, similar to the operation in the self-attention mechanism;

[0117] Output feature Y c (i,j) is:

[0118] Y c (i,j)=X c (i,j)·A c (i,j) (13)

[0119] In the formula, A c (i, j) represents the attention weight of each pixel, which is obtained based on the previous ReLU activation; this formula represents the attention weight of the input feature X c (i, j) is weighted so that important features (i.e. pixels with high sparsity) are enhanced and unimportant features are suppressed;

[0120] After the above steps, the generated output feature Y maintains the same shape as the input feature X, but the value of each pixel has been reweighted to highlight the part with higher sparsity in the feature;

[0121] Then, the feature maps processed by multiple PSABlocks are spliced ​​together to fuse the low-level and high-level feature information. This ensures that the model not only pays attention to local details, but also takes into account the overall fire smoke characteristics, ensuring the comprehensive collection of smoke shape, edge, concentration, etc.; while ensuring efficient calculation, the smoke characteristics at the fire scene are accurately identified; after being processed by multiple convolutions and attention mechanisms, the final output feature map will be passed to the feature fusion module to fuse the fire smoke features;

[0122] (4): In the feature fusion module of YOLOv11, the cross-channel feature fusion module CCFM and the deep residual spatial self-attention module DRSA (Deep Residual Spatial Attention) in RT-DETR are integrated into the neck part of the YOLOv11 model to ensure the accuracy of fire smoke feature extraction and target detection while reducing the number of parameters and computational cost;

[0123] Specifically, in the feature extraction network architecture, the second and third C3k2_DRSA modules were fine-tuned in structure, and new convolutional layers were introduced respectively. These convolutional layers serve as new input layers for the Concat module in the Neck layer.

[0124] In addition, for the C2PS_LWSM module, a convolution layer Conv is further added as the input layer of the Neck layer upsampling module;

[0125] Before the second upsampling process of the feature pyramid, a new convolution layer Conv is introduced again to provide richer feature information for subsequent processing;

[0126] Cross-fusion is performed on the feature map channel dimension so that feature maps at different levels can share information, thereby improving the network's sensitivity to fire smoke characteristics;

[0127] The DRSA module (Deep Residual Spatial Attention) further improves the feature map's ability to focus on the spatial dimension by introducing deep residual connections and spatial attention mechanisms;

[0128] This module can enhance the salient areas related to fire smoke in the feature map while suppressing irrelevant background noise; the DRSA module is embedded in the C3k2 module, which itself consists of 3 layers of convolution with a convolution kernel of k=2 and contains residual connections;

[0129] By combining with the DRSA module, the model's spatial attention to the target in complex scenes is improved; the optimization and integration of the above structures together form a new module CCFM_DRSA, such as Figure 6 As shown in the figure, the neck structure of YOLOv11 usually contains multiple feature fusion layers, which mainly realize the fusion of features of different resolutions through operations such as convolution, upsampling, and Concat. The CCFM module replaces some traditional feature fusion layers, so that the model can efficiently use cross-channel information during the fusion process.

[0130] The use of CCFM module also optimizes the overall parameter quantity of the model, so that the fused model can reduce the demand for computing resources while maintaining efficient feature extraction; at the same time, the C3k2 module processed by DRSA is inserted into the fusion layer, so that the fused feature map not only has rich cross-channel information, but also further improves the recognition of the target through the spatial attention mechanism; by combining the CCFM module and C3k2_DRSA module of RT-DETR, the network's ability to recognize fire smoke is enhanced; not only the detection accuracy of the model is improved, but also the parameters and computing overhead are reduced;

[0131] (5): During model training, a loss function Powerful-SIoU is proposed that combines Powerful-IoU and SIoU. SIoU improves the accuracy of bounding box regression by considering the scaling and overlap of the target box, especially when dealing with targets of different sizes and shapes.

[0132] The SIoU boundary loss function expression is:

[0133]

[0134] In the formula, A inter Represents the intersection area of ​​the predicted box and the true box; A union Represents the union area of ​​the predicted box and the true box; λ represents a hyperparameter that controls the impact of scale; scale(p,g) represents a scaling factor used to consider the size and shape of the box; ∈ represents avoiding small values ​​of zero;

[0135] Powerful-IoU enhances the sensitivity to overlapping areas and improves the accuracy of bounding box regression by introducing power and shape information, especially in small object detection.

[0136] The calculation formula of Powerful-IoU is as follows:

[0137]

[0138] In the formula, A inter Represents the intersection area of ​​the predicted box and the true box; A union Represents the union area of ​​the predicted box and the true box; shape(p,g) represents the function used to calculate the shape difference between the predicted box and the true box;

[0139] Combining the advantages of Powerful-IoU and SIoU, the bounding box regression performance in the target detection task is improved; the Powerful-SIoU loss function formula is as follows:

[0140]

[0141] In the formula, A inter Represents the intersection area of ​​the predicted box and the true box; A union Represents the union area of ​​the predicted box and the true box; α represents the hyperparameter controlling the IoU power; β represents the weight hyperparameter controlling the shape information; shape(p,g) represents the function used to calculate the shape difference between the predicted box and the true box; λ represents the hyperparameter controlling the scale effect; scale(p,g) represents the scaling factor used to consider the size and shape of the box; ∈ represents avoiding small values ​​of zero;

[0142] The Powerful-SIoU loss function uses the power and shape information introduced by Powerful-IoU to enhance the structure of overlapping areas while retaining SIoU's consideration of box size and proportion; it can simultaneously consider information such as intersection, union, shape, and size to obtain a more comprehensive loss calculation; Powerful-SIoU can be widely used in target detection models, especially when dealing with targets of different sizes, shapes, and complex backgrounds;

[0143] By combining the advantages of Powerful-IoU and SIoU, Powerful-SIoU aims to provide more accurate bounding box regression, thereby improving the overall detection performance of the model;

[0144] Step 4: Train the improved YOLOv11 model:

[0145] When collecting the data set used in the model training process, it is necessary to collect more high-quality fire smoke images, smoke images in different environments, different time periods, different concentrations, and different fire conditions; and use different data sets to train the model and evaluate the model performance; model performance evaluation includes precision, mean average precision (mAP), etc.

[0146] The accuracy of the target detected by the model is the precision rate, which is defined as the proportion of correctly predicted positive samples to all predicted positive samples:

[0147]

[0148] In the formula, Recall represents the recall rate, TP represents the number of samples correctly predicted as positive samples, FP represents the number of negative samples in the sample that are incorrectly predicted as positive samples; n represents the number of categories, AP i represents the average precision (AP) of the i-th category;

[0149] Input the training set image data in the fire data set into the improved network model in step 3 for training, save the model parameters with the highest accuracy on the validation set during the training process, and name the file best.pt;

[0150] In the present invention, the experimental environment is configured to use the Ubuntu 20.04 operating system, use the NVIDIA GeForce RTX4060Ti graphics card, and the deep learning framework is PyTorch; the specific configuration is shown in Table 1 below:

[0151] Table 1 Experimental environment configuration table

[0152]

[0153] Input the training set image data in the fire data set into the improved network model, use GPU training, set the parameters of the improved network model, the number of training rounds is 300, the momentum is 0.937, the initial learning rate is 0.01, the minimum learning rate is 0.0001, the weight decay coefficient is 0.00005, and the batch-size is 16; during the model training process, the model parameters with the best accuracy will be saved and named best.pt;

[0154] After the model training was completed, the validation set test showed that its validation set accuracy reached 75.3%, mAP reached 73%, the model size was 3.1M, the number of parameters was 1417561, and the amount of calculation was 4.8G;

[0155] Compared with the original YOLOv11, the mAP of the present invention is only improved by 1%, the model size is reduced by 42.6%, the number of parameters is reduced by 46.2%, and the computational amount GFLOPs is reduced by 1.7 points; the indicator comparison table is shown in Table 2, which proves the feasibility of the present invention.

[0156] Table 2 Comparison of experimental results of different models

[0157] Model mAP Recall Precision Model size Parameter quantity GFLOPS YOLOv11n 0.72 0.624 0.749 5.4 2590425 6.5 Ours 0.73 0.625 0.753 3.1 1417561 4.8

[0158] The present invention is suitable for fire smoke detection on mobile devices; due to its lightweight design, it not only has good feature extraction capability, but also has a small model size, small calculation amount, strong real-time performance, and improved calculation efficiency, which enables it to realize real-time or near real-time fire smoke detection on mobile devices, meet the requirements of deploying efficient fire smoke detection, provide a technical basis for fire smoke detection, and provide a convenient and efficient solution for fire prevention and control, which has certain scientific and practical significance.

[0159] The present invention aims to improve the detection accuracy and real-time performance of the model through lightweight convolution fitting and cross-channel feature fusion. Based on the YOLOv11 model, a lightweight convolution structure is adopted to reduce the computational overhead. At the same time, the cross-channel feature fusion mechanism is used to enhance the ability to distinguish fire smoke characteristics, further improving the detection robustness and accuracy of the model in complex scenarios.

Claims

1. An efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion, characterized in that: Use a camera to capture video signals, segment them into separate images, and perform image preprocessing; use the trained fire detection model to detect these images and display the detection results.

2. According to claim 1, the efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion is characterized in that: The operation steps are as follows: Step (1): Collect smoke signals and image data of relevant monitoring scenes, use a camera to receive video stream signals, and capture the recorded video frame by frame; Use the annotation tool CVAT to label the images in the dataset and automatically generate the corresponding TXT format annotation files; Step (2): Use random scaling, image flipping and Mosaic data image enhancement techniques to expand the training set and validation set samples; Step (3): Build an improved YOLOv11 model; Step (4): Train the improved YOLOv11 model.

3. The efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion according to claim 2 is characterized in that: In step (2), the labeled image samples are divided into training set, validation set and test set according to the proportion. The data folder is Datasets, which includes two subfolders, namely Images and Labels. Images stores the training images, and Labels stores the xml files corresponding to the training images. The training set image path is saved in train.txt, and the test set image path is saved in test.txt, which stores the relative paths of the training images and labels. The training set contains 8420 images, and the validation set and test set each contain 1915 images.

4. The efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion according to claim 2 is characterized in that: In step (3), the construction of the improved YOLOv11 model is specifically: The YOLOv11 network includes a feature extraction network, a feature fusion network and a detection head; the improvement steps are as follows: (3.1): Design a new convolution module DPGConv in the feature extraction network of the original YOLOv11 algorithm to extract fire smoke features and reduce the number of model parameters; (3.2): Based on step (3.1), the model is further improved and a new deep residual space self-attention module DRSA module is designed; (3.3): A new attention mechanism LWSM is designed in the C2PSA module in the feature extraction network to replace the original self-attention module to form a new module C2PS_LWSM; (3.4): In the feature fusion module of YOLOv11, the cross-channel feature fusion module CCFM and the deep residual space self-attention module DRSA module in RT-DETR are integrated into the neck part of the YOLOv11 model to ensure the accuracy of fire smoke feature extraction and target detection while reducing the number of parameters and computational cost; (3.5): During model training, a loss function Powerful-SioU is proposed that combines Powerful-IoU and SIoU.

5. The efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion according to claim 4 is characterized in that: In step (3.1), the DPGConv convolution module consists of two depthwise convolutions and pointwise convolutions, a GroupNorm and SiLU activation function; The DPGConv convolution uses two depth convolutions and point-by-point convolutions to reduce the number of parameters and calculations while maintaining the expressiveness of the model; the outputs of the two convolutions are concatenated, normalized using GroupNorm, and finally output a feature map through a SiLU activation function; In depthwise convolution, each input channel is convolved with a separate filter; each input channel generates a corresponding output channel; The point-by-point convolution is a 1×1 convolution operation that convolves all channels of the input at each position; Point-by-point convolution is regarded as a convolution operation performed on the channel dimension of the input data, which is used to linearly combine the feature maps of each channel generated by the deep convolution, thereby reducing the amount of calculation and the number of parameters by reducing the dimension of the input channel; While maintaining relatively high performance, GroupNorm is used to reduce the size and computational requirements of the model.

6. The efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion according to claim 4 is characterized in that: In step (3.2), the DRSA module combines deep convolution and spatial attention mechanisms, and spatially transforms the input feature map by combining horizontal and vertical self-attention mechanisms. The module is integrated into the improved CSP structure C3k2. C3k2 itself integrates multi-scale features through the CSP structure. After combining with DRSA, features of different scales are fused, so that the model maintains high performance when detecting small smoke or long-distance smoke. In addition, the DRSA module adopts depthwise separable convolution and the DSCN structure in DSCNPair to reduce the number of parameters by decomposing the standard convolution into channel-by-channel convolution and 1×1 convolution.

7. The efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion according to claim 4 is characterized in that: In step (3.3), the PSABlock in the C2PSA module introduces the LWSM attention mechanism. LWSM is a lightweight spatial attention module that improves SimAM to make it lightweight, replacing the Sigmoid in SimAM with ReLU. LWSM is used to capture the unevenness of smoke distribution in the spatial dimension, helping the model to focus on key tobacco features scattered in different areas. By focusing on feature enhancement at different locations, the spatial variation of smoke can be detected.

8. The efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion according to claim 4 is characterized in that: In step (3.4), the neck structure of YOLOv11 contains multiple feature fusion layers, which realize the fusion of features of different resolutions through convolution, upsampling, and Concat operations; The CCFM module replaces some traditional feature fusion layers. The use of the CCFM module optimizes the overall parameter quantity of the model. At the same time, the C3k2 module processed by DRSA is inserted into the fusion layer, so that the fused feature map has cross-channel information and improves the recognition of the target through the spatial attention mechanism; By combining the CCFM module and C3k2_DRSA module of RT-DETR, the network's ability to recognize fire smoke is enhanced.

9. The efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion according to claim 4 is characterized in that: In step (3.4), the Powerful-SIoU loss function uses the power and shape information introduced by Powerful-IoU to enhance the structure of the overlapping area while retaining SIoU's consideration of box size and scale; Consider information on intersection, union, shape, and size at the same time to obtain a more comprehensive loss calculation; By combining the advantages of Powerful-IoU and SIoU, Powerful-SIoU provides bounding box regression and improves the overall detection performance of the model.

10. The efficient fire smoke detection method based on lightweight convolution and cross-channel feature fusion according to claim 4 is characterized in that: In step (4), when collecting the data set used in the model training process, it is necessary to collect more fire smoke images, smoke images under different environments, different time periods, different concentrations, and different fire conditions; and use different data sets to train the model and evaluate the model's performance.

Images