Fire detection method for embedded equipment
By performing multi-scale optimization and pruning cutting Yolov8-N models, the EMG-YOLO model is obtained, and the problem of low operation efficiency of the existing fire detection model on low-power embedded equipment has been solved. Effect.
Patent Information
- Application Number
- CN202411358548.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2025-01-28
AI Technical Summary
The existing fire detection model is difficult to operate efficiently on low-power embedded devices, the multi-scale feature fusion is insufficient and the optimization effect is limited, which limits its application in urban environments.
By optimizing the Yolov8-N model, multi-scale attention modules, high-efficiency multi-scale convolutional modules, and broad-featured feature pyramid networks are introduced, and the Slimming pruning algorithm is used to obtain the EMG-YOLO model. This model achieves high -precision fire detection on embedded devices and improves the detection speed.
In the urban environment, the EMG-YOLO model can maintain high fire detection accuracy and significantly improve the detection speed, which is suitable for fast and real-time fire response on embedded devices.
Smart Images

Figure CN119380162A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence detection technology, and in particular to a fire detection method for embedded devices. Background Art
[0002] With the development of Internet of Things (IoT) and Artificial Intelligence (AI) technologies, especially in the field of fire detection in smart cities, more and more edge embedded devices are connected to the network to provide more efficient services and a safer living environment. Traditional fire detection methods mainly rely on photodetectors, ion detectors, thermocouples, and infrared sensors to identify smoke particles and flame spectral characteristics in the air. Although these methods are mature, they are usually limited by environmental conditions and are easily interfered by external factors, resulting in high false alarm rates and untimely responses. With the development of target detection technology in deep learning, fire detection methods based on computer vision have gradually become a more powerful and effective alternative.
[0003] In the prior art, the existing invention patent application document "A fire detection method and system" with publication number CN118397784A proposes a new fire detection method and system, which realizes fire detection and identification by combining an improved YOLOv5s lightweight target detection model with an optical flow model. The core of this method is to use the lightweight YOLOv5s model to analyze the input preset data set to obtain fire identification information, fire location and fire confidence. When the fire confidence is lower than the set threshold, the optical flow model is further introduced to estimate the inter-frame motion information to identify potential suspicious fire features, and the fire border and confidence are adaptively adjusted to improve the detection accuracy. Although this method enhances the robustness of fire detection to a certain extent, especially in dynamic scenes, it can reduce false alarms and missed alarms, but there are still some shortcomings. Although this invention patent provides a fire detection method combined with an optical flow model, its shortcomings in computational complexity, applicability of the optical flow model and multi-scale feature utilization limit its application effect on resource-constrained embedded devices, and further optimization and improvement are needed.
[0004] The existing invention patent application document "A YOLOv8 Lightweight Fire Recognition Method Based on Smoke Enhancement Algorithm" with publication number CN118379625A proposes a lightweight model design method for forest fire detection, which is suitable for the deployment of edge computing devices. The invention collects real-time screen images through lookouts, drones, etc., and inputs them into the smoke visibility enhancement module for data preprocessing, and then uses the YOLO-DE-Tiny model to perform target detection tasks. The method adopts group convolution technology, which significantly reduces the number of parameters and calculations of the backbone network, thereby reducing the complexity and computational cost of the model. In addition, the invention introduces the Enhanced-SPPF enhanced fast pooling module to improve the feature extraction and expression capabilities of the deep backbone network; at the same time, the GD mechanism is adopted to improve the exchange efficiency of feature information. In order to maintain high accuracy in foggy environments, the method also combines dark channel prior defogging technology to improve the clarity of foggy images, and enhances the robustness of smoke detection by smoke color enhancement. Although this technology shows high accuracy and good real-time performance in smoke detection of distant targets and foggy environments, and is suitable for deployment in forest fire smoke detection systems, it still has certain defects. First, this method is highly dependent on the smoke visibility enhancement module and defogging technology, which increases the complexity of the system to a certain extent. Especially under unfavorable environmental conditions, the defogging and color enhancement algorithms may not guarantee the optimal processing effect. Secondly, although the grouped convolution and fast pooling modules reduce the amount of calculation, they may lead to insufficient feature extraction integrity and detection accuracy when dealing with complex backgrounds or multi-target detection scenarios.
[0005] The existing invention patent application document "A Lightweight Target Learning Detection Method Based on Improved YOLOv8" with publication number CN 118212588A proposes an improved YOLOv8 lightweight target detection method, which is specifically used for fire detection tasks and is particularly suitable for the deployment of mobile devices. The method first collects the fire smoke images in the scene, generates a fire detection data set, and divides the data set into a training set, a test set, and a validation set according to a preset ratio. Subsequently, the YOLO detection model is optimized by introducing the MVB (multi-view enhancement) module, and a lightweight mobile device target detection model based on improved YOLOv8 is constructed. During the model training process, the moving exponential average (EMA) is used to smooth the parameters obtained by training to obtain the average value of the parameters to improve the stability and generalization ability of the model. Finally, the optimal fire detection model is selected through the performance evaluation index, and the model is used to detect the images in the test set, and the detection bounding box with the highest confidence is output to achieve high-accuracy fire image detection. Although this technology reduces the amount of calculation and model parameters in terms of lightweight processing, when dealing with large-scale scenes or complex fire characteristics, it is difficult for the model to fully capture all key information, which will lead to a decrease in detection accuracy.
[0006] However, although these deep learning-based models can significantly improve the accuracy of fire detection, the increase in accuracy also consumes too much computing resources, limiting their deployment and application on low-power embedded devices. In particular, with the growing number of edge embedded devices, the existing fire detection models are difficult to run efficiently on embedded devices, and the multi-scale feature fusion is insufficient and the optimization effect is limited, which has become a problem that needs to be solved. Therefore, a high-precision fire detection method that occupies very little computing resources of embedded devices is urgently needed. Summary of the invention
[0007] The object of the present invention is to provide a fire detection method for embedded devices to at least solve the above technical problems.
[0008] To achieve the above purpose, the technical solution adopted by the present invention is as follows:
[0009] A fire detection method for embedded devices, comprising the following steps:
[0010] S1. Obtain fire images in urban scenes and construct a training dataset;
[0011] S2. Optimize the structure of the YOLOv8-n model to obtain the EMG-YOLO model;
[0012] S3, train the EMG-YOLO model using the training data set;
[0013] S4. Deploy the trained EMG-YOLO model to the embedded device, then detect the fire image to be detected and output the detection results.
[0014] In some embodiments of the present invention, in S1, a web crawler is used to obtain fire images in urban scenes.
[0015] In some embodiments of the present invention, in S2, the structural optimization includes: introducing a multi-scale attention module and embedding it into the end of the YOLOv8-n model backbone network; using an efficient multi-scale convolution module to construct a C2f′ module and replace the original C2f module in the YOLOv8-n model backbone network; using a generalized feature pyramid network to replace FPN and PAN in the neck network of the YOLOv8-n model; and using a slimming pruning algorithm to prune the YOLOv8-n model to obtain an EMG-YOLO model.
[0016] In some embodiments of the present invention, the multi-scale attention module includes a trunk convolution layer and multiple parallel depth-separable convolution branches; the trunk convolution layer includes a first DWConv module, a first Conv module, a MaxPool module, an AvgPool module and a Sigmoid module, and the multiple parallel depth-separable convolution branches include multiple parallel second DWConv modules and multiple parallel third DWConv modules located between the first DWConv module and the first Conv module;
[0017] The working steps of the multi-scale attention module are as follows: the input is firstly subjected to preliminary feature extraction by the first DWConv module, the extracted features are sent to multiple parallel second DWConv modules for multi-scale feature extraction, and then sent to multiple parallel third DWConv modules for processing, the processing results are spliced with the features extracted by the first DWConv module, and the spliced results are sent to the first Conv module for processing to obtain transition features, the transition features are multiplied with the features extracted by the first DWConv module, and the multiplication results are sent to the MaxPool module and the AvgPool module for processing, and finally the output is obtained through the Sigmoid module.
[0018] In some embodiments of the present invention, the first DWConv module is a depth-separable convolution of size 5x 5, the first Conv module is a convolution of size 1x 1, the MaxPool module is a maximum pooling layer, the AvgPool module is an average pooling layer, and the Sigmoid module is an activation function; there are three of the multiple parallel second DWConv modules, namely, a depth-separable convolution of size 1x 7, a depth-separable convolution of size 1x 11, and a depth-separable convolution of size 1x 21; there are three of the multiple parallel third DWConv modules, namely, a depth-separable convolution of size 7x 1, a depth-separable convolution of size 11x 1, and a depth-separable convolution of size 21x 1.
[0019] In some embodiments of the present invention, the C2f′ module includes a second Conv module, a split module, a plurality of parallel efficient multi-scale convolution modules, a Concat module and a third Conv module;
[0020] The working steps of the C2f′ module are as follows: the input is passed through the second Conv module to adjust the number of channels, and then is divided into two branches by the split module, one of which is directly passed to the output, and the other is processed by multiple efficient multi-scale convolution modules;
[0021] The efficient multi-scale convolution module includes a fourth Conv module, a split module, multiple parallel fifth Conv modules, a Concat module and a MAM module;
[0022] The working steps of the efficient multi-scale convolution module are as follows: the input is passed through the fourth Conv module to adjust the number of channels, then the split module performs a segmentation operation, and then sent to multiple parallel fifth Conv modules for processing, the processing results are connected in the Concat module, and finally the output is obtained through the MAM module.
[0023] In some embodiments of the present invention, the second Conv module and the third Conv module are both 1x1 convolutions; the fourth Conv module is 1x1 convolution; there are four of the multiple parallel fifth Conv modules, namely, 1x1 convolution, 3x3 convolution, 5x5 convolution and 7x7 convolution.
[0024] In some embodiments of the present invention, the generalized feature pyramid network includes multiple parallel first CSPStage feature fusion modules and multiple parallel second CSPStage feature fusion modules; the first CSPStage feature fusion module and the second CSPStage feature fusion module have the same structure, both including a first convolution branch, a second convolution branch and a Concat module, the first convolution branch is a 1x 1 convolution, the second convolution branch is a 1x 1 convolution and N concatenated convolution modules, each convolution module includes a 3x 3 RepConv convolution module and a 1x 1 convolution; the RepConv convolution module includes two parallel sixth Conv modules and two parallel BN modules in the model training stage, the two parallel sixth Conv modules are 1x 1 convolution and 3x 3 convolution respectively; the RepConv convolution module includes a seventh Conv module and a BN module in the inference stage, the seventh Conv module is a 3x 3 convolution;
[0025] The working steps of the generalized feature pyramid network are as follows: the input first enters multiple parallel first CSPStage modules for preliminary feature extraction and fusion of multi-scale information, the output of each first CSPStage module is then sent to other first CSPStage modules except the current first CSPStage module and multiple parallel second CSPStage modules for further extraction and fusion, a part of the features processed by each second CSPStage module is directly output, and the other part is also sent to other second CSPStage modules except the current second CSPStage module for processing and then output;
[0026] After the input enters the first CSPStage module or the second CSPStage module, it is divided into two parts and then sent to the first convolution branch and the second convolution branch respectively: the features sent to the first convolution branch are processed by a 1x1 convolution; the features sent to the second convolution branch are processed by a 1x1 convolution and N series-connected convolution modules, and the features processed by the 1x1 convolution of the second convolution branch are spliced with the features processed by the convolution module, and the features processed by the first convolution branch and the second convolution branch are spliced in the Concat module; after the input features enter the RepConv module, they first enter the sixth Conv module or the seventh Conv module for processing, and then are processed by the BN module and output.
[0027] In some embodiments of the present invention, the process of the slimming pruning algorithm is as follows: after the input features are mean and variance normalized through a batch normalization layer, channel sparse regularization is used for training, and the slimming pruning algorithm prunes channels with scaling factors close to zero to prune the initial network of the YOLOv8-n model, and the above operations are repeated for repeated pruning to obtain an EMG-YOLO model.
[0028] In some embodiments of the present invention, the S4 includes: S41, deploying the trained EMG-YOLO model to an embedded device; S42, using the trained EMG-YOLO model to extract fire features in the fire picture to be detected, and generating the location coordinates of the fire and the corresponding confidence score; S43, when the confidence score is greater than a preset confidence threshold, judging that there is a fire and outputting a detection result, the detection result is a result picture, annotated with fire information, including the location, size and confidence of the fire.
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] The present invention combines a multi-scale attention module, an efficient multi-scale convolution module and a global feature pyramid feature fusion structure, and adopts a slimming pruning algorithm, which not only improves the accuracy of urban fire detection, but also greatly improves the detection speed. The model structure of the present invention is simple, and can ensure that it can maintain a high fire detection accuracy when running for a long time in an urban environment. The present invention also pays special attention to the reasoning speed of the model, so that urban fire detection can achieve fast and real-time response on embedded devices, which helps to take more timely prevention and rescue measures. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 The figure is a flow chart of the method of the present invention.
[0032] Figure 2This is a structural diagram of the multi-scale attention module of the present invention.
[0033] Figure 3 This is the structural diagram of the C2f′ module of the present invention.
[0034] Figure 4 This is a generalized feature pyramid network structure diagram of the present invention.
[0035] Figure 5 This is a flow chart of the slimming pruning algorithm of the present invention.
[0036] Figure 6 Schematic diagram of the slimming pruning algorithm used in the model.
[0037] Figure 7 This is a fire recognition comparison chart of the EMG-YOLO model of the present invention and the existing YOLOv8-n model. DETAILED DESCRIPTION
[0038] Terminology explanation:
[0039] The neck part is an intermediate module connecting the backbone network (Backbone) and the head network (Head);
[0040] FPN and PAN are both abbreviations of feature pyramid network, which is a feature extraction method that can be used for target detection tasks.
[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0042] like Figure 1 As shown, the present invention provides a fire detection method for embedded devices, comprising the following steps:
[0043] S1. Obtain fire images in urban scenes and construct a training dataset;
[0044] S2. Optimize the structure of the YOLOv8-n model to obtain the EMG-YOLO model;
[0045] S3, train the EMG-YOLO model using the training data set;
[0046] S4. Deploy the trained EMG-YOLO model to the embedded device, then detect the fire image to be detected and output the detection results.
[0047] The present invention adopts an improved YOLOv8-n model, which has a simple structure and can ensure high fire detection accuracy when operating for a long time in an urban environment.
[0048] In one embodiment, in S1, a web crawler is used to obtain fire images in urban scenes. A large number of fire images in urban scenes are obtained from the Internet through a web crawler to construct a rich fire training data set. This data set covers fire scenes in various complex urban environments, such as fire pictures of high-rise buildings, streets, shopping malls, etc., providing diverse data support for model training.
[0049] In one embodiment, in S2, the structural optimization includes: introducing a multi-scale attention module to embed into the end of the YOLOv8-n model backbone network; using an efficient multi-scale convolution module to construct a C2f′ module and replace the original C2f module in the YOLOv8-n model backbone network; using a generalized feature pyramid network to replace FPN and PAN in the neck network of the YOLOv8-n model; and using a slimming pruning algorithm to prune the YOLOv8-n model. In the existing fire detection methods, there are many improvements and applications based on the YOLOv8-n model, but most of these methods have deficiencies in processing multi-scale fire features, especially in complex urban environments, and are prone to missed detection or false detection. The present invention is different from the prior art, mainly in the multi-scale optimization of the YOLOv8-n model, including the introduction of a multi-scale attention module (MAM) to enhance the ability to extract fire features of different scales. The module uses convolution kernels of different sizes to capture multi-scale information in the feature map, and combines the deep separation convolution (DWConv) technology to effectively reduce the number of parameters and computational costs. The C2f module of YOLOv8-n is also reconstructed through the efficient multi-scale convolution module (EMCM), aiming to improve the model's accuracy in extracting fire features such as flames and smoke. The EMCM module adopts a multi-branch convolution structure, combined with multi-scale convolution and multi-scale attention modules, to effectively distinguish fire features from background information in complex or noisy scenes. In order to further enhance the feature fusion capability of the model, the generalized feature pyramid network (GFPN) is introduced, which allows features of different scales to interact and fuse more flexibly, reduces the loss of information during transmission, and improves the accuracy and robustness of fire detection.
[0050] In one embodiment, Figure 2As shown, the multi-scale attention module includes a trunk convolution layer and multiple parallel depth-separable convolution branches; the trunk convolution layer includes a first DWConv module, a first Conv module, a MaxPool module, an AvgPool module and a Sigmoid module, and the multiple parallel depth-separable convolution branches include multiple parallel second DWConv modules and multiple parallel third DWConv modules located between the first DWConv module and the first Conv module.
[0051] The working steps of the multi-scale attention module are as follows: the input is firstly subjected to preliminary feature extraction by the first DWConv module, the extracted features are sent to multiple parallel second DWConv modules for multi-scale feature extraction, and then sent to multiple parallel third DWConv modules for processing, the processing results are spliced with the features extracted by the first DWConv module, and the spliced results are sent to the first Conv module for processing to obtain transition features, the transition features are multiplied with the features extracted by the first DWConv module, and the multiplication results are sent to the MaxPool module and the AvgPool module for processing, and finally the output is obtained through the Sigmoid module.
[0052] Preferably, the first DWConv module is a depth-separable convolution of size 5x 5, the first Conv module is a convolution of size 1x 1, the MaxPool module is a maximum pooling layer, the AvgPool module is an average pooling layer, and the Sigmoid module is an activation function; there are three of the multiple parallel second DWConv modules, namely, a depth-separable convolution of size 1x 7, a depth-separable convolution of size 1x 11, and a depth-separable convolution of size 1x 21; there are three of the multiple parallel third DWConv modules, namely, a depth-separable convolution of size 7x 1, a depth-separable convolution of size 11x 1, and a depth-separable convolution of size 21x 1.
[0053] The multi-scale attention module of the present invention uses multiple parallel depth-separable convolution branches, each of which uses depth-separable convolution kernels of different sizes to process the input feature map. For example, 1x 7 and 7x 1 depth-separable convolutions are used to capture long features in the horizontal and vertical directions, respectively, thereby further improving the directionality of feature extraction. Larger convolution kernels ensure that large-scale features can also be fully recognized. After the above processing, the feature map enters the spatial attention module to enhance its representation ability in the spatial dimension while filtering out irrelevant background information.
[0054] In one embodiment, Figure 3As shown, the C2f′ module includes a second Conv module, a split module, multiple parallel efficient multi-scale convolution modules (EMCM), a Concat module and a third Conv module. The working steps of the C2f′ module are as follows: the input is adjusted through the second Conv module to adjust the number of channels, and then divided into two branches by the split module, one of which is directly passed to the output, and the other branch is processed by multiple efficient multi-scale convolution modules.
[0055] The efficient multi-scale convolution module (EMCM) includes a fourth Conv module, a split module, multiple parallel fifth Conv modules, a Concat module and a MAM module. The working steps of the efficient multi-scale convolution module are: the input is adjusted through the fourth Conv module to adjust the number of channels, then the split module performs a segmentation operation, and then sent to multiple parallel fifth Conv modules for processing, the processing results are connected in the Concat module, and finally the output is obtained through the MAM module.
[0056] Preferably, the second Conv module and the third Conv module are both 1x 1 convolutions; the fourth Conv module is 1x 1 convolution; the multiple parallel fifth Conv modules are four, namely 1x 1 convolutions, 3x 3 convolutions, 5x 5 convolutions and 7x 7 convolutions. The present invention uses an efficient multi-scale convolution module (EMCM) to construct a C2f′ module and replace the original C2f module in the YOLOv8-n model backbone network. It also introduces a multi-scale attention module to improve the feature extraction capability when processing complex or noisy scenes, thereby improving the ability to capture fire features. This improvement not only enhances the ability to capture features of different scales, but also effectively distinguishes interference information in flames and smoke from the background through the combination of multi-scale convolution and multi-scale attention.
[0057] In one embodiment, Figure 4As shown, the generalized feature pyramid network includes multiple parallel first CSPStage feature fusion modules and multiple parallel second CSPStage feature fusion modules; the first CSPStage feature fusion module and the second CSPStage feature fusion module have the same structure, both including a first convolution branch, a second convolution branch and a Concat module, the first convolution branch is a 1x 1 convolution, the second convolution branch is a 1x 1 convolution and N concatenated convolution modules, each convolution module includes a 3x 3 RepConv convolution module and a 1x 1 convolution; the RepConv convolution module includes two parallel sixth Conv modules and two parallel BN modules in the model training stage (train), the two parallel sixth Conv modules are 1x 1 convolution and 3x 3 convolution respectively; the RepConv convolution module includes a seventh Conv module and a BN module in the inference stage (infer), and the seventh Conv module is a 3x 3 convolution.
[0058] The working steps of the generalized feature pyramid network are as follows: the input features first enter multiple parallel first CSPStage modules for preliminary feature extraction and fusion of multi-scale information, the output of each first CSPStage module is then sent to other first CSPStage modules except the current first CSPStage module and multiple parallel second CSPStage modules for further extraction and fusion, a part of the features processed by each second CSPStage module is directly output, and the other part is also sent to other second CSPStage modules except the current second CSPStage module for processing and then output;
[0059] The working steps of the generalized feature pyramid network are as follows: the input first enters multiple parallel first CSPStage modules for preliminary feature extraction and fusion of multi-scale information. Each CSPStage module has the ability to separate and fuse features. Therefore, in this stage, different CSPStage modules perform preliminary feature extraction and multi-scale information fusion on the input in parallel. Specifically, each first CSPStage module processes the input feature map and generates a preliminary feature representation, while retaining a part of the original features to enhance the transmission of information; then, the output of each first CSPStage module is sent to other first CSPStage modules except the current first CSPStage module and multiple parallel second CSPStage modules for further extraction and fusion. At this time, multiple parallel paths form a cross-module feature interaction mechanism. Each first CSPStage module not only transmits its own processed features to its subsequent second The CSPStage module will also fuse the features of other parallel first CSPStage modules, thereby achieving direct interaction between features at different levels and further extraction of multi-scale features; then, each second CSPStage module will continue to further process these fused features. Through feature separation, convolution operations and cross-level feature interactions, each second CSPStage module can extract higher-level features while also obtaining information from other CSPStage modules, further enhancing feature diversity and representation capabilities. Part of the features processed by each second CSPStage module are directly output, and the other part is also sent to other second CSPStage modules except the current second CSPStage module for processing before output. Through this multiple cross-module interactions and fusions, the second CSPStage module can further enrich the expression of features, while providing multi-scale, fused feature output results for subsequent networks.
[0060] The input entering the first CSPStage module or the second CSPStage module is divided into two parts and then sent to the first convolution branch and the second convolution branch respectively: the features sent to the first convolution branch are processed by a 1x 1 convolution; the features sent to the second convolution branch are processed by a 1x 1 convolution and N convolution modules connected in series, and the features processed by the 1x 1 convolution of the second convolution branch are spliced with the features processed by the convolution module, and the features processed by the first convolution branch and the second convolution branch are spliced in the Concat module; after the input features enter the RepConv module, they first enter the sixth Conv module or the seventh Conv module for processing, and then are processed by the BN module and output.
[0061] The present invention adopts a generalized feature pyramid network (GFPN), which enhances feature representation through direct interaction of features of different scales and layer-by-layer feature fusion. GFPN realizes a more complex information exchange method between features at different levels. In the fire detection task, GFPN helps to solve the problems of capturing fuzzy features, reducing background interference, and accurately processing multi-scale fire features, thereby improving the accuracy and robustness of fire detection. Specifically, GFPN is composed of multiple CSPStage modules, each of which divides the input features into two parts through a channel separation strategy, one part is directly transmitted through a jump connection, and the other part is deeply extracted through a convolutional layer, thereby capturing multi-scale features at different levels. GFPN contains multiple parallel feature processing paths, so that features from different CSPStage modules can interact directly between different paths to achieve the integration of multi-scale features. Through the layer-by-layer fusion design, features at different levels in the network are repeatedly used and fused to form a richer feature representation. The aggregation of multi-scale information is further combined through cross-connections and convolution operations of parallel modules to effectively combine features of different scales, enabling the model to show higher detection accuracy and robustness in the detection tasks of complex backgrounds and multi-scale fire features.
[0062] In one embodiment, Figure 5 As shown, the process of the slimming pruning algorithm is: after the input features are normalized by mean and variance through the batch normalization layer, channel sparse regularization is used for training, the slimming pruning algorithm prunes the channels with scaling factors close to zero to prune the initial network of the YOLOv8-n model, repeats the above operation for repeated pruning, and fine-tunes the pruned model (retrains the pruned model) to obtain an EMG-YOLO model containing a compact network to restore some of the performance lost due to pruning. The present invention uses a slimming pruning algorithm to prune the model, cuts the existing redundant connection channels and reduces the amount of calculation and storage requirements, so that the model is more suitable for real-time applications on embedded devices. The slimming pruning algorithm reduces unnecessary parameters and calculations in the neural network model to reduce the complexity of the model, thereby making it more suitable for embedded devices or real-time application scenarios. The slimming pruning algorithm selectively removes redundant channels or neurons in the network by a sparse regularization method.
[0063] like Figure 6The figure shows a specific schematic diagram of the model using the slimming pruning algorithm. The initial network of the EMG-YOLO model is the initial state of the neural network. A certain convolution layer of the neural network (for example, the first convolution layer) contains multiple channels, marked as Ci1, Ci2,..., Cin, which represent multiple channels of the input feature map. These channels are processed by weights and activation functions and connected to multiple channels in subsequent convolution layers to form parameter connections of the network. The weight value of each channel, that is, the channel scaling factor, is expressed in numbers, such as 0.170, 0.201, 0.240, 0.820, etc. These values indicate the importance of the channel. The larger the weight of the channel, the more important it is in feature extraction. During the training process, the weight of each channel is constrained and optimized through sparse regularization (for example, L1 regularization), and some channels in the network are gradually sparse. In this process, channels with smaller weights are considered to have less impact on the final output, may contain redundant information, and are suitable for removal. In Figure 6 It can be seen that the weight values of different channels are quite different. For example, the weight of the Ci2 channel is 0.201, which is relatively small, while the weight of the Ci4 channel is 0.820, indicating that it is more important in the model. Through this evaluation of channel weights, the model will selectively retain more important channels and remove those redundant channels with smaller weights. The pruning process occurs in Figure 6 The middle part is called pruning, during which channels with smaller weights are pruned. Figure 6 In the example, the Ci2 channel is pruned because of its small weight. At this point, the remaining channels such as Ci1, Ci3, ..., Cin will continue to be retained as an important part of the model. The pruning process will not affect the main performance of the model, because the pruned channels provide relatively little feature information, and the model can continue to maintain high performance through the remaining important channels. The pruned network is called a compact model. Figure 6 The right part of the figure shows the state after pruning. Compared with the network before pruning, it has fewer channels, so its parameters and computation are greatly reduced. For example, after pruning, the Ci2 channel has been removed, the remaining channel weights have been redistributed, and the remaining channels (such as Ci1 and Ci3) are still connected to the output channels of the next convolutional layer through the corresponding weight values.
[0064] In one embodiment, S4 includes: S41, deploying the trained EMG-YOLO model to an embedded device; S42, using the trained EMG-YOLO model to extract fire features in a fire image to be detected, and generating the location coordinates of the fire and a corresponding confidence score; S43, when the confidence score is greater than a preset confidence threshold, determining that a fire exists and outputting a detection result, wherein the detection result is a result image, annotated with fire information, including the location, size and confidence of the fire.
[0065] The generation of the location coordinates of the fire and the corresponding confidence score specifically includes: the EMG-YOLO model extracts features from the fire image to be detected through its network structure and maps these features to feature maps of multiple scales; then, the detection layer uses these feature maps to predict whether each grid cell contains a fire target and generates a corresponding bounding box; each bounding box generates a confidence score, and the probability of the existence of a fire target in the box is calculated by the sigmoid function. At the same time, the model also predicts the location coordinates of the bounding box, including the x-coordinate and y-coordinate of the center point and the width and height of the bounding box. The width and height parameters of the bounding box directly reflect the size of the fire target; finally, post-processing steps such as non-maximum suppression (NMS) are applied to remove overlapping low-confidence boxes, and finally the fire location with the highest confidence and the corresponding confidence score are output.
[0066] Since the EMG-YOLO model generates a confidence score for each detected fire target, this score indicates the probability that the target is a fire, and the value range is usually between 0 and 1. In order to determine the existence of a fire, a confidence threshold (usually a value between 0 and 1) needs to be set. When the confidence score output by the model is higher than the preset confidence threshold, the detection result will be considered a valid fire target, that is, there is a fire; conversely, if the confidence score is lower than the preset confidence threshold, the detection result is considered unreliable and may be noise or a false alarm.
[0067] The EMG-YOLO and YOLOv8-n designed by the present invention are respectively deployed on the embedded device Jetson Orin Nano to test their detection accuracy and running speed. The test results are shown in Table 1.
[0068] Table 1 Test results of EMG-YOLO and YOLOv8-n models
[0069]
[0070] The urban fire detection model EMG-YOLO proposed in the present invention achieves 53.3 FPS on the embedded device Jetson Orin Nano, which is 86.3% higher than YOLOv8-n. Figure 7 is the detection result of EMG-YOLO and YOLOv8-n, Figure 7 It can be seen that the EMG-YOLO model of the present invention is more accurate in identifying fires.
[0071] Finally, it should be noted that the above embodiments are only preferred embodiments of the present invention to illustrate the technical solutions of the present invention, rather than limiting them, and certainly not limiting the patent scope of the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention. In other words, any changes or modifications made to the main design concept and spirit of the present invention that have no substantive significance, and the technical problems they solve are still consistent with the present invention, should be included in the protection scope of the present invention. In addition, the direct or indirect application of the technical solutions of the present invention in other related technical fields is also included in the patent protection scope of the present invention.
Claims
1. A fire detection method for embedded devices, characterized in that: The following steps are involved: S1. Obtain fire images in urban scenes and construct a training dataset; S2. Optimize the structure of the YOLOv8-n model to obtain the EMG-YOLO model; S3, train the EMG-YOLO model using the training data set; S4. Deploy the trained EMG-YOLO model to the embedded device, then detect the fire image to be detected and output the detection results.
2. A fire detection method for embedded devices according to claim 1, characterized in that: In S1, a web crawler is used to obtain fire images in urban scenes.
3. A fire detection method for embedded devices according to claim 1, characterized in that: In S2, the structural optimization includes: introducing a multi-scale attention module and embedding it into the end of the YOLOv8-n model backbone network; using an efficient multi-scale convolution module to construct a C2f′ module and replace the original C2f module in the YOLOv8-n model backbone network; using a generalized feature pyramid network to replace FPN and PAN in the neck network of the YOLOv8-n model; and using a slimming pruning algorithm to prune the YOLOv8-n model to obtain an EMG-YOLO model.
4. A fire detection method for embedded devices according to claim 3, characterized in that: The multi-scale attention module includes a trunk convolution layer and multiple parallel depth-separable convolution branches; the trunk convolution layer includes a first DWConv module, a first Conv module, a MaxPool module, an AvgPool module and a Sigmoid module, and the multiple parallel depth-separable convolution branches include multiple parallel second DWConv modules and multiple parallel third DWConv modules located between the first DWConv module and the first Conv module; The working steps of the multi-scale attention module are as follows: the input is firstly subjected to preliminary feature extraction by the first DWConv module, the extracted features are sent to multiple parallel second DWConv modules for multi-scale feature extraction, and then sent to multiple parallel third DWConv modules for processing, the processing results are spliced with the features extracted by the first DWConv module, and the spliced results are sent to the first Conv module for processing to obtain transition features, the transition features are multiplied with the features extracted by the first DWConv module, and the multiplication results are sent to the MaxPool module and the AvgPool module for processing, and finally the output is obtained through the Sigmoid module.
5. A fire detection method for embedded devices according to claim 4, characterized in that: The first DWConv module is a depth-separable convolution of size 5x 5, the first Conv module is a convolution of size 1x 1, the MaxPool module is a maximum pooling layer, the AvgPool module is an average pooling layer, and the Sigmoid module is an activation function; there are three of the multiple parallel second DWConv modules, namely, a depth-separable convolution of size 1x 7, a depth-separable convolution of size 1x 11, and a depth-separable convolution of size 1x 21; there are three of the multiple parallel third DWConv modules, namely, a depth-separable convolution of size 7x 1, a depth-separable convolution of size 11x 1, and a depth-separable convolution of size 21x 1.
6. A fire detection method for embedded devices according to claim 3, characterized in that: The C2f′ module includes a second Conv module, a split module, multiple parallel efficient multi-scale convolution modules, a Concat module and a third Conv module; The working steps of the C2f′ module are as follows: the input is passed through the second Conv module to adjust the number of channels, and then is divided into two branches by the split module, one of which is directly passed to the output, and the other is processed by multiple efficient multi-scale convolution modules; The efficient multi-scale convolution module includes a fourth Conv module, a split module, multiple parallel fifth Conv modules, a Concat module and a MAM module; The working steps of the efficient multi-scale convolution module are as follows: the input is passed through the fourth Conv module to adjust the number of channels, then the split module performs a segmentation operation, and then sent to multiple parallel fifth Conv modules for processing, the processing results are connected in the Concat module, and finally the output is obtained through the MAM module.
7. A fire detection method for embedded devices according to claim 6, characterized in that: The second Conv module and the third Conv module are both 1x 1 convolutions; the fourth Conv module is 1x 1 convolution; there are four of the multiple parallel fifth Conv modules, namely 1x 1 convolution, 3x 3 convolution, 5x 5 convolution and 7x 7 convolution.
8. A fire detection method for embedded devices according to claim 3, characterized in that: The generalized feature pyramid network includes multiple parallel first CSPStage feature fusion modules and multiple parallel second CSPStage feature fusion modules; the first CSPStage feature fusion module and the second CSPStage feature fusion module have the same structure, both including a first convolution branch, a second convolution branch and a Concat module, the first convolution branch is a 1x 1 convolution, the second convolution branch is a 1x 1 convolution and N concatenated convolution modules, each convolution module includes a 3x 3 RepConv convolution module and a 1x 1 convolution; the RepConv convolution module includes two parallel sixth Conv modules and two parallel BN modules in the model training stage, the two parallel sixth Conv modules are 1x 1 convolution and 3x 3 convolution respectively; the RepConv convolution module includes a seventh Conv module and a BN module in the inference stage, the seventh Conv module is a 3x 3 convolution; The working steps of the generalized feature pyramid network are as follows: the input first enters multiple parallel first CSPStage modules for preliminary feature extraction and fusion of multi-scale information, the output of each first CSPStage module is then sent to other first CSPStage modules except the current first CSPStage module and multiple parallel second CSPStage modules for further extraction and fusion, a part of the features processed by each second CSPStage module is directly output, and the other part is also sent to other second CSPStage modules except the current second CSPStage module for processing and then output; After the input enters the first CSPStage module or the second CSPStage module, it is divided into two parts and then sent to the first convolution branch and the second convolution branch respectively: the features sent to the first convolution branch are processed by a 1x1 convolution; the features sent to the second convolution branch are processed by a 1x1 convolution and N series-connected convolution modules, and the features processed by the 1x1 convolution of the second convolution branch are spliced with the features processed by the convolution module, and the features processed by the first convolution branch and the second convolution branch are spliced in the Concat module; after the input features enter the RepConv module, they first enter the sixth Conv module or the seventh Conv module for processing, and then are processed by the BN module and output.
9. A fire detection method for embedded devices according to claim 3, characterized in that: The process of the slimming pruning algorithm is as follows: after the input features are mean and variance normalized through the batch normalization layer, channel sparse regularization is used for training, and the slimming pruning algorithm prunes the channels with scaling factors close to zero to prune the initial network of the YOLOv8-n model, and repeats the above operations for repeated pruning to obtain the EMG-YOLO model.
10. A fire detection method for embedded devices according to claim 1, characterized in that: The S4 includes: S41, deploying the trained EMG-YOLO model to the embedded device; S42, using the trained EMG-YOLO model to extract fire features in the fire picture to be detected, and generating the location coordinates of the fire and the corresponding confidence score; S43, when the confidence score is greater than a preset confidence threshold, judging that there is a fire and outputting a detection result, the detection result is a result picture, annotated with fire information, including the location, size and confidence of the fire.
Citation Information
Patent Citations
Lightweight target learning detection method based on improved YOLOv8
CN118212588A
YOLOv8 lightweight fire identification method based on smoke enhancement algorithm
CN118379625A
Fire detection method and system
CN118397784A
Cited By
Fire smoke image detection method based on improved YOLOv8 network
CN121095735A
