A fire detection method with light weight and high robustness in a variety of environments

Abstract

The application discloses a fire detection method and device with light weight and high robustness in a diversified environment, and relates to the technical field of artificial intelligence vision and fire safety. The method comprises the following steps: constructing a fire detection model; the model comprises a heterogeneous multi-branch network structure module and a flame frequency domain perception module; obtaining a fire and smoke detection dataset to train the fire detection model; the heterogeneous multi-branch network structure module is used to extract features of different receptive fields; the features of different receptive fields are input into the flame frequency domain perception module to extract low-frequency features and high-frequency features of the flame, and the features are spliced to obtain spliced features; the fire detection model is trained based on the spliced features to obtain a trained fire detection model; obtaining a fire image to be detected; and inputting the fire image to be detected into the trained fire detection model to output a detection result. The application can improve the accuracy and stability of fire detection in a diversified environment.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence vision and fire safety technology, and in particular to a fire detection method and device with lightweight and high robustness in diverse environments. Background Technology

[0002] With the acceleration of urbanization and the increasing number of complex building environments, fire safety early warning and detection technologies have become a crucial line of defense for protecting people's lives and property. In diverse scenarios such as residential buildings, industrial parks, and forests, the early detection and accurate identification of fires directly determine the efficiency of rescue efforts and the effectiveness of disaster control. Therefore, how to achieve efficient and accurate fire monitoring using artificial intelligence vision technology has become a research hotspot in the field of public safety. In practical applications, if fire detection systems cannot adapt to complex and changing background environments, they are easily affected by changes in lighting, smoke obstruction, and interference from similar objects, leading to frequent false alarms or missed alarms, thus delaying the best time to extinguish fires and causing incalculable losses. Conversely, if a system possesses a lightweight model structure and highly robust recognition capabilities, it can not only run in real time on embedded devices or edge terminals such as drones, but also work stably and reliably under various harsh conditions, bringing revolutionary improvements to fire prevention and control. In view of this, how to achieve a fire detection method that is both lightweight and highly robust in diverse environments has attracted widespread attention from academia and industry.

[0003] In existing technologies, studies have improved fire detection accuracy by introducing deep learning-based target detection algorithms or by reducing computational complexity through model compression and pruning techniques to adapt to mobile deployments. However, most of these approaches fail to simultaneously achieve model miniaturization and robustness in complex environments. While some methods perform well under ideal laboratory conditions, their detection performance deteriorates sharply in real-world applications when faced with drastic changes in lighting, cluttered backgrounds, or multi-scale changes in the target. Therefore, developing a lightweight and robust fire detection method capable of deployment in diverse and complex environments is a pressing technical challenge in this field.

[0004] In diverse and complex environments, accurate and reliable fire detection relies on the strong adaptability of detection models to environmental factors such as changes in lighting, smoke obstruction, and background interference, as well as the real-time and accurate identification of flame and smoke targets. The implementation methods can be summarized as the optimization and application of deep learning-based target detection algorithms. For example, in forest fire prevention or urban security scenarios, by improving the YOLO series or SSD target detection framework and introducing attention mechanisms or multi-scale feature fusion modules, the model's ability to extract features of flames and smoke at different scales can be enhanced. These methods utilize the powerful feature representation capabilities of convolutional neural networks, improving the detection accuracy of fire targets to a certain extent. However, fire detection systems face the dual challenges of computational resources and detection performance in practical deployments. High-precision deep neural networks often have a large number of parameters and high computational complexity, making it difficult to achieve real-time operation on edge terminals such as drones and embedded devices. This leads to problems such as excessively large model size, slow inference speed, and difficulty in deployment on resource-constrained platforms. To address these challenges, existing technologies have introduced lightweight model techniques, such as network pruning, weight quantization, knowledge distillation, and lightweight network architecture design. By compressing model size and reducing floating-point computation, these techniques can adapt to the deployment requirements of mobile or embedded devices. While these lightweight methods have achieved significant results in reducing computational overhead, they still have obvious shortcomings in complex real-world environments: the excessive pursuit of lightweight models leads to a decline in feature extraction capabilities, and the model's robustness to drastic changes in lighting, dense smoke obstruction, or interference from objects similar to flames often decreases significantly, making it difficult to maintain high-precision detection results in diverse environments. Therefore, developing a fire detection method that can balance lightweight deployment requirements with high robustness in complex environments is a pressing technical problem in this field. Summary of the Invention

[0005] To address the technical problems of insufficient robustness of fire detection methods in complex environments and the difficulty in deploying effective fire detection methods on devices with limited computing resources, resulting in low detection accuracy in complex environments, this invention provides a lightweight and highly robust fire detection method and apparatus for diverse environments. The technical solution is as follows: On the one hand, a lightweight and robust fire detection method for diverse environments is provided. This method is implemented by a lightweight and robust fire detection device for diverse environments, and includes: S1. A fire detection model is constructed using YOLOv8 as the basic architecture. The model architecture includes: a heterogeneous multi-branch network structure module, a flame frequency domain sensing module, a C2f module, and a fast spatial pyramid pooling module. The heterogeneous multi-branch network structure module is embedded in the C2f module, and the flame frequency domain sensing module is embedded in the fast spatial pyramid pooling module. S2. Obtain the fire and smoke detection dataset; divide the dataset into a training set and input it into the fire detection model for training; extract feature maps of different receptive fields through a heterogeneous multi-branch network structure module; input the feature maps of different receptive fields into the flame frequency domain perception module to extract the low-frequency feature map and the high-frequency feature map of the flame, and stitch them together to obtain the stitched feature map; train the fire detection model based on the stitched feature map to obtain the trained fire detection model. S3. Obtain the fire image to be detected; input the fire image to be detected into the trained fire detection model and output the detection result.

[0006] On the other hand, a lightweight and robust fire detection device for diverse environments is provided, which is applied to a lightweight and robust fire detection method for diverse environments. The device includes: A building unit is used to construct a fire detection model based on YOLOv8. The model architecture includes: a heterogeneous multi-branch network structure module, a flame frequency domain sensing module, a C2f module, and a fast spatial pyramid pooling module. The heterogeneous multi-branch network structure module is embedded in the C2f module, and the flame frequency domain sensing module is embedded in the fast spatial pyramid pooling module. The training unit is used to acquire fire and smoke detection datasets; the datasets are divided into training sets and input into the fire detection model for training; feature maps of different receptive fields are extracted through a heterogeneous multi-branch network structure module; the feature maps of different receptive fields are input into the flame frequency domain perception module to extract low-frequency and high-frequency feature maps of the flame, and then spliced ​​to obtain a spliced ​​feature map; the fire detection model is trained based on the spliced ​​feature map to obtain a trained fire detection model. The detection unit is used to acquire the fire image to be detected; it inputs the fire image to be detected into the trained fire detection model and outputs the detection result.

[0007] On the other hand, a lightweight and robust fire detection device for diverse environments is provided, comprising: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement any of the methods described above for lightweight and robust fire detection in diverse environments.

[0008] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for fire detection in diverse environments that are lightweight and robust.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention, through the design of a heterogeneous multi-branch network structure module, introduces receptive field branches of different scales, which can simultaneously capture local details (such as the edge of a flame) and changes in brightness over a larger area. This effectively addresses complex backgrounds, lighting changes, and smoke interference, significantly improving the accuracy and stability of fire detection in diverse environments.

[0010] The embodiments of the present invention use depthwise separable convolution instead of standard convolution in heterogeneous multi-branch structures, which greatly reduces the number of model parameters and computational complexity, enabling the algorithm to run in real time on edge devices with limited computing resources (such as embedded cameras and IoT nodes), meeting the requirements of low latency and low power consumption in practical applications.

[0011] This invention innovatively introduces a flame frequency domain sensing module, which enhances the sensitivity to weak flame features and early signs by focusing on the high-frequency details (such as flickering texture) and low-frequency shape (such as overall outline) of the flame, thereby achieving very early fire detection, buying valuable time for early warning, and effectively reducing fire losses.

[0012] The improved embodiments of this invention maintain the high efficiency of the original YOLOv8 detection framework while synergistically enhancing detection accuracy and speed, especially in the identification of small target flames and early smoke, providing reliable technical support for intelligent fire protection systems. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a fire detection method with lightweight and high robustness in various environments provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a fire detection model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a heterogeneous multi-branch network structure module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a flame frequency domain sensing module provided in an embodiment of the present invention; Figure 5 This is a block diagram of a fire detection device with lightweight and high robustness in various environments provided by an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a fire detection device with lightweight and high robustness in various environments provided by an embodiment of the present invention. Detailed Implementation

[0015] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0016] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of the word "exemplary" is intended to present the concept in a specific manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0017] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0018] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0019] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0020] This invention provides a lightweight and robust fire detection method for diverse environments. This method can be implemented using a lightweight and robust fire detection device suitable for diverse environments, which can be a terminal or a server. Figure 1 The flowchart shown illustrates a lightweight and robust fire detection method for diverse environments. The process flow of this method may include the following steps: S1. A fire detection model is constructed using YOLOv8 as the basic architecture. The model architecture includes: a heterogeneous multi-branch network structure module, a flame frequency domain sensing module, a C2f module, and a fast spatial pyramid pooling module. The heterogeneous multi-branch network structure module is embedded in the C2f module, and the flame frequency domain sensing module is embedded in the fast spatial pyramid pooling module.

[0021] Among them, such as Figure 2The diagram shows a structural schematic of a fire detection model provided in an embodiment of the present invention; wherein, the C2f (Cross-stage partial blocks with 2 feature paths) module is the feature extraction module in the YOLOv8 model; and the SPPF (Spatial Pyramid Pooling - Fast) module.

[0022] Optionally, the heterogeneous multi-branch network structure module includes multiple heterogeneous branch structures; The first branch uses a 3×3 depthwise separable convolution; the second branch uses a 5×5 depthwise separable convolution; the third branch uses a dilated 3×3 convolution with a dilation rate of 2; and the fourth branch uses global average pooling to preserve global contextual information. Figure 3 The diagram shown is a schematic diagram of a heterogeneous multi-branch network structure module provided in an embodiment of the present invention; DSConv represents depthwise separable convolution; AvgPool represents average pooling; UpSample represents upsampling.

[0023] Among them, such as Figure 4 The diagram shown is a structural schematic of a flame frequency domain sensing module provided in an embodiment of the present invention; wherein, FAA (Frequency-Aware Activation, FAA) represents the flame frequency domain sensing module; BN represents batch normalization processing; Conv represents convolution processing; Maxpooll2d represents two-dimensional max pooling; wherein, the flame frequency domain sensing module includes a high-frequency detail path and a low-frequency shape path; Among them, the high-frequency characteristics of a flame are characterized by rapid changes and flashes at the edge of the flame and the existence of local high-temperature jumps; the low-frequency characteristics of a flame are characterized by stable heating in the core area of ​​the flame, large-scale thermal radiation diffusion, and an overall flame shape that is conical and diffused. Among them, the high-frequency detail path uses a 3×3 convolution kernel to capture local detail changes, while depthwise separable convolution is used to extract spatial details and maintain the independence of channels; Among them, the low-frequency shape path uses 5×5 flat pooling to blur details, preserve the overall shape, and effectively identify the overall outline and diffusion area of ​​the flame.

[0024] The heterogeneous multi-branch network structure module is a multi-scale receptive field enhancement module that improves the model's feature extraction capability by processing features from different receptive fields in parallel, while maintaining compatibility with the standard bottleneck. Additionally, depthwise separable convolutions are employed to optimize parameter efficiency and enhance the module's lightweight design.

[0025] S2. Obtain the fire and smoke detection dataset; divide the dataset into training, testing, and validation sets; input the training set into the fire detection model for training, and extract feature maps of different receptive fields through a heterogeneous multi-branch network structure module; input the feature maps of different receptive fields into the flame frequency domain perception module to extract the low-frequency and high-frequency feature maps of the flame, and then stitch them together to obtain the stitched feature map; train the fire detection model based on the stitched feature map to obtain the trained fire detection model.

[0026] Optionally, the specific implementation process of S2 includes S21-S25: S21. Input the training set into the fire detection model for training. Use a 3×3 depthwise separable convolution to extract features and obtain the feature map corresponding to the first branch. Use a 5×5 depthwise separable convolution to extract features and obtain the feature map corresponding to the second branch. Use a 3×3 dilated convolution to extract features and obtain the feature map corresponding to the third branch. Use global average pooling to extract features and obtain the feature map corresponding to the fourth branch. S22. Perform feature concatenation on the feature maps corresponding to the first branch, the second branch, the third branch, and the fourth branch, and output the first concatenated feature map through the residual structure. S23. Input the first stitched feature map into the flame frequency domain perception module. Extract the flame high-frequency features through the high-frequency detail path using a 3×3 depthwise separable convolution; extract the flame low-frequency features through the low-frequency shape path using a 5×5 average pooling. Alternatively, the high-frequency detail path can be represented by the following formula (1): (1) in, Indicates a high-frequency path; ( () indicates batch normalization processing; Represents input features; This represents a depthwise separable convolution of size 3×3; The implementation process of depthwise separable convolution is represented by the following formula (2): (2) Where c represents the channel index, ranging from (0 to C-1); i and j represent the spatial coordinates; m and n represent the offsets within the convolution kernel, ranging from (-1, 0, 1); the key point is that each channel is convolved independently, without cross-channel mixing, and the output is preserved. The shape is extracted, but only spatial details are extracted without changing the channel relationships. Finally, the high-frequency response is normalized and the training is stable.

[0027] Optionally, the batch normalization process is represented by the following formula (3): = + (3) in, This represents a feature map that has undergone batch normalization. Indicates the learnable scaling parameter; Indicates batch variance; Indicates the batch mean; This represents the element value of the input feature map, specifically the c-th channel at spatial location (i, j) (the pixel in the i-th row and j-th column of the feature map). This represents the learnable offset parameter; This indicates a small constant to prevent division by zero.

[0028] in, , indicating depthwise separable convolution; The batch variance is expressed by the following formula (4): (4) The batch mean is expressed by the following formula (5): (5) The low-frequency shape path is represented by the following formula (6): (6) in, Indicates a low-frequency path; This represents a 1×1 convolution; This represents average pooling with a size of 5×5.

[0029] Alternatively, the average pooling process can be represented by the following formula (7): (7) Where c represents the channel index, ranging from (0 to C-1); i and j represent the spatial coordinates; and m and n represent the offsets within the convolution kernel, ranging from (-1, 0, 1).

[0030] The convolution kernel is 5×5 with a stride of 1 and padding of 2 (keeping the size constant). The average of each 5×5 neighborhood is calculated to achieve a low-pass filtering effect, filtering out high-frequency noise and details while preserving the overall trend.

[0031] In one feasible implementation, the calculation process of 1×1 convolution is represented by the following formula (8): (8) in, , This represents the weight of a 1×1 convolution kernel, which linearly combines C channels into a new channel; its overall function is to re-integrate channel information and learn the channel relationships of low-frequency features.

[0032] S24. The high-frequency feature map and the low-frequency feature map of the flame are concatenated along the channel dimension to obtain the second concatenated feature map. In one feasible implementation, the process of splicing the high-frequency and low-frequency features of the flame in the channel dimension is represented by the following formula (9): (9) in, This represents the features after stitching; the stitching method is to stitch along the channel dimension, with each spatial location... There are 2C eigenvalues, including C high-frequency eigenvalues ​​and C low-frequency eigenvalues.

[0033] S25. Based on the second concatenated feature map, calculate the flame attention weights using 1×1 dimensionality reduction convolution, ReLU activation function, and Sigmoid activation function; adjust the input first concatenated feature map based on the flame attention weights, and output the adjusted feature map.

[0034] Optionally, based on the concatenated features of S25, the process of calculating the flame attention weights using 1×1 dimensionality-reduced convolution, ReLU activation function, and Sigmoid activation function is expressed by the following formulas (10)-(13): (10) (11) (12) (13) in, This represents the feature map that has been reduced and fused using 1×1 convolution; Indicates the features after splicing; This represents the feature map output after a nonlinear transformation using the ReLU activation function; denoted by ReLU activation function; C represents the number of channels in the input feature map; H represents the height of the feature map; W represents the width of the feature map; Let be the real number field, indicating that each element is a real number; This represents a feature map that has been upscaled back to the original number of channels after a 1×1 convolution. This represents the Sigmoid activation function; Indicates the flame attention weight; This represents a 1×1 dimension-reduced convolution.

[0035] In one feasible implementation, the calculated flame attention weights are used to adjust the input feature map, and the adjusted weight map is output. It can be expressed by the following formula (14): (14) in, Element-wise multiplication is represented by the following formula (15): (15) in, ,when When it approaches 1, the position The characteristics of channel c are likely flame-related; when When approaching 0, position The feature of channel c is most likely the background; when When the position is between 0 and 1, the characteristics of the position channel are uncertain or the flame characteristics are weak.

[0036] In one feasible implementation, to fully verify the feasibility and effectiveness of the scheme, experiments were conducted on a Linux system. The CPU and GPU used were an Intel 8481C and an RTX 4090 (24GB), respectively. The Python version was 3.8.10, the torch version was 2.0.0, and the CUDA version was 11.8. The same training parameters were used in all experiments. The number of epochs was set to 150 to ensure that each model fully converged. The image size was set to 640×640, and the batch size was set to 16. The optimization function was SGD. An initial learning rate of 0.01 was set for training, and weight decay with a value of 0.0005 was used.

[0037] The dataset comes from an open-source fire and smoke detection dataset. Fire scenes are extremely complex, with highly variable feature distributions and significant variations in image size and shape. The dataset contains images acquired from various visual sensors (surveillance cameras, drones, and satellites) at different distances, in different scenes, and under different light intensities. It includes images of fires, smoke, and easily confused non-fire and non-smoke images. A total of 7012 images were used for training (5612 images), testing (702 images), and validation (698 images), in a ratio of 8:1:1.

[0038] In one feasible implementation, Precision, Recall, and F1-Score are used as metrics to evaluate model performance during model training. Wherein, Precision represents the model's ability to correctly distinguish smoke and fire, expressed by the following formula (16): (16) Wherein, TP represents a sample whose actual value is fireworks and whose predicted result is fireworks, and FP represents a sample whose actual value is not fireworks but whose predicted result is fireworks.

[0039] Among them, Recall can be used to evaluate the model's ability to detect all smoke and fire, and is expressed by the following formula (17): (17) Here, FN represents a sample whose actual value is fireworks but whose prediction result is non-fireworks.

[0040] Wherein, F1-Score is the harmonic mean of the two indicators mentioned above, which is a comprehensive measure of the model's performance in both aspects, and is expressed by the following formula (18): (18) In one feasible implementation, the Average Precision (AP) is used to evaluate the model's accuracy in classification and detection for each category, examining the model's detection capability in a single category. Its value is the area under the Precision-Recall curve. The mean Average Precision (mAP) is used as the evaluation metric. mAP statistically reflects the model's detection accuracy in different categories, providing a more comprehensive view of the model's generalization and robustness. The calculation process is expressed by the following formula (19): (19) in, This represents the average precision of class C; C represents the total number of classes.

[0041] In order to examine the progress of the model in terms of lightweighting, the embodiments of the present invention use two indicators, the number of parameters and the number of floating-point operations (GFLOPs), for comparison, so as to intuitively understand the differences between different models.

[0042] In one feasible implementation, to clearly demonstrate the role of the heterogeneous multi-branch network structure module and the flame frequency domain sensing module in the overall model, several ablation experiments were designed. The results are shown in Table 1. The first set of experiments used YOLOv8 as the baseline model. Next, the original backbone component was replaced with the heterogeneous multi-branch network structure module in the C2f module. The results showed that the F1-score remained unchanged, mAP50 increased by 2%, mAP50-95 increased by 1.2%, and parameters and GFLOPs decreased by 11% and 10.6%, respectively. The third set of experiments replaced the original SPPF module only with the flame frequency domain sensing module. The comparison showed that the F1-score and mAP50 increased by 1.4% and 1.1%, respectively, parameters increased by 0.8%, and GFLOPs increased by 0.7%. The fourth set of experiments incorporated both the heterogeneous multi-branch network structure module and the flame frequency domain sensing module into the network structure. The results showed that the F1-score improved by 0.9%, mAP50 improved by 1.5%, mAP50-95 improved by 1%, and parameters and GFLOPs decreased by 3.7% and 9.4%, respectively.

[0043] Table 1

[0044] To further explore the fire detection capabilities of the improved model, comparative experiments were conducted in this embodiment of the invention. YOLOv3u is an upgraded version of YOLOv3, employing the anchorless design of YOLOv8 to further improve detection performance in complex scenes. YOLOv5u also incorporates relevant features of YOLOv8, significantly improving target detection capabilities. YOLOv8n, YOLOv8s, and YOLOv8m share the same network architecture, differing only in the number of stacked modules and the width of the channels. The results of the comparative experiments are shown in Table 2. Experimental results show that the model derived from this embodiment achieves F1-Score and maAP50 of 0.686 and 0.71, respectively, with detection accuracy indicators higher than other models. Furthermore, the model proposed in this embodiment has a parameter size of only 10.7M, demonstrating significant lightweight design while maintaining detection accuracy.

[0045] Table 2

[0046] S3. Obtain the fire image to be detected; input the fire image to be detected into the trained fire detection model and output the detection result.

[0047] This invention proposes a fire detection method based on an improved YOLOv8. To address the increasingly complex detection environment, a heterogeneous multi-branch network structure module is proposed. This module has receptive field branches with different scales, enabling simultaneous capture of local details and brightness variations over larger areas. Furthermore, depthwise separable convolutions are used for computation, effectively reducing computational load and meeting the deployment requirements of edge devices. To address the algorithm's insufficient flame perception capability, a flame frequency domain perception module is proposed. This module focuses on high-frequency details and low-frequency shapes of the flame, effectively improving the early warning capability for fires.

[0048] This invention, through the design of a heterogeneous multi-branch network structure module, introduces receptive field branches of different scales, which can simultaneously capture local details (such as the edge of a flame) and changes in brightness over a larger area. This effectively addresses complex backgrounds, lighting changes, and smoke interference, significantly improving the accuracy and stability of fire detection in diverse environments.

[0049] The embodiments of the present invention use depthwise separable convolution instead of standard convolution in heterogeneous multi-branch structures, which greatly reduces the number of model parameters and computational complexity, enabling the algorithm to run in real time on edge devices with limited computing resources (such as embedded cameras and IoT nodes), meeting the requirements of low latency and low power consumption in practical applications.

[0050] This invention innovatively introduces a flame frequency domain sensing module, which enhances the sensitivity to weak flame features and early signs by focusing on the high-frequency details (such as flickering texture) and low-frequency shape (such as overall outline) of the flame, thereby achieving very early fire detection, buying valuable time for early warning, and effectively reducing fire losses.

[0051] The improved embodiments of this invention maintain the high efficiency of the original YOLOv8 detection framework while synergistically enhancing detection accuracy and speed, especially in the identification of small target flames and early smoke, providing reliable technical support for intelligent fire protection systems.

[0052] Figure 5 This is a block diagram of a lightweight and robust fire detection device for diverse environments, provided by an embodiment of the present invention. This device is used for a lightweight and robust fire detection method in diverse environments. (Refer to...) Figure 5 The device includes a construction unit 510, a training unit 520, and a detection unit 530. Wherein: Construction unit 510 is used to build a fire detection model based on YOLOv8 as the basic architecture; the architecture includes: a heterogeneous multi-branch network structure module, a flame frequency domain sensing module, a C2f module, and a fast spatial pyramid pooling module; wherein, the heterogeneous multi-branch network structure module is embedded in the C2f module; and the flame frequency domain sensing module is embedded in the fast spatial pyramid pooling module. Training unit 520 is used to acquire fire and smoke detection datasets; the datasets are divided into training sets and input into the fire detection model for training; feature maps of different receptive fields are extracted through a heterogeneous multi-branch network structure module; the feature maps of different receptive fields are input into the flame frequency domain perception module to extract low-frequency and high-frequency feature maps of the flame, and then spliced ​​to obtain a spliced ​​feature map; the fire detection model is trained based on the spliced ​​feature map to obtain a trained fire detection model. The detection unit 530 is used to acquire the fire image to be detected; input the fire image to be detected into the trained fire detection model, and output the detection result.

[0053] Optionally, the heterogeneous multi-branch network structure module includes multiple heterogeneous branch structures; The first branch uses a 3×3 depthwise separable convolution; the second branch uses a 5×5 depthwise separable convolution; the third branch uses a dilated 3×3 convolution with an inflation rate of 2; and the fourth branch uses global average pooling to preserve global context information. The flame frequency domain sensing module includes a high-frequency detail path and a low-frequency shape path. Among them, the high-frequency detail path uses a 3×3 convolution kernel to capture local detail changes, while depthwise separable convolution is used to extract spatial details and maintain the independence of channels; Among them, the low-frequency shape path uses 5×5 flat pooling to blur details, preserve the overall shape, and effectively identify the overall outline and diffusion area of ​​the flame.

[0054] Optionally, the training unit 520 is used for: The training set is input into the fire detection model for training. Feature extraction is performed using a 3×3 depthwise separable convolution to obtain the feature map corresponding to the first branch; feature extraction is performed using a 5×5 depthwise separable convolution to obtain the feature map corresponding to the second branch; feature extraction is performed using a 3×3 dilated convolution to obtain the feature map corresponding to the third branch; and feature extraction is performed using global average pooling to obtain the feature map corresponding to the fourth branch. The feature maps corresponding to the first branch, the second branch, the third branch, and the fourth branch are concatenated, and the first concatenated feature is output through the residual structure. The first stitched feature map is input into the flame frequency domain perception module. High-frequency feature maps of the flame are extracted by using a 3×3 depthwise separable convolution through the high-frequency detail path; low-frequency feature maps of the flame are extracted by using a 5×5 average pooling through the low-frequency shape path. The high-frequency feature map and the low-frequency feature map of the flame are concatenated along the channel dimension to obtain the second concatenated feature map. Based on the second concatenated feature map, flame attention weights are calculated using 1×1 dimensionality reduction convolution, ReLU activation function, and Sigmoid activation function; the input first concatenated feature map is adjusted based on the flame attention weights, and the adjusted feature map is output.

[0055] Optionally, the high-frequency detail path is represented by the following formula (1): (1) in, Indicates a high-frequency path; ( () indicates batch normalization processing; Indicates the input feature map; This represents a depthwise separable convolution of size 3×3; The low-frequency shape path is represented by the following formula (2): (2) in, Indicates a low-frequency path; This represents a 1×1 convolution; This represents average pooling with a size of 5×5.

[0056] Optionally, the average pooling process is represented by the following formula (3): (3) Where c represents the channel index, ranging from (0 to C-1); i and j represent the spatial coordinates; and m and n represent the offsets within the convolution kernel, ranging from (-1, 0, 1).

[0057] Optionally, the batch normalization process is represented by the following formula (4): = + (4) in, This represents a feature map that has undergone batch normalization. Indicates the learnable scaling parameter; Indicates batch variance; Indicates the batch mean; This represents the input feature map, specifically the element value of the c-th channel at spatial location (i,j) (the pixel in the i-th row and j-th column of the feature map). This represents the learnable offset parameter.

[0058] Optionally, the process of calculating the flame attention weights based on the concatenated features using 1×1 dimensionality-reduced convolution, ReLU activation function, and Sigmoid activation function is represented by the following formulas (5)-(8): (5) (6) (7) (8) in, This represents the feature map that has been reduced and fused using 1×1 convolution; Indicates the features after splicing; This represents the feature map output after a nonlinear transformation using the ReLU activation function; denoted by ReLU activation function; C represents the number of channels in the input feature map; H represents the height of the feature map; W represents the width of the feature map; Let be the real number field, indicating that each element is a real number; This represents a feature map that has been upscaled back to the original number of channels after a 1×1 convolution. This represents the Sigmoid activation function; Indicates the flame attention weight; This represents a 1×1 dimension-reduced convolution.

[0059] This invention, through the design of a heterogeneous multi-branch network structure module, introduces receptive field branches of different scales, which can simultaneously capture local details (such as the edge of a flame) and changes in brightness over a larger area. This effectively addresses complex backgrounds, lighting changes, and smoke interference, significantly improving the accuracy and stability of fire detection in diverse environments.

[0060] The embodiments of the present invention use depthwise separable convolution instead of standard convolution in heterogeneous multi-branch structures, which greatly reduces the number of model parameters and computational complexity, enabling the algorithm to run in real time on edge devices with limited computing resources (such as embedded cameras and IoT nodes), meeting the requirements of low latency and low power consumption in practical applications.

[0061] This invention innovatively introduces a flame frequency domain sensing module, which enhances the sensitivity to weak flame features and early signs by focusing on the high-frequency details (such as flickering texture) and low-frequency shape (such as overall outline) of the flame, thereby achieving very early fire detection, buying valuable time for early warning, and effectively reducing fire losses.

[0062] The improved embodiments of this invention maintain the high efficiency of the original YOLOv8 detection framework while synergistically enhancing detection accuracy and speed, especially in the identification of small target flames and early smoke, providing reliable technical support for intelligent fire protection systems.

[0063] Figure 6 This is a structural schematic diagram of a lightweight and highly robust fire detection device for various environments provided by an embodiment of the present invention, as shown below. Figure 6 As shown, lightweight and robust fire detection equipment for diverse environments can include the above-mentioned... Figure 5 The fire detection device shown is lightweight and robust in diverse environments. Optionally, the lightweight and robust fire detection device 610 in diverse environments may include a first processor 2001.

[0064] Optionally, the fire detection device 610, which is lightweight and robust in diverse environments, may also include a memory 2002 and a transceiver 2003.

[0065] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0066] The following is combined with Figure 6 The components of the 610 fire detection device, which is lightweight and highly robust for diverse environments, are described in detail below: The first processor 2001 is the control center of the lightweight and robust fire detection device 610 in diverse environments. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0067] Optionally, the first processor 2001 can perform various functions of the lightweight and robust fire detection device 610 in diverse environments by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0068] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 6 CPU0 and CPU1 are shown in the diagram.

[0069] In a specific implementation, as one example, the lightweight and robust fire detection device 610 for diverse environments may also include multiple processors, for example... Figure 6 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0070] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0071] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently, and may be connected via the interface circuit of the lightweight and robust fire detection device 610 in various environments. Figure 6 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0072] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0073] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 6(Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0074] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the lightweight and robust fire detection device 610 for various environments. Figure 6 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0075] It should be noted that, Figure 6 The structure of the lightweight and robust fire detection device 610 shown in the diagram does not constitute a limitation on the router. Actual lightweight and robust fire detection devices in various environments may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0076] Furthermore, the technical effects of the lightweight and robust fire detection device 610 in diverse environments can be referenced from the technical effects of the lightweight and robust fire detection method in diverse environments described in the above method embodiments, and will not be repeated here.

[0077] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0078] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0079] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0080] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0081] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0082] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0083] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can employ different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0084] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0085] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0086] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0087] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0088] If the aforementioned functions are implemented as software functional units and sold or adopted as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for fire detection with lightweight and high robustness in diverse environments, characterized in that, The method includes: S1. A fire detection model is constructed using YOLOv8 as the basic architecture. The model architecture includes: a heterogeneous multi-branch network structure module, a flame frequency domain sensing module, a C2f module, and a fast spatial pyramid pooling module. The heterogeneous multi-branch network structure module is embedded in the C2f module, and the flame frequency domain sensing module is embedded in the fast spatial pyramid pooling module. S2. Obtain the fire and smoke detection dataset; divide the dataset into a training set and input it into the fire detection model for training; extract feature maps of different receptive fields through a heterogeneous multi-branch network structure module; input the feature maps of different receptive fields into the flame frequency domain perception module to extract the low-frequency feature map and the high-frequency feature map of the flame, and stitch them together to obtain the stitched feature map; train the fire detection model based on the stitched feature map to obtain the trained fire detection model. S3. Obtain the fire image to be detected; input the fire image to be detected into the trained fire detection model and output the detection result.

2. The method for fire detection with lightweight and high robustness in diverse environments according to claim 1, wherein, The heterogeneous multi-branch network structure module includes multiple heterogeneous branch structures; The first branch uses a 3×3 depthwise separable convolution; the second branch uses a 5×5 depthwise separable convolution; the third branch uses a dilated 3×3 convolution with an inflation rate of 2; and the fourth branch uses global average pooling to preserve global context information. The flame frequency domain sensing module includes a high-frequency detail path and a low-frequency shape path. Among them, the high-frequency detail path uses a 3×3 convolution kernel to capture local detail changes, while depthwise separable convolution is used to extract spatial details and maintain the independence of channels; Among them, the low-frequency shape path uses 5×5 flat pooling to blur details, preserve the overall shape, and effectively identify the overall outline and diffusion area of ​​the flame.

3. The method of claim 2, wherein the method is characterized by, The S2 process involves inputting the training set into the fire detection model for training, extracting features from different receptive fields using a heterogeneous multi-branch network structure module, and inputting the features from different receptive fields into the flame frequency domain perception module to extract low-frequency and high-frequency features of the flame. These features are then concatenated to obtain the concatenated features, including: S21. Input the training set into the fire detection model for training. Use a 3×3 depthwise separable convolution to extract features and obtain the feature map corresponding to the first branch. Use a 5×5 depthwise separable convolution to extract features and obtain the feature map corresponding to the second branch. Use a 3×3 dilated convolution to extract features and obtain the feature map corresponding to the third branch. Use global average pooling to extract features and obtain the feature map corresponding to the fourth branch. S22. Perform feature concatenation on the feature maps corresponding to the first branch, the second branch, the third branch, and the fourth branch, and output the first concatenated feature map through the residual structure. S23. Input the first stitched feature map into the flame frequency domain perception module. Extract the flame high-frequency feature map through the high-frequency detail path using a 3×3 depthwise separable convolution; extract the flame low-frequency feature map through the low-frequency shape path using a 5×5 average pooling. S24. The high-frequency feature map and the low-frequency feature map of the flame are concatenated along the channel dimension to obtain the second concatenated feature map. S25. Based on the second concatenated feature map, calculate the flame attention weights using 1×1 dimensionality reduction convolution, ReLU activation function, and Sigmoid activation function; adjust the input first concatenated feature map based on the flame attention weights, and output the adjusted feature map.

4. The method for fire detection with lightweight and high robustness in diverse environments as claimed in claim 1 wherein, The high-frequency detail path is represented by the following formula (1): （1） in, Indicates a high-frequency path; ( () indicates batch normalization processing; Indicates the input feature map; This represents a depthwise separable convolution of size 3×3; The low-frequency shape path is represented by the following formula (2): （2） in, Indicates a low-frequency path; This represents a 1×1 convolution; This represents average pooling with a size of 5×5.

5. The fire detection method with lightweight and high robustness in diverse environments according to claim 4, characterized in that, The average pooling process is represented by the following formula (3): （3） Where c represents the channel index, ranging from (0 to C-1); i and j represent the spatial coordinates; and m and n represent the offsets within the convolution kernel, ranging from (-1, 0, 1).

6. The fire detection method with lightweight and high robustness in diverse environments according to claim 4, characterized in that, The batch normalization process is represented by the following formula (4): = + （4） in, This represents a feature map that has undergone batch normalization. Indicates the learnable scaling parameter; Indicates batch variance; Indicates the batch mean; This represents the input feature map, specifically the element value of the c-th channel at spatial location (i,j) (the pixel in the i-th row and j-th column of the feature map). This represents the learnable offset parameter.

7. The fire detection method with lightweight and high robustness in diverse environments according to claim 3, characterized in that, The process of calculating the flame attention weights based on the concatenated features of S25 using 1×1 dimensionality-reduced convolution, ReLU activation function, and Sigmoid activation function is expressed by the following formulas (5)-(8): （5） （6） （7） （8） in, This represents the feature map that has been reduced and fused using 1×1 convolution; Indicates the features after splicing; This represents the feature map output after a nonlinear transformation using the ReLU activation function; denoted by ReLU activation function; C represents the number of channels in the input feature map; H represents the height of the feature map; W represents the width of the feature map; Let be the real number field, indicating that each element is a real number; This represents a feature map that has been upscaled back to the original number of channels after a 1×1 convolution. This represents the Sigmoid activation function; Indicates the flame attention weight; This represents a 1×1 dimension-reduced convolution.

8. A lightweight and robust fire detection device for diverse environments, wherein the lightweight and robust fire detection device for diverse environments is used to implement the lightweight and robust fire detection method for diverse environments as described in any one of claims 1-7, characterized in that, The device includes: A building unit is used to construct a fire detection model based on YOLOv8. The model architecture includes: a heterogeneous multi-branch network structure module, a flame frequency domain sensing module, a C2f module, and a fast spatial pyramid pooling module. The heterogeneous multi-branch network structure module is embedded in the C2f module, and the flame frequency domain sensing module is embedded in the fast spatial pyramid pooling module. The training unit is used to acquire fire and smoke detection datasets; the datasets are divided into training sets and input into the fire detection model for training; feature maps of different receptive fields are extracted through a heterogeneous multi-branch network structure module; the feature maps of different receptive fields are input into the flame frequency domain perception module to extract low-frequency feature maps and high-frequency feature maps of the flame, and then spliced ​​to obtain a spliced ​​feature map; the fire detection model is trained based on the spliced ​​feature map to obtain a trained fire detection model. The detection unit is used to acquire the fire image to be detected; it inputs the fire image to be detected into the trained fire detection model and outputs the detection result.

9. A fire detection device with lightweight and high robustness for diverse environments, characterized in that, The lightweight and robust fire detection equipment for diverse environments includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images