A flame detection method for an intelligent operation and maintenance fire patrol robot

Abstract

The present application relates to a kind of fire detection methods for intelligent operation and maintenance fire patrol robot, belong to computer vision field.It includes the following steps: obtaining fire scene image, constructs fire detection dataset;Fire detection model is constructed, the fire detection model takes YOLOv5s as benchmark model, including input layer, backbone network, neck network and detection head;The C3 module of the end in the backbone network of YOLOv5s model is replaced as DenseC3 module;Multidimensional cooperation attention module MCA is embedded in the output end of neck network;Group convolution module is introduced in neck network, and the CBS module of neck network is replaced as GSCBS module;Fire detection model is trained using dataset, and the trained model is obtained;The image to be detected is input into the trained model, and the fire detection result is obtained.The present application can improve the precision of flame detection while ensuring real-time.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a flame detection method for an intelligent operation and maintenance fire inspection robot. Background Technology

[0002] Flame detection is a core function of intelligent fire inspection robots, directly determining the timeliness and reliability of fire warnings. Current mainstream detection technologies include thermal imaging, smoke sensing, and optical image recognition. Among these, visible light image processing technology based on optical sensors has become the primary solution for robot vision inspection due to its low cost and rich information. Among existing target detection algorithms, the YOLO series of one-stage algorithms is widely used in the field of robot vision due to its balance between real-time performance and accuracy. YOLOv5, as a mature version of this series, embeds the CSPDarknet53 backbone network and the Mosaic data augmentation module, providing five scale models: n, s, m, l, and x. YOLOv5s, due to its lightweight characteristics, is suitable for edge computing scenarios and is often used as the benchmark model for fire inspection robot detection tasks.

[0003] However, in actual fire inspection scenarios, there are three key problems with flame targets: First, the initial target size is small, and the pixel ratio is often less than 32×32. Existing models are prone to losing detailed information when extracting features from high-rise buildings, leading to missed detections. Second, the aspect ratio is unbalanced, and irregularly spreading flames can easily introduce invalid background information, interfering with feature recognition and increasing the false detection rate. Third, there is a contradiction between performance and lightweight design. If the detection accuracy is improved by deepening the network, it will lead to a surge in the number of model parameters and computational load, which cannot meet the real-time requirements and hardware resource constraints (such as video memory and computing power) of fire inspection robots.

[0004] While existing technologies such as SE and CBAM can enhance feature discriminativeness and depthwise separable convolutions can reduce computational cost, their individual use cannot simultaneously solve the three major problems mentioned above: attention modules are not optimized for the aspect ratio imbalance of flames, and lightweight convolutions are prone to sacrificing feature accuracy. Therefore, there is an urgent need for a flame detection method that balances small target detection capability, aspect ratio adaptability, and lightweight characteristics to meet the engineering application requirements of intelligent operation and maintenance fire inspection robots. Summary of the Invention

[0005] To achieve the above objectives, the present invention employs the following technical solution:

[0006] This invention provides a flame detection method for an intelligent fire inspection robot, comprising the following steps:

[0007] S1. Acquire fire scene images and construct a flame detection dataset;

[0008] S2. Construct a flame detection model. The flame detection model is based on YOLOv5s and includes an input layer, a backbone network, a neck network, and a detection head. To address the issue of small target feature loss, the C3 module at the end of the backbone network of the YOLOv5s model is replaced with a DenseC3 module. To address the issue of flame aspect ratio imbalance, a multidimensional collaborative attention module (MCA) is embedded at the output of the neck network. Since the DenseC3 module achieves feature reuse through dense cross-layer connections, and the MCA module relies on a multi-branch parallel structure for context information fusion, the combined effect of these two modules significantly increases the computational load of the model on edge devices such as fire inspection robots, leading to a decrease in real-time target detection performance. Therefore, this invention introduces a grouped convolutional module into the neck network, replacing the CBS module of the neck network with a GSCBS module.

[0009] Furthermore, the input layer sets a fixed size for the input image and uses the Mosaic data augmentation method to randomly stitch together four images to expand the training samples.

[0010] Furthermore, the DenseC3 module employs a dual-branch parallel processing structure, comprising a main branch and an auxiliary branch. The main branch adopts the tightly connected architecture of the DenseNet network, including several CBS modules. The auxiliary branch includes one CBS module. Each CBS module includes a Conv2d convolutional module, a batch normalization layer, and a SiLU activation function. The input features of the DenseC3 module... Feature extraction is performed in the main branch, resulting in the output features of the main branch. Input characteristics of the DenseC3 module Feature extraction is performed through an auxiliary branch, resulting in the output features of the auxiliary branch. ; the output characteristics of the main branch Output characteristics of auxiliary branches After channel-dimensional fusion, the output features of the DenseC3 module are obtained through a CBS module. .

[0011] Furthermore, the GSCBS module includes a grouped convolutional module, a batch normalization layer, and a SiLU activation function. In the grouped convolutional module, the input features of the grouped convolutional module... Channel dimension compression is performed through convolution operations, reducing the number of feature channels to half of the original size, resulting in compressed features. ; Use depthwise separable convolution to compress features Spatial feature extraction is performed to obtain depthwise separable convolutional features. Compression features and depthwise separable convolutional features Channel-level fusion is performed to obtain fused features. The fusion feature By employing a channel shuffling algorithm, the inter-channel correlation constraints of convolution operations are broken, enhancing the cross-channel information exchange capability and obtaining the output features of the grouped convolution module. .

[0012] Furthermore, the output features of the grouped convolution module After processing with a batch normalization layer and the SiLU activation function, the output features of the GSCBS module are obtained.

[0013] Furthermore, the detection head adopts the multi-scale detection head structure of the YOLOv5s model, uses the anchor box mechanism to initialize the prediction box position, eliminates redundant boxes through non-maximum suppression, and outputs the prediction result with the highest confidence.

[0014] S3. Train the flame detection model using the dataset to obtain a trained model; input the image to be detected into the trained model to obtain the flame detection result.

[0015] The advantages of this invention are:

[0016] This invention proposes a flame detection algorithm based on an improved YOLOv5s architecture. A novel DenseC3 module is designed to replace the native C3 structure at the ends of the backbone network, enhancing the model's gradient propagation efficiency for small target features. A multi-dimensional collaborative attention module is integrated at the ends of the neck network to improve the model's feature representation ability for targets with abnormal aspect ratios. The neck network is reconstructed using the GSConv lightweight convolution operator, achieving computational complexity compression while maintaining detection accuracy. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0018] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0019] Figure 2 Original flame image;

[0020] Figure 3 A heatmap of detection for the YOLOv5s model;

[0021] Figure 4 This is a detection thermogram of the method of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1

[0024] In this embodiment, as Figure 1 As shown, this invention provides a flame detection method for an intelligent operation and maintenance fire inspection robot, the specific steps of which include:

[0025] S1. Acquire fire scene images and construct a flame detection dataset;

[0026] Specifically, to address the issues of insufficient scenario coverage and low sample quality in existing datasets, this step constructs a high-quality flame dataset through multi-source data collection, data cleaning, standardized labeling, and partitioning. The specific operations are as follows:

[0027] Multi-source data acquisition: Fire scene images were selected from publicly available security video databases and fire department case data, covering 6 major categories of scenarios including industrial fires, forest wildfires, and urban building fires. A total of 3,357 original images were collected to ensure that the samples contain multi-dimensional target features such as single fire source, multiple fire points, and regular / irregular fire patterns. Small flame samples with a pixel ratio of <32×32 were also included.

[0028] Data cleaning: The SSIM (Structural Similarity Index) is used to remove duplicate samples, highly similar samples (SSIM value > 0.95), and erroneous samples (such as pure smoke images without flames) to ensure the uniqueness and validity of the data.

[0029] Labeling and format conversion: The LabelImg tool is used to annotate the bounding box of the flame target. The coordinates of the annotation are normalized to generate a YOLO format annotation file (including target category, bounding box center point coordinates, width and height).

[0030] Dataset partitioning: A stratified random sampling strategy was adopted to maintain the consistency of sample distribution based on three dimensions: fire source type, scene complexity, and image resolution. The final dataset was divided into a training set of 2350 images, a validation set of 671 images, and a test set of 336 images, with the ratio strictly controlled at 7:2:1, providing standardized data support for model training and evaluation.

[0031] S2. Construct a flame detection model. The flame detection model is based on YOLOv5s and includes an input layer, a backbone network, a neck network, and a detection head. To address the issue of small target feature loss, the C3 module at the end of the backbone network of the YOLOv5s model is replaced with a DenseC3 module. To address the issue of flame aspect ratio imbalance, a multidimensional collaborative attention module (MCA) is embedded at the output of the neck network. Since the DenseC3 module achieves feature reuse through dense cross-layer connections, and the MCA module relies on a multi-branch parallel structure for context information fusion, the combined effect of these two modules significantly increases the computational load of the model on edge devices such as fire inspection robots, leading to a decrease in real-time target detection performance. Therefore, this invention introduces a grouped convolutional module into the neck network, replacing the CBS module of the neck network with a GSCBS module.

[0032] Specifically, the input layer sets a fixed size for the input image and uses the Mosaic data augmentation method to randomly stitch together four images to expand the training samples.

[0033] Specifically, the YOLOv5s model uses the CBS module to achieve multi-level feature extraction and leverages the C3 module to fuse cross-scale features to enhance representation capabilities. However, as network depth increases, the feature map resolution decreases, weakening pixel-level detail preservation (especially causing increased false negative rates for small object detection). Simultaneously, the vanishing gradient problem hinders effective updating of shallow parameters, affecting discriminative feature learning. To balance the contradiction between model depth and feature preservation, this invention proposes the DenseC3 module, optimizing the feature reuse mechanism to address the aforementioned structural defects. The DenseC3 module is a dual-branch parallel processing structure, including a main branch and an auxiliary branch. The main branch adopts the tightly connected architecture of the DenseNet network, including several CBS modules; the auxiliary branch includes one CBS module. The CBS module includes a Conv2d convolutional module, a batch normalization layer, and a SiLU activation function. In the DenseC3 module, the input features... Feature extraction is performed in the main branch, resulting in the output features of the main branch. Input characteristics of the DenseC3 module Feature extraction is performed through an auxiliary branch, resulting in the output features of the auxiliary branch. ; the output characteristics of the main branch Output characteristics of auxiliary branches After channel-dimensional fusion, the output features of the DenseC3 module are obtained through a CBS module. The formula is expressed as follows:

[0034] ,

[0035] ,

[0036]

[0037] in, Indicates the first One CBS module; the second; Indicates the first Output characteristics of each CBS module; express Module operations; This indicates a feature splicing operation.

[0038] Preferably, the main branch consists of three CBS modules, with the output of the first CBS module being... The input to the second CBS module is... and The features after channel-dimensional fusion are output by the second CBS module. The input to the third CBS module is... , as well as After channel-dimensional fusion, the output of the third CBS module is: .

[0039] Specifically, in the analysis of the flame dataset, one of the significant characteristics of flame targets is the prevalence of an imbalanced aspect ratio. This characteristic leads to the model not only introducing excessive invalid information during target feature extraction but also losing some key feature information, which adversely affects the model's detection accuracy. To address this issue, this invention embeds a multidimensional collaborative attention module (MCA) at the output of the neck network. The MCA module is a multidimensional, three-branch attention collaboration mechanism that effectively reweights the original feature map by comprehensively evaluating the attention weights in the three dimensions of height, width, and channel, thereby significantly improving the discriminability of target features and further optimizing the model's detection accuracy.

[0040] Specifically, the GSCBS module includes a grouped convolutional module, a batch normalization layer, and a SiLU activation function. In the grouped convolutional module, the input features... Channel dimension compression is performed through convolution operations, reducing the number of feature channels to half of the original size, resulting in compressed features. ; Use depthwise separable convolution to compress features Spatial feature extraction is performed to obtain depthwise separable convolutional features. Compression features and depthwise separable convolutional features Channel-level fusion is performed to obtain fused features. The fusion feature By employing a channel shuffling algorithm, the inter-channel correlation constraints of convolution operations are broken, enhancing the cross-channel information exchange capability and obtaining the output features of the grouped convolution module. .

[0041] Furthermore, the output features of the grouped convolution module After processing with a batch normalization layer and the SiLU activation function, the output features of the GSCBS module are obtained.

[0042] Furthermore, the detection head adopts the multi-scale detection head structure of the YOLOv5s model, uses the anchor box mechanism to initialize the prediction box position, eliminates redundant boxes through non-maximum suppression, and outputs the prediction result with the highest confidence.

[0043] S3. Train the flame detection model using the dataset to obtain a trained model; input the image to be detected into the trained model to obtain the flame detection result.

[0044] Example 2

[0045] In this embodiment, based on a self-built dataset and a specified hardware environment, the training and performance optimization of the DMG-YOLOv5 model are completed to ensure that the model meets the accuracy and real-time requirements of the fire inspection robot.

[0046] Training environment configuration: The hardware uses an NVIDIA GeForce RTX 4060 GPU (providing computing power support) and an Intel(R) Core (TM) i7-10700F CPU; the software environment is Python 3.8, PyTorch 1.12.1, CUDA 12.2, and Windows 10 operating system to ensure training efficiency and compatibility.

[0047] Hyperparameter settings: 150 training epochs (to ensure model convergence), batch size 16 (to balance memory usage and training speed), initial learning rate 0.01 (using SGD optimizer, weight decay 0.0005 to suppress overfitting), input image resolution 640×640 (matching model design).

[0048] Training process monitoring: Real-time tracking of bounding box regression loss, classification loss, and comprehensive detection loss to ensure that the losses on the training set and validation set converge to a low level; an early stopping strategy is adopted, and training is stopped if the validation set mAP@0.5 shows no improvement for 10 consecutive rounds to avoid overfitting.

[0049] The comparative experimental results are shown in Tables 1 and 2. On the self-built flame dataset, the DMG-YOLOv5 model exhibits a significant overall performance advantage. Specifically, YOLOv8s slightly outperforms this solution in recall by less than 1%, but the number of parameters and computational cost of YOLOv8s increases by more than 50% compared to the proposed DMG-YOLOv5 model. This excessively large number of parameters and computational cost can lead to problems such as high memory usage and excessive inference latency when deployed on edge computing devices. In contrast, compared to the baseline YOLOv5s model, the increase in the number of parameters and computational cost of DMG-YOLOv5 is less than 3%, a relatively small increase. In summary, the DMG-YOLOv5 model effectively controls the model size while ensuring detection accuracy and response speed, demonstrating its balance in practical applications. Therefore, the DMG-YOLOv5 model proposed in this invention meets the stringent requirements of fire inspection robots for real-time detection and low-power operation, providing a reliable solution for embedded flame detection in the robotics field.

[0050] Table 1. Comparison of detection performance of the method of the present invention with other popular models.

[0051]

[0052] Table 2 Comparison of parameters between the method of this invention and other popular models

[0053]

[0054] Grad-CAM, a gradient-based visualization method, was used to compare and analyze the improved DMG-YOLOv5 (this invention) with the original YOLOv5s model. The experimental results are as follows: Figure 3 , Figure 4 As shown, Figure 2 For the corresponding original image; from Figure 3 , Figure 4 A comparison of the third image in the first row and the first image in the second row shows that, when processing small-sized flame targets with abnormal aspect ratios, the feature distribution generated by DMG-YOLOv5 more closely matches the actual target location, and exhibits a more complete feature coverage compared to YOLOv5s. Combined with evaluation metrics such as mAP@0.5, the flame detection accuracy of DMG-YOLOv5 is significantly improved compared to the baseline model YOLOv5s, verifying the effectiveness of the proposed improvement strategy.

[0055] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A flame detection method for an intelligent operation and maintenance fire patrol robot, characterized in that, Includes the following steps: Acquire images of fire scenes and construct a flame detection dataset; A flame detection model is constructed, which is based on YOLOv5s and includes an input layer, a backbone network, a neck network, and a detection head. The C3 module at the end of the backbone network of the YOLOv5s model is replaced with a DenseC3 module. A multidimensional collaborative attention module (MCA) is embedded at the output of the neck network. A grouped convolution module is introduced into the neck network, and the CBS module of the neck network is replaced with a GSCBS module. The DenseC3 module is a double-branch parallel processing structure, including a main branch and an auxiliary branch; the main branch adopts a compact connection architecture in a DenseNet network and includes a plurality of CBS modules; the auxiliary branch includes one CBS module; the CBS module includes a Conv2d convolution module, a batch normalization layer, and a SiLU activation function; in the DenseC3 module, input features of the DenseC3 module are sequentially processed by the Conv2d convolution module, the batch normalization layer, and the SiLU activation function feature extraction is performed through the main branch to obtain output features of the main branch ​ Input characteristics of the DenseC3 module Feature extraction is performed using an auxiliary branch, resulting in the output features of the auxiliary branch. ; the output characteristics of the main branch Output characteristics of auxiliary branches After channel-dimensional fusion, the output features of the DenseC3 module are obtained through a CBS module. ; The GSCBS module includes a grouped convolutional module, a batch normalization layer, and a SiLU activation function; in the grouped convolutional module, the input features of the grouped convolutional module... Channel dimension compression is performed through convolution operations, reducing the number of feature channels to half of the original size, resulting in compressed features. ; Use depthwise separable convolution to compress features Spatial feature extraction is performed to obtain depthwise separable convolutional features. Compression features and depthwise separable convolutional features Channel-level fusion is performed to obtain fused features. The fusion feature The output features of the grouped convolutional module are obtained through a channel shuffling algorithm. ; The flame detection model is trained using a dataset to obtain a trained model; the image to be detected is then input into the trained model to obtain the flame detection result.

2. The flame detection method for an intelligent operation and maintenance fire inspection robot according to claim 1, characterized in that, The output features of the grouped convolution module After processing with a batch normalization layer and the SiLU activation function, the output features of the GSCBS module are obtained.

3. The flame detection method for an intelligent operation and maintenance fire inspection robot according to claim 2, characterized in that, The input layer sets a fixed size for the input image and uses the Mosaic data augmentation method to randomly stitch together four images to expand the training samples.

4. A flame detection method for an intelligent operation and maintenance fire inspection robot according to claim 3, characterized in that, The detection head adopts the multi-scale detection head structure of the YOLOv5s model, uses the anchor box mechanism to initialize the prediction box position, removes redundant boxes through non-maximum suppression, and outputs the prediction result with the highest confidence.

Images