Flame detection method based on improved YOLOv5 model

By improving the YOLOv5 model, embedding an attention mechanism and adding a new detection layer, and introducing a boundary loss function, the problems of slow flame detection speed and low accuracy were solved, achieving efficient identification of small flame areas and meeting the real-time detection needs of firefighting robots.

CN114662605BInactive Publication Date: 2024-11-19BEIJING INFORMATION SCI & TECH UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210321847.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-29
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing flame detection methods are slow and have low accuracy in complex urban environments, especially lacking precise identification of small flame areas, which cannot meet the real-time and accuracy requirements of fire-fighting robots for flame detection.

Method used

This paper improves upon the YOLOv5 model by embedding attention mechanism units into the baseline network module, adding a new detection layer in the detection module, and introducing a boundary loss function to optimize the model and enhance its ability to detect small targets.

Benefits of technology

It improves the accuracy and real-time performance of flame detection, enabling better application in firefighting robots and enhancing firefighting efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114662605B_ABST
    Figure CN114662605B_ABST
Patent Text Reader

Abstract

The invention discloses a flame detection method based on an improved YOLOv5 model, comprising: establishing a flame data set; taking the YOLOv5 model as a basic model, embedding an attention mechanism unit, adding a new detection layer, and introducing a boundary loss function to obtain a first YOLOv5 model; pre-training the first YOLOv5 model to obtain a second YOLOv5 model; optimizing the second YOLOv5 model to obtain a third YOLOv5 model; inputting a test image into the third YOLOv5 model to obtain a flame detection result. By embedding the attention mechanism unit, the relationship between channels and spatial information can be considered at the same time, so that the first YOLOv5 model can more accurately locate and identify the target area; adding a new detection layer for detecting small-sized targets; introducing a boundary loss function to be more robust to small data sets and noise, and more suitable for lightweight models.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of deep learning and target detection, and more specifically, to a flame detection method based on an improved YOLOv5 model. Background Art

[0002] Fire is one of the major disasters that threaten human life and property safety. Some inconspicuous sparks may cause serious economic losses and casualties. With the rapid development of the economy, the scale of urban buildings is getting larger and larger. When a fire occurs, the current means of firefighting are still mainly based on manual firefighting by firefighters. In this process, firefighters are often injured. Therefore, the use of firefighting robots to replace manual firefighting has become a development trend. Firefighting robots mainly use cameras to explore the fire scene, so real-time and accurate target detection of flames based on images has become the key to the smooth and efficient firefighting of firefighting robots.

[0003] Traditional flame detection technology mainly extracts features, for example: using RGB color space, analyzing the different characteristics of the three RGB channels for flame detection; using the flame characteristics of red and blue light generated when the flame burns, using the YCbCr color space with brightness and red and blue color information to extract flame features for flame detection; using the frame difference method, using the area growth ratio as the basis for judging the frame difference for flame detection. These traditional flame recognition methods meet the requirements of flame detection to a certain extent, but in complex urban environments, there are still shortcomings such as slow detection speed, low accuracy, and lack of accurate recognition of small flame areas. Therefore, these traditional detection methods cannot meet the requirements of firefighting robots for flame detection real-time and accuracy.

[0004] In recent years, various deep learning networks based on image processing have developed rapidly. In the field of target detection, deep learning algorithms represented by convolutional neural networks have improved detection accuracy and speed. Existing target detection algorithms are mainly divided into two categories: one is a two-stage detection algorithm. This type of algorithm first generates a pre-selected box that may contain the detected object in the candidate area, and then detects the object through a convolutional neural network. Common ones include R-CNN, Fast R-CNN, SPP-Net, etc. This type of algorithm has high detection accuracy, but in deep networks, the detection speed is slow and cannot meet the real-time requirements of flame detection. The other is a single-stage detection algorithm. This type of algorithm combines the two stages of candidate area division and object detection into one. Common ones include the Yolo series algorithm. This type of algorithm is characterized by a simple network structure and fast detection speed, which can meet the needs of real-time detection tasks, but the robustness and detection accuracy of small target detection need to be improved.

[0005] In summary, the present invention is based on the improvement of the YOLOv5 network model, which improves its detection accuracy and small target detection capability. The improved flame detection algorithm can meet the real-time and accuracy requirements of flame detection, and can be better applied to firefighting robots to help firefighters improve firefighting efficiency. Summary of the invention

[0006] In view of this, the present invention provides a flame detection method based on an improved YOLOv5 model, comprising:

[0007] Establishing a flame data set, wherein the flame data set includes a plurality of flame images;

[0008] The YOLOv5 model is used as the basic model. The YOLOv5 model includes an input end, a reference network module, a feature fusion module and a detection module which are arranged in sequence. An attention mechanism unit is embedded in the reference network module, a new detection layer is added to the detection module, and a boundary loss function is introduced in the detection module to obtain a first YOLOv5 model. The boundary loss function is calculated according to the following method:

[0009]

[0010] in, is the difference between the predicted border and the result border of the flame image; I O U is the interaction ratio, α is the adjustment parameter;

[0011] Pre-training the first YOLOv5 model using an image dataset to obtain a second YOLOv5 model;

[0012] The flame data set is divided into a training set, a validation set and a test set, the flame image in the test set is a test image, the training set and the validation set are input into the second YOLOv5 model for optimization, and a third YOLOv5 model is obtained;

[0013] The test image of the test set is input into the third YOLOv5 model to obtain a flame detection result.

[0014] Preferably, the establishing of the flame data set comprises:

[0015] Collecting a plurality of flame images, the plurality of flame images are all different, screening the plurality of flame images, and deleting the flame images whose flame areas exceed 80% of the flame image area to obtain a first data set;

[0016] Data enhancement is performed on the flame images in the first data set, and the data enhancement method includes: rotating, mirroring, scaling, splicing and brightness balancing the flame images in the first data set to obtain multiple expanded flame images, and merging the multiple expanded flame images with the first data set to form the flame data set.

[0017] Preferably, after establishing the flame data set, the method further includes labeling the flame image in the flame data set;

[0018] A flame portion of the flame image is encircled by a frame, information of the frame is recorded, and the information of the frame is converted into a first format for storage.

[0019] Preferably, the reference network module includes a first unit, a second unit, a third unit and a fourth unit; the first unit is a slice structure; the second unit is composed of a convolutional layer, batch normalization and a first activation function; the third unit is composed of the convolutional layer and a bottleneck layer; the fourth unit is composed of the convolutional layer and a pooling layer;

[0020] The reference network module extracts feature information of the test image to obtain a first feature map.

[0021] Preferably, embedding an attention mechanism unit in the reference network module comprises:

[0022] The attention mechanism unit obtains a first vector and a second vector by performing average pooling on the first feature map along a first direction and a second direction respectively, performs channel fusion on the first vector and the second vector, performs channel compression using a convolution block, encodes the spatial information of the first feature map along the first direction and the second direction, separates the channels, obtains a second feature map with the same number of channels as the first feature map through the convolution block, and performs normalization weighting through a second activation function.

[0023] Preferably, the adding a new detection layer to the detection module includes:

[0024] The size of the newly added detection layer is 160×160, which is used to detect targets with a receptive field size larger than 4×4;

[0025] The feature fusion module includes 24 layers of feature extraction layers, 2 layers of the feature extraction layers in the feature fusion module are added to match the detection layer, the second feature map is processed by the second layer of the feature extraction layer of the feature fusion module to obtain a third feature map, the second feature map is processed by the twenty-sixth layer of the feature extraction layer of the feature fusion module to obtain a fourth feature map, and the third feature map is fused with the fourth feature map to obtain a fifth feature map;

[0026] The newly added detection layer detects the fifth feature map.

[0027] Preferably, the third YOLOv5 model adopts a stochastic gradient descent optimizer.

[0028] Preferably, the method further includes evaluating the detection effect of the third YOLOv5 model, and calculating the result in the following manner:

[0029]

[0030] Among them, mAP is the accuracy detection value, m is the average, Precision(K) is the accuracy, Recall(K) is the recall, and AP is the integral of the accuracy index over the recall index in the range of 0 to 1.

[0031] Preferably, the integral of the accuracy index to the recall index in the range of 0 to 1 is calculated as follows:

[0032]

[0033] Among them, Precision is the accuracy, Recall is the recall rate, and AP is the integral of the accuracy index and the recall index in the range of 0 to 1.

[0034] Preferably, the accuracy and recall are calculated in the following manner:

[0035]

[0036]

[0037] Wherein, TP is the number of the enclosing boxes that correctly identify the flame, FP is the number of the enclosing boxes that incorrectly identify the flame, and FN is the number of flames that are not identified. Compared with the prior art, the flame detection method based on the improved YOLOv5 model provided by the present invention achieves at least the following beneficial effects:

[0038] In the flame detection method based on the improved YOLOv5 model provided by the present invention, an attention mechanism unit is embedded in the reference network module, which can simultaneously consider the relationship between channels and spatial information, so that the first YOLOv5 model can more accurately locate and identify the target area, and can also reduce the amount of calculation while enabling the first YOLOv5 model to obtain information of a larger area. A new detection layer is added to the detection module to detect small-sized targets, and the characteristic information of the target is fully learned, thereby improving the detection ability of the first YOLOv5 model for small flame areas. The boundary loss function is introduced in the detection module, which is obtained by improving the existing boundary loss function. By adjusting the parameters, it can have greater flexibility in terms of different levels of border regression accuracy. The introduced boundary loss function is more robust to small data sets and noise, and is more suitable for lightweight models.

[0039] Of course, any product implementing the present invention does not necessarily need to achieve all of the technical effects described above at the same time.

[0040] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0042] Figure 1 It is a flow chart of the flame detection method based on the improved YOLOv5 model proposed by the present invention;

[0043] Figure 2 It is a schematic diagram of a structure in which an attention mechanism unit is embedded in a benchmark network module;

[0044] Figure 3 It is a structural diagram of adding a new detection layer to the detection module;

[0045] Figure 4 It is a structural diagram of the feature fusion module;

[0046] Figure 5 It is a flow chart for building a flame data set;

[0047] Figure 6 It is a kind of structural diagram of the first unit;

[0048] Figure 7 It is a schematic diagram of a slicing operation in the first unit;

[0049] Figure 8 It is a structural diagram of the attention mechanism unit;

[0050] Fig. 9 It is the mAP value change curve of the original YOLOv5 model and the improved YOLOv5 model;

[0051] Fig.10 It is a graph of the loss function changes of the original YOLOv5 model and the improved YOLOv5 model. DETAILED DESCRIPTION

[0052] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0053] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0054] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0055] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0056] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0057] Combination Figure 1 , Figure 2 , Figure 3 and Figure 4 , Figure 1 It is a flow chart of the flame detection method based on the improved YOLOv5 model proposed by the present invention; Figure 2 It is a schematic diagram of a structure in which an attention mechanism unit is embedded in a benchmark network module; Figure 3 It is a structural diagram of adding a new detection layer to the detection module; Figure 4 : is a structural schematic diagram of a feature fusion module part, illustrating a specific embodiment of a flame detection method based on an improved YOLOv5 model proposed by the present invention, including:

[0058] S101: Establish a flame data set, where the flame data set includes multiple flame images;

[0059] S102: A YOLOv5 model is used as a basic model. The YOLOv5 model includes an input terminal, a reference network module, a feature fusion module, and a detection module that are sequentially arranged. An attention mechanism unit is embedded in the reference network module, a new detection layer is added to the detection module, and a boundary loss function is introduced in the detection module to obtain a first YOLOv5 model. The boundary loss function is calculated according to the following method:

[0060]

[0061] in, is the difference between the predicted bounding box and the result bounding box of the flame image; I O U is the interaction ratio, α is the adjustment parameter;

[0062] In step S102, the YOLOv5 model is improved based on the YOLOv3 model, including four models: YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. Here, the YOLOv5s model is used as the basic model. The benchmark network module is the backbone of the detection network, which extracts the high, medium, and low-level features of the test image. Figure 2 and Figure 3 The input end (Input) inputs the test image, and the data set can be processed at the input end stage; the benchmark network module (Backbone) is the backbone of the detection network, which is used to extract the high, medium and low-level features of the test image; the feature fusion module (Neck) is used to generate the feature pyramid; the detection module (Detect) is used to convolve again to obtain the prediction result. See Figure 4 , X represents upsampling, Y represents downsampling, and the feature fusion module is mainly used to generate feature pyramids. It adopts the structure of FPN layer + PAN layer. The FPN layer conveys strong semantic features from top to bottom, while the PAN layer conveys strong positioning features from bottom to top. The feature pyramid will enhance the model's detection of objects of different scales, so that it can recognize the same object of different sizes and scales. The detection module obtains the prediction result through convolution again.

[0063] It can be understood that embedding the attention mechanism unit in the benchmark network module can simultaneously consider the relationship between channels and spatial information, so that the first YOLOv5 model can more accurately locate and identify the target area, and can also enable the first YOLOv5 model to obtain information of a larger area while reducing the amount of calculation. A new detection layer is added to the detection module to detect small-sized targets and fully learn the characteristic information of the target, thereby improving the detection ability of the first YOLOv5 model for small flame areas. The boundary loss function introduced in the detection module is obtained by improving the existing boundary loss function. By adjusting the parameters, it can have greater flexibility in different levels of border regression accuracy. The introduced boundary loss function is more robust to small data sets and noise, and is more suitable for lightweight models.

[0064] It is understandable that the introduction of boundary loss function in the detection module can improve the robustness to small target data and noise, making the regression more accurate. The full name of IoU is interaction ratio, that is, the ratio of the intersection and union of the predicted border and the real border. IoU is a very important function for calculating the mAP value of the target detection algorithm performance, and is a standard for measuring the accuracy of detecting the corresponding object in a specific data set. The closer the predicted border and the real border are, the closer the IoU is to 1. In this way, the model can make the model obtain better prediction results by continuously reducing the Loss. In this embodiment, the CIou boundary loss function used by the original YOLOv5 model is replaced by a new boundary loss function, α-IoU, that is, the power transformation is introduced into the existing IoU_Loss, and by adjusting α, the detector has greater flexibility in achieving different levels of bbox regression accuracy. Experiments have shown that α-IoU is more robust to small data sets and noise, and is more suitable for lightweight models. After many experiments, it has been proved that the training effect is best when the value of α in the flame detection method of this embodiment is 3.

[0065] S103: Pre-training the first YOLOv5 model using the image data set to obtain a second YOLOv5 model;

[0066] In step S103, since the first YOLOv5 model is improved based on the YOLOv5 model, the network structure and the number of initial weight parameters are changed, so the first YOLOv5 model needs to be pre-trained. In this embodiment, the image data set uses the Microsoft COCO data set for pre-training. The Microsoft COCO data set is one of the most commonly used and widely used large-scale image data sets, containing more than 330,000 pictures, and is designed for target detection and semantic segmentation tasks. After pre-training, the second YOLOv5 model is obtained for migration for flame target detection, which can improve the generalization ability of flame target detection and improve the overfitting problem caused by insufficient flame data set to a certain extent, thereby promoting the establishment of recognition model and improving recognition accuracy.

[0067] S104: Divide the flame data set into a training set, a validation set, and a test set, wherein the flame image in the test set is the test image, and input the training set and the validation set into the second YOLOv5 model for optimization to obtain a third YOLOv5 model;

[0068] In step S104, the validation set can be used to verify the results obtained by the model later. Specifically, the flame data set can be divided into a ratio of 8:1:1 for training set: validation set: test set, but it is not limited thereto and can be set according to needs.

[0069] S105: Input the test image of the test set into the third YOLOv5 model to obtain a flame detection result.

[0070] It can be understood that in this embodiment, the YOLOv5 model is improved to obtain a model suitable for flame target detection, and the training set and the verification set are input into the second YOLOv5 model for optimization training. The training process is as follows: the input image will be feature extracted through the feature extraction network to obtain feature maps of different scales, and then these feature maps are classified and regressed, and the regression results are subjected to feature reconstruction operations to obtain more refined feature maps. On this basis, classification and regression operations are performed again, and the loss is calculated to complete the flame target detection based on the present invention.

[0071] In some optional embodiments, referring to Figure 5 , Figure 5 It is a flow chart for establishing a flame data set. Establishing a flame data set includes:

[0072] S1011: collecting a plurality of flame images, the plurality of flame images are all different, screening the plurality of flame images, deleting flame images whose flame areas exceed 80% of the flame image area to obtain a first data set;

[0073] In step S1011, the format of the multiple flame images collected is JPG format, and the collection method can be obtained through data set acquisition, self-photography or other methods, and there is no specific limitation here.

[0074] S1012: performing data enhancement on the flame images in the first data set, wherein the data enhancement method includes: rotating, mirroring, scaling, splicing and brightness balancing the flame images in the first data set to obtain multiple expanded flame images, and merging the multiple expanded flame images with the first data set to form a flame data set.

[0075] It is understandable that in this embodiment, OpenCV is used to perform data enhancement on the flame image in the first data set to enhance the generalization ability of the network model. OpenCV is a cross-platform computer vision and machine learning software library released based on the Apache2.0 license (open source). In step S1012, rotation and mirroring can improve the detection performance and robustness of the network; brightness balance can eliminate the impact of brightness deviation caused by ambient lighting changes and sensor differences on network performance, but the data enhancement method is not limited to this, and other data enhancement methods are also within the protection scope of this embodiment.

[0076] In some optional embodiments, after establishing the flame data set, the step further includes labeling the flame images in the flame data set;

[0077] A flame portion of the flame image is encircled by a frame, information of the frame is recorded, and the information of the frame is converted into a first format for storage.

[0078] It can be understood that in this embodiment, the flame image is annotated with the frame using the target detection open source software LabelImg, and the annotated content is the coordinates of the frame, etc., and the file generated by the annotation is converted into a file in the first format, and the first format is a format available to the network model. For example, the file format generated by the annotation is in the xml format, and the xml format file is converted into the txt text format through the data conversion code, and the txt text format is a format available to the network model. The conversion method of the file generated by the annotation in this embodiment and the first format of the txt text format are only for illustration, but are not limited to this.

[0079] In some optional embodiments, continue to refer to Figure 2 , Figure 3 , Figure 6 and Figure 7 , Figure 6 It is a kind of structural diagram of the first unit; Figure 7It is a slicing operation diagram of the first unit. The benchmark network module includes the first unit, the second unit, the third unit and the fourth unit. The first unit is a slicing structure; the second unit is composed of a convolutional layer, a batch normalization and a first activation function; the third unit is composed of a convolutional layer and a bottleneck layer; the fourth unit is composed of a convolutional layer and a pooling layer; the benchmark network module extracts the feature information of the test image to obtain a first feature map.

[0080] It can be understood that the baseline network module of the YOLOv5 model has a first unit. The first unit shown in box A is a slice structure (Focus slice structure) for slicing to extract common features. The structure is as follows: Figure 6 As shown in the figure, the original 3×640×640 image is converted into a 12×320×320 feature map through slicing and concatenation, and then a convolution operation with 64 convolution kernels is performed to finally convert it into a 64×320×320 feature map. The second unit shown in the B box is the Conv structure, which consists of a convolution layer (cv) + batch normalization (Batchnorm BN) + the first activation function, and the first activation function is the Hardswish activation function. The third unit shown in the C box is the C3 module, which contains 3 standard convolution layers and 2 Bottleneck modules. Its function is to extract rich information features from the input image. The fourth unit shown in the D box is the SPPF module, which mainly performs pooling operations, solves the problem of repeated feature extraction of images by convolutional neural networks, greatly improves the speed of generating candidate frames, and saves computing costs.

[0081] In some optional embodiments, continue to refer to Figure 2 , Figure 3 and Figure 8 , Figure 8 It is a structural diagram of an attention mechanism unit, in which an attention mechanism unit is embedded in a reference network module, including: the attention mechanism unit obtains a first vector and a second vector by averaging the first feature map along a first direction and a second direction respectively, performs channel fusion on the first vector and the second vector, performs channel compression using a convolution block, encodes the spatial information of the first feature map along the first direction and the second direction, and then separates the channels, obtains a second feature map with the same number of channels as the first feature map through a convolution block, and performs normalization weighting through a second activation function.

[0082] It can be understood that the attention mechanism (Coordinate Attention) averages the first feature map in the horizontal and vertical directions (X Avg Pool and Y Avg Pool) to obtain two 1D vectors, and then performs a Concat operation on the spatial dimension, and then compresses the channel through a 1x1 convolution block (Conv2d), and then encodes the spatial information in the vertical and horizontal directions through BatchNorm and Non-linear, and then performs a separation operation (split operation), and then each passes through a 1x1 convolution block (Conv2d) to obtain a second feature map with the same number of channels as the first feature map, and finally uses the second activation function for normalization and weighting, and the second activation function is the Sigmoid activation function. Specifically, the position where the attention mechanism unit is added is after the fourth unit in the benchmark network module, such as Figure 3 At the position shown, adding the attention mechanism unit can consider the relationship between channels and spatial information at the same time, so that the first YOLOv5 model can locate and identify the target area more accurately, and can also enable the first YOLOv5 model to obtain information of a larger area while reducing the amount of calculation. A new detection layer is added to the detection module to detect small-sized targets, fully learn the characteristic information of the target, and thus improve the detection ability of the first YOLOv5 model for small flame areas.

[0083] In some optional embodiments, referring to Figure 2 , add new detection layers to the detection module, including:

[0084] The size of the newly added detection layer is 160×160, which is used to detect targets with a receptive field size larger than 4×4; the feature fusion module includes 24 feature extraction layers, and 2 feature extraction layers in the feature fusion module are added to match the detection layer. The second feature map is processed by the second feature extraction layer of the feature fusion module to obtain the third feature map, and the second feature map is processed by the twenty-sixth feature extraction layer of the feature fusion module to obtain the fourth feature map. The third feature map is fused with the fourth feature map to obtain the fifth feature map; the new detection layer detects the fifth feature map.

[0085] It can be understood that this embodiment adds a 160×160 small target detection layer to the model to detect small-scale targets with a receptive field of more than 4×4, thereby improving the model's detection ability for small flame areas. In the original YOLOv5 model, there are only four detection layers, and the sizes of the detection layers are 80×80, 40×40, 20×20, and 10×10, respectively. Among them, the 80×80 detection layer is used to detect targets with a size of more than 8×8, and the 40×40 detection layer is used to detect targets with a size of more than 16*16. The 20×20 detection layer is used to detect targets with a size of more than 32×32, and the 10×10 detection layer is used to detect targets with a size of more than 64×64. The feature maps obtained by these four detection layers after 6 downsampling operations of the YOLOv5 network are 10×10, 20×20, 40×40, and 80×80, respectively. Among these four feature maps, the 80×80 feature map is responsible for detecting small targets, and corresponding to 640×640, the receptive field of each feature map is 640 / 80=8×8. That is, if the width or height of the target in the original image is less than 8 pixels, some information will be lost after layers of convolution, which makes it impossible to fully utilize the shallow special information, and the neural network cannot learn the characteristic information of the target, resulting in low detection accuracy of the small flame area. In order to improve the ability of the network to fuse multi-scale features, the present invention adds a 160×160 small target detection layer to detect targets with a receptive field of more than 4×4. At the same time, several feature extraction layers are added for small target detection: after the 24th layer of the backbone network, the feature map continues to be upsampled and other processes, so that the feature map continues to expand. At the same time, in the 26th layer, the acquired feature map of size 160×160 is concat fused with the feature map of the 2nd layer in the backbone network to obtain a larger feature map for small target detection.

[0086] In some optional embodiments, the third YOLOv5 model uses a stochastic gradient descent optimizer.

[0087] It is understandable that when the model is trained, the experimental environment is configured before training. The experimental platform used in this embodiment is a self-configured server, a 64-bit Windows 10 operating system, an Intel core i911900k CPU, 128GB of running memory, and an NVIDIA Geforce RTX 3090 GPU, 24GB of video memory. This embodiment builds a network model based on the PyTorch deep learning framework, and the development environment is PyTorch 1.7.1, CUDA 11.0, and Python 3.7. The training set and verification set of the flame image are input into the improved YOLOv5 network model for optimization training. The training process is: the input image will be feature extracted through the feature extraction network to obtain feature maps of different scales, and then these feature maps will be classified and regressed, and the regression results will be subjected to feature reconstruction operations to obtain more refined feature maps. On this basis, classification and regression operations are performed again, and losses are calculated to complete the flame target detection based on the present invention. Stochastic gradient descent optimizer. Also known as SGD optimizer, the training parameters are set as follows: batch size is 64, initial learning rate is 0.01, momentum factor is 0.937, weight decay rate is 0.0005, and the enhancement coefficients of hue (H), saturation (S) and brightness (V) are 0.015, 0.7 and 0.4 respectively. Single-scale training is used in all tests, and the image input size is 640×640 pixels. According to the characteristics of the model itself, the maximum number of iterations (epochs) of training the original YOLOv5 model and the improved YOLOv5 model are both set to 600. The pre-training model used by the original YOLOv5 model is YOLOv5s6, and the improved YOLOv5 model, that is, the third YOLOv5 model, is the YOLOv5_pre model.

[0088] This embodiment provides an improved third YOLOv5 model for flame detection test, which can clearly and accurately mark the flame position and flame size range in the picture in the form of a rectangular box, and display the word "fire" at the top of the rectangular box to indicate that this is the recognized flame. The number displayed behind is the recognition probability calculated by the model. If the word "fire 0.93" at the top of the box indicates that the probability of detecting a flame in this box is 93%.

[0089] In some optional embodiments, referring to Fig. 9 and Fig.10 , Fig. 9 This is the mAP value change curve of the original YOLOv5 model and the improved YOLOv5 model. Fig.10It is a loss function change curve diagram of the original YOLOv5 model and the improved YOLOv5 model, and also includes an evaluation of the detection effect of the third YOLOv5 model, which is calculated in the following way:

[0090]

[0091] Among them, mAP is the accuracy detection value, m is the average, Precision(K) is the accuracy, Recall(K) is the recall, and AP is the integral of the accuracy index over the recall index in the range of 0 to 1.

[0092] It can be understood that the evaluation index of the flame detection model: generally, objective evaluation indexes such as detection accuracy (mAP value) and detection speed (fps value) are used to evaluate the detection performance of the target detection model. Through the calculation of the detection model itself, the detection speed (fps value) of the flame target detection model of the third YOLOv5 model improved in this embodiment is 56 frames / s, which can meet the real-time detection requirements of the fire-fighting robot. When evaluating and analyzing the model detection performance, this embodiment compares the detection effect of the original YOLOv5 model with the performance (mAP value) of the improved YOLOv5 model (i.e., the third YOLOv5 model) of the present invention. Fig. 9 It can be seen that, under the premise that the threshold is set to 0.5, the mAP value of the improved YOLOv5 model provided in this embodiment can reach up to 96.6% and tends to be stable, while the maximum mAP value of the original YOLOv5 model is 93.3%. This result shows that the detection accuracy of the third YOLOv5 model provided in this embodiment is very high, and compared with the original model before improvement, the detection accuracy is also greatly improved. In addition, in addition to the average precision value (mAP value) being an important indicator for judging the performance of the target detection model, the loss function curve can also be used as an auxiliary evaluation indicator to intuitively reflect whether the network model converges stably as the number of iterations increases during the training process. The present invention also compares the loss functions of the YOLOv5 original model and the improved YOLOv5 flame target detection model of the present invention. The change curves of the two are shown in Figure 2. Fig.10 As shown. Fig.10 It can be seen that the loss function of the improved YOLOv5 model provided by this implementation converges faster than the loss function of the original YOLOv5 model. When the model iterates 500 times, the loss value is 0.02, which is close to 0, and the network basically converges.

[0093] Specifically, the integral of the accuracy index to the recall index in the range of 0 to 1 is calculated as follows:

[0094]

[0095] Among them, Precision is the accuracy, Recall is the recall rate, and AP is the integral of the accuracy index and the recall index in the range of 0 to 1.

[0096] It can be understood that AP is the integral of the accuracy index and the recall index in the range of 0 to 1, that is, the area under the accuracy-recall curve. Therefore, the higher the AP, the higher the accuracy of the network.

[0097] Specifically, the accuracy and recall are calculated as follows:

[0098]

[0099]

[0100] Among them, TP is the number of bounding boxes that correctly identify the flame, FP is the number of bounding boxes that incorrectly identify the flame, and FN is the number of unrecognized flames.

[0101] It can be understood that if the IoU calculated by the predicted box and the real flame box is greater than 0.5, the detection box is marked as TP, otherwise the detection box is marked as FP, and if the real flame target is not matched to the corresponding predicted box, it is marked as FN. Since the Precision and Recall do not affect each other, in order to better evaluate the detection accuracy, the mAP value is introduced to represent the detection accuracy.

[0102] It can be seen from the above embodiments that the flame detection method based on the improved YOLOv5 model provided by the present invention achieves at least the following beneficial effects:

[0103] In the flame detection method based on the improved YOLOv5 model provided by the present invention, an attention mechanism unit is embedded in the reference network module, which can simultaneously consider the relationship between channels and spatial information, so that the first YOLOv5 model can more accurately locate and identify the target area, and can also reduce the amount of calculation while enabling the first YOLOv5 model to obtain information of a larger area. A new detection layer is added to the detection module to detect small-sized targets, and the characteristic information of the target is fully learned, thereby improving the detection ability of the first YOLOv5 model for small flame areas. The boundary loss function is introduced in the detection module, which is obtained by improving the existing boundary loss function. By adjusting the parameters, it can have greater flexibility in terms of different levels of border regression accuracy. The introduced boundary loss function is more robust to small data sets and noise, and is more suitable for lightweight models.

[0104] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will be appreciated by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A flame detection method based on an improved YOLOv5 model, characterized in that: include: Establishing a flame data set, wherein the flame data set includes a plurality of flame images; The YOLOv5 model is used as the basic model. The YOLOv5 model includes an input end, a reference network module, a feature fusion module and a detection module which are arranged in sequence. An attention mechanism unit is embedded in the reference network module, a new detection layer is added to the detection module, and a boundary loss function is introduced in the detection module to obtain a first YOLOv5 model. The boundary loss function is calculated according to the following method: in, is the difference between the predicted border and the result border of the flame image; I O U is the interaction ratio, α is the adjustment parameter; The reference network module includes a first unit, a second unit, a third unit and a fourth unit; the first unit is a slice structure; the second unit is composed of a convolution layer, a batch normalization and a first activation function; the third unit is composed of the convolution layer and a bottleneck layer; the fourth unit is composed of the convolution layer and a pooling layer; The reference network module extracts feature information of the test image to obtain a first feature map; The embedding of the attention mechanism unit in the benchmark network module comprises: The attention mechanism unit obtains a first vector and a second vector by performing average pooling on the first feature map along a first direction and a second direction respectively, performs channel fusion on the first vector and the second vector, performs channel compression using a convolution block, encodes the spatial information of the first feature map along the first direction and the second direction, separates the channels, obtains a second feature map with the same number of channels as the first feature map through the convolution block, and performs normalization weighting through a second activation function; Adding a new detection layer to the detection module includes: The size of the newly added detection layer is 160×160, which is used to detect targets with a receptive field size larger than 4×4; The feature fusion module includes 24 layers of feature extraction layers, 2 layers of the feature extraction layers in the feature fusion module are added to match the detection layer, the second feature map is processed by the second layer of the feature extraction layer of the feature fusion module to obtain a third feature map, the second feature map is processed by the twenty-sixth layer of the feature extraction layer of the feature fusion module to obtain a fourth feature map, and the third feature map is fused with the fourth feature map to obtain a fifth feature map; The newly added detection layer detects the fifth feature map; Pre-training the first YOLOv5 model using an image dataset to obtain a second YOLOv5 model; The flame data set is divided into a training set, a validation set and a test set, the flame image in the test set is a test image, the training set and the validation set are input into the second YOLOv5 model for optimization, and a third YOLOv5 model is obtained; The test image of the test set is input into the third YOLOv5 model to obtain a flame detection result.

2. The flame detection method based on the improved YOLOv5 model according to claim 1, characterized in that: The flame data set is established, comprising: Collecting a plurality of flame images, the plurality of flame images are all different, screening the plurality of flame images, and deleting the flame images whose flame areas exceed 80% of the flame image area to obtain a first data set; Data enhancement is performed on the flame images in the first data set, and the data enhancement method includes: rotating, mirroring, scaling, splicing and brightness balancing the flame images in the first data set to obtain multiple expanded flame images, and merging the multiple expanded flame images with the first data set to form the flame data set.

3. The flame detection method based on the improved YOLOv5 model according to claim 1, characterized in that: After the flame data set is established, the method further includes labeling the flame image in the flame data set; A flame portion of the flame image is encircled by a frame, information of the frame is recorded, and the information of the frame is converted into a first format for storage.

4. The flame detection method based on the improved YOLOv5 model according to claim 1, characterized in that: The third YOLOv5 model uses a stochastic gradient descent optimizer.

5. The flame detection method based on the improved YOLOv5 model according to claim 1, characterized in that: It also includes evaluating the detection effect of the third YOLOv5 model, and calculating it in the following manner: Among them, mAP is the accuracy detection value, m is the average, Precision(K) is the accuracy, Recall(K) is the recall, and AP is the integral of the accuracy index over the recall index in the range of 0 to 1.

6. The flame detection method based on the improved YOLOv5 model according to claim 5, characterized in that: The integral of the accuracy index to the recall index in the range of 0 to 1 is calculated as follows: Among them, Precision is the accuracy, Recall is the recall rate, and AP is the integral of the accuracy index and the recall index in the range of 0 to 1.

7. The flame detection method based on the improved YOLOv5 model according to claim 5, characterized in that: The accuracy and recall are calculated as follows: Wherein, TP is the number of bounding boxes that correctly identify flames, FP is the number of bounding boxes that incorrectly identify flames, and FN is the number of flames that are not identified.