Fire image detection method and device based on improved YOLOv5

Abstract

The invention provides a fire image detection method and device based on improved YOLOv5, and belongs to the field of image detection. The method comprises the following steps: processing a fire image data set, and dividing the processed fire image data set into a training set and a test set according to a preset proportion; a YOLOv5 target detection model is constructed; wherein the YOLOv5 target detection model comprises a CBMA module and an upper and lower feature fusion module; performing model training on the YOLOv5 target detection model based on the sample data in the training set and the test set by adopting a Bayesian optimization algorithm to obtain an improved YOLOv5 target detection model; and performing fire image detection by using the improved YOLOv5 target detection model. According to the method provided by the invention, the target detection accuracy under a poor visible light imaging effect is optimized; and performing hyper-parameter tuning by using a Bayesian optimization algorithm to achieve the optimal detection performance.

Description

Technical Field

[0001] The present invention relates to the field of image detection, and in particular, to a fire image detection method based on improved YOLOv5, a fire image detection device based on improved YOLOv5, a computer device and a computer-readable storage medium. Background Art

[0002] The ethylene industry is the core of the petrochemical industry and one of the important indicators of a petrochemical development level. In 2021, the world's total ethylene production capacity reached 210 million tons / year, a year-on-year increase of 6.2%. 67% of the world's new ethylene production capacity came from China, and the corresponding safety issues are becoming increasingly prominent.

[0003] Recently, there have been many fire accidents in ethylene plants. Timely detection of sudden fires in ethylene production plants is an urgent problem to be solved. In order to reduce domino accidents and various losses caused by fire accidents, early fire detection in ethylene production plants is particularly important.

[0004] Currently, there are two main solutions to the problem of flame and smoke detection caused by accidents in ethylene production units. The first solution uses sensor technology, such as flame sensors or smoke sensors, to detect fire. The second solution uses real-time object detection algorithms provided by neural networks, such as AlexNet, RCNN, YOLO or SSD algorithms.

[0005] Traditional point-contact fire detectors based on sensors are mostly used in confined or semi-confined spaces. Due to the nature of contact detection of fire detectors, the sensors respond after smoke is generated by a fire or a large amount of heat is released by the combustion of combustibles, and they cannot accurately locate the size and detailed location of the fire. Misjudgments often occur, and fires cannot be detected in a timely manner. The application effect is poor in refining and chemical companies where production equipment is located in non-confined spaces. Therefore, companies will also arrange staff to inspect the equipment by patrolling, which is affected by subjective and objective factors, restricting the detection efficiency and the detection cost is also high.

[0006] Existing deep learning target detection algorithms can be divided into two-step target detection algorithms and single-step target detection algorithms. The two-step target detection algorithm takes a long time to train, has a slow model speed and a high false alarm rate. The single-step target detection algorithm has low accuracy, a short detection distance, and poor detection effect on small objects. At present, there are still some problems that need to be solved in fire identification methods, such as the scene is not perfect, most of the training set samples are flames under weak light sources, the accuracy of small flame detection is low, and it does not achieve full cycle and full coverage in time and space, which cannot effectively meet the characteristics of refining and chemical enterprise operation, wide distribution of plant equipment and small target detection. Summary of the invention

[0007] In view of the technical problem that the image recognition algorithm in the prior art cannot effectively detect, the present invention provides a fire image detection method and device based on improved YOLOv5, which can improve the accuracy of small target flame detection in ethylene plants.

[0008] To achieve the above-mentioned purpose, a fire image detection method provided by the present invention comprises the following steps: processing a fire image data set, and dividing the processed fire image data set into a training set and a test set according to a preset ratio; constructing a YOLOv5 target detection model; wherein the YOLOv5 target detection model comprises a CBMA module and an upper and lower feature fusion module; using a Bayesian optimization algorithm, training the YOLOv5 target detection model based on sample data in the training set and the test set to obtain an improved YOLOv5 target detection model; and using the improved YOLOv5 target detection model to perform fire image detection.

[0009] In an exemplary embodiment of the present invention, the method may further include: collecting fire image data, including: crawling and downloading fire pictures from a target website based on web crawler technology; and / or performing frame extraction processing on a fire video to obtain fire pictures as fire image data.

[0010] In an exemplary embodiment of the present invention, the processing of the fire image dataset may include: performing data enhancement and expansion processing on the fire image dataset using Gaussian noise, Mixup, CutMix and a generative adversarial network.

[0011] In an exemplary embodiment of the present invention, the fire image dataset includes a visible light fire image dataset and an infrared light fire image dataset; the processing of the fire image dataset further includes: performing grayscale processing on the infrared light fire image dataset using weighted average, the grayscale processing being as shown in formula (1); performing binarization processing on the infrared light fire image dataset after grayscale processing using large law threshold segmentation;

[0012] Gray=0.299R+0.587G+0.114B (1)

[0013] In formula (1), Gray is the gray value; R is the red value; G is the green value; B is the blue value

[0014] In an exemplary embodiment of the present invention, the processing of the fire image dataset may further include: performing code reading on the fire image dataset to obtain the fire image dataset of RGB values ​​and HSV values.

[0015] In an exemplary embodiment of the present invention, the method may further include: labeling the fire image dataset using a LabelImg labeling tool.

[0016] In an exemplary embodiment of the present invention, the CBMA module includes a channel attention mechanism and a spatial attention mechanism; the channel attention mechanism may include: performing maximum pooling and average pooling on the input feature layer to obtain feature data of the channel dimension and a feature layer of the channel attention; performing a shared multi-layer perceptron to extract feature data on the feature data of the channel dimension; adding the feature data, activating the function, obtaining the channel attention weight, and calculating the feature map of the channel attention mechanism by formula (2):

[0017]

[0018] In formula (2), F′ is the feature map output by the channel attention mechanism, F is the input feature matrix; M c is the channel compression weight matrix; Multiply the matrix elements one by one;

[0019] The spatial attention mechanism may include: performing maximum pooling and average pooling on the feature layer of the channel attention to obtain feature data of the spatial dimension and the feature layer of the spatial attention; stacking the feature data of the spatial dimension, extracting features, and activating the spatial attention weight through a function; and calculating the feature map of the spatial attention mechanism through formula (3) according to the feature map of the channel attention mechanism:

[0020]

[0021] In formula (3), F″ is the feature map output by the spatial attention mechanism, M s represents the feature matrix output by the spatial attention mechanism, F′ is the feature map output by the channel attention mechanism, Multiply the matrix elements one by one.

[0022] In an exemplary embodiment of the present invention, the upper and lower feature fusion module may include: using a neural network to obtain target features for feature maps of a channel attention mechanism and feature maps of a spatial attention mechanism; setting the upper and lower feature channels to half of the target features, performing normalization and function activation respectively, to obtain upper and lower feature channel data; splicing the upper and lower feature channel data and the target features to achieve upper and lower feature fusion.

[0023] In an exemplary embodiment of the present invention, the use of the Bayesian optimization algorithm to perform model training on the YOLOv5 target detection model based on sample data in the training set and the test set may include: if the loss function reaches a predetermined value under predetermined iteration conditions, the iteration ends and the optimized hyperparameters are obtained; if the loss function does not reach the predetermined value under predetermined iteration conditions, the loss function with the minimum value is selected to obtain the optimized hyperparameters; and based on the optimized hyperparameters, an improved YOLOv5 target detection model is obtained.

[0024] In an exemplary embodiment of the present invention, the use of the improved YOLOv5 target detection model to perform fire image detection may include: using the improved YOLOv5 target detection model to perform fire recognition detection on visible light image videos and infrared light image videos of key areas, and obtaining fire recognition detection results with the highest confidence levels for the visible light image videos and infrared light image videos; judging whether a fire has occurred based on the fire recognition detection results, if the fire recognition detection results are greater than a credible threshold, then a fire has occurred; if the fire recognition detection results are less than the credible threshold, then no fire has occurred.

[0025] Another aspect of the present invention provides a fire image detection device based on improved YOLOv5, which may include: a data processing module, used to process a fire image data set, and divide the processed fire image data set into a training set and a test set according to a preset ratio; a model construction module, used to construct a YOLOv5 target detection model; wherein the YOLOv5 target detection model includes a CBMA module and an upper and lower feature fusion module; a Bayesian optimization algorithm is used to train the YOLOv5 target detection model based on sample data in the training set and the test set to obtain an improved YOLOv5 target detection model; an image detection module, used to use the improved YOLOv5 target detection model to perform fire image detection.

[0026] Yet another aspect of the present invention provides a computer device, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the above-mentioned fire image detection method based on the improved YOLOv5.

[0027] Another aspect of the present invention provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above-mentioned fire image detection method based on improved YOLOv5.

[0028] Through the technical solution provided by the present invention, the present invention has at least the following technical effects:

[0029] 1) The present invention optimizes the target detection accuracy under poor visible light imaging effects;

[0030] 2) By using generative adversarial neural networks for data enhancement, the network is trained so that the generative network and the adversarial network reach Nash equilibrium, enriching the data enhancement methods and increasing the sample data set of small fire targets;

[0031] 3) Use the Bayesian optimization algorithm to tune hyperparameters and achieve the best detection performance by adjusting hyperparameters such as learning rate, batch size, number of iterations, activation function, and loss function coefficient;

[0032] 4) The present invention adds a convolutional attention mechanism and a top-bottom feature fusion mechanism. The convolutional attention mechanism can increase the weight of target features and reduce the weight of background features, so that the model can obtain the target area that needs to be focused on; the context feature fusion mechanism can fuse rich location features and semantic features, thereby improving the model's accuracy in detecting small target flames.

[0033] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0035] Figure 1 A flowchart of a fire image detection method according to a first exemplary embodiment of the invention;

[0036] Figure 2 A schematic diagram of a convolutional attention module of the first exemplary embodiment of the invention;

[0037] Figure 3 It is a schematic diagram of a context feature fusion module of the first exemplary embodiment of the invention. DETAILED DESCRIPTION

[0038] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0039] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0040] In the present invention, unless otherwise specified, directional words such as "up, down, top, bottom" are usually used with reference to the directions shown in the drawings or are used to describe the relative positional relationships of components in the vertical, perpendicular or gravity direction.

[0041] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0042] First Exemplary Embodiment

[0043] Please refer to Figure 1The embodiment of the present invention provides a fire image detection method, which includes the following steps:

[0044] S1. Process the fire image dataset, and divide the processed fire image dataset into a training set and a test set according to a preset ratio.

[0045] The method further includes: collecting fire image data, which may include: crawling and downloading fire images from a target website based on web crawler technology; and / or performing frame extraction processing on a fire video to obtain a fire image as fire image data. For example, a frame extraction processing is performed based on a fire video and a fire video shot during the production process of a simulated refinery to obtain an image.

[0046] Optionally, the processing of the fire image dataset may include: performing data enhancement and expansion processing on the fire image dataset using Gaussian noise, Mixup, CutMix and a generative adversarial network.

[0047] Specifically, Gaussian noise, Mixup and CutMix can be used to enhance the original fire image dataset, and the generative adversarial neural network model (also known as GAN) can be used to expand the data to obtain an enhanced fire image dataset.

[0048] Here, Gaussian noise refers to a type of noise whose probability density function follows a Gaussian distribution (i.e., normal distribution). If the amplitude distribution of a noise follows a Gaussian distribution and its power spectrum density is uniformly distributed, it is called Gaussian white noise. Mixup is an algorithm for mixed-class enhancement of images. It can mix images of different classes in a linear interpolation manner to construct new training samples and expand the training data set. CutMix selects two images from the data set, then cuts a part of one image and superimposes it on the other image as a new input image.

[0049] Here, the generative adversarial network model trains the generative network and the discriminative network, so that the images generated by the generative network tend to fire pictures. When the discriminative network cannot distinguish the generated fire pictures from the fire pictures obtained by video frame extraction, the generative network and the discriminative network reach Nash equilibrium. It can be considered that the fire pictures output by the generator at this time are very similar to the characteristics of real fire pictures. Therefore, by using the generative adversarial neural network for data enhancement, the network is trained to make the generative network and the adversarial network reach Nash equilibrium, enriching the data enhancement method and increasing the sample data set of small fire targets.

[0050] For example, the original fire image dataset has a total of 1,500 pictures. Gaussian noise processing, Mixup processing, and CutMix processing are performed on the 1,500 photos, and the dataset is expanded to 2,500 photos. The 2,500 photos are randomly divided into training set and validation set in a ratio of 7:3, and the generative adversarial neural network is used to enhance the image data, and finally a dataset of 3,000 fire images is obtained.

[0051] Optionally, the fire image dataset includes a visible light fire image dataset and an infrared light fire image dataset; the processing of the fire image dataset may also include: using weighted average to perform grayscale processing on the infrared light fire image dataset, and the grayscale processing is as follows; using large law threshold segmentation to binarize the infrared light fire image dataset after grayscale processing.

[0052] Gray=0.299R+0.587G+0.114B

[0053] Where Gray is the gray value; R is the red value; G is the green value; B is the blue value.

[0054] For example, the infrared image data set may include infrared pictures and / or pictures obtained by frame extraction of infrared videos.

[0055] Specifically, grayscale processing may include: converting an infrared light image data set into a grayscale image data set according to a weighted average method. For example, under environmental conditions such as low or even zero illumination, low contrast background, and bad weather, the imaging effect of a visible light camera is poor, and the image recognition algorithm cannot effectively detect fires. Therefore, infrared image processing can solve the imaging problem. Converting infrared video images into grayscale images can increase visual contrast and highlight the target area. Using a weighted average method, a certain weight is taken according to the sensitivity of the human eye to the three colors R, G, and B, and the R, G, and B values ​​are weighted averaged to convert the infrared video image into a grayscale image.

[0056] Optionally, the binarization process may include: binarizing the infrared image data set after grayscale processing using the large law threshold segmentation. Specifically, the histogram of the grayscale image is calculated to calculate the number of pixels occupied by each pixel value of 0-255; the threshold value 0-255 is traversed, and the pixels less than or equal to the threshold value are equal to the background, and the pixels greater than the threshold value are the foreground; the ratio of the number of background pixels to the total number of pixels and the average value of the background pixels are calculated; the number of foreground pixels to the total number of pixels is calculated; the inter-class variance or intra-class variance is calculated, and the threshold that maximizes the inter-class variance or minimizes the intra-class variance is the optimal threshold; the image is binarized using the optimal threshold value to make the image present an obvious black and white effect.

[0057] Optionally, the binary infrared fire image data set can also be subjected to frame-by-frame difference calculation. For example, in the early stage of a fire or when a fire has already occurred, the frame-by-frame difference calculation can obtain an image of a suspected fire area.

[0058] Optionally, the processing of the fire image dataset may further include: reading the code of the fire image dataset to obtain the fire image dataset of RGB value and HSV value. Here, RGB value and HSV value are attributes of the image. Here, the YOLOv5 model is used to perform fire detection based on the RGB channel, HSV channel in the visible light image and the infrared image binary image, and then the visible light image and the infrared image with the highest confidence are taken as the basis for judging whether a fire exists.

[0059] In this embodiment, the weighted mean method and the global binary method are used to preprocess the infrared image, and the suspicious fire area in the image is determined by frame-by-frame difference operation. The visible light video screenshots and the infrared video binary images are used for fire detection using the YOLOv5 model, thereby optimizing the target detection accuracy when the visible light imaging effect is poor.

[0060] Optionally, the method may further include: labeling the fire image dataset using a LabelImg labeling tool, wherein the labeling may be completed in a yolo label format and then saved as a txt file.

[0061] Optionally, the fire image data set is divided into a training set and a test set according to a preset ratio. For example, the annotation set may include a training set and a validation set that are divided into independent and non-repetitive sets in a ratio of 7:3. The training set is used to provide the YOLOv5 model for training, and the validation set is used to test the model effect to verify whether the model performance meets the requirements. Here, the original data set may have certain inherent orders and patterns, which may have a negative impact on the training and evaluation of the model, so by randomly shuffling, these orders and patterns in the data set can be eliminated, making the data more randomly and evenly distributed, so as to better train and evaluate the model. For example, in the present invention, the order of the data set is randomly shuffled and divided into a data set and a test set in a ratio of 7:3.

[0062] S2. Build a YOLOv5 target detection model; wherein the YOLOv5 target detection model includes a CBMA (also known as a convolutional attention module) module and an upper and lower feature fusion module.

[0063] Here, the construction of the YOLOv5 model may include: downloading the YOLOv5 open source model framework, modifying the configuration file, and performing basic settings on the development environment. Here, the YOLOv5 model is implemented using the TensorFlow or PyTorch framework.

[0064] Optionally, the CBMA module can be used to enhance the weight of target features and reduce the weight of background features. The channel attention mechanism is based on the RGB three-channel data of the image to calculate the weight values ​​of different channels. The spatial attention mechanism averages or pools different channels of the same pixel to obtain the overall distribution of different channels and discard singular channel values. Here, the CBMA module can be used to enhance the effect of the channel and spatial attention mechanisms, reduce the background RGB channel weights, and allow the YOLOv5 target detection model to obtain the target area that needs to be focused on.

[0065] Optionally, the CBMA module may include a channel attention mechanism and a spatial attention mechanism.

[0066] Among them, the channel attention mechanism may include: performing maximum pooling and average pooling on the input feature layer to obtain feature data of the channel dimension and the feature layer of the channel attention; performing a shared multi-layer perceptron to extract feature data from the feature data of the channel dimension; adding the feature data, activating it through a function, obtaining the channel attention weight, and calculating the feature map of the channel attention mechanism through the following formula:

[0067]

[0068] Where F′ is the feature map output by the channel attention mechanism, and F is the input feature matrix; M c is the channel compression weight matrix; Multiply the matrix elements one by one.

[0069] Among them, the spatial attention mechanism may include: performing maximum pooling and average pooling on the feature layer of the channel attention to obtain feature data of the spatial dimension and the feature layer of the spatial attention; stacking the feature data of the spatial dimension, extracting features, and activating the spatial attention weight through a function; according to the feature map of the channel attention mechanism, calculating the feature map of the spatial attention mechanism through the following formula:

[0070]

[0071] Where F″ is the feature map output by the spatial attention mechanism, M s represents the feature matrix output by the spatial attention mechanism, F′ is the feature map output by the channel attention mechanism, Multiply the matrix elements one by one.

[0072] Specifically, if Figure 2 As shown in Figure 2, the CBAM process is introduced as follows: CBAM combines the channel attention mechanism and the spatial attention mechanism.

[0073] Here, the upper part is the channel attention mechanism, which performs average pooling and maximum pooling on the feature layer with an input size of C×H×W, and obtains two feature layers with a size of C×1×1; then they pass through two shared multilayer perceptrons, the first one has C / r neurons, the activation function is ReLU, and the second one has C neurons; the two results are added together, and then passed through a Sigmoid function to obtain the weight value of each channel of the input feature layer, and finally the obtained weight value is multiplied by the original input feature layer to obtain a new feature, the feature F′ obtained through the channel attention mechanism. Here, C is the number of channels in the input feature layer, and r is the drop rate.

[0074] Among them, the activation function ReLU formula is as follows:

[0075] r(x)=max(0,x);

[0076] Where x is the output value of the upper node of the model. Here, r(x) is the RELU function.

[0077] Among them, the Sigmoid formula is as follows,

[0078]

[0079] In the formula, x is the output value of the upper node of the model. Here, s(x) is the Sigmoid function; e -x Perform power operation on the output value of the upper node of the model.

[0080] Specifically, the lower part is the spatial attention mechanism, which includes: performing maximum pooling and average pooling on the feature layer output by the channel attention mechanism to obtain two feature layers of size 1×H×W, which are stacked; then a convolution operation with a channel number of 1 and a Sigmoid function are performed to obtain the weight value; finally, the weight value is multiplied by the input feature layer to obtain the final feature, thereby introducing spatial attention in the input feature layer. The feature F″ obtained by the spatial attention mechanism.

[0081] Here, the channel attention mechanism in the CBMA mechanism calculates the weight values ​​of different channels based on the RGB three-channel data of the image and quantifies the importance of each channel. The spatial attention mechanism averages or pools different channels of the same pixel to obtain the overall distribution of different channels and discard singular channel values.

[0082] Optionally, the upper and lower feature fusion module can be used to combine the position information extracted by the shallow neural network and the semantic information extracted by the deep neural network, and can receive features of different scales of the YOLOv5 target detection model to avoid the loss of features of a single scale and improve the model's detection effect on small targets. The upper and lower feature fusion module may include: using a neural network to obtain target features for the feature mapping of the channel attention mechanism and the feature mapping of the spatial attention mechanism; setting the upper and lower feature channels to half of the target features, normalizing and function activating them respectively, to obtain upper and lower feature channel data; splicing the upper and lower feature channel data with the target features to achieve upper and lower feature fusion.

[0083] Specifically, if Figure 3 As shown in the figure, the process of introducing the context feature fusion mechanism includes: the feature maps in the head layer of the YOLOv5 network have different spatial sizes, and they need to be standardized before feature fusion. The context features are made to have the same spatial size as the target features through the transposed convolution (also known as Conv Transpose) operation. The transposed convolution is the inverse operation of the convolution, which can output low-resolution images as high-resolution images, retain more feature information, and give more feature information to small targets. The number of context feature channels is set to half of the number of target feature channels, and batch normalization and ReLU activation function are performed after each layer. Finally, the target features and context features are spliced ​​through cancat to achieve context feature fusion.

[0084] S3. Using the Bayesian optimization algorithm, the YOLOv5 target detection model is trained based on the sample data in the training set and the test set to obtain an improved YOLOv5 target detection model.

[0085] Here, the fire identification and detection method may further include: using a verification set to verify and enhance the YOLOv5 target detection model.

[0086] Optionally, a Bayesian optimization algorithm is used to train the YOLOv5 target detection model based on sample data in a training set and a test set, which may include: if the loss function reaches a predetermined value under predetermined iteration conditions, the iteration ends and an optimized hyperparameter is obtained; if the loss function does not reach a predetermined value under predetermined iteration conditions, a loss function with a minimum value is selected to obtain an optimized hyperparameter; and an improved YOLOv5 target detection model is obtained based on the optimized hyperparameter.

[0087] Specifically, the Bayesian optimization algorithm is used to tune hyperparameters during training. In the Bayesian framework, the sampling function (which can be called Acquisition Function or AF) is used to use historical information to find the next sample point. According to the posterior distribution, the local optimal solution is used to estimate the sample value of the global optimal solution and sample to find a smaller f(x). Common acquisition functions include Probability of improvement (which can be called PI) and Expected improvement (which can be called EI) functions. The present invention uses the EI function, and adds a balance parameter ξ to further process the relationship between the local and the global, so that the EI function can maintain a balance between development and exploration.

[0088] Optionally, the acquisition function is as follows,

[0089]

[0090] Where EI(x) is the acquisition function; Φ(Z) is the probability density function; φ(Z) is the cumulative distribution function; σ(x) is the variance; μ(x) is the expected value; f(x + ) is the current optimal value; ξ is a hyperparameter. Here, the hyperparameter is used to control exploration and utilization to obtain a better solution.

[0091] Optionally, Z is as follows,

[0092]

[0093] Where μ(x) is the expected value; f(x + ) is the optimal value; σ(x) is the variance; ξ is the hyperparameter. Here, Z is the improvement.

[0094] Therefore, the best detection performance is achieved by adjusting hyperparameters such as learning rate, batch size, number of iterations, activation function, etc.

[0095] Specifically, the steps of Bayesian optimization of YOLOv5 hyperparameters may include: (1) setting the range of YOLOv5 target detection model hyperparameters, inputting hyperparameters such as adjusted learning rate, batch size, number of iterations, activation function, etc. into the Gaussian process, training the YOLOv5 target detection model, and reducing the loss function to modify the Gaussian model; (2) using the sampling function EI(x) to select a better estimated point x in the Gaussian model i , update the sample library D = {(x1,y1),(x2,y2),...,(x n ,y n)} and Gaussian model; (3) The termination iteration algorithm can be divided into three parts: whether the maximum number of iterations is reached; whether the maximum running time is reached; whether the loss function reaches the preset value. If the termination iteration conditions are met, the algorithm stops, saves the YOLOv5 target detection model at this time and exits; (4) If the three conditions for termination iteration are not met, start again from the second step.

[0096] S4. Use the improved YOLOv5 target detection model for fire image detection.

[0097] Optionally, an improved YOLOv5 target detection model is used to perform fire recognition detection on the visible light image video and infrared light image video of the key area to obtain the fire recognition detection results with the highest confidence of the visible light image video and the infrared light image video; whether a fire has occurred is determined based on the fire recognition detection results. If the fire recognition detection result is greater than the credible threshold, a fire has occurred; if the fire recognition detection result is less than the credible threshold, no fire has occurred. Here, the lowest alarm threshold of the credible threshold is 0.7, and the highest alarm threshold of the credible threshold is 1. Increasing the credible threshold can reduce the false alarm rate of the model and improve the accuracy of the alarm. Here, an improved YOLOv5 target detection model is used to perform fire detection based on the RGB channel, HSV channel in the visible light image and the infrared image binary image, and the visible light image and the infrared image with the highest confidence are taken as the basis for judging whether a fire exists.

[0098] Specifically, the improved YOLOv5 target detection model is used to perform fire recognition detection on visible light image videos and infrared light image videos of key areas, and the fire recognition detection results with higher confidence in the two video sources are selected as the basis for abnormal alarm. When an abnormal situation occurs, the abnormal monitoring point will be highlighted in red in real time and the alarm will be activated. Through manual analysis, it will be confirmed whether a fire abnormality has occurred. If a fire occurs, the emergency response process will be immediately activated. Otherwise, it will be treated as a misjudgment and restored to normal.

[0099] In this implementation, in order to verify the performance of the improved YOLOv5 target detection model, precision (P), recall (R), and mean average precision (mAP) are used as performance evaluation indicators. Among them, mAP is calculated as mAP@0.5 and mAP@0.5:0.95 respectively.

[0100] Among them, the formula of precision (P) is as follows,

[0101] P = TP / (TP+FP);

[0102] Where P is the precision; TP is the number of correctly predicted positive samples; FP is the number of incorrectly predicted positive samples.

[0103] Here, P is the ratio of correct predictions among all the results predicted as positive samples.

[0104] The formula for recall (R) is as follows:

[0105] R = TP / (TP+FN);

[0106] Where R is the recall rate; TP is the number of correctly predicted positive samples; FN is the number of incorrectly predicted negative samples.

[0107] Here, R is the ratio of all even positive samples that are correctly predicted.

[0108] Among them, the formula of mean average precision (mAP) is as follows:

[0109]

[0110] In the formula, mAP is the mean average precision; j is the number of starting samples; c is the total number of samples; AP j is the sample accuracy.

[0111] Here, mAP refers to the performance in all categories. The larger the mAP, the better the network performance.

[0112] Optionally, according to the above analysis, improvements are made on the original YOLOv5 network structure in turn, and the improved network model is trained and tested. The ablation experiment results are shown in Table 1.

[0113] As shown in Table 1, adding the three improvement schemes separately can effectively improve the network performance. At the same time, in the whole experiment, it can be seen that experiment 5 has the best effect on mAP@0.5, and experiment 6 has a decrease in recall rate R and mAP@0.5 value, but the value of mAP@0.5:0.95 in experiment 6 is greater than that in experiment 5, indicating that the prediction box of experiment 6 is more accurate and can locate the fire point more accurately.

[0114] As shown in Table 1, adding the CBAM module and the feature fusion module to the YOLOv5 backbone network can not only improve the average precision, but also detect fires that YOLOv5 missed. After using the Bayesian optimizer to adaptively select parameters, although the image recall rate has decreased, the model's prediction box for the fire image is more accurate.

[0115]

[0116] Table 1 Ablation experiment results

[0117] Second Exemplary Embodiment

[0118] In a second exemplary embodiment of the present invention, a fire image detection device based on an improved YOLOv5 includes: a data processing module, used to process a fire image data set, and divide the processed fire image data set into a training set and a test set according to a preset ratio; a model construction module, used to construct a YOLOv5 target detection model; wherein the YOLOv5 target detection model includes a CBMA module and an upper and lower feature fusion module; a Bayesian optimization algorithm is used to train the YOLOv5 target detection model based on sample data in the training set and the test set to obtain an improved YOLOv5 target detection model; an image detection module, used to perform fire image detection using the improved YOLOv5 target detection model.

[0119] Third Exemplary Embodiment

[0120] In a third exemplary embodiment of the present invention, a computer device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing a fire image detection method based on an improved YOLOv5.

[0121] In this example, a desktop computer with Windows 10 64-bit operating system is used. The computer's CPU is Intel i9 10900X@3.75GHz, the GPU is NVIDIA GeForce GTX3090, the RAM is 128GB, PyTorchv1.12.1, and CUDA v11.3. Here, the training environment and the test environment are the same.

[0122] Fourth Exemplary Embodiment

[0123] In a fourth exemplary embodiment of the present invention, a computer-readable storage medium is provided.

[0124] The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the fire image detection method based on improved YOLOv5 of the above exemplary embodiment 1 is implemented.

[0125] The computer-readable storage medium may be any data storage device that stores data that can be read by a computer system. For example, examples of computer-readable storage media may include: read-only memory, random access memory, read-only optical disk, magnetic tape, floppy disk, optical data storage device, and carrier wave (such as data transmission via the Internet via a wired or wireless transmission path).

[0126] In summary, the present invention is based on the image recognition technology of YOLOv5, performs data enhancement through traditional data enhancement methods and generative adversarial neural networks, constructs a video image library of smelting area fires, adds a small target detection mechanism to the structure, uses the weighted mean method and the global binary method to pre-process the infrared images of smelting area fires, determines the suspicious fire areas in the image through frame-by-frame difference operations, and uses the YOLOv5 model to realize real-time detection of ethylene plant fires.

[0127] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

[0128] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0129] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A fire image detection method based on improved YOLOv5, characterized in that: The method comprises: Processing the fire image data set, and dividing the processed fire image data set into a training set and a test set according to a preset ratio; Constructing a YOLOv5 target detection model; wherein the YOLOv5 target detection model includes a CBMA module and an upper and lower feature fusion module; The Bayesian optimization algorithm is used to train the YOLOv5 target detection model based on the sample data in the training set and the test set to obtain an improved YOLOv5 target detection model. Use the improved YOLOv5 target detection model for fire image detection.

2. The fire image detection method based on improved YOLOv5 according to claim 1, characterized in that: The method further comprises: collecting fire image data, including: Crawl and download fire images from target websites using web crawler technology; and / or The fire video is subjected to frame extraction processing to obtain a fire picture as fire image data.

3. The fire image detection method based on improved YOLOv5 according to claim 2, characterized in that: The processing of the fire image data set includes: Gaussian noise, Mixup, CutMix and generative adversarial network are used to enhance and expand the fire image dataset.

4. The fire image detection method based on improved YOLOv5 according to claim 3, characterized in that: The fire image dataset includes a visible light fire image dataset and an infrared light fire image dataset; The processing of the fire image data set also includes: The infrared fire image dataset is gray-scale processed using weighted average, and the gray-scale processing is as shown in formula (1); The infrared fire image dataset after grayscale processing is binarized using the large-scale threshold segmentation method. Gray=0.299R+0.587G+0.114B (1) In formula (1), Gray is the gray value; R is the red value; G is the green value; and B is the blue value.

5. The fire image detection method based on improved YOLOv5 according to claim 4, characterized in that: The processing of the fire image data set also includes: The fire image dataset is read by code to obtain the fire image dataset of RGB value and HSV value.

6. The fire image detection method based on improved YOLOv5 according to claim 1, characterized in that: The method further comprises: The LabelImg annotation tool is used to annotate the fire image dataset.

7. The fire image detection method based on improved YOLOv5 according to claim 1, characterized in that: The CBMA module includes a channel attention mechanism and a spatial attention mechanism; The channel attention mechanism includes: Perform maximum pooling and average pooling on the input feature layer to obtain the feature data of the channel dimension and the feature layer of the channel attention; The feature data of the channel dimension is shared by a multi-layer perceptron to extract the feature data; Add the feature data and activate them through the function to get the channel attention weight. Then calculate the feature map of the channel attention mechanism through formula (2): In formula (2), F′ is the feature map output by the channel attention mechanism, F is the input feature matrix; M c is the channel compression weight matrix; Multiply the matrix elements one by one; The spatial attention mechanism includes: Perform maximum pooling and average pooling on the feature layer of channel attention to obtain feature data of spatial dimension and feature layer of spatial attention; The feature data of the spatial dimension are stacked, the features are extracted, and the spatial attention weight is obtained through function activation; According to the feature map of the channel attention mechanism, the feature map of the spatial attention mechanism is calculated by formula (3): In formula (3), F″ is the feature map output by the spatial attention mechanism, M s represents the feature matrix output by the spatial attention mechanism, F′ is the feature map output by the channel attention mechanism, Multiply the matrix elements one by one.

8. The fire image detection method based on improved YOLOv5 according to claim 7, characterized in that: The upper and lower feature fusion module includes: A neural network is used to obtain the target features from the feature maps of the channel attention mechanism and the feature maps of the spatial attention mechanism; Set the upper and lower feature channels to half of the target feature, perform normalization and function activation respectively, and obtain the upper and lower feature channel data; The upper and lower feature channel data are spliced ​​with the target features to achieve upper and lower feature fusion.

9. The fire image detection method based on improved YOLOv5 according to claim 1, characterized in that: The Bayesian optimization algorithm is used to train the YOLOv5 target detection model based on the sample data in the training set and the test set, including: If the loss function reaches the predetermined value under the predetermined iteration conditions, the iteration ends and the optimized hyperparameters are obtained; If the loss function does not reach the predetermined value under the predetermined iteration conditions, the minimum loss function is selected to obtain the optimized hyperparameters; By optimizing the hyperparameters, an improved YOLOv5 target detection model is obtained.

10. The fire image detection method based on improved YOLOv5 according to claim 1, characterized in that: The fire image detection using the improved YOLOv5 target detection model includes: Use the improved YOLOv5 target detection model to perform fire recognition detection on visible light image videos and infrared image videos of key areas, and obtain the fire recognition detection results with the highest confidence in visible light image videos and infrared image videos; Whether a fire has occurred is determined based on the fire identification detection result. If the fire identification detection result is greater than the credible threshold, a fire has occurred; if the fire identification detection result is less than the credible threshold, no fire has occurred.

11. A fire image detection device based on improved YOLOv5, characterized in that: include: A data processing module is used to process the fire image data set and divide the processed fire image data set into a training set and a test set according to a preset ratio; A model building module is used to build a YOLOv5 target detection model; wherein the YOLOv5 target detection model includes a CBMA module and an upper and lower feature fusion module; a Bayesian optimization algorithm is used to train the YOLOv5 target detection model based on sample data in a training set and a test set to obtain an improved YOLOv5 target detection model; Image detection module, used to detect fire images using the improved YOLOv5 target detection model.

12. A computer device, characterized in that: include: at least one processor; A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the fire image detection method based on improved YOLOv5 according to any one of claims 1 to 10.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the fire image detection method based on improved YOLOv5 according to any one of claims 1 to 10 is implemented.

Images