Intelligent construction site fire smoke early warning method based on improved yolov5
By improving the yolov5 algorithm, adding the Coordinate Attention module, separating convolutions using depth, replacing the Neck layer structure and optimizing the loss function, the problem of low detection efficiency and accuracy in construction site fire detection technology is solved, and more efficient and accurate fire smoke detection is achieved.
Patent Information
- Application Number
- CN202411453896.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-03-11
AI Technical Summary
The existing construction site fire detection technology based on network models has problems such as insufficient detection efficiency and detection accuracy, which may not be detected quickly, causing serious consequences.
A smart construction site fire smoke warning method based on the improved yolov5 algorithm is adopted. By inserting a Coordinate Attention module at the end of the YOLOv5 basic backbone network, a depth-separable convolution is used instead of the standard convolution layer, the Feature Pyramid Network and Path Aggregation Network of the Neck layer are replaced as Bi-directional Feature Pyramid Network, and the Prediction Head is optimized to adopt the Focal Loss loss function.
It improves the model's perception of position information, reduces the number of model parameters and calculation complexity, enhances the feature pyramid structure effect of target detection, improves the robustness and detection accuracy of the model, and adapts to the lightweight characteristics of mobile terminals and embedded devices.
Smart Images
Figure CN119600495A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a smart construction site fire detection technology, and specifically to a smart construction site fire smoke early warning technology based on improved yolov5 algorithm front-end perception. Background Art
[0002] As urbanization accelerates, construction sites play a vital role in cities. However, the fire risks that come with it are also becoming increasingly prominent. Traditional safety management methods can no longer meet the needs of modern construction sites. Fire prevention and control requires smarter and more efficient methods to meet the challenges brought by urbanization to ensure safe production on construction sites and the health of workers.
[0003] Traditional fire detection methods often rely on discrete sensors and manual monitoring, which are prone to missed detections and false alarms. This method makes it difficult to respond quickly in the early stages of a fire, increasing the probability of fire accidents. In high-risk environments such as construction sites, a more accurate and real-time detection method is needed to warn of fire risks in advance and minimize potential losses.
[0004] In recent years, with the rapid development of computer vision technology based on deep learning, fire warning has ushered in new opportunities. With the help of advanced target detection algorithms, such as YOLOv5, efficient identification of key information such as fire sources and smoke at construction sites can be achieved. This deep learning-based method not only improves the accuracy of detection, but also enables real-time monitoring. It has the characteristics of rapid deployment of detection, making the construction site fire warning system more intelligent and reliable, and protecting the safety of urban construction.
[0005] The main technologies used in the smart construction site fire smoke warning method and system based on the front-end perception of the improved YOLOv5 algorithm are target detection technology and deep learning technology to improve the accuracy and practicality of the model. Its main purpose is to achieve accurate and real-time detection and warning of fire smoke in smart construction sites through the video images captured by the front-end camera and combined with the improved YOLOv5 algorithm.
[0006] The existing traditional construction site fire detection system has the following disadvantages:
[0007] 1. The PAN+FPN structure is usually used as the feature fusion part of the Neck network. Such a structure uses a single bottom-up path for information transmission, which may limit the perception of the high-level feature map to the low-level feature map. In addition, the bottom-up information transmission may cause the performance of Neck to be weak when processing small targets.
[0008] 2. The traditional yolov5 backbone network pays relatively uniform attention to different areas in the image, rather than focusing on the area where the target is located. The model does not place special emphasis on the information of the target location. In the target detection task, accurate location information is very important for locating the target, which may affect the performance of the model in positioning. In addition, it usually uses deep convolutional neural networks (CNN) to extract features. The increase in network depth will cause gradient vanishing or gradient explosion problems, which will affect the transmission of information and cause information loss. In addition, traditional convolution operations have some unnecessary waste in parameter quantity and computational complexity, have more computational burden, and have information transmission delays, which affect the real-time performance of the system.
[0009] 3. The traditional cross entropy loss function does not address the problem of class imbalance. In the cross entropy loss, the gradient of samples that are easy to classify is relatively small and easily ignored, resulting in poor learning effects for minority categories, which is contrary to the rare abnormal situations in construction site safety issues. Summary of the invention
[0010] The purpose of the present invention is to solve the technical problem that the detection efficiency and detection accuracy of the existing construction site fire detection technology based on the network model are not ideal, which easily leads to the fire not being discovered quickly and causing serious consequences. An intelligent construction site fire smoke warning technology based on the front-end perception of the improved yolov5 algorithm is proposed.
[0011] In order to solve the above technical problems, the technical solution adopted by the present invention is:
[0012] A smart construction site fire smoke early warning method based on improved yolov5, comprising the following steps:
[0013] Step 1: Use the camera to obtain front-end perception video stream data;
[0014] Step 2: Perform image preprocessing on the obtained video stream data to improve image quality and obtain a set of images to be detected;
[0015] Step 3: Obtain training set, validation set, and test set from the image set sequence obtained in step 2;
[0016] Step 4: Construct a fire detection model based on yolov5, and then train the constructed fire detection model based on yolov5 with the training set obtained in step 3 to obtain a trained fire detection model based on yolov5;
[0017] Step 5: Use the fire detection model based on yolov5 trained in step 4 to detect the image set sequence to be detected and obtain the detection results;
[0018] The above steps can realize early warning of fire smoke in construction sites.
[0019] In step 3, the following steps are specifically included:
[0020] Step 3.1: Obtain images from video stream data and perform preprocessing, add random noise to the images and apply Mosaic enhancement to simulate the complexity in real scenes to improve the robustness of the model;
[0021] Step 3.2: Divide the preprocessed images into training set, validation set, and test set according to the specified ratio.
[0022] In step 3.1, when performing Mosaic enhancement, the following steps are taken:
[0023] 1) Randomly select four images: Let I1, I2, I3, and I4 be four randomly selected images respectively;
[0024] 2) Randomly select an anchor point: Let x, y be the coordinates of a randomly selected anchor point on the mosaic image;
[0025] 3) Overlay images at a certain ratio: Let r1, r2, r3 be three random ratios, representing the overlay ratio of images I2, I3, I4 relative to the anchor point I1;
[0026] 4) Generate mosaic image: The mosaic image M can be expressed as: M(x,y)=r1·I1(x,y)+r2·I2(x,y)+r3·I3(x,y)+(1-r1-r2-r3)·I4(x,y);
[0027] Where Ii(x,y) represents the pixel value of image Ii at coordinate (x,y);
[0028] 5) Adjust label information: adjust the coordinate information of the target bounding box in the mosaic image to ensure their correct position relative to the newly generated mosaic image;
[0029] The purpose of Mosaic enhancement is that the model needs to learn the ability to process multiple scenes simultaneously, which helps to improve the model's detection performance and generalization ability of the target.
[0030] In step 4, based on the validation set, the detection performance of the fire detection model based on yolov5 is verified;
[0031] Based on the test set, the fire smoke detection model based on YOLOv5, which has been verified by the detection performance, is tested to confirm the effectiveness of the trained fire smoke detection model.
[0032] In step 4, the fire detection model based on yolov5 is constructed as follows:
[0033] The input of the first ConV layer is used to input the initial image, the output of the first ConV layer is connected to the input of the first DeSep Conv, the output of the first DeSep Conv is connected to the input of the first C3 module, the output of the first C3 module is connected to the input of the second DeSep Conv, the output of the second DeSep Conv layer is connected to the input of the second C3 module, the output of the second C3 module is connected to the input of the third DeSep Conv layer, and the input of the second BiFPN module in the Neck part, the output of the third DeSep Conv layer is connected to the input of the third C3 module, the output of the third C3 module is connected to the input of the fourth DeSep Conv layer, the output of the fourth DeSep Conv layer is connected to the input of the fourth C3 module, the output of the fourth C3 module is connected to the input of the CA module, the output of the CA module is connected to the input of the SPPF module, the output of the SPPF module is connected to the input of the first layer of ConV layer, and the output of the first layer of ConV layer is connected to the input of the first BiFPN module in the Neck part.
[0034] In the Neck part, features from different layers are input to the first BiFPN module, and the output of the first BiFPN module is passed to the next level through the fifth C3 module. The output of the fifth C3 module is connected to the second ConV layer. On the one hand, the output of the second ConV layer is upsampled through the Upsample module, and on the other hand, the output of the second ConV layer is connected to the input of the third BiFPN module; the upsampled output is input to the second BiFPN module, and the output of the second BiFPN module is processed by the sixth C3 module and then passed to the first prediction head and connected to the input of the third ConV layer;
[0035] The outputs of the second ConV layer and the third ConV layer are connected to the input of the third BiFPN module, and the output of the third BiFPN module is processed by the seventh C3 module and passed to the second prediction head and the fourth ConV layer; the outputs of the first ConV layer and the fourth ConV layer are connected to the input of the fourth BiFPN module, and the output of the fourth BiFPN module is processed by the eighth C3 module and passed to the third prediction head, and multi-scale feature fusion is performed through processing at each layer.
[0036] Finally, in the Head part, the outputs from different BiFPN modules are passed through multiple prediction heads to generate the final detection results.
[0037] Compared with the prior art, the present invention has the following technical effects:
[0038] 1) By inserting the Coordinate Attention module at the end of the YOLOv5 basic backbone network, the attention weight of the position information is learned, emphasizing the attention to different positions, and improving the model's ability to perceive position information. This helps to better capture spatial relationships in visual tasks;
[0039] 2) Using depthwise separable convolution to replace the standard convolution layer in the backbone network effectively reduces the number of model parameters, improves computational efficiency, and accelerates the model training and reasoning process. This makes the model more adaptable to the lightweight characteristics of mobile and embedded devices, and helps to deploy in resource-constrained environments;
[0040] 3) The Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) of the Neck layer are replaced with the Bi-directional Feature Pyramid Network (BIFPN), and the low-scale to high-scale feature map transmission is realized through the cascade structure, and the low-scale feature map is continuously transmitted to the next high-scale convolution layer. This improvement improves the feature pyramid structure effect of the target detection model and further enhances the fusion and information transmission of features of different scales;
[0041] 4) The Prediction Head of the YOLOv5 detection model is optimized, and the Focal Loss loss function is used instead of the traditional CIOU_LOSS loss function. Focal Loss makes the model pay more attention to difficult samples, improves the robustness of the model, and makes it more focused on difficult-to-classify samples, thereby improving the learning effect on key samples. Focal Loss combines classification loss and adjustment factors to make the model pay more attention to difficult samples and improve the performance of the model in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:
[0043] Figure 1 is a flow chart of the method in the present invention;
[0044] Figure 2 It is a structural diagram of the improved yolov5 flame and smoke detection network in the present invention;
[0045] Figure 3 This is the structural diagram of the CA module added to the yolov5Backbone part of the present invention;
[0046] Figure 4 It is a network structure diagram of feature fusion BIPFN in the present invention;
[0047] Figure 5 It is a schematic diagram of the depth-wise separable convolution of the present invention. DETAILED DESCRIPTION
[0048] A smart construction site fire smoke early warning method based on improved yolov5 algorithm front-end perception includes the following steps:
[0049] Step 1: Use the camera to obtain the front-end perception video stream data;
[0050] Step 2: Perform image preprocessing on the obtained video stream data to improve image quality and obtain a set of images to be detected;
[0051] Step 3: Obtain training set, validation set, and test set from the image set sequence obtained in step 2;
[0052] Step 4: Construct a fire detection model based on yolov5, and then train the constructed fire detection model based on yolov5 with the training set obtained in step 3 to obtain a trained fire detection model based on yolov5;
[0053] Step 5: Use the fire detection model based on yolov5 trained in step 4 to detect the image set sequence to be detected and obtain the detection results;
[0054] The above steps can realize early warning of fire smoke at construction sites.
[0055] In step 3, the following steps are specifically included:
[0056] Step 3.1: Obtain images from video stream data and perform preprocessing, add random noise to the images and apply Mosaic enhancement to simulate the complexity in real scenes to improve the robustness of the model;
[0057] Step 3.2: Divide the preprocessed images into training set, validation set, and test set according to the specified ratio.
[0058] In step 3.1, when performing Mosaic enhancement, the following steps are taken:
[0059] 1) Randomly select four images: Let I1, I2, I3, and I4 be four randomly selected images respectively;
[0060] 2) Randomly select an anchor point: Let x, y be the coordinates of a randomly selected anchor point on the mosaic image;
[0061] 3) Overlay images at a certain ratio: Let r1, r2, r3 be three random ratios, representing the overlay ratio of images I2, I3, I4 relative to the anchor point I1;
[0062] 4) Generate mosaic image: The mosaic image M can be expressed as: M(x,y)=r1·I1(x,y)+r2·I2(x,y)+r3·I3(x,y)+(1-r1-r2-r3)·I4(x,y);
[0063] Where Ii(x,y) represents the pixel value of image Ii at coordinate (x,y);
[0064] 5) Adjust label information: adjust the coordinate information of the target bounding box in the mosaic image to ensure their correct position relative to the newly generated mosaic image;
[0065] The purpose of Mosaic enhancement is that the model needs to learn the ability to process multiple scenes simultaneously, which helps to improve the model's detection performance and generalization ability of the target.
[0066] In step 4, based on the validation set, the detection performance of the fire detection model based on yolov5 is verified;
[0067] Based on the test set, the fire smoke detection model based on YOLOv5, which has been verified by the detection performance, is tested to confirm the effectiveness of the trained fire smoke detection model.
[0068] In step 4, the fire detection model based on yolov5 is constructed as follows:
[0069] The input of the first ConV layer is used to input the initial image, the output of the first ConV layer is connected to the input of the first DeSep Conv, the output of the first DeSep Conv is connected to the input of the first C3 module, the output of the first C3 module is connected to the input of the second DeSep Conv, the output of the second DeSep Conv layer is connected to the input of the second C3 module, the output of the second C3 module is connected to the input of the third DeSep Conv layer, and the input of the second BiFPN module in the Neck part, the output of the third DeSep Conv layer is connected to the input of the third C3 module, the output of the third C3 module is connected to the input of the fourth DeSep Conv layer, the output of the fourth DeSep Conv layer is connected to the input of the fourth C3 module, the output of the fourth C3 module is connected to the input of the CA module, the output of the CA module is connected to the input of the SPPF module, the output of the SPPF module is connected to the input of the first layer of ConV layer, and the output of the first layer of ConV layer is connected to the input of the first BiFPN module in the Neck part.
[0070] In the Neck part, features from different layers are input to the first BiFPN module, and the output of the first BiFPN module is passed to the next level through the fifth C3 module. The output of the fifth C3 module is connected to the second ConV layer. On the one hand, the output of the second ConV layer is upsampled through the Upsample module, and on the other hand, the output of the second ConV layer is connected to the input of the third BiFPN module; the upsampled output is input to the second BiFPN module, and the output of the second BiFPN module is processed by the sixth C3 module and then passed to the first prediction head and connected to the input of the third ConV layer;
[0071] The outputs of the second ConV layer and the third ConV layer are connected to the input of the third BiFPN module, and the output of the third BiFPN module is processed by the seventh C3 module and passed to the second prediction head and the fourth ConV layer; the outputs of the first ConV layer and the fourth ConV layer are connected to the input of the fourth BiFPN module, and the output of the fourth BiFPN module is processed by the eighth C3 module and passed to the third prediction head, and multi-scale feature fusion is performed through processing at each layer.
[0072] Finally, in the Head part, the outputs from different BiFPN modules are passed through multiple prediction heads to generate the final detection results.
[0073] Example:
[0074] A fire warning method based on an improved YOLOv5 algorithm includes a front-end intelligent camera perception layer and a terminal display layer. The perception layer is an intelligent camera with embedded development capabilities, which is used to obtain detection images and perform fire detection. The terminal display layer includes mobile device monitoring and fixed device monitoring, which are used to receive and display the processing results of the server. The specific steps are as follows:
[0075] Step 1: Use the camera to obtain front-end perception video stream data;
[0076] Step 2: Perform image preprocessing on the obtained video stream data to improve image quality and obtain a set of images to be detected;
[0077] Step 3: Use the obtained image set sequence to train the fire detection model;
[0078] Furthermore, the specific steps of the training process of the above-mentioned smart construction site fire smoke detection model are as follows:
[0079] Step 3.1: Obtain images from video stream data and perform preprocessing, add random noise to the images and apply Mosaic enhancement to simulate the complexity in real scenes to improve the robustness of the model.
[0080] Furthermore, Mosaic enhancement can be described as:
[0081] 1. Randomly select four images: Let I1, I2, I3, and I4 be four randomly selected images.
[0082] 2. Randomly select an anchor point: Let x, y be the coordinates of a randomly selected anchor point on the mosaic image.
[0083] 3. Overlay images at a certain ratio: Let r1, r2, r3 be three random ratios, representing the overlay ratio of images I2, I3, I4 relative to the anchor point I1.
[0084] 4. Generate mosaic image: The mosaic image M can be expressed as: M(x,y)=r1·I1(x,y)+r2·I2(x,y)+r3·I3(x,y)+(1-r1-r2-r3)·I4(x,y);
[0085] Where Ii(x,y) represents the pixel value of image Ii at coordinate (x,y).
[0086] 5. Adjust label information: Adjust the coordinate information of the target bounding box in the mosaic image to ensure their correct position relative to the newly generated mosaic image.
[0087] The advantage of Mosaic enhancement is that the model needs to learn the ability to process multiple scenes at the same time, which helps to improve the model's detection performance and generalization ability of the target.
[0088] Step 3.2: Divide the preprocessed images into training set, validation set, and test set in a ratio of 8:1:1.
[0089] Step 3.3: Based on the training set, the fire smoke detection model is trained to obtain a trained fire smoke detection model, and based on the validation set, the detection performance of the fire smoke detection model is verified;
[0090] Step 3.4: Finally, based on the test set, fire smoke detection is performed on the fire smoke detection model that has passed the detection performance verification to confirm the effectiveness of the trained fire smoke detection model.
[0091] Step 4: Improve the yolov5 basic algorithm to adapt it to the requirements of smart construction site fire detection. The fire detection model is constructed based on the improved yolov5 basic algorithm, and its specific improvements include:
[0092] Step 4.1: Place a CA (Coordinate Attention) attention module at the end of the Yolov5 basic backbone network to increase attention to specific locations and enhance the model's perception of location information in the image, thereby improving the model's performance in visual tasks. The CA attention module is mainly divided into the following five processes:
[0093] 1. Get the final output feature map from the backbone network: Given the input feature map X∈R C×H×W , where C is the number of channels, H and W are the height and width.
[0094] 2. Define the mapping function of the coordinate attention module: f COORD and activation function σ.
[0095] 3. Calculate the coordinate attention weight: Calculate the coordinate attention weight for each position of the feature map
[0096] w i,j =σ(f COORD (i,j,X)
[0097] Among them, i and j represent the rows and columns of the feature map respectively.
[0098] 4. Generate weighted feature map: Use the coordinate attention weight to weight the original feature map to obtain the weighted feature map X COORD :X COORD (i,j)=w i,j ·X(i,j), where represents element-wise multiplication.
[0099] 5. X COORD It is input into the Neck layer as a weighted feature map.
[0100] Step 4.2: Replace the convolutional layer in the backbone network with a depth-wise separable convolution. The depth-wise separable convolution splits the standard convolution into two steps: depth-wise convolution and point-wise convolution. This reduces the number of model parameters, improves computational efficiency, and accelerates model training and reasoning, making it adaptable to the lightweight characteristics of mobile and embedded devices. It can significantly reduce the size of the model without losing too much performance, improve computational efficiency, adapt to mobile devices and embedded systems, and promote wider model deployment. The depth-wise separable convolution combines the two operations of depth-wise convolution and point-wise convolution. For the input feature map X, the operation of the depth-wise separable convolution is:
[0101]
[0102] Where K l,i is the convolution kernel of the depthwise convolution, P l,iis the convolution kernel of point-wise convolution.
[0103] Step 4.3: Replace the FPN and PAN of the Neck layer with BIFPN, and pass the low-scale feature map to the high-scale feature map input through the cascade structure, and continue to pass the low-scale feature map to the next high-scale convolution layer to achieve the fusion of features of different scales. Construct a better feature fusion module for multi-scale feature fusion to improve the effect of the feature pyramid structure of the target detection model, and further enhance the fusion and information transmission of features of different scales.
[0104] Step 4.4: The optimized yolov5 detection model Prediction Head uses the Focal Loss loss function instead of the traditional CIOU_LOSS loss function, so that the model pays more attention to difficult samples, improves the robustness of the model, focuses more on difficult-to-classify samples, and improves the learning effect on key samples. The Focal Loss loss function mainly consists of two parts: classification loss (Cross-Entropy Loss) and adjustment factor (Focusing Parameter).
[0105] 1. Classification loss: Classification loss measures the difference between the model's prediction and the actual label. For binary classification problems, it is usually expressed as negative log-likelihood loss. The formula is:
[0106] CE(p,y)=-(y·log(p)+(1-y)·log(1-p))
[0107] Here, p is the predicted probability of the model and y is the actual label (0 or 1).
[0108] 2. Adjustment factor: The adjustment factor introduces an additional term to adjust the weights of positive and negative categories, reducing the focus on easy-to-classify samples and making the model pay more attention to difficult samples. The formula for the adjustment factor is:
[0109] Focusing Parameter = (1-pt) γ
[0110] Among them, pt is the predicted probability of the model for the sample, and γ is the hyperparameter of the adjustment factor, which usually takes a positive value.
[0111] 3. Focal Loss overall formula: Focal Loss is obtained by multiplying the classification loss by the adjustment factor, and its formula is:
[0112] Focal Loss = CE(p,y)·(1-pt) γ
[0113] Step 5: Use the trained fire detection model algorithm in the front-end smart camera to detect the image set sequence to obtain the detection results.
[0114] Table 1 shows the performance of the improved YOLOv5 algorithm and the traditional YOLOv5 algorithm on the same construction site fire dataset. The two algorithms adopt the same parameter settings and train for 400 rounds. The results show that the improved YOLOv5 algorithm has different degrees of improvement in different indicators, and significantly increases the detection accuracy of the network on the basis of lightweight embedded deployment.
[0115] Table 1
[0116] Performance comparison between improved yolov5 algorithm and traditional yolov5 algorithm
[0117]
[0118] The present invention discloses a smart construction site fire detection method based on the YoloV5 network model. Before the YoLo network model was proposed, the field of target detection research was mainly based on two-stage algorithms headed by Faster-RCNN. These algorithms need to separate the extraction of candidate regions and the final network classification and regression into two independent parts, resulting in slower speed but relatively high detection accuracy. In contrast, the one-stage algorithm only needs to send the input image into the network once to obtain the location and category information of all prediction boxes. Although its accuracy is lower, the network runs faster and occupies less memory, which is very suitable for mobile terminals and environments with relatively simple hardware configurations.
[0119] The Yolo series network model is the most classic one-stage algorithm and is also the most widely used target detection network in the industrial field. The present invention uses YoloV5 for target detection, so that the detection results have better detection accuracy and faster reasoning speed.
[0120] Specific steps are as follows Figure 1 As shown, the key steps include:
[0121] 1) The front-end sensing device collects images, pre-processes and annotates them to obtain the site image
[0122] 2) Divide the labeled data set into training set, validation set, and test set according to the proportion to obtain the training data set
[0123] 3) Improve the yolov5 network model and input the data set into the improved yolov5 fire detection model for iterative training
[0124] 4) Embed the improved and trained model into the camera to obtain a front-end perception device with real-time detection function
[0125] 5) Use the fire detection model algorithm to detect the image set sequence to be detected and obtain the detection results
[0126] like Figure 2 As shown in the figure, it is the structural diagram of the improved yolov5 flame and smoke detection network in the present invention. The specific steps of inputting the picture through the network to the detection head to obtain the classification result are as follows:
[0127] 1) Take the input image as the initial input, use the Backbone network to extract features of the input image, capture information of different scales on the feature map of each stage, and build a feature pyramid structure.
[0128] 2) Use the Neck module for feature fusion to promote the integration of features at different scales. Use the BiFPN structure to construct a feature pyramid network.
[0129] 3) Use the Head network for target detection. The Head network usually includes multiple detection heads, each of which is responsible for target detection at different scales. Apply a convolutional layer to each scale in the feature pyramid to obtain relevant information for target detection. Perform anchor box prediction on the feature map of each scale to obtain the target's bounding box information and category probability. Then perform non-maximum suppression on the target box to remove redundant detection boxes.
[0130] 4) Set a threshold, perform probability screening, and determine whether it is fire smoke based on the category information of the detection box.
[0131] like Figure 3 As shown in the figure, in order to better enable the model to filter out a small amount of important information from a large amount of information, a coordinate attention mechanism (CA) is used to embed position information into channel attention, so that the model can more effectively perceive information at different positions in the image. This helps to improve the accuracy of the model for the target position.
[0132] When using the CA (Coordinate Attention) module, its specific working steps are implemented according to the following key points:
[0133] 1) Get the input feature map:
[0134] The features of each channel are averaged to obtain the global information of the channel.
[0135] 2) Positional encoding:
[0136] Perform position encoding on each position (i, j) of the feature map to generate a position encoding matrix P∈R H×W×2 .
[0137] 3) Mapping function calculation:
[0138] Define the mapping function f CA , maps the position encoding matrix P to the attention score matrix A∈R H×W×1 .
[0139] 4) Activation function application:
[0140] Apply the activation function to normalize the attention score matrix A to the range of 0-1 and obtain the attention weight matrix W∈R H ×W×1 .
[0141] 5) Generate weighted feature map:
[0142] Use the attention weight matrix W to weight the original feature map X to obtain the weighted feature map
[0143] X CA (i,j)=W(i,j)·X(i,j)
[0144] Here, · represents element-wise multiplication.
[0145] 6) Input the weighted feature map into the next layer of the network: CA Passed as input to the Neck layer.
[0146] Through the above key points, the Coordinate Attention module can emphasize the attention to different positions by learning the attention weights of position information, which helps to improve the model's perception of position information. This helps to better capture spatial relationships in visual tasks.
[0147] like Figure 4 In the BIFPN structure shown in the figure, the FPN+PAN structure of the original Neck layer is replaced by the BIPFN architecture. The low-scale feature map output is passed to the high-scale feature map input, and the low-scale feature map is further transmitted to the lower high-scale convolution layer through the cascade structure, thereby realizing the fusion of features of different scales.
[0148] like Figure 5 As shown in the figure, the flow chart of depthwise separable convolution is as follows: First, four convolution kernels of size 3x3 are used to traverse the data of three channels respectively to generate three feature layers. Then, four convolution kernels of size 1x1 are used to traverse these three feature layers to finally get the output result. The number of parameters in the whole process is calculated as 4x3x3+1x1x3x4=48. From this example, it can be seen that the use of depthwise separable convolution can indeed effectively reduce the number of parameters, making the model more lightweight and more suitable for deployment in embedded devices.
Claims
1. A smart construction site fire smoke early warning method based on improved yolov5, characterized in that: The following steps are involved: Step 1: Use the camera to obtain front-end perception video stream data; Step 2: Perform image preprocessing on the obtained video stream data to improve image quality and obtain a set of images to be detected; Step 3: Obtain training set, validation set, and test set from the image set sequence obtained in step 2; Step 4: Construct a fire detection model based on yolov5, and then train the constructed fire detection model based on yolov5 with the training set obtained in step 3 to obtain a trained fire detection model based on yolov5; Step 5: Use the fire detection model based on yolov5 trained in step 4 to detect the image set sequence to be detected and obtain the detection results; The above steps can realize early warning of fire smoke at construction sites.
2. The method according to claim 1, characterized in that In step 3, the following steps are specifically included: Step 3.1: Obtain images from video stream data and perform preprocessing, add random noise to the images and apply Mosaic enhancement to simulate the complexity in real scenes to improve the robustness of the model; Step 3.2: Divide the preprocessed images into training set, validation set, and test set according to the specified ratio.
3. The method according to claim 1, characterized in that In step 3.1, when performing Mosaic enhancement, the following steps are taken: 1) Randomly select four images: Let I1, I2, I3, and I4 be four randomly selected images respectively; 2) Randomly select an anchor point: Let x, y be the coordinates of a randomly selected anchor point on the mosaic image; 3) Overlay images at a certain ratio: Let r1, r2, r3 be three random ratios, representing the overlay ratio of images I2, I3, I4 relative to the anchor point I1; 4) Generate mosaic image: The mosaic image M can be expressed as: M(x,y)=r1·I1(x,y)+r2·I2(x,y)+r3·I3(x,y)+(1-r1-r2-r3)·I4(x,y); Where Ii(x,y) represents the pixel value of image Ii at coordinate (x,y); 5) Adjust label information: adjust the coordinate information of the target bounding box in the mosaic image to ensure their correct position relative to the newly generated mosaic image; The purpose of Mosaic enhancement is that the model needs to learn the ability to process multiple scenes simultaneously, which helps to improve the model's detection performance and generalization ability of the target.
4. The method according to claim 1, characterized in that: In step 4, based on the validation set, the detection performance of the fire detection model based on yolov5 is verified; Based on the test set, the fire smoke detection model based on YOLOv5, which has been verified by the detection performance, is tested to confirm the effectiveness of the trained fire smoke detection model.
5. The method according to claim 1, characterized in that The fire detection model constructed based on yolov5 is as follows: The input of the first ConV layer is used to input the initial image, the output of the first ConV layer is connected to the input of the first DeSep Conv, the output of the first DeSep Conv is connected to the input of the first C3 module, the output of the first C3 module is connected to the input of the second DeSepConv, the output of the second DeSep Conv layer is connected to the input of the second C3 module, the output of the second C3 module is connected to the input of the third DeSep Conv layer and the input of the second BiFPN module of the Neck part, the output of the third DeSep Conv layer is connected to the input of the third C3 module, the output of the third C3 module is connected to the input of the fourth DeSep Conv layer, the output of the fourth DeSep Conv layer is connected to the input of the fourth C3 module, the output of the fourth C3 module is connected to the input of the CA module, the output of the CA module is connected to the input of the SPPF module, the output of the SPPF module is connected to the input of the first ConV layer, and the output of the first ConV layer is input to the first BiFPN module of the Neck part after upsampling; In the Neck part, the output of the first BiFPN module is passed to the next level through the fifth C3 module. The output of the fifth C3 module is connected to the second ConV layer. On the one hand, the output of the second ConV layer is upsampled through the Upsample module. On the other hand, the output of the second ConV layer is connected to the input of the third BiFPN module. The upsampled output is input to the second BiFPN module. The output of the second BiFPN module is processed by the sixth C3 module and then input to the first prediction head and connected to the input of the third ConV layer. The outputs of the second ConV layer and the third ConV layer are connected to the input of the third BiFPN module, and the output of the third BiFPN module is processed by the seventh C3 module and then passed to the second prediction head and the fourth ConV layer; the outputs of the first ConV layer and the fourth ConV layer are connected to the input of the fourth BiFPN module, and the output of the fourth BiFPN module is processed by the eighth C3 module and then passed to the third prediction head; Finally, in the Head part, the outputs from different BiFPN modules are passed through multiple prediction heads to generate the final detection results.