Real-time detection method for live working point based on deep learning

Through the real-time detection method of fire operation points based on deep learning, combined with weighted integration and convolutional attention module, the detection accuracy of fire operation points is improved, the problem of low detection accuracy in the existing technology is solved, and high precision is achieved and rapid detection effects.

CN118941487APending Publication Date: 2024-11-12PETROCHINA CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202310522030.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing real-time detection method of fire operation points based on deep learning is not very accurate, and requires manual intervention to correct the results, which cannot meet the safety control requirements of high-risk industries.

Method used

Using the real-time detection method of fire operation points based on deep learning, the model training is carried out by establishing the data set of the fire point color area and the rising smoke area, and using the deep learning network to detect whether the fire point exists in the original picture, and through weighted integration and volume The attention module improves detection accuracy.

Benefits of technology

The accuracy of detection is improved, the missed judgment rate is reduced, and the detection accuracy is improved to about 95.5%, meeting the requirements of real-time processing and high precision, and can quickly and accurately identify the fire points and frame them on the original picture.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941487A_ABST
    Figure CN118941487A_ABST
Patent Text Reader

Abstract

The invention provides a real-time detection method for a live working point based on deep learning, and the method is higher in detection precision. Comprising the following steps: respectively preprocessing a dynamic fire point color region data set and a rising smoke region data set to form respective training sets, inputting the dynamic fire point color region training set into a dynamic fire point color region deep learning network, and inputting the rising smoke region training set into a rising smoke region deep learning network, carrying out respective model training; the method comprises the following steps: segmenting an input original picture into a plurality of S * S rectangular grid units through a deep learning network model, and automatically extracting features in each grid; detecting whether a target object of a movable fire point color area or a rising smoke area exists in an input original picture through a deep learning network model, and predicting a target bounding box and confidence degree scores of the movable fire point color area and the rising smoke area detected in the target bounding box; and judging whether the input original picture has a live point or not.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of hot work detection, and specifically relates to a real-time detection method for hot work operation points based on deep learning. Background Art

[0002] The oil and gas industry is a flammable and explosive high-risk industry. Hot work is a special operation and a high-risk operation in the oil and gas industry. Unsafe behaviors and unsafe conditions on site can easily lead to major safety accidents, causing huge property losses and casualties. Real-time detection of hot work points can help monitor, judge and warn of unsafe behaviors and conditions on the hot work site. For high-risk industries such as petrochemicals and construction, the need to grasp the unsafe behaviors and conditions of hot work in key areas in real time is becoming more and more urgent.

[0003] At present, the detection of hot work is mainly achieved by monitoring open smoke and open flames. There are two common methods: one is the detection method using sensors, and the other is the detection method based on deep learning. The detection method using sensors is usually greatly affected by the production and construction site environment and is limited by the amount of calculation. The actual detection accuracy is low, and the application effect cannot meet the safety management requirements of the production and construction site. With the development of deep learning technology, the detection of hot work can reduce the impact of environmental factors, and obtain higher detection accuracy than the sensor detection method through a smaller amount of calculation and parameters.

[0004] The existing real-time detection method of hot work points based on deep learning includes the following steps: S1, establishing a dataset of hot work areas based on the construction site image, preprocessing the data, and inputting it into the deep learning network of the hot work area for model training; S2, using the deep learning network model to divide the input source image into S×S rectangular grid units, and automatically extracting features in each grid; S3, using the deep learning network model to detect whether there is a hot work area in the input original image, and predicting the target bounding box (Bounding Box) and the confidence score of the detected hot work area, and then judging whether there is a hot work point in the input original image. According to the results shown by the experimental data, the detection accuracy of the above method is 70%, and there is still a defect of low detection accuracy, and manual intervention is required to correct the results in the later stage. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a real-time detection method for hot work points based on deep learning, which has high detection accuracy.

[0006] The technical solution adopted by the present invention to solve the technical problem is: a real-time detection method for hot work points based on deep learning, comprising the steps of:

[0007] S1, establishing a fire point color area dataset and a rising smoke area dataset based on the construction site image; preprocessing the fire point color area dataset and the rising smoke area dataset to form their own training sets, inputting the fire point color area training set into the fire point color area deep learning network, and inputting the rising smoke area training set into the rising smoke area deep learning network, and performing their own model training;

[0008] S2, divides the input original image into several S×S rectangular grid units through the deep learning network model, and automatically extracts features in each grid;

[0009] S3, using a deep learning network model to detect whether the input original image contains a target object in the color area of ​​a hot fire point or in the area of ​​rising smoke, and predict the target bounding box and the confidence score of the detected hot fire point color area and the rising smoke area in the target bounding box; determine whether the input original image contains a hot fire point.

[0010] Furthermore, before determining whether there is a fire point in the input original image in step S3, step S3a is also included to obtain the weight of the deep learning network detection algorithm for the fire point color area and the weight of the deep learning network detection algorithm for the smoke rising area, and then perform weighted integration.

[0011] Furthermore, in step S3a, the least mean square algorithm LMS is used for weighted integration;

[0012]

[0013] Among them, W i (n) is the current weight vector value;

[0014] D(x,n) is the decision function of the least mean square algorithm weighted dynamic fire point color area sub-algorithm and rising smoke area sub-algorithm;

[0015] e(x,n) is the error between the actual value and the estimated value;

[0016] μ is a value between 0 and 2;

[0017] After obtaining the hot spot area y(x,n) detected by the deep learning network, a matrix of the results is formed, where "1" represents a hot spot and "-1" represents no hot spot.

[0018] Furthermore, the step S3 further includes a step S4, marking the detected hot spot area with a frame on the original image, and outputting the marked result.

[0019] Furthermore, in the process of automatically extracting features, step S2 embeds a convolutional attention module CBAM, which automatically obtains the importance of each feature channel through learning, and uses the obtained importance to enhance features and suppress features that are not important to the current task.

[0020] Furthermore, in step S2, when processing a video, a frame of the video is read as the input original image, and after the detection is completed, the next frame of the video is read for detection.

[0021] Furthermore, in step S3, the target bounding box is a rectangular box represented by (x, y, w, h);

[0022] Among them, x, y are the coordinates of the center point of the target bounding box;

[0023] w is the ratio of the width of the target bounding box to the width of the original image;

[0024] h is the ratio of the height of the target bounding box to the height of the original image.

[0025] Furthermore, in step S3, when there are features of the color region of the moving fire point or the features of the rising smoke region in the grid cell, the confidence score is equal to the DIOU_nms between the truth value and the target bounding box; when there are no features of the color region of the moving fire point or the features of the rising smoke region in the grid cell, the confidence score is zero; the algorithm is:

[0026] Pr(Classi|Object)×Pr(Object)×DIOU_nms=Pr(Object)×DIOU_nms

[0027] Among them, Pr(Classi|Object) represents the probability that an object belongs to a certain category when given an object;

[0028] Pr(Object) represents the probability of occurrence of this object;

[0029] DIOU_nms is a measure of the degree of overlap of the target bounding box, and nms stands for the non-maximum suppression algorithm.

[0030] Furthermore, in step S3, if the grid cells have the characteristics of the color area of ​​the ignition point and the characteristics of the rising smoke area, the confidence scores of all the grid cells of the color area of ​​the ignition point on the original image and the confidence scores of all the grid cells of the maximum circumscribed rectangular area of ​​the rising smoke area on the original image are calculated.

[0031] Furthermore, the deep learning network model is YOLOv5.

[0032] Compared with the prior art, the invention has the following beneficial effects: the invention provides a real-time detection method for hot work points based on deep learning, which has high detection accuracy. It can improve the regression effect of the small target boundary box and increase the weight of the small target in the loss function. The detected hot work point area is marked with a frame on the original picture, and the marked result is output. The detection speed is fast, the accuracy is high, and the requirements of real-time processing can be met. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the present invention; DETAILED DESCRIPTION

[0034] The present invention is further described below in conjunction with embodiments.

[0035] A real-time detection method for hot work points based on deep learning includes the following steps:

[0036] S1. Establish a ignition point color area dataset and a rising smoke area dataset based on the construction site image; pre-process the ignition point color area dataset and the rising smoke area dataset to form their own training sets, input the ignition point color area training set into the ignition point color area deep learning network, and input the rising smoke area training set into the rising smoke area deep learning network to perform their own model training.

[0037] S2, through the deep learning network model, the input original image is divided into several S×S rectangular grid units, and features are automatically extracted in each grid; in the process of automatically extracting features, the convolutional attention module CBAM is embedded, which automatically obtains the importance of each feature channel through learning, uses the obtained importance to enhance features and suppress features that are not important to the current task. The convolutional attention module CBAM can obtain more detailed features, including improving the regression effect of the small target bounding box in steps S3 and S3a, and increasing the weight of small targets in the loss function.

[0038] When processing a video, read a frame of the video as the input original image. After the detection is completed, read the next frame of the video for detection.

[0039] S3, through the deep learning network model, detects whether there is a target object in the color area of ​​the fire point or the rising smoke area in the input original image, and predicts the target bounding box and the confidence score of the detected color area of ​​the fire point and the rising smoke area in the target bounding box.

[0040] Specifically, the target bounding box is a rectangular box, represented by (x, y, w, h);

[0041] Among them, x, y are the coordinates of the center point of the target bounding box;

[0042] w is the ratio of the width of the target bounding box to the width of the original image;

[0043] h is the ratio of the height of the target bounding box to the height of the original image.

[0044] The confidence algorithm is then used to calculate the probability of the target feature existing in each grid cell.

[0045] Each grid cell predicts only one class at a time, and obtains a specific confidence score for each grid cell. When there are features of the color area of ​​the moving fire point or the rising smoke area in the grid cell, the confidence score is equal to the DIOU_nms (Diatance Intersection over Union with Non-MaxSuppression) between the truth value and the target bounding box, that is, the intersection and union of distances with non-maximum suppression; when there are no features of the color area of ​​the moving fire point or the rising smoke area in the grid cell, the confidence score is zero; the algorithm is:

[0046] Pr(Classi|Object)×Pr(Object)×DIOU_nms=Pr(Object)×DIOU_nms

[0047] Among them, Pr(Classi|Object) represents the probability that an object belongs to a certain category when given an object;

[0048] Pr(Object) represents the probability of occurrence of this object;

[0049] DIOU_nms is a measure of the degree of overlap of the target bounding box, and nms stands for the non-maximum suppression algorithm.

[0050] DIOU takes into account the overlapping area of ​​the intersection and union and the distance between the center points. The role of non-maximum suppression selection nms (Non-Max Suppression) in the test is to mask the predicted rectangular box of the non-maximum DIOU value, so that only the boundary prediction box data with the largest DIOU score is selected for the calculation of the confidence score. In this way, if the grid unit has the characteristics of the color area of ​​the dynamic fire point or the characteristics of the rising smoke area, the confidence score of the network grid of all the dynamic fire point color areas of the image or the confidence score of all the network grids of the maximum circumscribed rectangular area of ​​the rising smoke area is calculated.

[0051] If the grid cell has the color area feature of the ignition point and the rising smoke area feature, the confidence scores of all the grid cells of the color area of ​​the ignition point on the original image and the confidence scores of all the grid cells of the maximum circumscribed rectangular area of ​​the rising smoke area on the original image are calculated.

[0052] According to the results shown in the experimental data, the probability of missed judgment can be reduced by comprehensive judgment based on the two dimensions of the color area of ​​the ignition point and the rising smoke area. The detection accuracy is about 82%, but the detection precision is still insufficient.

[0053] S3a, after obtaining the weight of the deep learning network detection algorithm for the color area of ​​the ignition point and the weight of the deep learning network detection algorithm for the smoke rising area, weighted integration is performed.

[0054] Specifically, the least mean square algorithm LMS (Least Mean Square) is used for weighted integration. Because the dynamic fire point detection algorithm is actually a dynamic process. There are a lot of regional differences in the oil and gas field operations in many places, and there are many temporary changes in the same area. The least mean square algorithm LMS has good adaptability.

[0055] The decision value of a given area is obtained by linear combination of two sub-algorithms, the deep learning network detection algorithm function of the hot spot color area and the deep learning network detection algorithm function of the smoke rising area, to determine whether there is a hot spot in the input original image. The decision value is obtained by combining the functions of the two sub-algorithms through LMS, and the weights of each sub-algorithm are automatically updated by the LMS algorithm.

[0056]

[0057] Among them, W i (n) is the current weight vector value;

[0058] D(x,n) is the decision function of the least mean square algorithm weighted dynamic fire point color area sub-algorithm and rising smoke area sub-algorithm;

[0059] e(x,n) is the error between the actual value and the estimated value;

[0060] μ is a value between 0 and 2;

[0061] After obtaining the hot spot area y(x,n) detected by the deep learning network, a matrix of the results is formed, where "1" represents a hot spot and "-1" represents no hot spot. Then it is determined whether there is a hot spot in the input original image.

[0062] According to the results shown in the experimental data, the weights of the deep learning network detection algorithm for the color area of ​​the ignition point and the weights of the deep learning network detection algorithm for the smoke rising area were obtained, and then weighted integration was performed. The detection accuracy was about 95.5%, which further improved the accuracy of the judgment.

[0063] S4, marking the detected hot spot area with a frame on the original image, and outputting the marked result.

[0064] The deep learning network model can be SSD, YOLOv3, etc., preferably YOLOv5. YOLOv5 can not only meet the requirements of real-time processing, but also meet the detection of multiple classes at each location at the same time. It also has the advantages of small framework and fast detection speed, and is easy to deploy on edge computing devices.

[0065] The above is a specific implementation of the present invention. From the implementation process, it can be seen that the present invention provides a real-time detection method for hot work points based on deep learning, which has high detection accuracy. It can improve the regression effect of the small target boundary box and increase the weight of the small target in the loss function. The detected hot work point area is marked with a frame on the original image, and the marked result is output. The detection speed is fast, the accuracy is high, and it can meet the requirements of real-time processing.

Claims

1. A real-time detection method for hot work points based on deep learning, characterized in that: Includes steps: S1, establishing a fire point color area dataset and a rising smoke area dataset based on the construction site image; preprocessing the fire point color area dataset and the rising smoke area dataset to form their own training sets, inputting the fire point color area training set into the fire point color area deep learning network, and inputting the rising smoke area training set into the rising smoke area deep learning network, and performing their own model training; S2, divides the input original image into several S×S rectangular grid units through the deep learning network model, and automatically extracts features in each grid; S3, using a deep learning network model to detect whether the input original image contains a target object in the color area of ​​a hot fire point or in the area of ​​rising smoke, and predict the target bounding box and the confidence score of the detected hot fire point color area and the rising smoke area in the target bounding box; determine whether the input original image contains a hot fire point.

2. The real-time detection method for hot work points based on deep learning according to claim 1 is characterized in that: Before determining whether there is a hotspot in the input original image in step S3, step S3a is also included to obtain the weight of the deep learning network detection algorithm for the hotspot color area and the weight of the deep learning network detection algorithm for the smoke rising area, and then perform weighted integration.

3. The real-time detection method for hot work points based on deep learning according to claim 2 is characterized in that: In the step S3a, the least mean square algorithm LMS is used for weighted integration; Among them, W i (n is the current weight vector value; Dx,n) is the decision function of the least mean square algorithm weighted dynamic fire point color area sub-algorithm and rising smoke area sub-algorithm; ex,n) is the error between the actual value and the estimated value; μ is a value between 0 and 2; After obtaining the hot spot area yx,n) detected by the deep learning network, a matrix of the results is formed, where "1" represents a hot spot and "-1" represents no hot spot.

4. The real-time detection method for hot work points based on deep learning according to claim 1, characterized in that: The step S3 also includes a step S4, marking the detected hot spot area with a frame on the original picture, and outputting the marked result.

5. The method for real-time detection of hot work points based on deep learning according to claim 1, characterized in that: In the process of automatically extracting features, step S2 embeds a convolutional attention module CBAM, which automatically obtains the importance of each feature channel through learning, and uses the obtained importance to enhance features and suppress features that are not important to the current task.

6. The method for real-time detection of hot work points based on deep learning according to claim 1, characterized in that: In step S2, when processing a video, a frame of the video is read as the input original image, and after the detection is completed, the next frame of the video is read for detection.

7. The real-time detection method for hot work points based on deep learning according to claim 1, characterized in that: In step S3, the target bounding box is a rectangular box, represented by (x, y, w, h); Among them, x, y are the coordinates of the center point of the target bounding box; w is the ratio of the width of the target bounding box to the width of the original image; h is the ratio of the height of the target bounding box to the height of the original image.

8. The method for real-time detection of hot work points based on deep learning according to claim 1, characterized in that: In step S3, when there are features of the color region of the hot spot or the features of the rising smoke region in the grid cell, the confidence score is equal to the DIOU_nms between the truth value and the target bounding box; when there are no features of the color region of the hot spot or the features of the rising smoke region in the grid cell, the confidence score is zero; the algorithm is: Pr(Classi|Object)×Pr(Object)×DIOU_nms=Pr(Object)×DIOU_nms Among them, Pr(Classi|Object) represents the probability that an object belongs to a certain category when given an object; Pr(Object) represents the probability of occurrence of this object; DIOU_nms is a measure of the degree of overlap of the target bounding box, and nms stands for the non-maximum suppression algorithm.

9. The method for real-time detection of hot work points based on deep learning according to claim 1, characterized in that: In step S3, if the grid cells have the characteristics of the color area of ​​the ignition point and the characteristics of the rising smoke area, the confidence scores of all the grid cells of the color area of ​​the ignition point on the original image and the confidence scores of all the grid cells of the maximum circumscribed rectangular area of ​​the rising smoke area on the original image are calculated.

10. The method for real-time detection of hot work points based on deep learning according to any one of claims 1 to 9, characterized in that: The deep learning network model is YOLOv5.

Citation Information

Patent Citations

  • Forest fire identification method based on cascade classification

    CN110211323A

  • Embedded smoke and flame video alarm system and method based on deep learning framework

    CN110517441A

  • Smoke and fire detection early warning method and system based on YOLOV5 network

    CN114677629A

  • Smoke and fire detection method based on convolutional neural network

    CN115294520A