Gas station oil discharge standard behavior detection method based on video stream

Abstract

The invention discloses a gas station oil discharge standard behavior detection method based on a video stream. The method specifically comprises the following steps: setting an image acquisition module to carry out image acquisition on a detection area as required; constructing and training a deep learning detection model for identifying a target object in the detection area; judging the credibility of a detection result output by the detection model according to an experience confidence threshold; and carrying out logical judgment on the credible detection result to determine whether the oil unloading process is a standard behavior or not.

Description

Technical Field

[0001] The present invention is applied to the field of artificial intelligence technology, and specifically is a method for detecting standardized oil unloading behavior at a gas station based on video stream. Background Art

[0002] The key to detecting the standard behavior in the unloading area of ​​a gas station is to identify the actions or objects of each standard behavior. The traditional methods of monitoring the unloading process at a gas station mainly include the following:

[0003] 1. Manual supervision: The oil unloading process at the gas station is supervised on-site by staff to ensure that the oil unloading process complies with regulations and safety requirements.

[0004] 2. Video surveillance: By installing video surveillance equipment in the oil unloading area, the oil unloading process can be monitored in real time, but it requires manual video viewing, making it difficult to achieve automated supervision.

[0005] 3. Sensor monitoring: Install sensors, such as liquid level sensors and temperature sensors, in the oil unloading area to monitor parameters such as the oil tank liquid level and temperature to prevent oil overflow and leakage during the unloading process.

[0006] 4. Regulatory requirements: Gas stations are required to formulate oil unloading operation specifications, clarify oil unloading procedures and safety requirements, and staff must comply with relevant specifications when performing oil unloading operations.

[0007] These traditional methods have some shortcomings. For example, manual supervision and video monitoring require on-site operation or real-time monitoring by personnel, which makes it difficult to achieve all-weather, unmanned supervision. Sensor monitoring can only monitor parameters such as oil tank liquid level and temperature, and cannot comprehensively monitor the entire oil unloading process. Although the regulatory requirements can clarify the oil unloading procedures and safety requirements, it is difficult to ensure that every staff member complies with the regulations.

[0008] The method and system of the eight-step oil unloading standard behavior detection of gas stations based on video streams in the present invention reduces manpower and avoids violations caused by human errors compared to the above methods. It can also observe the situation on site around the clock. There is no need to install additional sensors, and only the existing surveillance cameras on site can be used, which greatly saves costs. It can also have functions such as violation reminders and saving historical oil unloading records, which empowers gas stations intelligently. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide a method for detecting the standard behavior of oil unloading at a gas station based on video stream in view of the deficiencies of the prior art.

[0010] In order to solve the above technical problems, a method for detecting standard oil unloading behavior at a gas station based on video stream of the present invention specifically comprises the following steps:

[0011] Set the image acquisition module to acquire images of the detection area as required;

[0012] Build and train a deep learning detection model to identify target objects in the detection area;

[0013] The credibility of the detection results output by the detection model is judged according to the empirical confidence threshold;

[0014] Make logical judgments on the credible test results to determine whether the oil unloading process is a standard behavior.

[0015] As a possible implementation, further, the step of constructing and training a deep learning detection model for identifying target objects in the detection area specifically includes:

[0016] Collect diverse image data to build a dataset;

[0017] Build an improved detection model and complete training;

[0018] Using the trained detection model, the location coordinates and corresponding categories of target objects in the oil unloading area of ​​the gas station are output.

[0019] As a possible implementation mode, further, the construction of an improved detection model includes: selecting a YOLO series detection model to improve model performance through optimization and compression, specifically improving the detection head part in the detection model, using a decoupled detection head, so that classification, positioning, and confidence prediction are processed by different convolutional layers.

[0020] As a possible implementation method, further, the training process of the detection model adopts a combination of pre-labeling and manual review. Specifically, a pre-labeling model is trained by a small amount of data for pre-labeling, and the data labeling is corrected in combination with manual review. The reviewed data is added to the pre-trained model for fine-tuning and then the unlabeled data is labeled, and this process is repeated until the data labeling is completed.

[0021] As a possible implementation, further, the step of judging the credibility of the detection result output by the detection model according to the empirical confidence threshold specifically includes:

[0022] Set an empirical confidence threshold for each target category output by the detection model;

[0023] By comparing the empirical confidence with the threshold, it is judged whether there is a corresponding target category in the oil unloading area of ​​the gas station. If the empirical confidence of the target box is greater than the empirical confidence threshold of the category, the target box and category are considered to be credible, and if it is less than the threshold, it is considered to be unreliable.

[0024] Filter out unreliable test results to ensure the accuracy of the test results.

[0025] As a possible implementation mode, further, the logical judgment of the credible detection result to determine whether the oil unloading process is a standard behavior step specifically includes:

[0026] The specific logic judgment process is as follows: when a tanker is detected in the oil unloading area, a monitoring picture is uploaded. If wheel chocks, static clamps, and fire extinguishers are not identified within a preset time, the corresponding steps are considered missing. If they are identified, the steps are considered satisfied.

[0027] When the oil pipeline is detected in the oil unloading area, it is determined that the pre-unloading preparation steps are completed;

[0028] The unloading supervision step is considered satisfied only when workers are detected after the pre-unloading preparation step is completed;

[0029] Within the preset time after the vehicle leaves the oil unloading area, no fire extinguisher, oil pipeline, or static clamp is detected, and the site cleaning is completed;

[0030] When the above steps are fully met, the oil unloading process is judged to be a standard behavior.

[0031] As a possible implementation mode, further, the setting of the image acquisition module specifically includes: setting up a camera in the oil unloading area of ​​the gas station, and the position should ensure that the shooting range covers the entire oil unloading area, including the tanker truck, wheel chocks, electrostatic clamps, fire extinguishers, oil pipelines and the activity area of ​​​​the staff; and the camera should be installed to avoid facing the front or rear of the tanker truck to ensure that the shooting angle can fully capture the entire oil unloading process.

[0032] As a possible implementation, further, it also includes:

[0033] When the oil unloading process is irregular, the control alarm module will be linked and save the missing step records and screenshots of the correct steps.

[0034] A gas station oil unloading standard behavior detection system based on video stream, comprising:

[0035] An image acquisition module is used to acquire images of the detection area as required;

[0036] An image detection module is used to identify target objects in the detection area and output detection results;

[0037] The discrimination module determines the credibility of the detection result output by the image detection module according to the empirical confidence threshold;

[0038] The logical reasoning module performs logical judgment on the credible detection results to determine whether the oil unloading process is a standard behavior.

[0039] The present invention adopts the above technical solution and has the following beneficial effects:

[0040] Realizing all-weather supervision and alarm of the oil unloading area of ​​a gas station can reduce labor costs and time costs, and can also reduce the investment and maintenance costs of hardware equipment. It only needs to use the existing cameras in the oil unloading area of ​​the gas station, without installing other sensor equipment, avoiding the difficulty of sensor maintenance, and has more functions than using sensors, such as saving image records during oil unloading. It can adapt to the interference of different weather, light and other external environments in the scene. The present invention has real-time performance and can promptly alarm and remind staff of illegal behaviors. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0042] Example 1

[0043] A method for detecting standard oil unloading behavior at a gas station based on video stream, specifically comprising:

[0044] Set up the image acquisition module to obtain images of the inspection area as needed; set up cameras in the oil unloading area of ​​the gas station, and the position should ensure that the shooting range covers the entire oil unloading area, including tank trucks, wheel chocks, static clamps, fire extinguishers, oil pipelines and staff activity areas; and the camera should be installed so as not to face the front or rear of the tank truck to ensure that the shooting angle can fully capture the entire oil unloading process.

[0045] Build and train a deep learning detection model to identify target objects in the detection area; specifically:

[0046] Collect diverse image data to build a dataset;

[0047] Build an improved detection model and complete the training; in the process of building the improved detection model, the YOLO series detection model is selected to improve the model performance through optimization and compression. Specifically, the detection head part is improved in the detection model, and a decoupled detection head is used so that classification, positioning, and confidence prediction are processed by different convolutional layers. The training process uses a combination of pre-labeling and manual review. Specifically, a pre-labeling model is trained with a small amount of data for pre-labeling, and the data labeling is corrected in combination with manual review. The approved data is added to the pre-trained model for fine-tuning and then the unlabeled data is labeled. This process is repeated until the data labeling is completed.

[0048] Using the trained detection model, the location coordinates and corresponding categories of target objects in the oil unloading area of ​​the gas station are output.

[0049] The credibility of the detection results output by the detection model is judged according to the empirical confidence threshold; specifically, the following steps are performed: setting an empirical confidence threshold for each target category output by the detection model; judging whether the corresponding target category exists in the oil unloading area of ​​the gas station by comparing the empirical confidence with the threshold; if the empirical confidence of the target box is greater than the empirical confidence threshold of the category, the target box and category are considered to be credible; if less than the empirical confidence threshold, the target box and category are considered to be uncredible; filtering out unreliable detection results to ensure the accuracy of the detection results.

[0050] Logical judgment is performed on the credible detection results to determine whether the oil unloading process is a standard behavior. The specific logical judgment process is as follows: when a tanker is detected in the oil unloading area, the monitoring picture is uploaded. If the wheel chocks, electrostatic clamps, and fire extinguishers are not identified within the preset time, the corresponding steps are determined to be missing. If they are identified, the steps are considered to be satisfied; when the oil pipeline is detected in the oil unloading area, the pre-unloading preparation steps are determined to be completed; it is necessary to detect staff after the pre-unloading preparation steps are completed before the loading and unloading supervision steps are considered to be satisfied; within the preset time after the vehicle leaves the oil unloading area, no fire extinguisher, oil pipeline, or electrostatic clamp is detected to determine that the site cleaning is completed; when the above steps are fully satisfied, the oil unloading process is determined to be a standard behavior.

[0051] When the oil unloading process is irregular, the control alarm module will be linked and save the missing step records and screenshots of the correct steps.

[0052] It involves a gas station oil unloading standard behavior detection system based on video stream, including:

[0053] An image acquisition module is used to acquire images of the detection area as required;

[0054] An image detection module is used to identify target objects in the detection area and output detection results;

[0055] The discrimination module determines the credibility of the detection result output by the image detection module according to the empirical confidence threshold;

[0056] The logical reasoning module performs logical judgment on the credible detection results to determine whether the oil unloading process is a standard behavior.

[0057] Alarm module.

[0058] Example 2

[0059] The recognition method of the standardized behavior detection of oil unloading at a gas station based on video stream includes:

[0060] 1. Set up a camera at a certain height to collect images of the oil unloading area.

[0061] 2. Use the trained detection model to output the location coordinates and corresponding categories of the target objects (tank trucks, wheel chocks, static clamps, fire extinguishers, oil pipelines, and staff) in the unloading area of ​​the gas station. The advanced deep learning detection network in recent years is used to carry out this process. The main processes include the construction of the data set, the construction of the model network, and the training and optimization of the model network;

[0062] 3. The output of the detection model includes an empirical confidence threshold, which is used to determine whether the corresponding target category exists in the oil unloading area of ​​the gas station. If the empirical confidence of the target box is greater than the empirical confidence threshold of the category, the target box and category are considered to be credible. If it is less than the empirical confidence threshold, it is considered to be unreliable.

[0063] 4. Perform logical judgment by outputting the target box and category in the image to complete the following process

[0064] (1) Guided vehicle: A tanker truck is detected in the oil unloading area

[0065] (2) Wheel chock placement: Wheel chock detected

[0066] (3) Safety check: Electrostatic clamp detected

[0067] (4) Safety protection: Fire extinguisher detected

[0068] (5) Preparation before unloading: oil pipeline is detected

[0069] (6) Loading and unloading supervision: Detection of staff

[0070] (7) Cleaning up the scene: No fire extinguishers, oil pipelines, or static clamps were detected in the area for a period of time after the vehicle left.

[0071] The specific logic judgment process is that when a tanker is found, the monitoring picture is uploaded. If the wheel chocks, electrostatic clamps, and fire extinguishers are not identified within the set time, the corresponding steps are considered missing and an alarm is issued. If they are identified, the steps are considered to be met. When the oil pipeline is identified, the pre-unloading preparation step is considered to be completed. The loading and unloading supervision needs to detect the staff after the pre-unloading preparation step is completed before the loading and unloading supervision step is considered to be met. The final cleaning site is that the fire extinguisher, oil pipeline, and electrostatic clamp are not detected within the set time after the vehicle leaves the unloading area. When the above steps are fully met, it is considered to be a complete unloading process.

[0072] Example 3

[0073] The invention relates to a recognition system for detecting the standardized behavior of oil unloading at a gas station based on video stream, which includes: an image acquisition module, which is mainly used to collect images of the oil unloading area of ​​the gas station. An image detection module, which is used to detect various key objects in the image: tank trucks, wheel chocks, electrostatic clamps, fire extinguishers, oil pipelines, and staff. A discrimination module, which is used to determine whether the target frame detected in the image is credible. A logical reasoning module, which infers whether the behavior specified in the specification has been completed through the image information obtained during the image detection and discrimination process. An alarm module, which alarms for standardized behaviors that have not been completed.

[0074] in:

[0075] The image acquisition module is mainly used to collect images of the oil unloading area of ​​the gas station. During the actual deployment test, the camera should be located in front of or behind the oil tanker in the direction of oil unloading and at a certain angle. It should not be facing the front or rear of the oil tanker. The wheel chocks, electrostatic clamps, fire extinguishers, oil pipelines, and staff should be visible within the field of view.

[0076] The image detection module is used to detect various key objects in the image: tank trucks, wheel chocks, electrostatic clamps, fire extinguishers, oil pipelines, and staff; the detection module can use machine learning, deep learning and other methods to obtain detection models, such as HOG+SVM detectors, RCNN, Fast RCNN, etc. In this invention, it is intended to optimize and compress the network model based on the existing more advanced target detection network. The existing network models are represented by YOLOv3, YOLOv5, and Yolox, and the RCNN series represented by RCNN, Fast RCNN, and Faster RCNN. These network models have shown extremely excellent performance in the coco data set and highlight the possibility of using neural networks in real scenes. The detection module of this solution is intended to be optimized on the basis of YOLO. The training of the neural network depends on data. To obtain an excellent detection model, extremely rich data is required. In order to adapt to different scenes, weather conditions, lighting conditions, image quality, and interference caused by camera angles, when constructing the data set, a camera is set up in a real gas station scene to collect data as a training set and a test set. The collected data includes the complete oil unloading process during the day and night, regular color images, and infrared images. In terms of data labeling, in order to reduce the workload of manual labeling, we first train a detection model with a small amount of data to perform pre-labeling, and then manually review and correct the data labeling. The approved data is added to the pre-trained model for fine-tuning and then the unlabeled data is labeled. This process is repeated until the data labeling is completed. This process can greatly reduce the workload of labeling. In addition to the aforementioned labeling categories, the non-staff category is also labeled, so that this module can distinguish between staff and non-staff.

[0077] The discrimination module is used to determine whether the target box detected in the image is credible. When the trained detection model is applied to the actual scene, the detection model will detect the objects in the image. There will be some other objects similar to the target object in the scene. Although these objects are similar to the target object, their confidence is low. These objects can be filtered out through the confidence threshold.

[0078] The logical reasoning module uses the image information obtained during the image detection and discrimination process to infer whether the behavior specified in the specification has been completed; when the tanker enters the monitoring area, the monitoring process starts. When the tanker is detected, the monitoring picture is uploaded. If the wheel chocks, electrostatic clamps, fire extinguishers, and oil pipelines are not identified within the set time, the corresponding steps are considered to be missing and an alarm is issued. After the oil pipeline is identified, if the loading and unloading personnel are not identified, it is considered that the loading and unloading personnel are not present and an alarm is issued. When the tanker leaves the unloading area within the set time range, and the fire extinguisher, oil pipeline, and electrostatic clamp are not identified in the unloading area, it is considered that the scene has been cleaned up.

[0079] The alarm module issues an alarm for incomplete standardized behaviors; in order to enable illegal behaviors to be corrected in a timely manner, this system can be linked with the sound column installed on site. When the system issues an alarm, the sound column emits an alarm sound to remind on-site personnel of the illegal steps, and saves the missing steps and screenshots of the correct steps on the system platform.

[0080] The above are embodiments of the present invention. For ordinary technicians in this field, according to the teachings of the present invention, all equivalent changes, modifications, substitutions and variations made within the scope of the patent application of the present invention without departing from the principles and spirit of the present invention should fall within the scope of the present invention.

Claims

1. A method for detecting standard oil unloading behavior at a gas station based on video stream, characterized in that: The specific steps include: Set the image acquisition module to acquire images of the detection area as required; Build and train a deep learning detection model to identify target objects in the detection area; The credibility of the detection results output by the detection model is judged according to the empirical confidence threshold; Make logical judgments on the credible test results to determine whether the oil unloading process is a standard behavior.

2. According to claim 1, a method for detecting standard oil unloading behavior at a gas station based on video stream, characterized in that: The steps of constructing and training a deep learning detection model for identifying target objects in the detection area specifically include: Collect diverse image data to build a dataset; Build an improved detection model and complete training; Using the trained detection model, the location coordinates and corresponding categories of the target objects in the oil unloading area of ​​the gas station are output.

3. The method for detecting the normal behavior of unloading oil at a gas station based on video stream according to claim 2 is characterized by: The construction of the improved detection model includes: selecting a YOLO series detection model to improve model performance through optimization and compression, specifically improving the detection head part in the detection model, using a decoupled detection head, so that classification, positioning, and confidence prediction are processed by different convolutional layers.

4. The method for detecting the normal behavior of unloading oil at a gas station based on video stream according to claim 2 is characterized by: The training process of the detection model adopts a combination of pre-labeling and manual review. Specifically, a pre-labeling model is trained with a small amount of data for pre-labeling, and the data labeling is corrected in combination with manual review. The approved data is added to the pre-trained model for fine-tuning and then the unlabeled data is labeled, and this process is repeated until the data labeling is completed.

5. The method for detecting the normal behavior of unloading oil at a gas station based on video stream according to claim 1, characterized in that: The step of judging the credibility of the detection result output by the detection model according to the empirical confidence threshold specifically includes: Set an empirical confidence threshold for each target category output by the detection model; By comparing the empirical confidence with the threshold, it is judged whether there is a corresponding target category in the oil unloading area of ​​the gas station. If the empirical confidence of the target box is greater than the empirical confidence threshold of the category, the target box and category are considered to be credible, and if it is less than the threshold, it is considered to be unreliable. Filter out unreliable test results to ensure the accuracy of the test results.

6. The method for detecting the normal behavior of unloading oil at a gas station based on video stream according to claim 1, characterized in that: The logical judgment of the credible detection results to determine whether the oil unloading process is a standard behavior step specifically includes: The specific logic judgment process is as follows: when a tanker is detected in the oil unloading area, a monitoring picture is uploaded. If wheel chocks, static clamps, and fire extinguishers are not identified within a preset time, the corresponding steps are considered missing. If they are identified, the steps are considered satisfied. When the oil pipeline is detected in the oil unloading area, it is determined that the pre-unloading preparation steps are completed; The unloading supervision step is considered satisfied only when workers are detected after the pre-unloading preparation step is completed; Within the preset time after the vehicle leaves the oil unloading area, no fire extinguisher, oil pipeline, or static clamp is detected, and the site cleaning is completed; When the above steps are fully met, the oil unloading process is judged to be a standard behavior.

7. The method for detecting the normal behavior of unloading oil at a gas station based on video stream according to claim 1, characterized in that: The setting of the image acquisition module specifically includes: setting up a camera in the oil unloading area of ​​the gas station, and the position should ensure that the shooting range covers the entire oil unloading area, including the tanker truck, wheel chocks, static clamps, fire extinguishers, oil pipelines and the activity area of ​​​​the staff; and the camera should be installed to avoid facing the front or rear of the tanker truck to ensure that the shooting angle can fully capture the entire oil unloading process.

8. The method for detecting the normal behavior of unloading oil at a gas station based on video stream according to claim 1 is characterized in that: Also includes: When the oil unloading process is irregular, the control alarm module will be linked and save the missing step records and screenshots of the correct steps.

9. A video stream-based gas station oil unloading standard behavior detection system, characterized in that: include: An image acquisition module is used to acquire images of the detection area as required; An image detection module is used to identify target objects in the detection area and output detection results; The discrimination module determines the credibility of the detection result output by the image detection module according to the empirical confidence threshold; The logical reasoning module performs logical judgment on the credible detection results to determine whether the oil unloading process is a standard behavior.