A monitoring method for personnel's illegal behavior during tank cleaning process based on image recognition

By using image recognition technology during the cabin washing process, the violations of the cabin washing operators are automatically monitored and judged, and the problem of manual supervision errors in the existing technology is solved, and the ship safety and monitoring accuracy are improved.

CN114581757BActive Publication Date: 2024-11-26CHONGQING SOUTHWEST WATER TRANSPORT ENG RES INST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210216848.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-07
Publication Date
2024-11-26
Estimated Expiration
2042-03-07

AI Technical Summary

Technical Problem

In the prior art, manual supervision and video surveillance during cabin washing are prone to errors and fatigue, resulting in insufficient safety guarantees and it is difficult to effectively monitor and prevent violations.

Method used

Using image recognition-based monitoring methods, by installing cameras in the cabin, building wearable object object detection and human posture recognition models, establishing a knowledge base for people-object interaction behavior recognition, automatically identifying and judging violations of cabin washing workers, and reducing artificial Monitoring error.

Benefits of technology

It improves the safety guarantee of the cabin washing process, reduces human risk factors, reduces the labor intensity of supervisors, and continuously improves the accuracy of judgments on violations through deep machine learning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114581757B_ABST
    Figure CN114581757B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for monitoring illegal behaviors of personnel in a tank cleaning process based on image recognition, which is characterized in that video data of key operation areas in the tank cleaning process are obtained by video acquisition equipment, and image processing is performed on the video data. Personnel behavior is identified by using a pre-built target detection and personnel behavior recognition model for the tank cleaning process and a related knowledge base, and it is determined whether the operating personnel have illegal behaviors based on the recognition results, and an alarm is immediately issued if there are illegal behaviors. The present invention can avoid manual monitoring errors and has the advantages of good safety and high security.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of ship safety maintenance, and in particular to a method for monitoring illegal behaviors of personnel in a tank washing process based on image recognition. Background Art

[0002] For oil tankers and bulk chemical ships, ship safety is particularly important because their transported goods are usually flammable and explosive. In order to ensure the quality of cargo loaded on oil tankers and bulk chemical ships or safety during repairs, oil tankers and bulk chemical ships need to clean the cargo holds before changing cargo types or entering the factory or docking for repairs. The main methods of tank cleaning include steaming, water washing, chemical washing, crude oil washing, etc. The tank cleaning process is complicated and cumbersome, and it is very easy to cause fire and explosion, leakage of hazardous chemicals and even personal injury accidents. The whole process is full of dangers. Manual operation is required in all links of the tank cleaning operation process, so human factors have become an important risk factor in the ship tank cleaning process. Standardized operation of tank cleaning operators is of great significance to reducing the risk of tank cleaning. During the tank cleaning process, real-time monitoring of the behavior of tank cleaning operators can avoid illegal operations by personnel, or timely intervention after illegal behavior occurs, which can effectively reduce the risk of accidents and improve the safety of the tank cleaning process.

[0003] Currently, there are two main ways to monitor the behavior of operators during tank cleaning: manual supervision and video monitoring. Manual supervision mainly involves visual supervision of the behavior of tank cleaning personnel by managers such as the tank cleaning team leader on site, while video monitoring mainly involves obtaining real-time images of the operation site through cameras arranged on site, and arranging full-time personnel to observe the video images for supervision. Both methods require personnel to pay attention to the situation on site or in the video monitoring images at all times, which places certain requirements on the concentration and professionalism of the supervisors, and are also prone to fatigue and omissions of the supervisors.

[0004] Therefore, how to design a method for monitoring illegal behaviors of tank cleaning personnel that can better improve safety assurance has become a problem that technical personnel in this field need to consider and solve. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is: how to provide a method for monitoring personnel's illegal behaviors in the tank washing process based on image recognition, which can avoid manual monitoring errors and better improve safety assurance.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A method for monitoring illegal behaviors of personnel in a tank cleaning process based on image recognition, characterized in that it comprises the following steps:

[0008] Step 1: Install a camera at a fixed position in the cabin, collect information through the camera, establish a training data set for wearable object detection in the cabin cleaning process, and build and train a wearable object detection model based on a neural network algorithm;

[0009] Step 2: Construct a training dataset for human behavior recognition and use it to train a human posture recognition model.

[0010] Step 3: Construct a knowledge base for identifying the human-object interaction behavior during the tank cleaning process. The knowledge base for identifying the human-object interaction behavior during the tank cleaning process predefines the rules for the wearing characteristics of personnel and the characteristics of personnel actions;

[0011] Step 4: Collect video data of each key area during the tank cleaning operation and extract key frame images;

[0012] Step 5: Extract features from each key frame image to obtain the action features and wearing features of the person;

[0013] Step 6: Compare the acquired motion features and wearing features of the personnel with the preset rules in the knowledge base of human-object interaction behavior recognition during the tank cleaning process to determine whether the personnel have violated the regulations;

[0014] Step 7: If a person violates the rules, an alarm is issued based on the risk level of the violation and the corresponding feature description.

[0015] In this way, this method uses camera photography to obtain data, relies on computer algorithm training to obtain an accurate violation judgment model, and then relies on camera monitoring to automatically identify whether the operator has violated the rules. Therefore, it can avoid manual monitoring errors and better improve ship safety protection.

[0016] Furthermore, the neural network algorithm adopts the YOLOV3 algorithm, which is more applicable and reliable.

[0017] Furthermore, the human posture recognition model adopts the OpenPose model. Using OpenPose for human posture recognition can better identify the operation actions in the cabin, and can be better matched and combined with the neural network algorithm to improve the calculation accuracy.

[0018] Furthermore, in step 1, the objects to be worn that need to be detected during the tank cleaning process include, but are not limited to, safety helmets, work clothes, work pants, safety shoes, mobile phones, lighters and cigarettes.

[0019] Some of the above-mentioned wearable objects are prohibited items, and some are items that need to be worn in accordance with regulations. Monitoring the above-mentioned objects can ensure safety more comprehensively. During implementation, a target detection data set is constructed based on these categories of objects. The data set should preferably contain more than 1,000 images, of which 80% are training sets and 20% are test sets. In addition to the actual collection by the camera, you can also search for images on the Internet to obtain these types of target images, and then uniformly scale them to a size of 640×480 pixels, and mark the images. When marking, the IDs of each category are numbered one by one starting from 1 according to natural numbers, which can be marked as 1 for safety helmets, 2 for work clothes, 3 for work pants, 4 for labor protection shoes, 5 for mobile phones, 6 for lighters, and 7 for cigarettes.

[0020] Furthermore, in step 1, when training the wearable object detection model, the database is expanded by color transformation, random translation, random cropping or random rotation.

[0021] This is because a small amount of data may cause the model to overfit, so this method can increase the amount of data, expand the distribution of training data, enhance the generalization ability of the model, and help improve model accuracy.

[0022] Furthermore, in step 2, when constructing the personnel behavior recognition training data set, designated personnel perform standing, walking, lying, sitting, squatting, climbing and carrying actions in the cabin scene. Each group of actions lasts about 5 seconds, and the frame rate is about 25 frames / second. The cameras are used to shoot from three fixed angles, and pictures are extracted from the video files at a certain frame interval. Then the pictures are grouped, and behavior labels are set for each group to constitute the personnel behavior recognition training data.

[0023] In this way, the recognition of various actions can be better achieved and the recognition reliability can be guaranteed.

[0024] Furthermore, in step 3, when constructing the knowledge base for human-object interaction behavior recognition, human behavior recognition uses a human posture recognition model to obtain human skeleton information in the image, and the joint point numbering uses the COCO model numbering method. The specific numbering rules are shown in the following table:

[0025] serial number 0 1 2 3 4 5 Articulation Points nose neck Right shoulder right elbow right wrist Left shoulder serial number 6 7 8 9 10 11 Articulation Points left elbow left wrist Right hip right knee Right ankle Left hip serial number 12 13 14 15 16 17 Articulation Points Left knee Left ankle Right eye Left Eye Right ear left ear .

[0026] Furthermore, in step 3, the knowledge base for identifying human-object interaction behaviors presets rules for wearing characteristics and human action characteristics of personnel, including whether the personnel wear safety helmets, work clothes, work pants and safety shoes, and whether the personnel play with mobile phones, make phone calls, use lighters and smoke. This can better facilitate the subsequent identification of violations.

[0027] Furthermore, in step 6, the trained target detection model is used to detect several types of targets of interest in the image, and the position (ix, iy, iw, ih) and category i of each target in the image are obtained. Then, new coordinate points are generated according to the skeleton node coordinates (x, y) output by the human posture recognition model, and a judgment is made based on the target position information and the new coordinate information to obtain the detection result.

[0028] In this way, it is possible to better determine whether the clothing is worn correctly first, and then detect whether the action process is illegal.

[0029] Furthermore, when judging whether a helmet is worn, the helmet is first detected from the picture, and then it is judged whether the helmet appears at the head position; if the helmet is not detected, or the helmet does not appear at the head position of the operator, it is judged as a violation of not wearing the helmet as required; when making position judgments, the two detected ear nodes of the human body are selected as references, the left ear coordinates are set to (Lx, Ly), the right ear coordinates are set to (Rx, Ry), and the helmet position information is (hat.x, hat.y, hat.w, hat.h); when judging, the range of the helmet detection frame is appropriately expanded and translated (up to twice the original size), and when the left and right ears are located in the adjusted detection frame, it is considered that the helmet is worn correctly; due to the camera shooting angle problem, only one of the left and right ears may be recognized in the picture, so the judgment conditions for the correct wearing of the helmet are as follows:

[0030]

[0031] or

[0032]

[0033] In this way, it is possible to better determine whether a helmet is worn, and other worn items can be determined using a similar method.

[0034] Therefore, the present invention uses video acquisition equipment to obtain video data of each key operation area during the tank cleaning process, and performs image processing on the video data. The pre-built target detection and personnel behavior recognition model for the tank cleaning process and the relevant knowledge base are used to recognize the personnel behavior, and the identification results are used to determine whether the operating personnel have violated the regulations. If there is any violation, an alarm is immediately issued.

[0035] The advantages of the present invention are: 1. It proposes an intelligent monitoring method for personnel violations during the tank washing process based on image recognition, changing the traditional manual supervision method to an unmanned intelligent monitoring method. It can enhance the supervision effect on the operating personnel, reduce the labor intensity of on-site supervisors, and reduce the human risk factors in the tank washing operation. 2. By establishing a knowledge base of violations during the tank washing process, the judgment of violations does not need to rely on the professional knowledge and technical level of on-site supervisors. And through deep machine learning, the knowledge base can be continuously expanded and improved to improve the accuracy of judgments on violations in tank washing.

[0036] Therefore, the present invention can avoid manual monitoring errors and has the advantages of good safety and high security. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the helmet position determination process in the embodiment. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below in conjunction with specific implementation modes.

[0039] Specific embodiment: A method for monitoring illegal behaviors of personnel in a tank cleaning process based on image recognition, comprising the following steps:

[0040] Step 1: Install a camera at a fixed position in the cabin, collect information through the camera, establish a training data set for wearable object detection in the cabin cleaning process, and build and train a wearable object detection model based on a neural network algorithm;

[0041] Step 2: Construct a training dataset for human behavior recognition and use it to train a human posture recognition model.

[0042] Step 3: Construct a knowledge base for identifying the human-object interaction behavior during the tank cleaning process. The knowledge base for identifying the human-object interaction behavior during the tank cleaning process predefines the rules for the wearing characteristics of personnel and the characteristics of personnel actions;

[0043] Step 4: Collect video data of each key area during the tank cleaning operation and extract key frame images;

[0044] Step 5: Extract features from each key frame image to obtain the action features and wearing features of the person;

[0045] Step 6: Compare the acquired motion features and wearing features of the personnel with the preset rules in the knowledge base of human-object interaction behavior recognition during the tank cleaning process to determine whether the personnel have violated the regulations;

[0046] Step 7: If a person violates the rules, an alarm is issued based on the risk level of the violation and the corresponding feature description.

[0047] In this way, this method uses camera photography to obtain data, relies on computer algorithm training to obtain an accurate violation judgment model, and then relies on camera monitoring to automatically identify whether the operator has violated the rules. Therefore, it can avoid manual monitoring errors and better improve ship safety protection.

[0048] The neural network algorithm adopts the YOLOV3 algorithm, which is more applicable and reliable.

[0049] The human posture recognition model adopts the OpenPose model. Using OpenPose for human posture recognition can better identify the operation actions in the cabin, and can be better matched and combined with the neural network algorithm to improve the calculation accuracy.

[0050] In step 1, the objects to be worn during the tank cleaning process include, but are not limited to, safety helmets, work clothes, work pants, safety shoes, mobile phones, lighters and cigarettes.

[0051] Some of the above-mentioned wearable objects are prohibited items, and some are items that need to be worn in accordance with regulations. Monitoring the above-mentioned objects can ensure safety more comprehensively. During implementation, a target detection data set is constructed based on these categories of objects. The data set should preferably contain more than 1,000 images, of which 80% are training sets and 20% are test sets. In addition to the actual collection by the camera, you can also search for images on the Internet to obtain these types of target images, and then uniformly scale them to a size of 640×480 pixels, and mark the images. When marking, the IDs of each category are numbered one by one starting from 1 according to natural numbers, which can be marked as 1 for safety helmets, 2 for work clothes, 3 for work pants, 4 for labor protection shoes, 5 for mobile phones, 6 for lighters, and 7 for cigarettes.

[0052] In step 1, when training the wearable object detection model, the database is expanded by color transformation, random translation, random cropping or random rotation.

[0053] This is because a small amount of data may cause the model to overfit, so this method can increase the amount of data, expand the distribution of training data, enhance the generalization ability of the model, and help improve model accuracy.

[0054] In step 2, when constructing the personnel behavior recognition training data set, designated personnel perform standing, walking, lying, sitting, squatting, climbing and carrying actions in the cabin scene. Each group of actions lasts about 5 seconds, and the frame rate is about 25 frames / second. The cameras are used to shoot from three fixed angles, and pictures are extracted from the video files at a certain frame interval. Then the pictures are grouped, and behavior labels are set for each group to form the personnel behavior recognition training data.

[0055] In this way, the recognition of various actions can be better achieved and the recognition reliability can be guaranteed.

[0056] In step 3, when constructing the knowledge base for human-object interaction behavior recognition, human behavior recognition uses a human posture recognition model to obtain human skeleton information in the image, and the joint point numbering uses the COCO model numbering method. The specific numbering rules are shown in the following table:

[0057] serial number 0 1 2 3 4 5 Articulation Points nose neck Right shoulder right elbow right wrist Left shoulder serial number 6 7 8 9 10 11 Articulation Points left elbow left wrist Right hip right knee Right ankle Left hip serial number 12 13 14 15 16 17 Articulation Points Left knee Left ankle Right eye Left Eye Right ear left ear .

[0058] In step 3, the knowledge base for identifying human-object interaction behaviors presets rules for the characteristics of people's clothing and human action characteristics, including whether people wear helmets, work clothes, work pants and safety shoes, and whether people play with mobile phones, make phone calls, use lighters and smoke. This can better facilitate the subsequent identification of violations.

[0059] In step 6, the trained target detection model detects several types of targets in the image, and obtains the position (ix, iy, iw, ih) and category i of each target in the image. Then, new coordinate points are generated according to the skeleton node coordinates (x, y) output by the human posture recognition model, and the target position information and the new coordinate information are combined for judgment to obtain the detection result.

[0060] In this way, it is possible to better determine whether the clothing is worn correctly first, and then detect whether the action process is illegal.

[0061] When judging whether to wear a helmet, see Figure 1 , first detect the helmet from the picture, and then determine whether the helmet appears at the head position; if the helmet is not detected, or the helmet does not appear at the head position of the operator, it is determined to be a violation of not wearing the helmet as required; when making position judgments, the two detected ear nodes of the human body are selected as references, and the left ear coordinates are set to (Lx, Ly), the right ear coordinates are set to (Rx, Ry), and the helmet position information is (hat.x, hat.y, hat.w, hat.h); when judging, the range of the helmet detection frame is appropriately expanded and translated (up to twice the original size). When the left and right ears are located in the adjusted detection frame, it is considered that the helmet is worn correctly ( Figure 1 ); Due to the camera shooting angle, only one of the left and right ears may be recognized in the picture, so the judgment conditions for correct wearing of the helmet are as follows:

[0062]

[0063] or

[0064]

[0065] In this way, it is possible to better determine whether a helmet is worn, and other worn items can be determined using a similar method.

Claims

1. A method for monitoring illegal behaviors of personnel in tank cleaning process based on image recognition, characterized in that: The following steps are involved: Step 1: Install a camera at a fixed position in the cabin, collect information through the camera, establish a training data set for wearable object detection in the cabin cleaning process, and build and train a wearable object detection model based on a neural network algorithm; Step 2: Construct a training dataset for human behavior recognition and use it to train a human posture recognition model. Step 3: Construct a knowledge base for identifying the human-object interaction behavior during the tank cleaning process. The knowledge base for identifying the human-object interaction behavior during the tank cleaning process pre-sets the rules for the wearing characteristics of personnel and the characteristics of personnel actions; Step 4: Collect video data of each key area during the tank cleaning operation and extract key frame images; Step 5: Extract features from each key frame image to obtain the action features and wearing features of the person; Step 6: Compare the acquired motion features and wearing features of the personnel with the preset rules in the knowledge base of human-object interaction behavior recognition during the tank cleaning process to determine whether the personnel have violated the regulations; Step 7: If a person violates the rules, an alarm is issued based on the risk level of the violation and the corresponding feature description.

2. The method for monitoring illegal behaviors of personnel in the tank cleaning process based on image recognition according to claim 1 is characterized in that: The neural network algorithm adopts the YOLOV3 algorithm.

3. The method for monitoring illegal behaviors of personnel in the tank cleaning process based on image recognition according to claim 1 is characterized in that: The human posture recognition model adopts the OpenPose model.

4. The method for monitoring illegal behaviors of personnel in the tank cleaning process based on image recognition according to claim 1 is characterized in that: In step 1, the objects to be worn during the tank cleaning process include, but are not limited to, safety helmets, work clothes, work pants, safety shoes, mobile phones, lighters and cigarettes.

5. The method for monitoring illegal behaviors of personnel in the tank cleaning process based on image recognition according to claim 1, characterized in that: In step 1, when training the wearable object detection model, the database is expanded by color transformation, random translation, random cropping or random rotation.

6. The method for monitoring illegal behaviors of personnel in the tank cleaning process based on image recognition according to claim 1, characterized in that: In step 2, when constructing the personnel behavior recognition training data set, designated personnel perform standing, walking, lying, sitting, squatting, climbing and carrying actions in the cabin scene. Each group of actions lasts about 5 seconds, and the frame rate is about 25 frames / second. The cameras are used to shoot from three fixed angles, and pictures are extracted from the video files at a certain frame interval. Then the pictures are grouped, and behavior labels are set for each group to form the personnel behavior recognition training data.

7. The method for monitoring illegal behaviors of personnel in the tank cleaning process based on image recognition according to claim 1 is characterized in that: In step 3, when constructing the knowledge base for human-object interaction behavior recognition, human behavior recognition uses a human posture recognition model to obtain human skeleton information in the image, and the joint point numbering uses the COCO model numbering method. The specific numbering rules are shown in the following table: 。 8. The method for monitoring illegal behaviors of personnel in the tank washing process based on image recognition according to claim 7 is characterized in that: In step 3, the human-object interaction behavior recognition knowledge base presets the rules for the wearing characteristics of personnel and the characteristics of personnel actions, including whether the personnel wear a safety helmet, work clothes, work pants and safety shoes, and whether the personnel play with mobile phones, make phone calls, use lighters and smoke.

9. The method for monitoring illegal behaviors of personnel in the tank cleaning process based on image recognition according to claim 8, characterized in that: In step 6, the trained target detection model detects several types of targets in the image, and obtains the position (ix, iy, iw, ih) and category i of each target in the image. Then, new coordinate points are generated according to the skeleton node coordinates (x, y) output by the human posture recognition model, and the target position information and the new coordinate information are combined for judgment to obtain the detection result.

10. The method for monitoring illegal behaviors of personnel in the tank cleaning process based on image recognition according to claim 9, characterized in that: When judging whether a helmet is worn, the system first detects the helmet from the image, and then determines whether the helmet appears on the head. If the helmet is not detected, or the helmet does not appear on the operator's head, it is considered a violation of not wearing the helmet as required. When making position judgment, the two detected ear nodes of the human body are selected as references, the left ear coordinates are set to (Lx, Ly), the right ear coordinates are set to (Rx, Ry), and the helmet position information is set to (hat.x, hat.y, hat.w, hat.h); when judging, the range of the helmet detection frame is appropriately expanded and translated. When the left and right ears are located in the adjusted detection frame, it is considered that the helmet is worn correctly; due to the camera shooting angle, only one of the left and right ears may be recognized in the picture, so the judgment conditions for the correct wearing of the helmet are as follows: or