A rocker drill illegal operation detection method based on human posture recognition

Abstract

The application discloses a kind of based on human posture recognition's rocker arm drilling machine rule violation detection method, by recognizing human posture, obtain human key joint, by judging module, logic judgment is carried out to joint and mark point, whether the operation of the worker on rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm drilling machine is checked before rocker arm drilling machine starts, whether the operation of the rocker arm

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of rocker drill bed operation detection, in particular to a rocker drill bed illegal operation detection method based on human body posture recognition. BACKGROUND

[0002] The rocker drill bed is a general machine tool with wide application, which is a rocker drill bed with a rocker that can rotate around a column and lift, and a main shaft box that moves horizontally on the rocker. Among various types of rocker drill beds, the rocker drill bed is convenient and flexible to operate, has a wide range of applications, and is typical, especially suitable for single-piece or batch production of large parts with multiple holes. It is a common machine tool in general mechanical processing workshops. With the extensive use of rocker drill beds, people are increasingly concerned about some safety issues of rocker drill beds. Safety accidents caused by workers violating safety operation procedures occur frequently. It is difficult to achieve real-time monitoring of workers' illegal operation of rocker drill beds during use by selecting manual supervision of workers' operation of rocker drill beds, and it is difficult to effectively reduce the probability of safety accidents. SUMMARY

[0003] The present application aims to provide a rocker drill bed illegal operation detection method based on human body posture recognition to solve the problems of low efficiency, poor effect and poor safety in traditional rocker drill bed operation workshops.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0005] A rocker drill bed illegal operation detection method based on human body posture recognition, comprising a posture detection computing device, a main camera, a secondary camera and a PLC module, wherein the posture detection computing device is internally integrated with a posture detection model, a judgment module and a warning module.

[0006] Specifically comprising the following steps:

[0007] Step 1, camera installation:

[0008] A main camera is installed obliquely above the working area to shoot the entire working area, and a secondary camera is installed on one side of the drill bed rocker to shoot the area below the rocker.

[0009] Step 2, mark the area coordinates:

[0010] Record the two-dimensional coordinates of the rocker drill bed lifting mechanism, clamping mechanism, electrical system, workbench, area below the rocker, tool rotation and working area through the main camera and secondary camera; set the rocker drill bed lifting mechanism as P1, the clamping mechanism as P2, the electrical system as P3, the working area as G1, the tool rotation as G2, the workbench area as G3, and the area below the rocker as G4.

[0011] Step 3: Obtain the coordinates of human joints:

[0012] The judgment module is always running. Once it detects the two-dimensional skeletal joints of the human body, it determines whether the coordinates of the neck joint are within G1. If they are, it means that the person has entered the work area, and a radial drilling machine work detection process is initiated. At this time, the two-dimensional RGB image information of the human body collected by the main camera and the secondary camera is used to detect the worker's posture through the posture detection model, obtain the two-dimensional coordinates of the worker's joints, record these two-dimensional coordinates of the joints as shared variables, and pass them to the judgment module. Moreover, the two-dimensional coordinates of the worker's joints are updated in real time along with the real-time video acquired by the main camera and the secondary camera.

[0013] Step 4: Mark the coordinates of key joints in the human body:

[0014] Set the left wrist joint point to PL, the right wrist joint point to PR, the line from the right wrist to the right elbow to XR, the line from the left wrist to the left elbow to XL, the line from the nose to the neck to XN, and the line from the neck to the mid-hip to XH.

[0015] Step 5: Standard worker operation diagram of the rocker arm joint:

[0016] Footage of workers routinely operating the rocker arm at different times and angles on-site is captured. Using a posture detection and calculation device, joint information from the video data is collected in real time. A set of coordinates for 12 nodes—neck 1, right shoulder 2, right elbow 3, right wrist 4, left shoulder 5, left elbow 6, left wrist 7, mid-hip 8, left knee 13, right knee 10, left ankle 14, and right ankle 11—is created. Twelve circular regions (R1, R2, R3, R4, R5, R6, R7, R8, R9, R10, R11, R12) are defined, with the radius of each region set as a threshold value of 90px, 100px, 110px, or 120px. This results in a standard joint diagram of a worker operating the rocker arm.

[0017] The conditions under which step five does not involve judgment include: within a single workflow, the judgment described in step five is only performed when the radial drilling machine is started for the first time; after the first start, the judgment described in step five is not performed during the stopping and starting operations of the radial drilling machine.

[0018] Step Six: Inspection Process Judgment

[0019] The PLC module calculates the distances from the marker point PR (right wrist joint) to the radial drilling machine's lifting mechanism P1, clamping mechanism P2, and electrical system P3 for each frame captured by the main and secondary cameras. If the distance is less than a set threshold (40, 50, 60, or 70 pixels), it is determined that the worker has performed an inspection of the radial drilling machine's lifting mechanism, clamping mechanism, and electrical system. If the worker does not pass all three threshold checks, the PLC module will prevent the radial drilling machine from starting and will issue a warning to the worker via the warning module, while recording the current scene. The radial drilling machine can only start normally after the worker completes these inspection operations.

[0020] The conditions under which step six will not be judged include: when the worker manually stops the radial drilling machine during the workflow to change tools, adjust the position of the workpiece on the worktable, change tools on the worktable, go to the toilet, or temporarily leave the work area for any reason, step six will stop judging.

[0021] Step 7: Detection of Work Violations

[0022] After the radial drilling machine is turned on, it detects whether the worker's left wrist joint point PL and right wrist joint point PR are in the tool rotation area G2 and the worktable area G3. If the left wrist joint point PL and right wrist joint point PR are detected in the area, the judgment module determines that the worker has violated the operation rules. The PLC module will stop the radial drilling machine. At this time, the warning module will give a voice warning to the worker, record the violation, and send the violation information to the safety management personnel.

[0023] When the radial drilling machine is turned on, the system detects whether the worker's right wrist to right elbow line XR and left wrist to left elbow line XL have entered the area under the radial arm G4 beyond the threshold. This threshold is set to 60px, 70px, or 80px. If one or more exceed the threshold, it is determined that the worker has put their hand under the radial arm while the radial drilling machine is turned on. The PLC module will stop the radial drilling machine. At this time, the warning module will give the worker a voice warning, record the violation, and send the violation information to the safety management personnel.

[0024] When the radial drilling machine is started, the system detects whether the straight line XN from the worker's nose to his neck and the straight line XH from his neck to his mid-hip are within the work area G1. If both XN and XH are not within the G1 area, it will be determined that the worker has left the work area when the radial drilling machine is started. The PLC module will stop the radial drilling machine. At this time, the warning module will issue a voice warning to the worker, record the violation, and send the violation information to the safety management personnel.

[0025] When the radial drilling machine is started, the joint information in the data is collected in real time using a posture detection and calculation device. The coordinates of 12 nodes are extracted: neck 1, right shoulder 2, right elbow 3, right wrist 4, left shoulder 5, left elbow 6, left wrist 7, mid-hip 8, left knee 13, right knee 10, left ankle 14, and right ankle 11. These coordinates are then compared with the standard worker operation diagram of the radial drilling machine obtained in step 5 to determine whether the worker is operating the radial drilling machine normally. Taking the neck (circular area R1) in the standard worker operation diagram of the radial drilling machine as an example, during the workflow, the joints in the video data are collected in real time using a posture detection model. Point information, such as the coordinates of the neck joint point PN1 and the coordinates of the center of the neck circle in the R1 circular area r1, is obtained. The straight-line distance between r1 and PN1 is calculated using the formula X=((x1-x2)^2+(y1-y2)^2)^(1 / 2). If the result does not exceed the set radius threshold, which is set to 80px, 90px, or 100px, then the neck joint point is considered to be in place. If the number of 12 joint points in place exceeds the threshold, which is set to 8, 9, or 10, then the worker's action can be considered to be in place, and the worker's normal operating arm posture can be identified.

[0026] Step 8: End the testing process:

[0027] After the radial drilling machine stops, if the straight line XN from the worker's nose to his neck and the straight line XH from his neck to his mid-hip are not within the G1 area for more than the threshold duration, which is set to 20 minutes, 30 minutes or 40 minutes, the current inspection process ends and waits for the next work process to restart.

[0028] The judgment module calculates the distances from the right wrist joint point PR marker to P1, P2, and P3 in step six. The specific method used is as follows: if the distance X from P1 to PR is calculated, record the right wrist joint points PR(x1, y1) and P1(x2, y2), and get: X = ((x1-x2)^2 + (y1-y2)^2)^(1 / 2); the distances from P2 and P3 to PR are calculated in the same way.

[0029] Whether the left wrist joint point PL and the right wrist joint point PR in step seven appear in the tool rotation area G2 and the worktable area G3, the specific steps include:

[0030] (1) Taking the determination of PL in region G2 as an example, P1, P2, P3, and P4 are the coordinates of the four vertices of the rectangular region G2, and P is the coordinate of PL. The cross product method is used to determine whether (P2-P1)X(P-P1)*(P4-P3)X(P-P3)>=0.

[0031] If (P3-P2)X(P-P2)*(P1-P4)X(P-P4)>=0, then the point is determined to be within the rectangular region, and vice versa.

[0032] (2) Determine whether the straight line XR from the right wrist to the right elbow and the straight line XL from the left wrist to the left elbow intrude into the lower region G4 of the rocker arm. The specific steps include:

[0033] Taking whether the XR line intrudes into the area G4 under the rocker arm as an example, divide the XR line into ten equal parts, obtaining 10 coordinate points P1 to P10 proportionally. For the four coordinate points GP1, GP2, GP3, and GP4 in the G4 area, taking P1 as an example, using the cross product method, if (GP2-GP1)×(P1-GP1)*(GP4-GP3)×(P1-GP3)>=0, then...

[0034] If (GP3-GP2)X(P1-GP2)*(GP1-GP4)X(P1-GP4)>=0, then P1 is considered to be inside the G4 region. The judgment method for P2 to P10 is as described above. If the number of the ten coordinate points from P1 to P10 located in the G4 region exceeds the threshold, then the XR line is considered to have invaded the G4 region. Here, the threshold is set to 6, 7, or 8, and X is the cross product symbol.

[0035] (3) Determine whether the straight line XN from the worker's nose to his neck and the straight line XH from his neck to his mid-hip are within the work area G1. The specific steps include:

[0036] Take the center points of XN and XH, P1 and P2, and the four coordinate points of region G1, GP1, GP2, GP3, and GP4. If (GP2-GP1)X(P1-GP1)*(GP4-GP3)X(P1-GP3)>=0, then...

[0037] If (GP3-GP2)X(P1-GP2)*(GP1-GP4)X(P1-GP4)>=0 is satisfied simultaneously, then P1 is determined to be inside region G1. The same applies to P2. Then, it is determined whether P1 and P2 are both inside or outside G1.

[0038] The PLC module connects the attitude detection and calculation equipment and the radial drilling rig. Specifically, when the PLC module receives instructions from the judgment module in the attitude detection and calculation equipment, it controls the radial drilling rig. Specifically, when the PLC module receives a stop instruction from the judgment module, it will power off the radial drilling rig; when the PLC module receives a power-off instruction from the judgment module, it will prevent the radial drilling rig from being powered on.

[0039] The warning module receives signals from the judgment module. If a signal is received before the radial drilling machine is started, it will issue a voice alarm to remind the worker that the pre-start checks of the radial drilling machine have not been completed. At the same time, the judgment module will issue a command to prevent the radial drilling machine from starting, and will record the images from the main and secondary cameras. When the radial drilling machine is started, if a signal is received from the judgment module, it will issue a voice alarm to remind the worker that the radial drilling machine has violated the safety operating procedures. At the same time, the PLC module will control the radial drilling machine to stop, and will record the images from the main and secondary cameras. Optionally, it can notify safety management personnel that a worker has violated the operating procedures.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] This invention identifies key joints of the human body by recognizing human posture. A judgment module then logically compares these joints with marked points to identify whether a worker checks the lifting mechanism, clamping mechanism, electrical system, or other devices on the radial drilling machine before it starts; whether the worker touches the cutting tool or worktable area, or passes or retrieves tools across the rotating parts of the machine after it starts; whether the worker leaves the work area while the radial drilling machine is running; and whether the worker's normal operating posture on the radial drilling machine is correct. After identification, the invention judges the worker's unsafe operation of the radial drilling machine. When a violation is detected, a voice alarm is triggered, reminding the worker of the violation of the radial drilling machine's safety operating procedures. The PLC module controls the radial drilling machine to stop and records the images from the main and secondary cameras. Optionally, safety management personnel can be notified of the worker's violation, effectively reducing worker violations and improving safety. Attached Figure Description

[0042] Figure 1 This is a three-dimensional structural schematic diagram of the radial drilling machine of the present invention;

[0043] Figure 2 This is a schematic diagram of the overall structure of the radial drilling machine of the present invention;

[0044] Figure 3 This is a schematic diagram of the radial drilling machine violation detection method of the present invention;

[0045] Figure 4 This is a schematic diagram of the matrix used to determine point P in this invention;

[0046] Figure 5 This is a schematic diagram of the workflow of the present invention;

[0047] Figure 6 This is a schematic diagram of the attitude detection model of the present invention;

[0048] Figure 7 This is a standard worker's diagram of the rocker arm joint for this invention.

[0049] In the picture:

[0050] 01-Lifting mechanism, 02-Clamping mechanism, 03-Electrical system, 04-Workbench, 05-Under-rocker area, 06-Tool rotation area, 07-Working area, 08-Attitude detection and calculation device, 09-Main camera, 010-Secondary camera, 011-PLC module. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example 1

[0052] like Figure 3 As shown, a method for detecting violations of radial drilling machine operation based on human posture recognition includes a posture detection and computing device 08, a main camera 09, a secondary camera 010, and a PLC module 011. The posture detection and computing device 08 internally deploys an OpenPose posture detection model and an Nvidia Jetson Agx Xavier 16G judgment module with a logic judgment program.

[0053] Specifically, the following steps are included:

[0054] Step 1: Camera Installation

[0055] like Figures 1-3 As shown, a main camera 09 is installed diagonally above the working area 07 to capture the entire working area 07. A secondary camera 010 is installed on one side of the drilling machine arm to capture the area 05 below the arm and record the two-dimensional coordinates of the drilling machine lifting mechanism 01, clamping mechanism 02, electrical system 03, worktable 04, area 05 below the arm, tool rotation point 06, and working area 07. Real-time video of the working area 07 is obtained through the cameras.

[0056] Step 2: Mark the coordinates of the region:

[0057] The main camera 09 and the secondary camera 010 record the two-dimensional coordinates of the radial drilling machine's lifting mechanism 01, clamping mechanism 02, electrical system 03, worktable 04, lower area of ​​the radial arm 05, tool rotation area 06, and working area 07; and set the radial drilling machine's lifting mechanism as P1, clamping mechanism as P2, electrical system 03 as P3, working area as G1, tool rotation area as G2, worktable area as G3, and lower area of ​​the radial arm as G4.

[0058] Step 3: Obtain the coordinates of human joints:

[0059] The judgment module is always running. Once the two-dimensional skeleton joints of the human body are detected, it determines whether the coordinates of the neck joint are within G1. If they are, it means that the person has entered the work area 07 and the radial drilling machine work detection process is started. The judgment module obtains the two-dimensional RGB image of the human body in the current video frame and uses the posture detection model to obtain the coordinates of the two-dimensional skeleton joints of the human body.

[0060] The pose detection model specifically uses OpenPose body_25, an open-source library developed by Carnegie Mellon University (CMU) based on convolutional neural networks and supervised learning, using Caffe as the framework. It can estimate poses such as human movements, facial expressions, and finger movements. body_25 is a built-in detection library for 25 key points of human pose, namely: nose 0, neck 1, right shoulder 2, right elbow 3, right wrist 4, left shoulder 5, left elbow 6, left wrist 7, mid-hip 8, right hip 9, right knee 10, right ankle 11, left hip 12, left knee 13, left ankle 14, right eye 15, left eye 16, right ear 17, left ear 18, left thumb 19, left little finger 20, left heel 21, right thumb 22, right little finger 23, right heel 24 (e.g., ...). Figure 6 What is seen).

[0061] This implementation uses 12 nodes as key joints: neck 1, right shoulder 2, right elbow 3, right wrist 4, left shoulder 5, left elbow 6, left wrist 7, mid-hip 8, left knee 13, right knee 10, left ankle 14, and right ankle 11. The two-dimensional coordinates of these 12 joints are recorded as shared variables and passed to the judgment module. The two-dimensional coordinates of the worker's joints are updated in real time as the real-time video acquired by the main camera 09 and the secondary camera 010 is obtained.

[0062] Step 4: Mark the coordinates of key joints in the human body:

[0063] Set the left wrist joint point to PL, the right wrist joint point to PR, the line from the right wrist to the right elbow to XR, the line from the left wrist to the left elbow to XL, the line from the nose to the neck to XN, and the line from the neck to the mid-hip to XH.

[0064] Step 5:

[0065] Footage of workers routinely operating the rocker arm at different times and angles on-site is captured. Using a posture detection and calculation device (08), joint information from the video data is collected in real time. A set of coordinates for 12 nodes is created: neck 1, right shoulder 2, right elbow 3, right wrist 4, left shoulder 5, left elbow 6, left wrist 7, mid-hip 8, left knee 13, right knee 10, left ankle 14, and right ankle 11. Twelve circular regions (R1, R2, R3, R4, R5, R6, R7, R8, R9, R10, R11, R12) are defined, with the radius of each region set as a threshold of 100 pixels. This forms a standard joint diagram of a worker operating the rocker arm (e.g., ...). Figure 7 (What is seen);

[0066] When the radial drilling machine is started, the posture detection and calculation device 08 collects joint point information in real time, extracting the coordinates of 12 nodes: neck 1, right shoulder 2, right elbow 3, right wrist 4, left shoulder 5, left elbow 6, left wrist 7, mid-hip 8, left knee 13, right knee 10, left ankle 14, and right ankle 11. These coordinates are then compared with the standard worker's radial drilling machine joint diagram to determine if the worker is operating the radial drilling machine normally. Taking the neck area (R1 circle) in the standard worker's radial drilling machine joint diagram as an example, the posture detection model is used in the workflow. Real-time acquisition of joint point information from video data, such as obtaining the coordinates of the neck joint PN1 and the coordinates of the center of the neck circle r1 in the R1 circular area, calculating the straight-line distance between r1 and PN1 using the formula X=((x1-x2)^2+(y1-y2)^2)^(1 / 2), if the result does not exceed the set radius threshold, which is set to 90px, then it is determined that the neck joint is in place; if the number of 12 joints in place exceeds the threshold, which is 9, then it can be determined that the worker's action is in place, and the worker's normal operating arm posture can be identified.

[0067] The conditions under which step five does not involve judgment include: within a single workflow, the judgment described in step five is only performed when the radial drilling machine is started for the first time; after the first start, the judgment described in step five is not performed during the stopping and starting operations of the radial drilling machine.

[0068] Step Six: Detection Process Judgment

[0069] The judgment module calculates the distances from the marker point PR of the right wrist joint to the lifting mechanism P1, clamping mechanism P2, and electrical system P3 of the radial drilling machine, respectively, for each frame of images captured by the main camera 09 and the secondary camera 010.

[0070] The specific calculation method is as follows: If we calculate the distance X from P1 to PR, we record the right wrist joint points PR(x1, y1) and P1(x2, y2), and get: X=((x1-x2)^2+(y1-y2)^2)^(1 / 2); the calculation of the distances from P2 and P3 to PR is the same as above.

[0071] If the distance is less than the set threshold, which is set to 50px, it is determined that the worker has performed an inspection operation on the radial drilling machine's lifting mechanism 01, clamping mechanism 02, and electrical system 03. If the worker starts the radial drilling machine without passing all three threshold judgments, the PLC module 011 will prevent the radial drilling machine from starting, and the warning module will issue a warning to the worker and record the current screen. The radial drilling machine can only start normally after the worker completes these inspection operations.

[0072] The conditions under which step six will not be judged include: when the worker manually stops the radial drilling machine during the workflow to change tools, adjust the position of the workpiece in worktable 04, change the tools on worktable 04, go to the toilet, or temporarily leave the work area 07 for some reason, step six will stop the judgment operation.

[0073] Step 7: Detection of Work Violations

[0074] (1) After the radial drilling machine is turned on, check whether the worker's left wrist joint PL and right wrist joint PR are in the tool rotation area G2 and the worktable area G3.

[0075] The specific calculation method is as follows: Taking the determination that PL is in region G2 as an example, P1, P2, P3, and P4 are the coordinates of the four vertices of the rectangular region G2, and P is the coordinate of PL (e.g., ...). Figure 4 As shown), the cross product method is used to determine whether (P2-P1)X(P-P1)*(P4-P3)X(P-P3)>=0.

[0076] If (P3-P2)X(P-P2)*(P1-P4)X(P-P4)>=0, then the point is determined to be within the rectangular region, and vice versa.

[0077] If the left wrist joint PL and right wrist joint PR of a human are detected in the area, the judgment module determines that the worker has violated the operation rules. The PLC module 011 will stop the radial drilling machine. At this time, the warning module will issue a voice warning to the worker, record the violation, and send the violation information to the safety management personnel.

[0078] (2) When the radial drilling machine is turned on, detect whether the range of the worker's right wrist to right elbow line XR and left wrist to left elbow line XL intrusion into the area G4 under the radial arm exceeds the threshold, which is set to 70px.

[0079] The specific calculation method is as follows: Taking whether the XR line intrudes into the area G4 under the rocker arm as an example, the XR line is divided into ten equal parts, and ten coordinate points P1 to P10 are obtained proportionally. For the four coordinate points GP1, GP2, GP3, and GP4 in the G4 area, taking P1 as an example, the cross product method is used. If (GP2-GP1)X(P1-GP1)*(GP4-GP3)X(P1-GP3)>=0 and (GP3-GP2)X(P1-GP2)*(GP1-GP4)X(P1-GP4)>=0 are satisfied at the same time, then it is determined that P1 is inside the G4 area. The judgment method for P2 to P10 is as described above. If the number of the ten coordinate points from P1 to P10 located in the G4 area exceeds the threshold, it is determined that the XR line intrudes into the G4 area. The PLC module 011 will stop the rocker arm drilling machine. At this time, the warning module will give a voice warning to the worker, record the violation scene, and send the violation information to the safety management personnel.

[0080] Here, the threshold is set to 7, and X represents the cross product symbol.

[0081] (3) When the radial drilling machine is turned on, check whether the straight line XN from the worker's nose to his neck and the straight line XH from his neck to his middle hip are within the working area G1.

[0082] The specific calculation method is as follows: Take the center point coordinates P1 and P2 of XN and XH, and the four coordinate points GP1, GP2, GP3, and GP4 of region G1. If (GP2-GP1)X(P1-GP1)*(GP4-GP3)X(P1-GP3)>=0 and (GP3-GP2)X(P1-GP2)*(GP1-GP4)X(P1-GP4)>=0 are satisfied at the same time, then it is determined that P1 is inside region G1. The same applies to P2. Then, it is determined whether P1 and P2 are both inside or outside of region G1.

[0083] If it is determined that the XN and XH lines are not within the G1 area at the same time, it will be determined that the worker left the work area 07 when the radial drilling machine was turned on. The PLC module 011 will stop the radial drilling machine. At this time, the warning module will give a voice warning to the worker, record the violation, and send the violation information to the safety management personnel.

[0084] Step 8: End the testing process:

[0085] After the radial drilling machine stops, if the straight line XN from the worker's nose to his neck and the straight line XH from his neck to his mid-hip are not within the G1 area for more than the threshold duration, the threshold duration is set to 30 minutes. The current inspection process ends and waits for the next work process to restart.

[0086] PLC module 011 connects to attitude detection and calculation device 08 and radial drilling rig. Specifically, when PLC module 011 receives instructions from the judgment module in attitude detection and calculation device 08, it controls the radial drilling rig. Specifically, when PLC module 011 receives a stop instruction from the judgment module, it will power off the radial drilling rig; when PLC module 011 receives a power-on prevention instruction from the judgment module, it will prevent the radial drilling rig from being powered on.

[0087] The warning module receives signals from the judgment module. If a signal is received before the radial drilling machine is started, it will issue a voice alarm to remind the worker that the pre-start checks of the radial drilling machine have not been completed. At the same time, the judgment module will issue a command to prevent the radial drilling machine from starting, and will record the images from the main camera 09 and the auxiliary camera 010. When the radial drilling machine is started, if a signal is received from the judgment module, it will issue a voice alarm to remind the worker that the radial drilling machine has violated the safety operating procedures. At the same time, the PLC module 011 will control the radial drilling machine to stop and record the images from the main camera 09 and the auxiliary camera 010. Optionally, it can notify the safety management personnel that a worker has violated the operating procedures.

[0088] Since the data for program judgment comes from images, distance judgments are measured in pixels (px), while quantity judgments are measured in units of "pieces" and duration judgments in units of "minutes." Additionally, if the relative distance between the camera and the machine tool and operator changes, the threshold should be adjusted accordingly based on the proportion of image change caused by the distance change, such as increasing or decreasing by 10-20 pixels, increasing the number of judgments by 1-2 pieces, and increasing or decreasing the judgment duration by 10-30 minutes depending on the specific factory conditions.

[0089] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting improper operation of a radial drilling machine based on human posture recognition, characterized in that: It includes an attitude detection and calculation device (08), a main camera (09), a secondary camera (010), and a PLC module (011), wherein the attitude detection and calculation device (08) integrates an attitude detection model, a judgment module, and a warning module; and includes the following steps: Step 1: Camera Installation A main camera (09) is installed diagonally above the work area (07) to capture the entire work area (07), and a secondary camera (010) is installed on one side of the drill press rocker arm to capture the area below the rocker arm (05). Step 2: Mark the coordinates of the region: The main camera (09) and the secondary camera (010) record the two-dimensional coordinates of the radial drilling machine's lifting mechanism (01), clamping mechanism (02), electrical system (03), worktable (04), lower area of ​​the radial arm (05), tool rotation area (06), and working area (07); and set the radial drilling machine's lifting mechanism as P1, clamping mechanism as P2, electrical system as P3, working area as G1, tool rotation area as G2, worktable area as G3, and lower area of ​​the radial arm as G4. Step 3: Obtain the coordinates of human joints: The judgment module is always running. Once the two-dimensional skeleton joints of the human body are detected, it determines whether the coordinates of the neck joints are within G1. If they are, it means that the person has entered the work area (07) and the radial drilling machine work detection process is started. At this time, the two-dimensional RGB image information of the human body collected by the main camera (09) and the secondary camera (010) is used to detect the worker's posture through the posture detection model, obtain the two-dimensional coordinates of the worker's joints, record these two-dimensional coordinates of the joints as shared variables, and pass them to the judgment module. The two-dimensional coordinates of the worker's joints are updated in real time with the real-time video obtained by the main camera (09) and the secondary camera (010). Step 4: Mark the coordinates of key joints in the human body: Set the left wrist joint point to PL, the right wrist joint point to PR, the straight line from the right wrist to the right elbow to XR, the straight line from the left wrist to the left elbow to XL, the straight line from the nose to the neck to XN, and the straight line from the neck to the mid-hip to XH. Step 5: Standard worker operation diagram of the rocker arm joint: Capture images of workers routinely operating the rocker arm at different times and angles on the work site. Use a posture detection and calculation device (08) to collect joint information from the video data in real time. Create a set of coordinates for 12 nodes: neck 1, right shoulder 2, right elbow 3, right wrist 4, left shoulder 5, left elbow 6, left wrist 7, mid-hip 8, left knee 13, right knee 10, left ankle 14, and right ankle 11. Define 12 circular regions R1, R2, R3, R4, R5, R6, R7, R8, R9, R10, R11, and R12. Set the radius of the circular region (90-120px) as the judgment threshold to form a standard worker operating rocker arm joint diagram. Step Six: Inspection Process Judgment The judgment module calculates the distances from the marker point right wrist joint PR to the radial drilling machine lifting mechanism P1, clamping mechanism P2 and electrical system P3 respectively through the judgment module of each frame image captured by the main camera (09) and the secondary camera (010). If the distance is less than the set threshold of 40-70px, it is determined that the worker has performed an inspection operation on the radial drilling machine lifting mechanism (01), clamping mechanism (02) and electrical system (03). Step 7: Detection of Work Violations After the radial drilling machine is turned on, the system detects whether the worker's left wrist joint point PL and right wrist joint point PR are located in the tool rotation area G2 and the worktable area G3. The judgment module then determines whether the worker has violated the rules. When the radial drilling machine is turned on, the line XR from the worker's right wrist to his right elbow and the line XL from his left wrist to his left elbow are checked to see if the area encroaching on the area G4 under the radial arm exceeds the threshold of 60-80px; when the radial drilling machine is turned on, the line XN from the worker's nose to his neck and the line XH from his neck to his mid-hip are checked to see if they are within the working area G1. When the radial drilling machine is turned on, the joint information in the data is collected in real time using the posture detection and calculation device (08). The coordinates of 12 nodes, namely neck 1, right shoulder 2, right elbow 3, right wrist 4, left shoulder 5, left elbow 6, left wrist 7, mid-hip 8, left knee 13, right knee 10, left ankle 14, and right ankle 11, are extracted and compared with the standard worker operation radial drilling joint diagram obtained in step 5 to determine whether the worker is operating the radial drilling machine in a routine manner. When the above detection determines that the worker has violated regulations or operated abnormally, the PLC module (011) will stop the radial drilling machine. At this time, the warning module will issue a voice warning to the worker, record the violation, and send the violation information to the safety management personnel. Step 8: End the testing process: After the radial drilling machine stops, if the straight line XN from the worker's nose to his neck and the straight line XH from his neck to his mid-hip are not within the G1 area for more than 20-40 minutes, the current inspection process ends and the worker waits for the next work process to restart.

2. The method for detecting improper operation of a radial drilling machine based on human posture recognition according to claim 1, characterized in that: In step six, the judgment module calculates the distance from the right wrist joint point PR marker to P1, P2, and P3, calculates the distance X from P1 to PR, and records the right wrist joint points PR(x1, y1) and P1(x2, y2), resulting in: X = ((x1-x2)^2 + (y1-y2)^2)^(1 / 2); the calculation of the distances from P2 and P3 to PR is the same as above.

3. The method for detecting improper operation of a radial drilling machine based on human posture recognition according to claim 1, characterized in that: In step seven, it is checked whether the worker's left wrist joint point PL and right wrist joint point PR appear in the tool rotation area G2 and the worktable area G3; whether the straight line XR from the right wrist to the right elbow and the straight line XL from the left wrist to the left elbow intrude into the area under the rocker arm G4; and whether the straight line XN from the worker's nose to the neck and the straight line XH from the neck to the middle hip are located in the working area G1. The calculation method used for all three is the cross product method.

4. The method for detecting improper operation of a radial drilling machine based on human posture recognition according to claim 1, characterized in that: The conditions under which step five does not involve judgment include: within a single workflow, the judgment described in step five is only performed when the radial drilling machine is started for the first time; after the first start, the judgment described in step five is not performed during the stopping and starting operations of the radial drilling machine.

5. The method for detecting improper operation of a radial drilling machine based on human posture recognition according to claim 1, characterized in that: The conditions under which step six is ​​not to make a judgment include: when the worker manually stops the radial drilling machine during the workflow to change the tool, adjust the workpiece position on the worktable (04), change the tool on the worktable (04), go to the toilet, or temporarily leave the work area (07) for some reason, step six stops the judgment operation.

6. The method for detecting improper operation of a radial drilling machine based on human posture recognition according to claim 1, characterized in that: The PLC module (011) is connected to the attitude detection and calculation device (08) and the rocker arm drilling rig. Specifically, the PLC module (011) receives instructions from the judgment module in the attitude detection and calculation device (08) and controls the rocker arm drilling rig.

Images