A welding production line safety protection method based on visual AI

Abstract

The present invention provides a welding production line safety protection method based on visual AI, the method includes equipment deployment, upper and lower piece opening safety protection, upper and lower piece opening safety warning, full-line safety protection and a matching mechanism of safety perspective and face perspective. Through the deployment of visual equipment in the welding scene, the welding production line is protected in cooperation with the method of full-line safety protection, the upper and lower piece opening safety warning mechanism method and the upper and lower piece opening safety protection method. The use of visual artificial intelligence technology for personnel protection makes it possible to effectively distinguish between people and objects, thereby providing more accurate protection measures. Multiple protection logics are used to control the entire line body to ensure that when there are maintenance personnel inside, the line body cannot be reset and started, providing safety protection for all personnel on the line. The introduction of an innovative visual warning mechanism can warn before danger occurs, thereby effectively reducing production safety risks.

Description

Technical Field

[0001] The present invention relates to the technical field of automated welding production, and in particular to a welding production line safety protection method based on visual AI. Background Art

[0002] In the process of automated welding production, there are safety hazards in the human-machine interaction area, and appropriate safety measures need to be taken. At present, traditional safety measures mainly use safety gratings, safety laser scanners and other equipment for protection. However, the traditional means use infrared technology for detection, which cannot effectively distinguish between people and objects, and there is a risk of false triggering. For example, the published patent 2017112237384 discloses a welding robot production line, which uses four robot-side safety gratings to protect workers, but it cannot effectively distinguish between people and objects; in addition, the traditional protection means is a two-dimensional plane protection means, which is prone to blind spots and protection dead angles. For example, when a worker is operating in the human-machine interaction area, when an external worker mistakenly operates the reset function, the traditional protection means cannot effectively protect the people inside the area. In addition, the traditional means cannot confirm whether there are maintenance personnel inside the automated production line. If someone is repairing inside the production line, the external personnel close the safety door and reset the production line, the internal staff may be in danger. Therefore, it is particularly important to solve the problem that there is a risk of false triggering in the safety protection detection of ordinary infrared technology and the traditional means cannot confirm whether there are maintenance personnel inside the automated production line. Summary of the invention

[0003] The purpose of the present invention is to provide a welding production line safety protection method based on visual AI, which adopts visual artificial intelligence technology for personnel protection, so that it can effectively distinguish between people and objects, thereby providing more accurate protection measures, and at the same time adopts multiple protection logics to control the entire line body, to ensure that when there are maintenance personnel inside, the line body cannot be reset and started, to provide safety protection for all personnel on the line, and solves the risk of false triggering of ordinary infrared technology safety protection detection and the problem that traditional means cannot confirm whether there are maintenance personnel inside the automated production line.

[0004] The present invention provides a welding production line safety protection method based on visual AI, the method comprising equipment deployment, upper and lower piece openings safety protection, upper and lower piece openings safety warning, full-line safety protection and a matching mechanism between a safety perspective and a face perspective. The welding production line is protected by deploying visual equipment in a welding scenario in combination with a full-line safety protection method, an upper and lower piece openings safety warning mechanism method and an upper and lower piece openings safety protection method.

[0005] A further improvement is that the equipment deployment includes the following: setting up overhead security cameras and facial cameras facing both sides of the entry and exit at all locations where humans interact with or enter the interior of a line.

[0006] Further improvements are: the positions where human-machine interaction or entry into the line body include upper and lower openings, safety doors, and isolation passages; the safety camera monitoring coverage area is the entire area of ​​human-machine interaction at the upper and lower openings, the safety door monitors the area of ​​the door entrance, and the isolation passage monitors the entrance area; the safety camera is deployed above the line body to monitor and cover the entire interior of the line body.

[0007] A further improvement is that the upper and lower opening safety protection includes the following steps:

[0008] Step 2.1: Use the human detection algorithm to constantly monitor whether there are people in the picture and their location. Once a person overlaps with the dangerous area, proceed to step 2.2;

[0009] Step 2.2: Once a person overlaps with the dangerous area, an alarm message is immediately sent to the PLC, and then the PLC determines whether to perform an emergency stop operation on the robot based on the current working status of the upper and lower parts ports. If yes, it goes to step 2.3, otherwise it goes to step 2.4;

[0010] Step 2.3: When the upper and lower parts port robots are in working state, make the robots stop urgently;

[0011] Step 2.4: If the upper and lower parts port robots are not in working condition, no processing will be performed.

[0012] A further improvement is that the safety warning of the upper and lower openings includes the following steps:

[0013] Step 3.1: Draw a safety warning area outside the danger zone under the monitoring area of ​​the safety camera at the upper and lower parts openings, and use the human detection algorithm to determine whether a person intersects with the warning area;

[0014] Step 3.2: If there is an intersection, a signal will be sent to the PLC, which will reduce the robot's movement speed according to the degree of danger, and the buzzer at the upper and lower parts opening will sound an alarm to remind workers to stay away from the dangerous area.

[0015] A further improvement is that the full-line safety protection includes the following steps:

[0016] Step 4.1: The high-altitude safety camera detects human figures in the internal area of ​​the safety partition. If a person is detected inside the safety partition, the PLC is notified that the reset and start-up work cannot be performed if there is someone in the area;

[0017] Step 4.2: When any person passes through the loading and unloading port, safety door, or isolation passage to enter the safety partition, the number of people protection logic is triggered and the process goes to step 4.3;

[0018] Step 4.3: Take a face-linked snapshot and cache the person's face as a record of people entering the internal area. At the same time, when someone passes through the loading and unloading port, security door, or isolation passage to leave the security partition, take a face-linked snapshot and compare the features with the previously cached internal personnel record face.

[0019] Step 4.4: After the comparison is completed, the record is deleted until all internal personnel records are cleared, then a signal is sent to the PLC, the personnel in the area are cleared, and reset is allowed;

[0020] Step 4.5: High-altitude safety camera protection logic: The partitioned people counter and the face recording protection logic are compared. If the recording status of the three does not match, the security manager is notified for manual confirmation.

[0021] A further improvement is that the matching mechanism of the security perspective and the face perspective includes the following steps:

[0022] Step 5.1: The monitoring areas of the safety view and the face view are partially overlapped. Select the same reference object in the face and safety view and draw a reference line segment: in the welding scene, it is the line between the two gratings. In the safety view, when the human target passes through the reference line, the linkage of the face view is triggered. The linkage mapping relationship is divided into two types: entry and exit.

[0023] Step 5.2: When entering, since the safety view and the face view are mirror images, the relationship between the left and right endpoints of the reference line of the humanoid target in the safety view needs to be flipped in the face view. The ratios r1 and r2 calculated from left to right between the leftmost point x_safe1 and the rightmost point x_safe2 of the humanoid target frame at the two ends of the reference line are recorded respectively. In the face view, the two ends of the reference line l1 and l2 are calculated from right to left according to the ratios r1 and r2. Only when the face target extracted from the face view is between x_face1 and x_face_2, the face is successfully matched.

[0024] Step 5.3: When leaving, since the safe perspective and the leaving face perspective are not in a mirror relationship, there is no need to perform mirror flip mapping. In the safe perspective, r1 and r2 are calculated based on the reference line coordinates and x_safe1 and x_safe2. Then x_safe1 and x_safe2 are directly calculated from left to right in the leaving face perspective. The face is successfully matched only when the face target is between x_face1 and x_face_2.

[0025] A further improvement is that in step 3.2, the movement speed of the robot is reduced from 100% to 70%.

[0026] Further improvement lies in: the number of people protection logic in step 4.2 includes the following contents: the upper and lower access ports, safety doors and isolation passages bound to the same safety partition share a partition number counter; the number of people increases when crossing the dangerous area into the interior of the line body; the number of people decreases when crossing the dangerous area from the interior of the line body to leave; when the partition number counter is not zero, a safety partition alarm signal is sent to the PLC, and the PLC will not allow the reset operation of the entire safety partition.

[0027] The beneficial effects of the present invention are: using visual artificial intelligence technology for personnel protection, it is possible to effectively distinguish between people and objects, thereby providing more accurate protection measures. Using multiple protection logics to control the entire line body, ensuring that when there are maintenance personnel inside, the line body cannot be reset and start working, providing safety protection for all personnel on the line body. Introducing an innovative visual early warning mechanism can warn before danger occurs, thereby effectively reducing safety production risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is the line deployment diagram of the present invention.

[0029] Figure 2 It is a single point deployment diagram of the present invention.

[0030] Figure 3 It is a schematic diagram of face matching of the present invention.

[0031] Figure 4 It is a schematic diagram of visual safety protection of the present invention. Implementation

[0032] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0033] like Figure 1 As shown, this embodiment provides a welding production line safety protection method based on visual AI, the method includes equipment deployment, upper and lower piece openings safety protection, upper and lower piece openings safety warning, full-line safety protection and a matching mechanism for safety perspective and face perspective. Through the deployment of visual equipment in the welding scenario, the welding production line is protected in conjunction with the full-line safety protection method, the upper and lower piece openings safety warning mechanism method and the upper and lower piece openings safety protection method.

[0034] Equipment deployment: A security camera with an overhead view and a facial camera facing both sides of the entrance and exit are installed at all locations where humans interact or can enter the interior of a line, including the loading and unloading ports, security doors, and isolation passages. The security camera monitoring coverage area is the entire area where humans and machines can interact at the loading and unloading ports, the area where the security door monitors the entrance, and the area where the isolation passage monitors the entrance. Multiple security cameras are deployed above the line to monitor the entire interior of the line.

[0035] Safety protection of the upper and lower parts ports: Under the monitoring area of ​​the safety camera of the upper and lower parts ports, the danger zone (the area where human-machine interaction may occur when workers are working) is drawn according to the safety plan of the production process. The presence of people in the picture and the position of people are constantly monitored through the human detection algorithm. Once a person overlaps with the danger zone, an alarm message will be immediately sent to the PLC, and then the PLC will determine whether the robot needs to be emergency stopped based on the current working status of the upper and lower parts ports. In addition, if there is a person in the danger zone, the robot cannot be reset, which solves the blind spot problem of traditional infrared protection methods.

[0036] Safety warning mechanism for loading and unloading ports: Under the monitoring area of ​​the safety camera at the loading and unloading ports, a safety warning area is drawn outside the danger zone, and the human detection algorithm is used to determine whether a person intersects with the warning area. If there is an intersection, a signal will be sent to the PLC, which will reduce the robot's movement speed (from 100% to 70%) according to the degree of danger, and the buzzer at the loading and unloading ports will sound an alarm to remind workers to stay away from the danger zone.

[0037] Full-line safety protection method: Based on the safety partition dimension of the line body as the smallest unit of line body protection, a safety camera and an entry and exit face camera are set up at each entrance of the safety partition where people can enter, and one or more safety cameras are set up at the height of the entire safety partition to monitor and cover the entire safety partition area. Based on the above hardware deployment solution, a dual internal safety protection logic of the line body is provided. (1) The high-altitude safety camera detects human figures in the internal area of ​​the safety partition. If a person is detected inside the safety partition, the PLC will be notified, and the work cannot be reset and started because there are people in the area. (2) When any person passes through the loading and unloading port, safety door, or isolation channel to enter the internal safety partition, the number of people protection logic will be triggered. The loading and unloading ports, safety doors, and isolation channels bound to the same safety partition share a partition people counter. When someone crosses the dangerous area drawn by the safety perspective camera at each position (entering from outside the line body to inside the line body), the partition people counter will add the corresponding number of people crossed. When someone crosses the dangerous area drawn by the safety perspective camera at each position (leaving the line body from inside the line), the partition people counter will subtract the corresponding number of people crossed. When the number of people counter in a partition is not zero, a safety partition alarm signal will be sent to the PLC, and the PLC will not allow the reset operation of the entire safety partition. (3) When any person passes through the loading and unloading port, safety door, or isolation channel to enter the safety partition, the face protection logic will be triggered, and the face linkage capture will be performed, and the person's face will be cached as a record of entering the internal personnel. At the same time, when someone passes through the loading and unloading port, safety door, or isolation channel to leave the safety partition, the face capture will be linked, and the feature will be compared with the previously cached internal personnel record face. If the comparison is completed, the record will be deleted until all internal personnel records are cleared, and a signal will be sent to the PLC, the personnel in the area will be cleared, and the reset can be started. (4) The high-altitude safety camera protection logic, the partitioned number counter and the face record protection logic are compared. If the record status of the three does not match, the security manager will be notified for manual confirmation.

[0038] Matching mechanism between security perspective and face perspective: In order to solve the problem that it is difficult to match the corresponding person's face when multiple people enter the area at the same time, a face matching method based on mapping relationship is proposed. The areas monitored by the security perspective and the face perspective have some overlap. In the face perspective, a line segment is selected as a reference (the line between the upper and lower openings, the safety door or the two fences of the isolation channel). In the security perspective, when the human target passes through the reference line, it will trigger the linkage of the face perspective. The linkage mapping relationship is divided into two types: entry and exit. (1) When entering, since the security perspective and the face perspective of entering are mirror images, the relationship between the left end point and the right end point of the reference line of the human target in the security perspective needs to be flipped in the face perspective. The ratios r1 and r2 calculated from left to right between the leftmost point x_safe1 and the rightmost point x_safe2 of the human target frame are recorded respectively. In the face perspective, the two ends of the reference line l1 and l2 are calculated from right to left according to the ratios r1 and r2 to obtain two coordinate points x_face1 and x_face_2. The face is matched successfully only when the face target extracted from the face perspective is between x_face1 and x_face_2. (2) When leaving, since the safe perspective and the leaving face perspective are not in a mirror relationship, mirror flip mapping is not required. In the safe perspective, r1 and r2 are calculated based on the reference line coordinates and x_safe1 and x_safe2. x_safe1 and x_safe2 can be directly calculated from left to right in the leaving face perspective. And the face is matched successfully only when the face target is between x_face1 and x_face_2.

Claims

1. A welding production line safety protection method based on visual AI, characterized by: The method includes equipment deployment, safety protection of loading and unloading ports, safety warning of loading and unloading ports, full-line safety protection, and a matching mechanism of safety perspective and face perspective. The welding production line is protected by deploying visual equipment in the welding production line scenario, in conjunction with a method of full-line safety protection, a method of safety warning of loading and unloading ports, and a method of safety protection of loading and unloading ports; The equipment deployment includes the following: setting up overhead security cameras and facial cameras facing both sides of the entry and exit at all locations where human-machine interactions or entry into the line are located; The positions where humans and machines interact or enter the interior of the line include the upper and lower parts openings, the safety door, and the isolation channel; the areas covered by the safety camera monitoring are: the entire area of ​​the upper and lower parts openings for human-machine interaction, the area of ​​the safety door monitoring entrance, and the area of ​​the isolation channel monitoring entrance; Security cameras are also deployed above the line to monitor the entire line. The upper and lower opening safety protection comprises the following steps: Step 2.1: Use the human detection algorithm to constantly monitor whether there are people in the picture and their location. Once a person overlaps with the dangerous area, proceed to step 2.2; Step 2.2: Once a person overlaps with the dangerous area, an alarm message is immediately sent to the PLC, and then the PLC determines whether to perform an emergency stop operation on the robot based on the current working status of the upper and lower parts ports. If yes, it goes to step 2.3, otherwise it goes to step 2.4; Step 2.3: When the upper and lower parts port robots are in working state, make the robots stop urgently; Step 2.4: If the upper and lower parts port robots are not in working state, no processing is done; The safety warning of the loading and unloading ports includes the following steps: Step 3.1: Draw a safety warning area outside the danger zone under the monitoring area of ​​the safety camera at the upper and lower parts openings, and use the human detection algorithm to determine whether a person intersects with the warning area; Step 3.2: If they intersect, a signal will be sent to the PLC, which will reduce the robot's movement speed according to the degree of danger, and the buzzer at the upper and lower parts opening will sound an alarm to remind workers to stay away from the dangerous area; The full-line safety protection includes the following steps: Step 4.1: The high-altitude safety camera detects human figures in the internal area of ​​the safety partition. If a person is detected inside the safety partition, the PLC is notified that the reset and start-up work cannot be performed if there is someone in the area; Step 4.2: When any person passes through the loading and unloading port, safety door, or isolation passage to enter the safety partition, the number of people protection logic is triggered and the process goes to step 4.3; The number protection logic in step 4.2 includes the following: the upper and lower ports, safety doors, and isolation channels bound to the same safety partition share a partition number counter; the number of people increases when crossing the dangerous area and entering the inside of the line; the number of people decreases when crossing the dangerous area from the inside of the line and leaving; when the partition number counter is not zero, a safety partition alarm signal is sent to the PLC, and the PLC will not allow the reset operation of the entire safety partition; Step 4.3: Take a face-linked snapshot and cache the person's face as a record of people entering the internal area. At the same time, when someone passes through the loading and unloading port, security door, or isolation passage to leave the security partition, take a face-linked snapshot and compare the features with the previously cached internal personnel record face. Step 4.4: After the comparison is completed, the record is deleted until all internal personnel records are cleared, and a signal is sent to the PLC: the personnel in the area are cleared and reset is allowed; Step 4.5: High-altitude safety camera protection logic: The partitioned people counter and the face recording protection logic are compared. If the recording status of the two does not match, the security manager is notified for manual confirmation; The matching mechanism of the security perspective and the face perspective includes the following steps: Step 5.1: The monitoring areas of the safety view and the face view partially overlap. Select the same reference object in the face view and the safety view, and draw a reference line segment: in the welding scene, it is the line between the two gratings. In the safety view, when the human target passes through the reference line, the linkage of the face view is triggered. The linkage mapping relationship is divided into two types: entry and exit. Step 5.2: When entering, since the safety view and the face view are mirror images, the relationship between the left and right endpoints of the reference line of the humanoid target in the safety view needs to be flipped in the face view. The ratios r1 and r2 calculated from left to right between the leftmost point x_safe1 and the rightmost point x_safe2 of the humanoid target frame at the two ends of the reference line are recorded respectively. In the face view, the two ends of the reference line l1 and l2 are calculated from right to left according to the ratios r1 and r2 to calculate the two coordinate points x_face1 and x_face_2. Only when the face target extracted from the face view is between x_face1 and x_face_2, the face is successfully matched. Step 5.3: When leaving, since the safety perspective and the leaving face perspective are not in a mirror relationship, there is no need to perform mirror flip mapping. In the safety perspective, r1 and r2 are calculated based on the reference line coordinates and x_safe1 and x_safe2. In the leaving face perspective, x_face1 and x_face_2 are directly calculated from left to right. The face is successfully matched only when the face target is between x_face1 and x_face_2.

2. A welding production line safety protection method based on visual AI as claimed in claim 1, characterized in that: In the step 3.2, the movement speed of the robot is reduced from 100% speed to 70% speed.

Images