A Safety Protection and Control System for a Robot Arm Based on Video Surveillance

Abstract

The present invention discloses a safety protection and control system for a robotic arm based on video surveillance, which relates to the technical field of robotic arm safety protection. The present invention periodically analyzes the video surveillance data of several robotic arm working areas through a video analysis module to determine the time periods when personnel are active and the corresponding active robotic arm working areas. The identification and control module, in the case where there are operators in the operation area at the current moment and a human contour is recognized in the robotic arm working area at the current moment, selects the type of robotic arm locking. For the robotic arm working areas in specific risk areas, the robotic arm is forcibly locked to ensure that the safety of personnel is not threatened. Otherwise, the robotic arm is temporarily locked and a locking time is set to avoid potential safety hazards caused by non-standard operations of operators in some areas with frequent personnel access, and at the same time, potential non-standard operations are also prevented.

Description

Technical Field

[0001] The present invention relates to the technical field of robot arm safety protection, and in particular to a robot arm safety protection and control system based on video monitoring. Background Art

[0002] In the context of the rapid development of industrial automation and robotics technology, the application scope of robotic arms is rapidly expanding. These high-precision and high-flexibility mechanical equipment play a vital role in improving production efficiency and quality. However, along with this come the potential safety hazards they may bring during operation, especially in the complex and ever-changing construction site environment;

[0003] The high mobility of personnel at the construction site, the changing environment, and the unpredictability of human operations all pose challenges to the safe operation of the robotic arm. A common practice at present is to lock the robotic arm before the operator arrives at the operating area until the operator arrives and is ready to take over control;

[0004] In order to accurately identify the operator and give other personnel on site enough time to evacuate, real-time facial recognition technology was used to monitor personnel within the robotic arm’s working area; this technology not only enables quick and accurate positioning of the operator who is about to take over the robotic arm, but also ensures that other personnel have sufficient time to safely leave the danger zone;

[0005] However, in actual operation, operators usually follow a predetermined path from entering the working area of ​​the robot arm to finally reaching the operating position. If the system performs face recognition on all personnel in the working area, this will undoubtedly lead to a significant consumption of computing resources and increase processing time, resulting in unnecessary waste of resources.

[0006] In order to solve the above problems, the present invention proposes a solution. Summary of the invention

[0007] The purpose of the present invention is to provide a robotic arm safety protection and control system based on video surveillance. In order to solve the problem in the prior art that in order to accurately identify the operator before the operator arrives at the operating area and give other personnel on site enough time to evacuate, real-time face recognition technology is used on site to monitor the personnel in the robotic arm working area without considering that the operator usually follows a predetermined path from entering the robotic arm working area to finally reaching the operating position. If the system performs face recognition on all personnel in the working area, this will undoubtedly lead to a significant consumption of computing resources, and will also increase processing time, resulting in unnecessary waste of resources.

[0008] The purpose of the present invention can be achieved through the following technical solutions:

[0009] A safety protection and control system for a robotic arm based on video surveillance, comprising:

[0010] In the current analysis period, the video analysis module extracts several sets of frame images from the video surveillance data of each robotic arm working area stored during the current analysis period.

[0011] By analyzing the shooting moment corresponding to each frame image in several sets of frame images and the robotic arm working area, a trajectory list of the operator in the current analysis period is determined. The trajectory list stores several robotic arm working areas from left to right and stores them.

[0012] After receiving the video surveillance data of each robotic arm working area at the current moment, the recognition and determination module first determines whether there is an operator in the operation area of the robotic arm:

[0013] If there is no operator in the operation area, all the robotic arm working areas stored in the trajectory list are read sequentially from left to right. According to the reading sequence, human contour recognition is performed on each frame image in the video surveillance data of each robotic arm working area read at the current moment. If a human contour is recognized, face recognition is performed on each person corresponding to the human contour to determine whether it is an operator. If a certain person is determined to be an operator, that person is locked, and the human form feature data of that person is extracted. The human form feature data includes height, type and color of the upper garment, type and color of the trousers, and gender.

[0014] After a certain person is locked, starting from the moment when that person is locked, the recognition and determination module locks that person from the video surveillance of each robotic arm working area received at each subsequent moment until that person enters the operation area. When locking that person, face recognition is not performed on the remaining persons.

[0015] Further, when there is no operator in the operation area, the robotic arm is locked.

[0016] Further, it further includes a regional surveillance module for real-time collecting media data of the robotic arm working area, and the media data includes video surveillance data of each robotic arm working area.

[0017] Advantages of the present invention:

[0018] (1) By dividing the robotic arm working area into several robotic arm working areas and arranging a high-definition network camera for each robotic arm working area, the present invention obtains the video content of the corresponding robotic arm working area, so as to ensure full-round and non-blind-zone monitoring of the robotic arm working area;

[0019] (2) The present invention determines the active time period of personnel and the corresponding active working area of the robotic arm by setting up a video analysis module to periodically analyze the video monitoring data of several working areas of the robotic arm, and based on the camera moments corresponding to all frame images containing recognizable human silhouettes in it and the number of human silhouettes in the frame images. The recognition control module is set to, when there is an operator in the operation area at the current moment and it is recognized that a human silhouette is identified in the working area of the robotic arm at the current moment, select the robotic arm locking type according to the calibration type corresponding to the current moment and the working area of the robotic arm corresponding to the frame image in which the human silhouette is identified. For the working area of the robotic arm in a specific risk area, the robotic arm is forcibly locked to ensure that the safety of personnel is not threatened. On the contrary, the robotic arm is temporarily locked and the locking time is set to avoid potential safety hazards caused by non-standard operations of operators in some areas with frequent personnel access, and also prevent potential non-standard operations;

[0020] (3) The present invention analyzes the travel path of the operator from entering the working area of the robotic arm to finally reaching the operation area through the video analysis module to determine the travel trajectory list. When there is no operator in the operation area at the current moment, the recognition control module preferentially performs face recognition on the personnel in several working areas included in the travel trajectory list and locks the personnel. In this way, excessive waste of computing resources and time resources is avoided. And when the personnel are locked, an alarm is issued to give the remaining personnel time to evacuate, avoiding the situation where the operator arrives at the operation area and waits for other personnel to evacuate. Description of the Drawings

[0021] The present invention will be further described below with reference to the accompanying drawings.

[0022] Figure 1 is the system block diagram of the present invention. Detailed Embodiments

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] As Figure 1 shown, a robotic arm safety protection and control system based on video monitoring includes a regional monitoring module, a recognition control module, and a video analysis module;

[0025] The area monitoring module is used to perform video monitoring on the working area of the robotic arm. The area monitoring module includes several area monitoring units. One area monitoring unit corresponds to one working partition of the robotic arm. The area monitoring unit collects video monitoring data of the corresponding working partition of the robotic arm in real time. It should be noted that the real time here refers to each moment;

[0026] In this embodiment, in order to monitor the working area of the robotic arm comprehensively and without dead angles, the management personnel divide the working area of the robotic arm into several working partitions of the robotic arm, and deploy a high-definition network camera for each working partition of the robotic arm to obtain the video content of the corresponding working partition of the robotic arm;

[0027] In this embodiment, the specifications of the high-definition network cameras deployed for each working partition of the robotic arm are the same;

[0028] The area monitoring module generates real-time media data of the robotic arm working area based on the video monitoring data of all working partitions of the robotic arm collected in real time, and transmits the real-time media data of the robotic arm working area to the recognition control module and the video analysis module respectively;

[0029] The recognition control module is used to recognize the media data of the robotic arm working area and perform safety control on the robotic arm based on the recognition result. The facial image data of the operator operating the robotic arm, the risk analysis table, and the list of the operator's travel trajectories are pre-stored in the recognition control module;

[0030] After receiving the media data of the robotic arm working area at the current moment transmitted, the recognition control module first determines whether there is an operator in the operation area of the robotic arm;

[0031] If there is an operator in the operation area of the robotic arm at the current moment, the robotic arm is subjected to safety control according to the preset first control rule. The preset first control rule is as follows:

[0032] S21: Obtain the type corresponding to the time period to which the current moment belongs from the risk analysis table stored in the recognition control module. If the time period to which the current moment belongs is a risk time period, obtain all the working partitions of the robotic arm that are marked as specific risk areas corresponding to the time period to which the current moment belongs from the risk analysis table;

[0033] S22: Extract the video monitoring data of all the working partitions of the robotic arm that are marked as specific risk areas corresponding to the time period to which the current moment belongs from the received media data of the robotic arm working area at the current moment, and perform recognition on it:

[0034] S221: If a human body contour is recognized in any of the video surveillance data, a 10S audible alarm will be immediately issued, and the operation permission of the robotic arm will be forcibly locked. The forced locking duration is set to P2. This method can avoid potential hazards and mechanical damage caused by repeated startups. P2 is the threshold value of the forced locking duration for the preset specific risk area. It should be noted that during the forced locking of the robotic arm, even authorized personnel cannot forcibly unlock it, and human body contour recognition will not be performed on all video surveillance data of the robotic arm working area received during the locking period;

[0035] S222: If no human body contour is recognized in any of the video surveillance data, human body contour recognition will be performed on all remaining unextracted video surveillance data of the robotic arm working area. If a human body contour is recognized, a 10S audible alarm will be immediately issued, and the operation permission of the robotic arm will be temporarily locked. The temporary locking duration is set to P3. This method can avoid potential hazards and mechanical damage caused by repeated startups. P3 is the threshold value of the temporary locking duration for the preset specific safety area. It should be noted that during the temporary locking of the robotic arm, authorized personnel can forcibly unlock it, and human body contour recognition will not be performed on all video surveillance data of the robotic arm working area received during the locking period;

[0036] If there is no operator in the operation area of the robotic arm at the current moment, it means that the robotic arm is not working at the current moment. At this time, the robotic arm is locked and requires the operator to unlock it. The robotic arm is safely controlled according to the preset second control rule. The preset second control rule is as follows:

[0037] S31: Based on the list of the operator's movement trajectories stored in the recognition control module, in ascending order of the element subscripts therein, all robotic arm working areas included therein are sequentially marked as T1, T2,..., Tt, where t≥1;

[0038] S32: First, extract the video surveillance data of the robotic arm working area T1 at the current moment from the media data of the robotic arm working area received at the current moment, and perform human body contour recognition on each frame of the video surveillance data of the robotic arm working area T1 at the current moment according to the preset determination rule. The preset determination rule is as follows:

[0039] S321: Extract the facial feature data of the operator from the facial image data of the operator stored in the recognition control module;

[0040] S322: According to the order of the camera shooting moments, first perform human body contour recognition on the first frame of the video surveillance data of the robotic arm working area T1 at the current moment:

[0041] If a human body contour is recognized, facial feature data of all personnel within the first frame image is extracted, and the similarity is calculated in sequence with the facial feature data of the extracted operator. If the similarity between the facial feature data of a certain person and the facial feature data of the operator is greater than or equal to P5, then it is determined that this person is the operator and this person is locked. On the contrary, if the similarity between the facial feature data of each person and the facial feature data of the operator is calculated to be less than P5, it is determined that there is no operator in the first frame image, where P5 is a preset facial recognition comparison threshold;

[0042] S323: On the premise that it is determined in S322 that there is no operator in the first frame image, in the order of the shooting time, the human body contour recognition is performed again on the second frame image included in the video surveillance data of the robotic arm working area T1 at the current moment according to the steps of S322 until a certain person is locked, or all the frame images included in the video surveillance data of the robotic arm working area T1 at the current moment are recognized and no person is locked yet;

[0043] S33: For the situation in S323 where all the frame images included in the video surveillance data of the robotic arm working area T1 at the current moment are recognized and no person is locked yet, in the order of T1, T2,..., Tt, according to step S32, the video surveillance data of the robotic arm working areas T2, T3,..., Tt at the current moment is extracted from the media data of the robotic arm working area at the current moment in sequence, and the human body contour recognition is performed on each frame image included therein until a certain person is locked;

[0044] If after the human body contour recognition of each frame image included in the video surveillance data of the robotic arm working area Tt at the current moment is completed, no person is locked yet, it is determined that the operator does not appear at the current moment, and no processing is performed;

[0045] After a certain person is locked, first a sound alarm lasting for 10S is issued to notify other people to quickly evacuate the robotic arm working area, and then the human form feature data of this person is extracted from the video surveillance data to which the frame image determining this person as the operator belongs. The human form feature data includes height, the type and color of the upper garment, the type and color of the trousers, and gender;

[0046] And this person is locked in the subsequent received media data of the robotic arm working area based on the human form feature data of this person until this person enters the operation area. When locking this person, facial recognition is not performed on the remaining people, and in this way, unnecessary facial recognition can be avoided and computing resources can be saved;

[0047] A video analysis module, which is used to store and analyze media data in the working area of the robotic arm. After receiving the media data in the working area of the robotic arm transmitted in real time, the video analysis module stores it;

[0048] The video analysis module analyzes the media data in the working area of the robotic arm stored during each analysis period at intervals of one analysis period. In this embodiment, the interval duration of the analysis period is P1, and the P1 is the threshold value of the interval duration of the preset analysis period;

[0049] In the current analysis period, the video analysis module analyzes the media data in the working area of the robotic arm stored during the current analysis period according to the preset first analysis rule and second analysis rule. The preset first analysis rule is as follows:

[0050] S11: Mark several robotic arm working partitions divided from the working area of the robotic arm, and mark them as A1, A2,..., Aa respectively, where a≥1, and a refers to the total number of robotic arm working partitions;

[0051] S12: Arrange all the media data in the working area of the robotic arm stored in the video analysis module during the current analysis period in the order of their storage, and mark all the media data in the working area of the robotic arm stored in the video analysis module during the current analysis period as B1, B2,..., Bb in turn, where b≥1;

[0052] S13: Divide the interval duration of the current analysis period into several monitoring periods, and mark them as Y1, Y2,..., Yy respectively, where y≥1; then perform monitoring segment division, divide the time of one monitoring period into c equal-duration monitoring segments, and mark the c monitoring segments of one monitoring period as C1, C2,..., Cc in turn, where c≥1; In this embodiment, one monitoring period is 1 day and one monitoring segment is 1 hour;

[0053] S14: Calculate and obtain the personnel activity index J1 of the robotic arm working partition A1 according to the preset calculation rule, specifically as follows:

[0054] S141: Sequentially obtain the video surveillance data of the robotic arm working partition A1 from the media data B1, B2,..., Bb in the working area of the robotic arm, and mark them as D1, D2,..., Db correspondingly;

[0055] S142: Select the monitoring period Y1 as the screening period, extract all the video surveillance data with the acquisition time within the monitoring segment C1 of the screening period from the video surveillance data D1, D2,..., Db, and arrange the extracted all video surveillance data as Z1, Z2,..., Zz in the order of D1, D2,..., Db, where 1≤z≤b;

[0056] S142: Perform human contour recognition on each frame of video surveillance data Z1, Z2, ..., Zz in sequence to obtain all the images in which human contours can be recognized;

[0057] In the order of the distance from the shooting time of each frame of image to the current time, from far to near, sequentially label all the images in which human contours can be recognized as F1, F2, ..., Ff, f≥1, where the shooting time corresponding to each frame of image refers to the time when each frame of image is taken;

[0058] S143: Sequentially obtain the interval durations between the shooting times corresponding to images F1 and F2, F2 and F3, ..., Ff-1 and Ff, and label them as G1, G2, ..., Gg, g = 1, 2, ..., f - 1;

[0059] S144: Use the formula to calculate and obtain the discrete value H1 of the interval durations G1, G2, ..., Gg, and compare the size of H1 and H, where G is the average value of Gh, and H is the preset discrete threshold of the interval duration;

[0060] If H1≥H, then delete the corresponding Gh in the order of |Gh - G| from large to small, calculate the discrete value H1 of the remaining Gh, compare the size of H1 and H again until H1 < H, obtain the average value of the remaining Gh participating in the calculation of the discrete value H1 at this time, and label it as the personnel frequency I1 in the working area A1 of the robotic arm under the screening period monitoring section C1;

[0061] S145: Use the formula to calculate and obtain the personnel activity index J1 in the working area A1 of the robotic arm under the screening period monitoring section C1:

[0062] S15: Calculate and obtain the personnel activity indices J1, J2, ..., Ja in the working areas A1, A2, ..., Aa of the robotic arm under the screening period monitoring section C1 in sequence according to S14, and use the summation and averaging formula to calculate and obtain their average value, and label the calculated average value as the average personnel activity index K1 of the screening period monitoring section C1;

[0063] S16: Sequentially select the monitoring periods Y1, Y2, ..., Yy as the screening periods, and calculate and obtain the average personnel activity indices K1, K2, ..., Ky of the monitoring periods Y1, Y2, ..., Yy monitoring section C1 in sequence according to S15;

[0064] Use the formula to calculate and obtain the discrete value M1 of the average personnel activity indices K1, K2, ..., Ky, and compare the size of M1 and M, where K is the average value of Km, and M is the preset discrete threshold of the average personnel activity index;

[0065] If M1 ≥ M, then delete the corresponding Km in descending order of |Km - K| and calculate the discrete value M1 of the remaining Km. Compare M1 with M again until M1 < M. Obtain the average value of the remaining Km participating in the calculation of the discrete value M1 at this time, and calibrate it as the activity evaluation index L1 of the monitoring section C1.

[0066] S17: Compare L1 with L. If L1 ≥ L, it is determined that the personnel access in each robotic arm working area within the monitoring section C1 is active, and the monitoring section C1 is calibrated as a risk time period. Otherwise, it is determined that the personnel access in each robotic arm working area within the monitoring section C1 is scarce, and the monitoring section C1 is calibrated as a safe time period.

[0067] S18: According to S11 to S17, determine the personnel access situation in each robotic arm working area within the monitoring sections C1, C2,..., Cc in sequence, and determine whether the corresponding monitoring section is a risk time period or a safe time period.

[0068] S19: According to S14, calculate and obtain the personnel activity indices N1, N2, N3,..., Ny of the robotic arm working area A1 under the monitoring section C1 of the monitoring cycle Y1, under the monitoring section C1 of the monitoring cycle Y2,..., under the monitoring section C1 of the monitoring cycle Yy in sequence. Among them, the personnel activity index of the robotic arm working area A1 under the monitoring section C1 of the monitoring cycle Y1 has been marked as J1 in S145. Here, for the convenience of subsequent step operations, it is re-marked and re-marked as N1.

[0069] S110: Use the formula Calculate and obtain the discrete value O1 of the personnel activity indices N1, N2, N3,..., Ny, and compare O1 with O, where N is the average value of Nn, and O is the preset discrete threshold of the personnel activity index.

[0070] If O1 ≥ O, then delete the corresponding Nn in descending order of |Nn - N| and calculate the discrete value O1 of the remaining Nn. Compare O1 with O again until O1 < O. Obtain the average value of the remaining Nn participating in the calculation of the discrete value O1 at this time, and calibrate it as the activity index Q1 of the robotic arm working area A1 under the monitoring section C1.

[0071] S111: Compare Q1 with Q. If Q1 ≥ Q, it is determined that the personnel access in the robotic arm working area A1 under the monitoring section C1 is active, and the robotic arm working area A1 under the monitoring section C1 is calibrated as a specific risk area. Otherwise, it is determined that the personnel access in the robotic arm working area A1 under the monitoring section C1 is scarce, and the robotic arm working area A1 under the monitoring section C1 is calibrated as a specific safe area.

[0072] S112: Calculate and obtain the activity indicators of the robotic arm working area A1 under the monitoring sections C1, C2, ..., Cc in sequence according to S19 to S111, and based on the activity indicators of the robotic arm working area A1 under the monitoring sections C1, C2, ..., Cc, select whether to label the robotic arm working area A1 under the corresponding monitoring section as a specific risk area or a specific safety area;

[0073] S113: According to S112, calculate the activity indicators of the robotic arm working areas A1, A2, ..., Aa under the monitoring sections C1, C2, ..., Cc in sequence, and based on the activity indicators of each robotic arm working area under each monitoring section, select whether to label each robotic arm working area under each monitoring section as a specific risk area or a specific safety area;

[0074] The video analysis module generates a risk analysis table for the current analysis period based on all the robotic arm working areas labeled as specific risk areas and specific safety areas under each monitoring section, and all the time periods labeled as risk time periods and safety time periods, and transmits it to the recognition control module for update and storage;

[0075] The risk analysis table for the current analysis period includes a time period field, a risk area field, a safety area field, and a type field;

[0076] The time period field stores the monitoring sections C1, C2, ..., Cc respectively; the type field stores the types labeled for the corresponding monitoring sections, where the types include risk time periods and safety time periods;

[0077] The risk area field stores all the robotic arm working areas labeled as specific risk areas under the corresponding monitoring section; the safety area field stores all the robotic arm working areas labeled as specific safety areas under the corresponding monitoring section;

[0078] The video analysis module pre-stores the facial image data of the operator of the current operating robotic arm,

[0079] The second analysis step is as follows:

[0080] S41: Extract a set of frame images for several track recognition periods from all the robotic arm working area media data stored in the video analysis module during the current analysis period;

[0081] A set of frame images for one track recognition period contains several frame images that can recognize the human contour of the operator. All the frame images in a set of frame images for one track recognition period are arranged from left to right in the order of the shooting time;

[0082] The time interval difference between two adjacent frame images arranged in a set of frame images for one track recognition period is 1 s;

[0083] S42: Obtain the working partition of the robotic arm corresponding to the inner frame images from all the frame images included in the frame image set of one orbit recognition period, and sequentially label the corresponding working partitions of the robotic arm as R1, R2, ..., Rr in the order of the shooting times of the frame images, where r ≥ 1;

[0084] Among them, for the working partitions of the robotic arm corresponding to multiple frame images that are the same, only label this working partition of the robotic arm once, and when labeling, select the earliest shooting time as the calibration order of the labeling sequence;

[0085] The working partition of the robotic arm corresponding to the frame image refers to the working partition of the robotic arm corresponding to the high-definition network camera that shoots the frame image;

[0086] S43: Sequentially add the working partitions of the robotic arm R1, R2, ..., Rr into an empty list to obtain the trajectory list of the operator in this orbit recognition period;

[0087] S44: Calculate and obtain the trajectory lists of the operators in all orbit recognition periods in sequence according to S41 to S43, and select the trajectory list with the most occurrences from them, and label this trajectory list as the travel trajectory list of the operator in the current analysis period, where the trajectory list with the most occurrences refers to the trajectory list that is exactly the same as this trajectory list in the trajectory lists of the operators in all orbit recognition periods is the most;

[0088] The video analysis module transmits the travel trajectory list of the operator in the current analysis period to the recognition control module, and the recognition control module updates and stores it after receiving the travel trajectory list of the operator in the current analysis period;

[0089] In the description of the specification, the descriptions with reference to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0090] The above content is only an example and explanation of the present invention. Those skilled in the art of this technology make various modifications or supplements to the described specific embodiments or use similar ways to replace them, as long as they do not deviate from the invention or exceed the scope defined by the claims of the present invention, they should all fall within the protection scope of the present invention.

[0091] The above has described in detail an embodiment of the present invention, but the above content is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of application of the present invention should still fall within the scope covered by the patent of the present invention.

Claims

1. A robotic arm safety protection and control system based on video surveillance, characterized in that, Including: In the current analysis period, the video analysis module extracts several sets of frame images from the video monitoring data of each robotic arm working partition stored during the current analysis period; According to the shooting time corresponding to each frame image in the several sets of frame images and the robotic arm working partition, determine the trajectory list of the operator in the current analysis period. The trajectory list stores several robotic arm working partitions from left to right; After receiving the video monitoring data of each robotic arm working partition at the current moment, the recognition and determination module first determines whether there is an operator in the operation area of the robotic arm: If it is determined that there is no operator in the operation area, read all the robotic arm working partitions stored in the trajectory list from left to right, and perform human contour recognition on each frame image in the video monitoring data of each robotic arm working partition read at the current moment in the order of reading. If a human contour is recognized, perform face recognition on each person corresponding to the human contour to determine whether it is an operator. If a certain person is determined to be an operator, lock that person and extract the human form feature data of that person; After a certain person is locked, starting from the moment when that person is locked, the recognition and determination module locks that person from the video monitoring of each robotic arm working partition received at each subsequent moment until the person enters the operation area.

2. The safety protection and control system for a robotic arm based on video surveillance according to claim 1, wherein When there is no operator in the operation area, the robotic arm is locked.

3. A robotic arm safety protection and control system based on video surveillance according to claim 1, characterized in that, It also includes a regional monitoring module for real-time collecting media data of the robotic arm working area, and the media data contains video monitoring data of each robotic arm working partition.

4. A safety protection and control system for a robotic arm based on video surveillance according to claim 1, characterized in that, A set of frame images contains several frame images that can recognize the human contour of the operator. The difference in shooting time between two adjacent frame images in a set of frame images is 1s, and all the frame images in a set of frame images are arranged from left to right in the order of the corresponding shooting time.

5. A robotic arm safety protection and control system based on video surveillance according to claim 1, characterized in that, The video analysis module is also used to divide several monitoring segments of equal duration according to the interval duration of the current analysis period; Then extract all the frame images that can recognize the human contour from the video monitoring data of each robotic arm working partition stored during the current analysis period, and calibrate the types of the monitoring segments and the robotic arm working partitions according to the shooting time corresponding to each extracted frame image and the total number of human contours that can be recognized in each frame image, to obtain the risk analysis table of the current analysis period; The calibration types of the monitoring segments are two types: risk time period and safe time period, and the calibration types of the robotic arm working partitions are two types: specific risk area and specific safe area.

6. The safety protection and control system of a robotic arm based on video surveillance according to claim 5, characterized in that, When the recognition and determination module determines that there is an operator in the operation area of the robotic arm after receiving the video monitoring data of each robotic arm working partition at the current moment, the robotic arm is safely controlled according to the preset first control rule. The preset first control rule is as follows: S21: Obtain the type corresponding to the time period to which the current moment belongs from the risk analysis table stored in the recognition control module. If the type corresponding to the time period to which the current moment belongs is a risk time period, obtain all the robotic arm working partitions marked as specific risk areas corresponding to the time period to which the current moment belongs from the said risk analysis table; S22: Extract the video surveillance data of all the robotic arm working partitions marked as specific risk areas corresponding to the time period to which the current moment belongs from the media data of the robotic arm working area received at the current moment, and identify it: S221: If a human contour is identified in any one of the video surveillance data, immediately issue a sound alarm that lasts for 10S, and forcibly lock the robotic arm. Set the forced lock duration to P2, where P2 is the forced lock duration threshold for the preset specific risk area. During the forced lock of the robotic arm, even authorized personnel cannot forcibly unlock it, and no human contour recognition is performed on the video surveillance data of all the robotic arm working partitions received during the lock period; S222: If no human contour is identified in any one of the video surveillance data, perform human contour recognition on the video surveillance data of all the remaining unextracted robotic arm working partitions. If a human contour is identified, immediately issue a sound alarm that lasts for 10S, and temporarily lock the robotic arm. Set the temporary lock duration to P3, where P3 is the temporary lock duration threshold for the preset specific safety area. During the temporary lock of the robotic arm, authorized personnel can perform forced unlocking, and no human contour recognition is performed on the video surveillance data of all the robotic arm working partitions received during the lock period.

7. A robotic arm safety protection and control system based on video surveillance according to claim 1, characterized in that When the recognition and determination module determines that there is no operator in the operation area of the robotic arm after receiving the video surveillance data of each robotic arm working partition at the current moment, the steps to lock the said person until they enter the operation area are as follows: S31: Based on the list of the operator's travel trajectories stored in the recognition control module, in the order of the element subscripts from smallest to largest inside it, sequentially mark all the robotic arm working partitions contained in it as T1, T2,..., Tt, where t≥1; S32: First, extract the video surveillance data of the robotic arm working partition T1 at the current moment from the media data of the robotic arm working area received at the current moment, and perform human contour recognition on each frame of the video surveillance data of the robotic arm working partition T1 at the current moment according to the preset determination rules. The preset determination rules are as follows: S321: Extract the facial feature data of the operator from the facial image data of the operator stored in the recognition control module; S322: In the order of the shooting times, first perform human contour recognition on the first frame of the video surveillance data contained in the robotic arm working partition T1 at the current moment; If a human silhouette is recognized, facial feature data of all persons within it are extracted from the first frame image, and similarity calculations are performed in sequence with the facial feature data of the extracted operator. If the similarity between the facial feature data of a certain person and the facial feature data of the operator is greater than or equal to P5, then this person is determined to be the operator and is locked. Conversely, if the similarity between the facial feature data of each person and the facial feature data of the operator is calculated to be less than P5, it is determined that there is no operator in the first frame image. P5 is a preset facial recognition comparison threshold; S323: On the premise that it is determined in S322 that there is no operator in the first frame image, in the order of the shooting times, the second frame image included in the video surveillance data of the robotic arm working area T1 at the current moment is again subjected to human silhouette recognition according to the steps of S322 until a certain person is locked, or all the frame images included in the video surveillance data of the robotic arm working area T1 at the current moment have been recognized and still no person has been locked; S33: For the situation in S323 where all the frame images included in the video surveillance data of the robotic arm working area T1 at the current moment have been recognized and still no person has been locked, in the order of T1, T2,..., Tt, according to step S32, the video surveillance data of the robotic arm working areas T2, T3,..., Tt at the current moment are sequentially extracted from the media data of the robotic arm working area at the current moment, and human silhouette recognition is performed on each frame image included therein until a certain person is locked; If no person has been locked after the human silhouette recognition of each frame image included in the video surveillance data of the robotic arm working area Tt at the current moment is completed, no processing is performed.

Images