A method for abnormal behavior identification and risk control in human-machine collaboration

Abstract

The invention discloses a method for abnormal behavior recognition and risk control in human-machine collaboration, in the field of robot technology, comprising the following steps: S10: marking key parts of human-machine interaction and creating a special data set for human-machine collaboration; S20: training data with a YOLOv11‑Pose model and constructing a real-time detection model after optimization; S30: tracking key parts of humans and robots in a collaborative area using a kernel correlation filter of a structured SVM and recording motion trajectories; S40: establishing a human-machine collaborative motion analysis model based on the motion trajectory; S50: constructing a human-machine collaborative safety risk prediction model based on a combination of multiple factors; S60: classifying the risks of collaborative scenarios and taking corresponding measures based on risk prediction; the invention has the following beneficial effects: achieving real-time monitoring of collaborative scenarios between robots and operators, preventing accidents caused by close contact, effectively improving safety in a human-machine collaborative environment, reducing the probability of accidents, and ensuring the continuity and effectiveness of operations.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and more specifically, to a method for abnormal behavior recognition and risk management in human-machine collaboration. Background Art

[0002] In the wave of intelligent manufacturing, human-machine collaboration is gradually becoming a core trend of future industrial development as a key path to improve production efficiency, enhance operational flexibility and innovation capabilities. This concept aims to achieve efficient collaboration and intelligent upgrading of production processes by seamlessly integrating the advantages of human intelligence and robot precision execution. However, in a complex and changeable unstructured dynamic production environment, ensuring efficient cooperation between humans and machines while ensuring safety has become a major challenge that needs to be solved urgently. Especially in highly flexible and task-changing operating scenarios, traditional sensor technology and existing visual recognition methods face many limitations and are difficult to meet the high standards of human-machine collaboration.

[0003] Traditionally, in order to ensure the safety of human-machine collaboration, the industry has widely used ultrasonic, photoelectric and other sensors for physical obstacle detection and distance measurement. Although these technologies can effectively avoid direct safety risks such as collisions to a certain extent, their detection range is limited and it is difficult to fully capture the dynamic changes in the working environment, especially the lack of understanding of human behavioral intentions, subtle movements and complex scenes. For example, ultrasonic sensors may be interfered by environmental noise, while photoelectric sensors are easily affected by changes in light conditions, resulting in false alarms or missed alarms, which limits their widespread application in complex human-machine collaboration environments.

[0004] With the rapid development of computer vision technology, visual recognition is regarded as an important breakthrough in improving the level of intelligence in human-machine collaboration. However, current vision-based abnormal behavior recognition and risk control technologies still face multiple challenges. First, the processing of high-dimensional image data requires huge computing resources, which places extremely high demands on hardware performance, especially the reliance on high-performance graphics processing units (GPUs), which greatly increases costs. Secondly, although algorithms such as deep learning have made significant progress in the field of visual recognition, in actual applications, facing complex and changing working environments, different lighting conditions, human posture diversity, and fast-moving objects, the accuracy of recognition and prediction is still difficult to meet the high reliability requirements of industrial-grade applications. Misidentification or delayed recognition may lead to safety hazards and affect the smoothness and safety of human-machine collaboration. Summary of the invention

[0005] In order to overcome the shortcomings of the existing technology, a method for abnormal behavior identification and risk control in human-machine collaboration is proposed. The method realizes real-time monitoring of the collaborative scenarios of robots and operators to prevent accidents caused by close contact. The risk calculation model based on various factors of key parts can effectively improve the safety in the human-machine collaborative environment, reduce the probability of accidents, and ensure the continuity and effectiveness of operations.

[0006] The technical solution adopted by the present invention to solve its technical problem is: a method for abnormal behavior identification and risk control in human-machine collaboration, the improvement of which is that the method includes the following steps:

[0007] S10: Based on the interaction characteristics between operators and robots, key parts are marked to build a data set suitable for human-robot collaboration scenarios;

[0008] S20: The YOLOv11-Pose model is used to train the data set, and the model is quantized and pruned to build a real-time detection model for real-time detection of key parts of operators and robots;

[0009] S30: In the limited area of ​​human-robot collaboration, the kernel correlation filter of structured SVM is used to track the key parts of the operator and the robot in real time to obtain their spatiotemporal motion trajectories;

[0010] S40: According to the spatiotemporal trajectories of the operator and the key parts of the robot, a human-machine collaborative motion element calculation model based on spatiotemporal analysis is established;

[0011] S50: Based on the elements of collaborative motion, a human-machine collaborative safety risk prediction model is constructed with multi-factor coupling calculation;

[0012] S60: Based on the risk coefficient calculated by the risk prediction model, the risk of human-machine collaboration scenarios is divided and corresponding control measures are implemented according to the risk level;

[0013] In step S40, the human-machine collaborative motion elements include the minimum relative distance, relative speed, motion direction, and the operator's line of sight direction angle; wherein:

[0014] Minimum relative distance: the minimum spatial distance between a person and a robot;

[0015] Relative speed: the speed difference between the human and the robot;

[0016] Movement direction: Differences in movement direction between humans and robots;

[0017] The worker's sight angle: the worker's attention to the robot;

[0018] The specific steps of step S40 are:

[0019] S401: extracting motion elements such as the minimum relative distance, relative speed, relative motion direction and sight angle of key parts;

[0020] S402: Calculate the sight direction of the operator and the sight direction angle between the operator and the key part of the robot according to the left and right eye positions or shoulder positions of the operator and the key part of the robot;

[0021] S403: Calculate the relative speed and direction between the robot and the key parts of the operator, and dynamically analyze the human-machine interaction characteristics according to the two-frame time interval;

[0022] The specific steps of step S50 are:

[0023] S501: A multi-factor risk assessment model based on the minimum relative distance, relative speed, relative speed direction and line of sight angle;

[0024] S502: introducing a weight factor of the sight direction angle into the model, and dynamically adjusting the risk assessment result in combination with the visual range of the operator, wherein the visual range includes a first visual area, a second visual area, and a blind area;

[0025] S503: Quantify the relative speed direction, and increase the risk factor when the operator and the robot are moving in opposite directions, and reduce the risk factor when they are in the same direction;

[0026] S504: Calculate the risk level comprehensively through the risk calculation formula and output the risk coefficient in real time.

[0027] Furthermore, the specific steps of step S10 are:

[0028] S101: According to the needs of human-machine collaboration scenarios, the robot's base, waist joint and execution end are the key detection parts of the robot;

[0029] S102: Construct a human-machine collaboration scenario dataset with the operator's left eye, right eye, mouth, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles as key parts;

[0030] S103: By labeling the above key parts, a high-quality labeling data set of the operator and the robot under different environments, postures and dynamic conditions is obtained.

[0031] Furthermore, the specific steps of step S20 are:

[0032] S201: According to the key parts detection requirements of the operator and the robot, the number of output channels is increased to detect the key parts, and the size and proportion of the anchor frame are adjusted to adapt to the size and shape of the key parts;

[0033] S202: Add a spatial pyramid pooling layer before the output layer to improve the detection performance of key parts of different scales. Use the feature pyramid network to perform multi-level feature fusion to enhance the recognition ability of smaller key parts in complex scenes.

[0034] S203: Add an integrated convolution module attention mechanism to enhance the real-time detection model’s ability to focus on key parts:

[0035] S204: adjusting the loss function according to the requirements of the key part detection task, and improving the detection accuracy by increasing the weight of the key point positioning loss;

[0036] S205: Set the learning rate, batch size, and training cycle, and perform training on the GPU, monitoring the loss value and detection accuracy during the training process;

[0037] S206: Using a static quantization method to perform low-precision conversion on the trained weights, and implementing dynamic quantization in the inference stage;

[0038] S207: Remove inefficient channels through structured pruning, and then randomly remove weights through unstructured pruning;

[0039] S208: Test the real-time detection model after quantization and pruning, and deploy the chip; evaluate its accuracy, recall rate, and inference speed to ensure that the real-time detection model has real-time inference capabilities while meeting the detection accuracy requirements.

[0040] Furthermore, in step S50, the human-machine collaboration safety risk prediction model is specifically as follows:

[0041]

[0042] Among them, d represents the minimum relative distance, v represents the relative speed, θ represents the direction of movement, and α represents the direction of the staff's line of sight; ρ(αi) represents the quantized processing function value of the staff's line of sight direction α at the i-th frame, g(θi) represents the quantized processing function value of the movement direction θ at the i-th frame, vi represents the relative speed between the person and the robot at the i-th frame, and di represents the minimum relative distance between the person and the robot at the i-th frame.

[0043] Furthermore, the specific steps of step S60 are:

[0044] S601: The system continuously monitors in safe and low-risk states;

[0045] S602: In a general dangerous state, an alarm signal is issued to prompt the operator or robot to adjust its behavior;

[0046] S603: In a severe danger state, the system issues an emergency alarm and triggers the robot to stop or adjust the path to avoid safety accidents.

[0047] Furthermore, the risk levels can be subdivided into:

[0048] Severe danger: The risk factor is in the highest range of 0.8-1.0, indicating extremely high collision risk and serious consequences;

[0049] General danger: The risk factor is in the medium to high range of 0.5-0.8, indicating a clear risk of collision, which may lead to medium consequences;

[0050] Lower risk: The risk factor is in the medium range of 0.2-0.5, indicating potential risks, but the consequences are limited;

[0051] Safety: The risk factor is in the lowest range of 0.0-0.2, which means that the risk is negligible and the operation is safe.

[0052] The beneficial effects of the present invention are: the method realizes real-time monitoring of the collaborative scene between the robot and the operator, prevents accidents caused by close contact, and the risk calculation model based on various factors of key parts can effectively improve the safety in the human-machine collaborative environment, reduce the probability of accidents, and ensure the continuity and effectiveness of operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of a method for abnormal behavior identification and risk management in human-machine collaboration of the present invention;

[0054] Figure 2 It is a schematic diagram of an implementation method of abnormal behavior identification and risk management in human-machine collaboration of the present invention;

[0055] Figure 3 A framework diagram of a human-machine collaboration safety risk assessment model for a method of abnormal behavior identification and risk management in human-machine collaboration of the present invention;

[0056] Figure 4 This is a human-machine position reference diagram of a method for abnormal behavior identification and risk management in human-machine collaboration of the present invention. DETAILED DESCRIPTION

[0057] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0058] The following will clearly and completely describe the concept, specific structure and technical effects of the present invention in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present invention. In addition, all the connection / connection relationships involved in the patent do not refer to the direct connection of components, but refer to the formation of a better connection structure by adding or reducing connection accessories according to the specific implementation situation. The various technical features in the invention can be combined interchangeably without conflicting with each other.

[0059] Human-robot collaboration is an inevitable trend in the development of intelligent manufacturing. However, it is difficult to ensure efficient and safe collaboration between humans and robots in a non-structural and dynamic production environment, and the detection function range of traditional ultrasonic, photoelectric and other sensors is limited. In addition, the recognition of abnormal behavior in human-robot collaboration is not just about capturing simple actions, but more importantly, it is about understanding people's behavioral intentions, predicting potential risks, and making timely and accurate responses accordingly. This requires the system to have a high degree of situational awareness, a rapid decision-making mechanism, and natural interaction with people, which are difficult to fully achieve under the current technical framework.

[0060] See also Figure 1-Figure 3 As shown, the present invention provides a method for abnormal behavior identification and risk management in human-machine collaboration, the method comprising the following steps:

[0061] S10: Based on the interaction characteristics between operators and robots, key parts are marked to build a data set suitable for human-robot collaboration scenarios;

[0062] S20: The YOLOv11-Pose model is used to train the data set, and the model is quantized and pruned to build a real-time detection model for real-time detection of key parts of operators and robots;

[0063] S30: In the limited area of ​​human-robot collaboration, the kernel correlation filter of structured SVM is used to track the key parts of the operator and the robot in real time to obtain their spatiotemporal motion trajectories;

[0064] S40: According to the spatiotemporal trajectories of the operator and the key parts of the robot, a human-machine collaborative motion element calculation model based on spatiotemporal analysis is established;

[0065] S50: Based on the elements of collaborative motion, a human-machine collaborative safety risk prediction model is constructed with multi-factor coupling calculation;

[0066] S60: Based on the risk coefficient calculated by the risk prediction model, the human-machine collaboration scenario is divided into risks, and corresponding management and control measures are implemented according to the risk level.

[0067] Furthermore, the specific steps of step S10 are:

[0068] S101: According to the needs of human-machine collaboration scenarios, the robot's base, waist joint and execution end are the key detection parts of the robot;

[0069] S102: Construct a human-machine collaboration scenario dataset with the operator's left eye, right eye, mouth, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles as key parts;

[0070] S103: By labeling the above key parts, a high-quality labeling data set of the operator and the robot under different environments, postures and dynamic conditions is obtained.

[0071] In an intelligent manufacturing environment, human-machine collaboration is the key to improving production efficiency and flexibility. However, due to unstructured workplaces and dynamic working conditions, uncertainty in collaboration leads to potential safety risks. In this case, a risk calculation model based on key parts can help monitor and evaluate the safety of human-machine interaction in real time. In this embodiment, the electronic product assembly workshop of the intelligent manufacturing workshop is mainly responsible for the assembly and testing of electronic products, including the installation, detection and assembly of PCB boards. This type of work scene involves multiple workstations, usually with assembly lines and multiple robots collaborating on different tasks. In an automobile manufacturing workshop, high-definition video data of human-machine interaction is collected. These data cover the interaction between operators and different types of robots (such as assembly robots and handling robots) in various work scenarios. A professional annotation team is used to annotate the key parts of the human body and robot in the video, and record their spatiotemporal trajectories. Key parts include human heads, arms, legs, and end effectors and joints of robots. Selecting reasonable key detection parts can accurately reflect the changes in the movements of operators and robots, which helps to improve the safety and efficiency of human-machine collaboration scenarios. Data cleaning and preprocessing, such as frame rate adjustment and resolution unification, are performed to meet model training requirements. Through high-quality labeled data sets, the difficulty of model training is reduced and the real-time and accuracy of key part detection is improved.

[0072] The specific steps of step S20 are:

[0073] S201: According to the key parts detection requirements of the operator and the robot, the number of output channels is increased to detect the key parts, and the size and proportion of the anchor frame are adjusted to adapt to the size and shape of the key parts;

[0074] S202: Add a spatial pyramid pooling layer before the output layer to improve the detection performance of key parts of different scales. Use the feature pyramid network to perform multi-level feature fusion to enhance the recognition ability of smaller key parts in complex scenes.

[0075] S203: Add an integrated convolution module attention mechanism to enhance the real-time detection model’s ability to focus on key parts:

[0076] S204: adjusting the loss function according to the requirements of the key part detection task, and improving the detection accuracy by increasing the weight of the key point positioning loss;

[0077] S205: Set the learning rate, batch size, and training cycle, and perform training on the GPU, monitoring the loss value and detection accuracy during the training process;

[0078] S206: Using a static quantization method to perform low-precision conversion on the trained weights, and implementing dynamic quantization in the inference stage;

[0079] S207: Remove inefficient channels through structured pruning, and then randomly remove weights through unstructured pruning;

[0080] S208: Test the real-time detection model after quantization and pruning, and deploy the chip; evaluate its accuracy, recall rate, and inference speed to ensure that the real-time detection model has real-time inference capabilities while meeting the detection accuracy requirements.

[0081] The YOLOv11-pose model is based on the YOLO (You Only Look Once) architecture, which usually includes a feature extraction layer (e.g. based on CSPNet or other convolutional networks), a multi-scale feature fusion layer, and a final prediction layer. The last few layers of the model are responsible for generating predictions, including class probabilities, bounding box coordinates, and object pose key points. By increasing the number of output channels, the model can detect multiple key parts of operators and robots in more detail, while adjusting the size and scale of the anchor box to adapt to the size and shape of different key parts to ensure the accuracy and robustness of the detection results. Spatial pyramid pooling (SPP) is a feature extraction technique that is widely used in deep learning tasks (especially object detection and image classification). It extracts feature information of different scales by performing multi-scale pooling operations on the input feature map, thereby enhancing the detection ability of the model, especially the adaptability to objects of different sizes.

[0082] Through the integration of the attention mechanism, the model can automatically focus on key parts and improve the attention to target detection, especially in complex backgrounds or occlusion scenes, significantly improving the robustness and accuracy of detection. Then adjust the loss function, increase the weight of key point positioning loss, strengthen the model's optimization goal for key part detection, further improve the positioning accuracy of key parts, and adapt to the high-precision requirements in the interaction scene between operators and robots. Reasonably set the learning rate, batch size and training cycle, monitor the loss value and detection accuracy, ensure the stable convergence of the training process, prevent overfitting or underfitting problems, and use GPU to accelerate model training, significantly shorten the training time, and improve development efficiency. Using the static quantization method, by converting the trained model weights to low precision, further dynamically quantize the activation value in the inference stage to improve the inference speed. Then, remove redundant network channels through structured pruning to further optimize the network structure; remove fine-grained weights through unstructured pruning to further compress the model size, which not only improves the model efficiency but also retains the stability of performance indicators. Finally, the quantized and pruned model is fully tested (including accuracy, recall, and inference speed) to ensure that the model meets the real-time requirements in actual scenarios while maintaining high detection accuracy. This testing and evaluation process ensures that the final deployed model can run efficiently on low-cost hardware.

[0083] In step S30, the kernel correlation filter of the structured SVM combined with machine learning can more efficiently predict the position and track the trajectory of the target (such as the operator and the key parts of the robot). In complex scenes, the filter can use the local features of the key parts, such as shape, texture and color, to reliably lock the target and reduce the errors caused by complex background or changing lighting. By optimizing the weights of global and local features, the structured SVM can provide more stable tracking of key parts and avoid jitter and drift during the detection process.

[0084] Furthermore, in step S40, the human-machine collaborative motion elements include the minimum relative distance, relative speed, motion direction, and the operator's line of sight direction angle; wherein:

[0085] Minimum relative distance: the minimum spatial distance between a person and a robot;

[0086] Relative speed: the speed difference between the human and the robot;

[0087] Movement direction: Differences in movement direction between humans and robots;

[0088] Worker's sight direction angle: people's attention to the robot.

[0089] The specific steps of step S40 are:

[0090] S401: extracting motion elements such as the minimum relative distance, relative speed, relative motion direction and sight angle of key parts;

[0091] S402: Calculate the sight direction of the operator and the sight direction angle between the operator and the key part of the robot according to the left and right eye positions or shoulder positions of the operator and the key part of the robot;

[0092] S403: Calculate the relative speed and direction between the robot and the key parts of the operator, and dynamically analyze the human-machine interaction characteristics based on the two-frame time interval.

[0093] like Figure 4 As shown in the figure, according to the spatiotemporal trajectories of the key parts of the operator and the robot, a human-machine collaborative motion element calculation model based on spatiotemporal analysis is established, as follows:

[0094] Assume that the position of the key part of the operator in the i-th frame is P i k (x, y), k = 0, 1, ... 14 represents the position of the key parts of the operator in the i-th frame time map, and its P i k The x coordinate of , the y coordinate is .

[0095] Where k = 0, 1, .. 14 represent the left eye, right eye, mouth, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left foot, and right foot of the human body respectively. The positions of the key parts of the robot in the i-th frame are Where l=0, 1, and 2 represent the robot's base, waist joint, and execution end, respectively.

[0096] Assume that a target (a key part of a robot or a key part of an operator) has position information in two frames of real-time video. Assume that the position coordinates of the target at time t1 of the first frame are (x1, y1), and the position coordinates of the target at time t2 of the second frame are (x2, y2), and the time interval between the two frames is Δt = t2-t1. Calculate the distance between the three key parts of the robot and the key parts of the operator, and obtain the key part of the robot with the smallest distance l M And the key parts of the operators M ,but

[0097] The minimum relative distance is:

[0098] The relative speed calculation object is the key parts of the robot and the operator at the minimum distance.

[0099] The relative speed is:

[0100]

[0101] The relative speed is:

[0102] The relative velocity direction is:

[0103] in, Indicates the key part k of the operator in the second frame M The X coordinate position of the . Indicates the key part l of the robot in the second frame M The X coordinate position of the . Indicates the key part k of the operator in the second frame M The y coordinate position of . Indicates the key part l of the robot in the second frame M The y coordinate position of .

[0104] Assuming that the left eye and right eye are detected by the model, the left eye coordinates (x el ,y el )、right eye coordinates (x er ,y er ), generally speaking, the sight direction can be approximated as the vertical direction of the midpoint of the line connecting the left eye and the right eye. According to the principle that the dot product of two vertical vectors is 0, the sight direction of the operator can be calculated as Assuming that the left eye and the right eye cannot be detected by the model, the left shoulder (x sl ,y sl )、Right shoulder(x sr ,y sr ) roughly estimates the sight direction, which is (Note: The judgment basis for the left eye and right eye to pass the model detection is that the confidence of the left eye and right eye detection box results is greater than 0.5).

[0105] Through the robot's key parts with the smallest human-machine distance M And the key parts of the operators M The human-machine direction can be calculated as Then the sight direction angle is:

[0106]

[0107] The specific steps of step S50 are:

[0108] S501: A multi-factor risk assessment model based on the minimum relative distance, relative speed, relative speed direction and line of sight angle;

[0109] S502: introducing a weight factor of the sight direction angle into the model, and dynamically adjusting the risk assessment result in combination with the visual range of the operator, wherein the visual range includes a first visual area, a second visual area, and a blind area;

[0110] S503: Quantify the relative speed direction, and increase the risk factor when the operator and the robot are moving in opposite directions, and reduce the risk factor when they are in the same direction;

[0111] S504: Calculate the risk level comprehensively through the risk calculation formula and output the risk coefficient in real time.

[0112] (1) In this embodiment, it is assumed that is the risk coefficient of human-machine collaboration at the i-th frame, and its specific model calculation formula is as follows:

[0113]

[0114] (a) Sight angle factor

[0115] The maximum horizontal visual angle of human eyes can reach about 188 degrees. The overlapping visual field of human eyes is about 124 degrees. When people concentrate, the visual field range will be reduced to about one-fifth of the overlapping visual field, and the central angle of sight is about 25 degrees. Therefore, when the robot is within the central visual range of 25 degrees, the human-computer interaction state is normal under the same conditions, and people can respond quickly to possible abnormal events; on the contrary, when the robot deviates from the central visual angle area of ​​the human eye, the operator's sensitivity to the interactive events in human-computer collaboration will deteriorate, and the greater the deviation angle, the worse the perception. Therefore, the human eye field of vision is divided into three areas: (1) The area where the human eye focuses attention (the first visual area), that is, the area less than 25 / 2 or 12.5 degrees. When the robot is in the area where the human eye focuses attention, the danger level is the lowest and its danger coefficient is 0; (2) The second visual area of ​​the human eye, that is, 12.5 degrees to 94 degrees. When the robot is in the second visual area of ​​the human eye, the danger level gradually increases as the line of sight angle increases; (3) The blind area of ​​the human eye, that is, greater than 94 degrees. When the robot is in the blind area of ​​the human eye, the danger level is the highest. Establish a weight factor term ρ based on the human eye line of sight angle. The details are as follows:

[0116]

[0117] (b) Relative velocity direction factor

[0118] When the relative speed direction between the human and the robot is θ = 0°, it means they are moving in the same direction, and the risk level is the lowest at this time; when θ = 180°, it means the human and the robot are moving in opposite directions, and the probability of human-robot collision is the highest when the distance between the human and the robot is the same, and the risk level is the highest. The risk factor based on the relative speed direction can be expressed as:

[0119]

[0120] At this time, when θ=0°, g(θ)=0 (safest), and when θ=180°, g(θ)=1 (most dangerous).

[0121] According to the characteristics of human-machine collaboration scenarios, it is easy to know that the greater the relative speed of human-machine collaboration, the greater the risk of collision; at the same time, the smaller the minimum relative distance between humans and robots, the greater the risk of collision. Therefore, the degree of danger of human-machine collaboration is proportional to the relative speed of humans and robots, and inversely proportional to the minimum relative distance. Taking into account the relative speed direction factor, the formula for calculating the risk of human-machine collaboration is expressed as follows:

[0122]

[0123] In addition, under normal human-machine collaboration, the robot is in the human eye's highly focused visual area. However, in order to further improve the reliability of risk calculation and prevent the robot from deviating from the human eye's visual area in the event of irregular operation or sudden abnormality, it is necessary to add a weight factor item based on the human eye's line of sight to the original human-machine collaboration risk calculation item. Therefore, the human-machine collaboration safety risk prediction model is revised as follows:

[0124]

[0125] Among them, d represents the minimum relative distance, v represents the relative speed, θ represents the direction of movement, and α represents the direction of the staff's line of sight; ρ(α i ) represents the quantized processing function value of the staff's line of sight direction α at the i-th frame, g(θ i ) represents the quantization processing function value of the motion direction θ at the i-th frame, v i represents the relative speed between the human and the robot at the i-th frame, d i Indicates the minimum relative distance between the human and the robot at the i-th frame.

[0126] Once the risk factor is obtained, it can be normalized and mapped to different risk levels based on pre-set thresholds. These levels can be broken down into:

[0127] Severe danger: The risk factor is in the highest range of 0.8-1.0, indicating extremely high collision risk and serious consequences;

[0128] General danger: The risk factor is in the medium to high range of 0.5-0.8, indicating a clear risk of collision, which may lead to medium consequences;

[0129] Lower risk: The risk factor is in the medium range of 0.2-0.5, indicating potential risks, but the consequences are limited;

[0130] Safety: The risk factor is in the lowest range of 0.0-0.2, which means that the risk is negligible and the operation is safe.

[0131] When monitoring risk factors, the system continuously monitors in safe and less dangerous conditions; in general dangerous conditions, an alarm signal is issued to prompt the operator or robot to adjust its behavior; in severe dangerous conditions, the system issues an emergency alarm and triggers the robot to shut down or adjust its path to avoid safety accidents. In this way, the modeling of risk factors not only provides a quantifiable standard, but also facilitates the automated implementation of safety decisions and management in the intelligent manufacturing environment.

[0132] The present invention realizes real-time monitoring of the collaborative scenarios between robots and operators to prevent accidents caused by close contact. The risk calculation model based on various factors of key parts can effectively improve the safety in the human-machine collaborative environment, reduce the probability of accidents, and ensure the continuity and effectiveness of operations.

[0133] The preferred implementation of the present invention has been specifically described, but the invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for abnormal behavior identification and risk control in human-machine collaboration, characterized in that: The method comprises the following steps: S10: Based on the interaction characteristics between operators and robots, key parts are marked to build a data set suitable for human-robot collaboration scenarios; S20: The YOLOv11-Pose model is used to train the data set, and the model is quantized and pruned to build a real-time detection model for real-time detection of key parts of operators and robots; S30: In the limited area of ​​human-robot collaboration, the kernel correlation filter of structured SVM is used to track the key parts of the operator and the robot in real time to obtain their spatiotemporal motion trajectories; S40: According to the spatiotemporal trajectories of the operator and the key parts of the robot, a human-machine collaborative motion element calculation model based on spatiotemporal analysis is established; S50: Based on the elements of collaborative motion, a human-machine collaborative safety risk prediction model is constructed with multi-factor coupling calculation; S60: Based on the risk coefficient calculated by the risk prediction model, the risk of human-machine collaboration scenarios is divided and corresponding control measures are implemented according to the risk level; In step S40, the human-machine collaborative motion elements include the minimum relative distance, relative speed, motion direction, and the operator's line of sight direction angle; wherein: Minimum relative distance: the minimum spatial distance between a person and a robot; Relative speed: the speed difference between the human and the robot; Movement direction: Differences in movement direction between humans and robots; The worker's sight angle: the worker's attention to the robot; The specific steps of step S40 are: S401: extracting the minimum relative distance, relative speed, relative motion direction and line of sight direction angle motion elements of key parts; S402: Calculate the sight direction of the operator and the sight direction angle between the operator and the key part of the robot according to the left and right eye positions or shoulder positions of the operator and the key part of the robot; S403: Calculate the relative speed and direction between the robot and the key parts of the operator, and dynamically analyze the human-machine interaction characteristics according to the two-frame time interval; The specific steps of step S50 are: S501: A multi-factor risk assessment model based on the minimum relative distance, relative speed, relative speed direction and line of sight angle; S502: introducing a weight factor of the sight direction angle into the model, and dynamically adjusting the risk assessment result in combination with the visual range of the operator, wherein the visual range includes a first visual area, a second visual area, and a blind area; S503: Quantify the relative speed direction, and increase the risk factor when the operator and the robot are moving in opposite directions, and reduce the risk factor when they are in the same direction; S504: Calculate the risk level comprehensively through the risk calculation formula and output the risk coefficient in real time.

2. The method for abnormal behavior identification and risk management in human-machine collaboration according to claim 1, characterized in that: The specific steps of step S10 are: S101: According to the needs of human-machine collaboration scenarios, the robot's base, waist joint and execution end are the key detection parts of the robot; S102: Construct a human-machine collaboration scenario dataset with the operator's left eye, right eye, mouth, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles as key parts; S103: By labeling the above key parts, a high-quality labeling data set of the operator and the robot under different environments, postures and dynamic conditions is obtained.

3. The method for abnormal behavior identification and risk management in human-machine collaboration according to claim 1, characterized in that: The specific steps of step S20 are: S201: According to the key parts detection requirements of the operator and the robot, the number of output channels is increased to detect the key parts, and the size and proportion of the anchor frame are adjusted to adapt to the size and shape of the key parts; S202: Add a spatial pyramid pooling layer before the output layer to improve the detection performance of key parts of different scales. Use the feature pyramid network to perform multi-level feature fusion to enhance the recognition ability of smaller key parts in complex scenes. S203: Add an integrated convolution module attention mechanism to enhance the real-time detection model’s ability to focus on key parts: S204: adjusting the loss function according to the requirements of the key part detection task, and improving the detection accuracy by increasing the weight of the key point positioning loss; S205: Set the learning rate, batch size, and training cycle, and perform training on the GPU, monitoring the loss value and detection accuracy during the training process; S206: Using a static quantization method to perform low-precision conversion on the trained weights, and implementing dynamic quantization in the inference stage; S207: Remove inefficient channels through structured pruning, and then randomly remove weights through unstructured pruning; S208: Test the real-time detection model after quantization and pruning, and deploy the chip; evaluate its accuracy, recall rate, and inference speed to ensure that the real-time detection model has real-time inference capabilities while meeting the detection accuracy requirements.

4. The method for abnormal behavior identification and risk management in human-machine collaboration according to claim 1, characterized in that: In step S50, the human-machine collaboration safety risk prediction model is specifically as follows: Among them, d represents the minimum relative distance, v represents the relative speed, θ represents the direction of movement, and α represents the direction of the staff's line of sight; ρ(α i ) represents the quantized processing function value of the staff's line of sight direction α at the i-th frame, g(θ i ) represents the quantization processing function value of the motion direction θ at the i-th frame, v i represents the relative speed between the human and the robot at the i-th frame, d i Indicates the minimum relative distance between the human and the robot at the i-th frame.

5. The method for abnormal behavior identification and risk management in human-machine collaboration according to claim 1, characterized in that: The specific steps of step S60 are: S601: The system continuously monitors in safe and low-risk states; S602: In a general dangerous state, an alarm signal is issued to prompt the operator or robot to adjust its behavior; S603: In a severe danger state, the system issues an emergency alarm and triggers the robot to stop or adjust its path. To avoid safety accidents.

6. The method for abnormal behavior identification and risk management in human-machine collaboration according to claim 1, characterized in that: The risk levels can be broken down into: Severe danger: The risk factor is in the highest range of 0.8-1.0, indicating extremely high collision risk and serious consequences; General danger: The risk factor is in the medium to high range of 0.5-0.8, indicating a clear risk of collision, which may lead to medium consequences; Lower risk: The risk factor is in the medium range of 0.2-0.5, indicating potential risks, but the consequences are limited; Safety: The risk factor is in the lowest range of 0.0-0.2, which means that the risk is negligible and the operation is safe.

Images