Heavy-load industrial robot space safety man-machine cooperation control method

Abstract

The invention relates to the technical field of heavy-load industrial robot safety assembly, solves the technical problem that a traditional method lacks effective judgment on real-time tracking of human body posture details and invasion of human body joints into dangerous areas, and particularly relates to a heavy-load industrial robot space safety man-machine cooperation control method. Comprising the steps of obtaining a transformation operator of a left camera in the depth camera relative to base coordinates; the human body posture is analyzed based on a YOLOv8-pose model to obtain a current output result containing the human body key point part, and a translation operator of the real-time visible human body key point part relative to a left camera in the depth camera is obtained. Compared with a traditional human-machine distance identification method, the human-machine distance identification method has the advantages that accurate position identification can be carried out on human joints, judgment is carried out through force sense and position estimation, the human-machine cooperation state can be sensed in an omnibearing mode, safety risks of human-machine cooperation are avoided to a certain extent, and the safety of operators is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of safe assembly of heavy-load industrial robots, and in particular to a method for spatially safe human-machine collaborative control of heavy-load industrial robots. Background Art

[0002] As modern manufacturing becomes more intelligent, industrial robots are gradually being introduced into production processes to reduce labor burdens and improve precision and efficiency. Industrial robots capable of heavy-duty operations are being introduced on a large scale. As the use of humans and heavy-duty industrial robots for collaborative work becomes increasingly common, traditional human-robot collaboration models face multiple technical bottlenecks.

[0003] Early human-robot separation ensured safety through physical fencing, but this compromised collaborative flexibility and task response efficiency, making it unable to adapt to the dynamic adjustments required in complex assembly scenarios. As collaborative robot technology evolved, lightweight designs and power-limiting solutions reduced the risk of collision damage, but also limited load capacity, making it difficult to meet the demands of heavy-load operations.

[0004] To solve the above problems, the patent with publication number CN116079761A discloses a safe interactive human-machine collaborative robot. The system includes a processing module, an execution module, an identification module and an instruction module. The operation warning range is set by the historical instruction set of the total space set by the robot, the robot position information and the manual position information, and is corrected based on the actual production environment to obtain a new relatively safe area. When an operation instruction within the safe range is received, the execution module is controlled to work. Although this method can identify the human-machine safety distance to a certain extent, it mainly relies on the historical instruction set to generate a fixed warning range and cannot perceive the manual position information and changes in the human body state in real time. For complex heavy-load industrial robot operation scenarios, especially when human limbs suddenly intrude into dangerous areas, this method is difficult to respond quickly and there are safety risks.

[0005] Patent publication number CN110900601A discloses a method for autonomous control of robot operation for human-robot collaboration safety. This method first uses a depth camera and robot controller to perceive and detect the state of the industrial robot's workspace, constructs a human-robot collision detection model, then models the problem of safe human-robot collaboration as a Markov decision process, designs a reward function based on the human-robot distance, and finally establishes a deep neural network safety controller, which is trained, optimized, and simulated in a simulation environment. Although this method can, to a certain extent, complete relevant tasks in a human-robot collaborative environment and under the premise of ensuring personnel safety, it still has shortcomings in terms of human-robot distance and lacks accurate tracking of key parts of the human body. In particular, when the robot's end moves rapidly, subtle changes in the human torso and joints are easily overlooked, leading to safety assessment errors.

[0006] In summary, considering the human-machine distance, existing methods for determining human-machine distance have played a role in protecting human safety to a certain extent. However, these methods cannot track the details of human posture in real time, especially when certain human joints enter dangerous areas. Furthermore, single-source visual judgment is still insufficient to ensure safety, requiring the introduction of multiple sources of information, such as force. Therefore, a human-machine collaborative safety control method that combines multi-source information from force, vision, and position can fill the gaps in current technology and provide an additional layer of security for human-machine collaboration. Summary of the Invention

[0007] In response to the shortcomings of the existing technology, the present invention provides a method for spatial safety human-machine collaborative control of heavy-duty industrial robots, which solves the technical problems of traditional methods in real-time tracking of human posture details and lack of effective judgment when human joints invade dangerous areas.

[0008] To solve the above technical problems, the present invention provides the following technical solution: a method for spatial safety human-machine collaborative control of a heavy-duty industrial robot, the method comprising the following steps:

[0009] S1. Establish a spatial coordinate system with the base of the heavy-duty industrial robot as the base coordinate, and obtain the transformation operator of the left camera in the depth camera relative to the base coordinate in the spatial coordinate system

[0010] S2. Analyze the human body posture based on the YOLOv8-pose model to obtain the output result containing the current human body key points, and obtain the real-time translation operator of the human body key points relative to the left camera in the depth camera based on the output result. B P (i) ;

[0011] S3, obtain the six degrees of freedom coordinates (Xg 、Y g 、Z g 、A g 、B g 、C g ), and calculate the real-time translation operator of the farthest part of the end workpiece relative to the base coordinates A P;

[0012] S4. According to the transformation operator Translation operator B P (i) and the translation operator A P calculates the distance r(t) between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body;

[0013] S5. Establish a spatial safety envelope including the non-collaborative space, the collaborative space, and the dangerous space based on the distance r(t), and construct an admittance control model to realize human-machine collaboration in the collaborative space through a variable admittance control algorithm;

[0014] S6. In the collaborative space, human-machine collaborative control is performed based on the admittance control model of the human-machine collaborative mechanism.

[0015] Furthermore, in step S1, the specific steps include:

[0016] S11. In the spatial coordinate system, calibrate the heavy-duty industrial robot base through the optical dynamic tracking system and determine the transformation operator of the optical dynamic tracking system relative to the heavy-duty industrial robot base. The formula is as follows:

[0017]

[0018] Where, It is the transformation operator of the optical dynamic tracking system of the heavy-load industrial robot base;

[0019] S12, calibrating the left camera of the depth camera through the optical dynamic tracking system, and determining the transformation operator of the left camera in the depth camera relative to the optical dynamic tracking system The formula is as follows:

[0020]

[0021] Where, is the transformation operator of the left camera in the optical dynamic tracking system relative to the depth camera;

[0022] S13, according to the transformation operator and transformation operators Determine the transformation operator of the left camera in the depth camera relative to the heavy-load industrial robot base. The formula is as follows:

[0023]

[0024] Where, is the transformation operator of the left camera of depth camera 1 relative to the base of the heavy-load industrial robot.

[0025] Furthermore, in step S2, the specific steps include:

[0026] S21. Configure the acquisition environment of the depth camera and obtain real-time RGB images and depth maps through the communication instructions of the depth camera;

[0027] S22. Preprocess the RGB image and analyze the human posture using the YOLOv8-pose model. Output the output Result, which includes the bounding box coordinates, category labels, and the image coordinates of 17 key points of the human body. The expression is:

[0028] Result=[(B img ,Class,{k (1) ,…,k (17)})]

[0029] Where B img is the bounding box coordinate; Class is the category label; {k (1) ,…,k (17)} are the image coordinates and visibility of 17 key points of the human body; where {k (1) ,…,k (17)} can be expressed as the following formula:

[0030] k (i) =(x kpt ,y kpt ,v)

[0031] Where k (i) is the image coordinates and visibility of the i-th human key point; x kpt and y kpt is the RGB image grid coordinate; v is the visibility score of the human key point;

[0032] S23, according to the RGB image grid coordinates of the human key points visible in the output result Result, and combined with the corresponding depth map, position matching is performed to obtain the real-time translation operator of the i-th human key point relative to the left camera in the depth camera B P (i) .

[0033] Furthermore, in step S3, the specific steps include:

[0034] S31, according to the six-degree-of-freedom coordinates (X g 、Y g、z g 、A g 、B g 、C g ) Obtain the real-time transformation operator of the heavy-duty industrial robot end relative to the base coordinates The formula is as follows:

[0035]

[0036] S32, according to the size of the end workpiece (L x 、L y 、L z ), calculate the real-time translation operator of the farthest part of the end workpiece relative to the end of the heavy-duty industrial robot L P, the formula is as follows:

[0037]

[0038] S33. Combined transformation operator Calculate the real-time translation operator of the farthest part of the end workpiece relative to the base coordinates. The formula is as follows:

[0039]

[0040] Where, A P is the real-time translation operator of the farthest part of the end workpiece 4 relative to the base coordinates.

[0041] Furthermore, in step S4, the specific steps include:

[0042] S41. According to the translation operator B P (i) Calculate the translation operator of the i-th human key point relative to the base coordinates that is visible in real time Z P (i) , the formula is as follows:

[0043]

[0044] Where, Z P (i) is the translation operator of the i-th human key point relative to the base coordinates that is visible in real time;

[0045] S42. According to the translation operator Z P (i) Translation operator of the farthest part of the end workpiece relative to the base coordinates A P, calculate the distance r(t) between the farthest part of the visible end workpiece and the key point of the i-th human body (i) ;

[0046] S43, from the distance r(t) (i)Excluding the key points of the human arm, calculate the distance r(t) between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body. The formula is as follows:

[0047] r(t)=min{r(t) (1) ,…,r(t) (17)}

[0048] Where, {r(t) (1) ,…,r(t) (17)} are 17 key points of the human body.

[0049] Furthermore, in step S5, the specific steps include:

[0050] S51. Define the relationship between the end workpiece and the spatial safety envelope, namely:

[0051] When the farthest part of the end workpiece is in the non-collaborative space, the heavy-load industrial robot does not move; when the farthest part of the end workpiece is in the collaborative space but not in the dangerous space, the heavy-load industrial robot and the operator perform human-machine collaboration, and the operator can guide the heavy-load industrial robot to move by applying force to the end workpiece; when the farthest part of the end workpiece suddenly enters the dangerous space, the heavy-load industrial robot immediately stops, stops working, and issues an alarm;

[0052] S52. Establish a nonlinear coupling relationship between the end contact force and the motion state, that is:

[0053] When the end workpiece comes into contact with the environment, the dynamic response of the system is described as follows:

[0054]

[0055] The virtual stiffness k and the desired motion acceleration speed With position x d are all set to 0, so the above formula becomes:

[0056]

[0057] Where, f ext is the interaction force after gravity compensation of the end; m is the virtual mass; b is the virtual damping; k is the virtual stiffness; x represents the acceleration, velocity, and position of the end of the heavy-load industrial robot; x d Represent the desired motion acceleration, velocity and position respectively;

[0058] S53. Obtain the force signal of the six-dimensional force sensor installed at the end of the heavy-duty industrial robot and calculate the instantaneous average power spectrum P. The formula is as follows:

[0059]

[0060] Where P is the instantaneous average power spectrum; T is the sampling period; A is the instantaneous amplitude;

[0061] S54, based on the instantaneous average power spectrum P, obtain the critical node between vibration and no vibration by adjusting the admittance parameter, and obtain the critical node value by connection;

[0062] S55. Based on the spatial safety envelope, the one-dimensional admittance control model, and the critical node value, an admittance control model for realizing human-machine collaboration through the admittance parameter change control rule is obtained, namely:

[0063] When the heavy-loaded industrial robot is in a non-collaborative space and a dangerous space, the heavy-loaded industrial robot does not move; when the heavy-loaded industrial robot enters a dangerous space, the heavy-loaded industrial robot makes an emergency stop to ensure the safety of the operator; when the heavy-loaded industrial robot is in a collaborative space, the heavy-loaded industrial robot collaborates with the operator. The closer the heavy-loaded industrial robot is to the operator, the greater the damping; conversely, the farther the heavy-loaded industrial robot is from the operator, the smaller the damping.

[0064] Furthermore, the admittance parameter change control rule includes changing and controlling the real-time effective virtual damping b(r) and the real-time effective virtual mass m(r), wherein:

[0065] The rule for the change of real-time effective virtual damping b(r) is as follows:

[0066]

[0067] Where z is a constant parameter, which is used to adjust the smoothness of the change of damping due to distance change; b min and b max are the minimum virtual damping and the maximum virtual damping, respectively, which are used to roughly estimate the effective virtual damping range; r(t) is the distance between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body; r0 is the shortest distance between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body, that is, the distance at which the heavy-loaded industrial robot will immediately enter the dangerous space; ξ is a variable parameter used to fine-tune the damping according to changes in human intentions;

[0068] Take any position of the change curves O0-O3 of the real-time effective virtual mass m(r) and the real-time effective virtual damping b(r) in the farthest stationary or non-contact state in the collaborative space, and the rule of the change of the real-time effective virtual mass m(r) is as follows:

[0069]

[0070] Where b' min is a constant parameter, the virtual damping when the collaborative space is at its farthest distance or without contact; m m is a constant parameter, which serves as the virtual mass when the space distance is farthest and the object is stationary or without contact; β is a constant parameter, which is used to adjust the convergence degree of the acceleration curve O0-O2; α is a constant parameter, which is used to adjust the rise and fall degree of the curve O0-O2 under acceleration; τ is a constant parameter, which is used to adjust the smoothness of the change of the curve O0-O1 under deceleration; σ is a constant parameter, which is used to adjust the rise and fall degree of the curve O0-O1 under deceleration.

[0071] Furthermore, the change of the real-time effective virtual damping b(r) is adjusted by changing the parameter ξ, so as to realize the analysis of human intention and the interaction force f ext 5 speed industrial robot with heavy loads The direction is judged by the following formula:

[0072]

[0073] Combining the human intention analyzed above, the setting rule for the change parameter ξ is as follows:

[0074]

[0075] Where η is a constant parameter with a value of 0<η<1, which is used to adjust the variable parameter ξ according to human intention; f max is the maximum interaction force, the purpose of which is to limit the interaction force f ext The size of the interaction force f min .

[0076] By means of the above technical solution, the present invention provides a method for spatially safe human-machine collaborative control of a heavy-duty industrial robot, which has at least the following beneficial effects:

[0077] 1. This invention combines the YOLOv8-pose model to dynamically capture the 3D spatial position information of key points on the human body, establishing non-collaborative, collaborative, and dangerous spaces, and creating a spatial safety envelope for the human body. Compared to traditional methods for identifying human-machine distance, this method can accurately identify the position of human joints, significantly improving operator safety.

[0078] 2. The present invention fully considers the integration of visual, force perception and other information. For places that cannot be determined by vision, force perception and position estimation are used to judge, which can fully perceive the state of human-machine collaboration and avoid the safety risks of human-machine collaboration to a considerable extent.

[0079] 3. The present invention designs a variable admittance control model combined with a spatial safety envelope. Compared with traditional passive protection systems, it can achieve active and compliant control and improve the adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0081] Figure 1 Schematic diagram of the equipment composition of the human-machine collaborative control system of the present invention;

[0082] Figure 2 Schematic diagram of the overall transformation operator of the human-machine collaborative control system in the present invention;

[0083] Figure 3 Schematic diagram of the spatial safety envelope in the present invention;

[0084] Figure 4 This is a schematic diagram of the principle of determining the admittance parameters in the present invention.

[0085] In the figure: 1. Depth camera; 2. Optical dynamic tracking system; 3. Six-dimensional force sensor; 4. End workpiece; 5. Heavy-load industrial robot. DETAILED DESCRIPTION

[0086] To make the above-mentioned objectives, features, and advantages of the present invention more clearly understood, the present invention is further described below in detail with reference to the accompanying drawings and specific embodiments. This will enable a full understanding of how this application uses technical means to solve technical problems and achieve technical effects, and to implement the invention accordingly.

[0087] Please refer to Figures 1-4 , this embodiment proposes a method for human-machine collaborative control of heavy-duty industrial robots in space safety. Figure 1 As shown, the method proposed by the present invention is applicable to a system equipped with a six-degree-of-freedom heavy-duty industrial robot 5 equipped with a depth camera 1, an optical dynamic tracking system 2, a six-dimensional force sensor 3, and an end workpiece 4. This embodiment uses a KUKA KR60 six-degree-of-freedom heavy-duty industrial robot 5, a ZED 2i binocular stereo depth camera 1, a C-Track optical dynamic tracking system 2, and a Yuli six-dimensional force sensor 3. This embodiment only illustrates the method of use of the present invention and does not constitute a constraint on the claims of the present invention. The details are as follows:

[0088] The heavy-load industrial robot 5 is a high-precision heavy-duty industrial automation equipment. Its overall structure adopts a serial six-axis robotic arm configuration and is rigidly connected to the AGV car workstation through a base.

[0089] The depth camera 1 is a high-precision three-dimensional visual perception device that supports the output of images such as depth maps and RGB images. It is fixed to the outside of the heavy-duty industrial robot 5 through a tripod without affecting the normal working area of ​​the heavy-duty industrial robot 5. It can collect all the distance information between the heavy-duty industrial robot 5 and the operator.

[0090] The optical dynamic tracking system 2 is a high-precision six-degree-of-freedom real-time motion capture and posture measurement device. It uses a modular binocular infrared camera group to build an expandable stereo vision network. It is fixed to the outside of the heavy-duty industrial robot 5 via a tripod without affecting the normal working area of ​​the heavy-duty industrial robot 5. It calibrates the ZED 2i binocular stereo depth camera 1 and the base coordinates of the heavy-duty industrial robot 5 to determine the spatial position coordinate information of the system.

[0091] The six-dimensional force sensor 3 is a high-precision omnidirectional force / torque measurement device. Its sub-millisecond delay characteristics can capture micro-newton-level contact force / torque changes in real time. It is connected to the end of the heavy-duty industrial robot 5 through a connecting flange to sense the force / torque changes of the components below the force sensor.

[0092] The system of a six-degree-of-freedom heavy-duty industrial robot 5 based on a depth camera 1, an optical dynamic tracking system 2, a six-dimensional force sensor 3, and an end workpiece 4, and the method for spatially safe human-machine collaborative control of the heavy-duty industrial robot proposed in this embodiment includes the following steps:

[0093] S1. Establish a spatial coordinate system with the base of the heavy-duty industrial robot 5 as the base coordinate, and obtain the transformation operator of the left camera in the depth camera relative to the base coordinate in the spatial coordinate system like Figure 2 As shown, a spatial coordinate system with the base of the six-degree-of-freedom heavy-duty industrial robot 5 as the base coordinate is constructed through the optical dynamic tracking system 2 and the depth camera 1. As a preferred embodiment of step S1, the specific steps include:

[0094] S11. In the spatial coordinate system, the base of the heavy-load industrial robot 5 is calibrated by the optical dynamic tracking system 2 to determine the transformation operator of the optical dynamic tracking system 2 relative to the base of the heavy-load industrial robot 5. The formula is as follows:

[0095]

[0096] Where, It is the transformation operator of the base of the heavy-load industrial robot 5 relative to the optical dynamic tracking system 2;

[0097] S12, calibrate the left camera of the depth camera 1 through the optical dynamic tracking system 2, and determine the transformation operator of the left camera in the depth camera 1 relative to the optical dynamic tracking system 2 The formula is as follows:

[0098]

[0099] Where, is the transformation operator of the left camera in the optical dynamic tracking system 2 relative to the depth camera 1;

[0100] S13, according to the transformation operator and transformation operators Determine the transformation operator of the left camera in the depth camera 1 relative to the base of the heavy-load industrial robot 5. The formula is as follows:

[0101]

[0102] Where, It is the transformation operator of the left camera of the depth camera 1 relative to the base of the heavy-load industrial robot 5.

[0103] It should be noted that the transformation operator of the left camera of the depth camera 1 relative to the base of the heavy-load industrial robot 5 obtained in this embodiment is a fixed value. If the position of the depth camera 1 or the position of the base of the heavy-load industrial robot 5 changes, recalibration is required.

[0104] S2. Analyze the human body posture based on the YOLOv8-pose model to obtain the output result containing the current human body key points, and obtain the real-time translation operator of the human body key points relative to the left camera in depth camera 1 based on the output result. B P (i) .like Figure 2 As shown in the figure, the RGB image obtained in real time by the left camera in the depth camera 1 is combined with the YOLOv8-pose model to analyze the human posture and obtain the current key points of the human body in real time. Then, the depth map obtained in real time by the left camera in the depth camera 1 is matched with the two to obtain the translation operator of the real-time visible key points of the human body relative to the left camera in the depth camera 1. B P (i) As a preferred implementation of step S2, the specific steps include:

[0105] S21, configure the acquisition environment of the depth camera 1, and obtain real-time RGB images and depth maps through the communication instructions of the depth camera 1;

[0106] S22. Preprocess the RGB image and analyze the human posture using the YOLOv8-pose model. Output the output Result, which includes the bounding box coordinates, category labels, and the image coordinates of 17 key points of the human body. The expression is:

[0107] Result=[(B img ,Class,{k (1) ,…,k(17)})]

[0108] Where B img is the bounding box coordinate; Class is the category label; {k (1) ,…,k (17)} is the image coordinates and visibility of 17 key points of the human body;

[0109] Among them, {k (1) ,…,k (17)} can be expressed as the following formula:

[0110] k (i) =(x kpt ,y kpt ,v)

[0111] Where k (i) is the image coordinates and visibility of the i-th human key point; x kpt and y kpt is the RGB image grid coordinate; v is the visibility score of the human key point, the threshold is set to 0.5, and only the threshold greater than 0.5 is visible;

[0112] S23, according to the RGB image grid coordinates of the human key points visible in the output result Result, and combined with the corresponding depth map, position matching is performed to obtain the real-time translation operator of the i-th human key point relative to the left camera in the depth camera 1 B P (i) .

[0113] S3, obtain the six-degree-of-freedom coordinates (X g 、Y g 、Z g 、A g 、B g 、C g ), and calculate the real-time translation operator of the farthest part of the end workpiece 4 relative to the base coordinates A P. As Figure 2 As shown, by performing UDP communication with the control cabinet of the heavy-duty industrial robot 5, the real-time six-degree-of-freedom coordinates of the end of the heavy-duty industrial robot 5 are obtained, and the coordinates of the farthest part of the end workpiece 4 are calculated in combination with the size of the end workpiece 4. As a preferred embodiment of step S3, the specific steps include:

[0114] S31, according to the six-degree-of-freedom coordinates (X g 、Y g 、Z g 、A g 、B g 、C g ) Obtain the real-time transformation operator of the heavy-duty industrial robot 5 end relative to the base coordinates The formula is as follows:

[0115]

[0116] S32, according to the size of the end workpiece 4 (L x 、L y 、L z ), calculate the real-time translation operator of the farthest part of the end workpiece 4 relative to the end of the heavy-duty industrial robot 5 L P, the formula is as follows:

[0117]

[0118] S33. Combined transformation operator The real-time translation operator of the farthest part of the end workpiece 4 relative to the base coordinate is calculated using the following formula:

[0119]

[0120] Where, A P is the real-time translation operator of the farthest part of the end workpiece 4 relative to the base coordinates.

[0121] S4. According to the transformation operator Translation operator B P (i) and the translation operator A P calculates the distance r(t) between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body. As a preferred embodiment of step S4, the specific steps include:

[0122] S41. According to the translation operator B P (i) Calculate the translation operator of the i-th human key point relative to the base coordinates that is visible in real time Z P (i) , the formula is as follows:

[0123]

[0124] Where, Z P (i) is the translation operator of the i-th human key point relative to the base coordinates that is visible in real time;

[0125] S42. According to the translation operator Z P (i) The translation operator of the farthest part of the end workpiece 4 relative to the base coordinates A P, calculate the distance r(t) between the farthest part of the visible end workpiece 4 and the key point of the i-th human body (i) ;

[0126] S43, from the distance r(t) (i)Excluding the key points of the human arm, the distance r(t) between the coordinates of the farthest part of the end workpiece 4 and the key point closest to the key point of the human body is calculated as follows:

[0127] r(t)=min{r(t) (1) ,…,r(t) (17)}

[0128] Where, {r(t) (1) ,…,r(t) (17)} are 17 key points of the human body.

[0129] S5. Based on the distance r(t), a spatial safety envelope is established, including the non-collaborative space, the collaborative space, and the dangerous space. A variable admittance control algorithm is then used to construct an admittance control model for human-machine collaboration within the collaborative space. After constructing the spatial safety envelope, this embodiment establishes a human-machine collaboration mechanism using the variable admittance control algorithm. As a preferred implementation of step S5, the specific steps include:

[0130] S51. The distance r(t) between the coordinates of the farthest part of the end workpiece 4 and the key point closest to the key point of the human body is used to establish a spatial safety envelope. The spatial safety envelope is initially defined as three spaces: non-collaborative space, collaborative space, and dangerous space. Figure 3 As shown in the figure, the relationship between the end workpiece and the spatial safety envelope is defined, including:

[0131] When the farthest part of the end workpiece 4 is in the non-collaborative space, the heavy-loaded industrial robot 5 does not move; when the farthest part of the end workpiece 4 is in the collaborative space but not in the dangerous space, the heavy-loaded industrial robot 5 performs human-machine collaboration with the operator, and the operator can guide the movement of the heavy-loaded industrial robot 5 by applying force to the end workpiece 4; when the farthest part of the end workpiece 4 suddenly enters the dangerous space, the heavy-loaded industrial robot 5 immediately stops, stops working, and sounds an alarm.

[0132] S52. Establish a nonlinear coupling relationship between the end contact force and the motion state. When constructing a human-machine collaborative interaction model, the end workpiece 4 can be abstracted as a "mass-damper-spring" admittance control model with equivalent mechanical impedance characteristics to establish a nonlinear coupling relationship between the end contact force and the motion state. To facilitate analysis of this model and without loss of generality, this embodiment will use a one-dimensional motion admittance control model for description. A similar description can be used for a one-dimensional rotational admittance control model, namely:

[0133] When the end workpiece 4 comes into contact with the environment, the dynamic response of the system can be described as follows:

[0134]

[0135] Where, f ext is the interaction force after gravity compensation of the end; m is the virtual mass; b is the virtual damping; k is the virtual stiffness; x represents the acceleration, velocity, and position of the end of the heavy-load industrial robot; x d Represent the desired motion acceleration, velocity and position respectively.

[0136] This one-dimensional admittance control model essentially constructs the force-displacement dynamic response characteristics. By dynamically adjusting parameters such as m, b, and k, the system can present adjustable impedance characteristics from rigid collision to flexible contact. This embodiment intends to use the one-dimensional admittance control model to simulate free motion under human-machine collaboration, so the virtual stiffness k and the desired motion acceleration are speed With position x d are all set to 0, so the above formula becomes:

[0137]

[0138] For speed control, the Laplace transform of the speed of the heavy-duty industrial robot 5 can be written as follows:

[0139]

[0140] Where, For speed Laplace transform of F H (s) is f ext Laplace transform of ; H(s) is the transfer equation, and the formula is as follows:

[0141]

[0142] Where s is the variable in the complex frequency domain after Laplace transform, which converts the differential equation in the time domain into an algebraic equation in the complex frequency domain, thereby simplifying the analysis of the dynamic characteristics of the system.

[0143] As is clear from the above equation, the maximum steady-state velocity output is solely a function of the virtual damping parameters. Furthermore, the dynamic model is influenced by the ratio of virtual mass m to virtual damping b (admittance parameter). To achieve proper human-robot collaboration, these parameters must be determined so that the heavy-load industrial robot 5 exhibits human-like compliance.

[0144] S53. Obtain the force signal from the six-dimensional force sensor 3 installed at the end of the heavy-duty industrial robot 5 and calculate the instantaneous average power spectrum P. To determine the stability of the system, install the six-dimensional force sensor 3 at the end and simulate the test results of the six-dimensional force sensor 3 in the X direction when it is unloaded. Obtain and process the force signal from the six-dimensional force sensor 3 to calculate the instantaneous average power spectrum. The formula is as follows:

[0145]

[0146] Where P is the instantaneous average power spectrum; T is the sampling period; A is the instantaneous amplitude.

[0147] S54. Based on the instantaneous average power spectrum P, the admittance parameter is adjusted to obtain a critical node between vibration and no vibration, and the obtained critical node value is obtained by concatenation. Specifically, when the instantaneous average power spectrum P is greater than a given threshold, it is considered that the heavy-load industrial robot 5 is vibrating. By setting multiple virtual damping values ​​and then continuously adjusting the admittance parameter m / b during the experiment, the critical node between vibration and no vibration is obtained, and the obtained critical node value is obtained by concatenation.

[0148] like Figure 4 As shown in the figure, the horizontal axis represents the size of the virtual damping b, the vertical axis represents the size of the admittance parameter m / b, the shaded area represents the unstable area with vibration, and the blank area represents the stable area without vibration. In the process of human-machine collaboration, when the operator applies the same force, if the selected parameter is closer to the vertical axis ( Figure 4 A0→A1), the heavy-load industrial robot 5 has a larger acceleration and a faster speed; if the selected parameter is closer to the horizontal axis ( Figure 4 A0→A2), the heavy-load industrial robot 5 moves more flexibly.

[0149] S55. Based on the spatial safety envelope, the one-dimensional admittance control model, and the critical node value, an admittance control model for realizing human-machine collaboration through the admittance parameter change control rule is obtained. The admittance parameter change control rule includes the change control of the real-time effective virtual damping b(r) and the real-time effective virtual mass m(r), namely:

[0150] When the heavy-loaded industrial robot 5 is in a non-collaborative space or a dangerous space, the heavy-loaded industrial robot 5 does not move. In particular, once the heavy-loaded industrial robot 5 enters a dangerous space, the heavy-loaded industrial robot 5 makes an emergency stop to ensure the safety of the operator. When the heavy-loaded industrial robot 5 is in a collaborative space, the heavy-loaded industrial robot 5 collaborates with the operator. The closer the heavy-loaded industrial robot 5 is to the operator, the greater the damping. Conversely, the farther the heavy-loaded industrial robot 5 is from the operator, the smaller the damping. Comprehensive analysis shows that the rule for the change of the real-time effective virtual damping b(r) is as follows:

[0151]

[0152] Where z is a constant parameter, which is used to adjust the smoothness of the change of damping due to distance change; b min and b max are the minimum virtual damping and the maximum virtual damping, respectively, which are used to roughly estimate the effective virtual damping range; r(t) is the distance between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body; r0 is the shortest distance between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body, that is, the distance at which the heavy-loaded industrial robot 5 immediately enters the dangerous space; ξ is a variable parameter, which is used to fine-tune the damping according to changes in human intentions.

[0153] When the heavy-load industrial robot 5 cooperates with the operator, if the heavy-load industrial robot 5 needs to accelerate, the real-time effective virtual damping b(r) should be reduced. Conversely, if the heavy-load industrial robot 5 needs to decelerate, the real-time effective virtual damping b(r) should be increased. The change of the real-time effective virtual damping b(r) can be adjusted by changing the parameter ξ to realize the analysis of human intention. The interaction force f ext 5 speed industrial robot with heavy loads The direction is judged by the following formula:

[0154]

[0155] Combining the human intention analyzed above, the setting rule for the change parameter ξ is as follows:

[0156]

[0157] Where η is a constant parameter with a value of 0<η<1, which is used to adjust the variable parameter ξ according to human intention; f max is the maximum interaction force, the purpose of which is to limit the interaction force f ext The size of the interaction force f min .

[0158] Considering that the real-time effective virtual damping b(r) in the collaborative space is time-varying, in order to better analyze, the real-time effective virtual mass m(r) and the real-time effective virtual damping b(r) variation curves in the static or non-contact state are taken ( Figure 4 For a certain position (O0-O3) in the collaborative space, taking the situation where the collaboration space is at its farthest distance and stationary or without contact as an example, the rule for the change of the real-time effective virtual mass m(r) is as follows:

[0159]

[0160] Where m(r) is the real-time effective virtual mass; b' min is a constant parameter, the virtual damping when the collaborative space is at its farthest distance or without contact; m m is a constant parameter, which is used as the virtual mass when the space distance is farthest and stationary or without contact; β is a constant parameter, which is used to adjust the acceleration condition ( Figure 4 The convergence degree of the curve O0-O2 in the middle; α is a constant parameter, which is used to adjust the rise and fall degree of the curve O0-O2 in the acceleration case; τ is a constant parameter, which is used to adjust the Figure 4 The smoothness of the change of the curve O0-O1 (in the figure); σ is a constant parameter used to adjust the degree of rise and fall of the curve O0-O1 under deceleration.

[0161] S6. Human-robot collaboration is performed within the collaborative space using an admittance control model based on the human-robot collaboration mechanism. Specifically, based on the admittance control model with admittance parameter change control rules, a heavy-load industrial robot and an operator perform human-robot collaboration within the collaborative space. The operator can guide the heavy-load industrial robot's movement by applying force to the end workpiece.

[0162] According to the human-machine collaborative control method proposed in this embodiment, a spatial coordinate system with the base of the six-degree-of-freedom heavy-duty industrial robot 5 as the base coordinate is constructed. Through the depth camera 1 and the optical dynamic tracking system 2, combined with the YOLOv8-pose model, the distance between the coordinates of the farthest part of the end workpiece 4 and the key point closest to the key point of the human body is calculated and a spatial safety envelope is constructed. When the farthest part of the end workpiece 4 is in the non-collaborative space, the industrial robot 5 does not move; when the farthest part of the end workpiece 4 is in the collaborative space, a human-machine collaborative control mechanism is established through the variable admittance control algorithm for human-machine collaboration; when the farthest part of the end workpiece 4 suddenly enters the dangerous space, the industrial robot 5 immediately stops, stops working and alarms.

[0163] Those skilled in the art will appreciate that all or part of the steps in the above-mentioned embodiment methods can be accomplished by instructing the relevant hardware through a program. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] The above embodiments provide a detailed introduction to the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A method for spatial safety human-machine collaborative control of a heavy-duty industrial robot, characterized in that: The method comprises the following steps: S1. Establish a spatial coordinate system with the base of the heavy-duty industrial robot as the base coordinate, and obtain the transformation operator of the left camera in the depth camera relative to the base coordinate in the spatial coordinate system S2. Analyze the human body posture based on the YOLOv8-pose model to obtain the output result containing the current human body key points, and obtain the real-time translation operator of the human body key points relative to the left camera in the depth camera based on the output result. B P (i) ; S3, obtain the six degrees of freedom coordinates (X g 、Y g 、Z g 、A g 、B g 、C g ), and calculate the real-time translation operator of the farthest part of the end workpiece relative to the base coordinates A P; S4. According to the transformation operator Translation operator B P (i) and the translation operator A P calculates the distance r(t) between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body; S5. Establish a spatial safety envelope including the non-collaborative space, the collaborative space, and the dangerous space based on the distance r(t), and construct an admittance control model to realize human-machine collaboration in the collaborative space through a variable admittance control algorithm; S6. In the collaborative space, human-machine collaborative control is performed based on the admittance control model of the human-machine collaborative mechanism.

2. The human-machine collaborative control method according to claim 1, characterized in that: In step S1, the specific steps include: S11. In the spatial coordinate system, calibrate the heavy-duty industrial robot base through the optical dynamic tracking system and determine the transformation operator of the optical dynamic tracking system relative to the heavy-duty industrial robot base. The formula is as follows: Where, It is the transformation operator of the optical dynamic tracking system of the heavy-load industrial robot base; S12, calibrating the left camera of the depth camera through the optical dynamic tracking system, and determining the transformation operator of the left camera in the depth camera relative to the optical dynamic tracking system The formula is as follows: Where, is the transformation operator of the left camera in the optical dynamic tracking system relative to the depth camera; S13, according to the transformation operator and transformation operators Determine the transformation operator of the left camera in the depth camera relative to the heavy-load industrial robot base. The formula is as follows: Where, is the transformation operator of the left camera of depth camera 1 relative to the base of the heavy-load industrial robot.

3. The human-machine collaborative control method according to claim 1, characterized in that: In step S2, the specific steps include: S21. Configure the acquisition environment of the depth camera and obtain real-time RGB images and depth maps through the communication instructions of the depth camera; S22. Preprocess the RGB image and analyze the human posture using the YOLOv8-pose model. Output the output Result, which includes the bounding box coordinates, category labels, and the image coordinates of 17 key points of the human body. The expression is: Result=[(B img ,Class,{k (1) ,…,k (17) })] Where B img is the bounding box coordinate; Class is the category label; {k (1) ,…,k (17) } are the image coordinates and visibility of 17 key points of the human body; where {k (1) ,…,k (17) } can be expressed as the following formula: k (i) =(x kpt ,y kpt ,v) Where k (i) is the image coordinates and visibility of the i-th human key point; x kpt and y kpt is the RGB image grid coordinate; v is the visibility score of the human key point; S23, according to the RGB image grid coordinates of the human key points visible in the output result Result, and combined with the corresponding depth map, position matching is performed to obtain the real-time translation operator of the i-th human key point relative to the left camera in the depth camera B P (i) .

4. The human-machine collaborative control method according to claim 1, characterized in that: In step S3, the specific steps include: S31, according to the six-degree-of-freedom coordinates (X g 、Y g 、Z g 、A g 、B g 、C g ) Obtain the real-time transformation operator of the heavy-duty industrial robot end relative to the base coordinates The formula is as follows: S32, according to the size of the end workpiece (L x 、L y 、L z ), calculate the real-time translation operator of the farthest part of the end workpiece relative to the end of the heavy-duty industrial robot L P, the formula is as follows: S33. Combined transformation operator Calculate the real-time translation operator of the farthest part of the end workpiece relative to the base coordinates. The formula is as follows: Where, A P is the real-time translation operator of the farthest part of the end workpiece 4 relative to the base coordinates.

5. The human-machine collaborative control method according to claim 1, characterized in that: In step S4, the specific steps include: S41. According to the translation operator B P (i) Calculate the translation operator of the i-th human key point relative to the base coordinates that is visible in real time Z P (i) , the formula is as follows: Where, Z P (i) is the translation operator of the i-th human key point relative to the base coordinates that is visible in real time; S42. According to the translation operator Z P (i) Translation operator of the farthest part of the end workpiece relative to the base coordinates A P, calculate the distance r(t) between the farthest part of the visible end workpiece and the key point of the i-th human body (i) ; S43, from the distance r(t) (i) Excluding the key points of the human arm, calculate the distance r(t) between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body. The formula is as follows: r(t)=min{r(t) (1) ,…,r(t) (17) } Where, {r(t) (1) ,…,r(t) (17) } are 17 key points of the human body.

6. The human-machine collaborative control method according to claim 1, characterized in that: In step S5, the specific steps include: S51. Define the relationship between the end workpiece and the spatial safety envelope, namely: When the farthest part of the end workpiece is in the non-collaborative space, the heavy-load industrial robot does not move; when the farthest part of the end workpiece is in the collaborative space but not in the dangerous space, the heavy-load industrial robot and the operator perform human-machine collaboration, and the operator can guide the heavy-load industrial robot to move by applying force to the end workpiece; when the farthest part of the end workpiece suddenly enters the dangerous space, the heavy-load industrial robot immediately stops, stops working, and issues an alarm; S52. Establish a nonlinear coupling relationship between the end contact force and the motion state, that is: When the end workpiece comes into contact with the environment, the dynamic response of the system is described as follows: The virtual stiffness k and the desired motion acceleration speed With position x d are all set to 0, so the above formula becomes: Where, f ext is the interaction force after gravity compensation of the end; m is the virtual mass; b is the virtual damping; k is the virtual stiffness; x represents the acceleration, velocity, and position of the end of the heavy-load industrial robot; x d Represent the desired motion acceleration, velocity and position respectively; S53. Obtain the force signal of the six-dimensional force sensor installed at the end of the heavy-duty industrial robot and calculate the instantaneous average power spectrum P. The formula is as follows: Where P is the instantaneous average power spectrum; T is the sampling period; A is the instantaneous amplitude; S54, based on the instantaneous average power spectrum P, obtain the critical node between vibration and no vibration by adjusting the admittance parameter, and obtain the critical node value by connection; S55. Based on the spatial safety envelope, the one-dimensional admittance control model, and the critical node value, an admittance control model for realizing human-machine collaboration through the admittance parameter change control rule is obtained, namely: When the heavy-loaded industrial robot is in a non-collaborative space and a dangerous space, the heavy-loaded industrial robot does not move; when the heavy-loaded industrial robot enters a dangerous space, the heavy-loaded industrial robot makes an emergency stop to ensure the safety of the operator; when the heavy-loaded industrial robot is in a collaborative space, the heavy-loaded industrial robot collaborates with the operator. The closer the heavy-loaded industrial robot is to the operator, the greater the damping; conversely, the farther the heavy-loaded industrial robot is from the operator, the smaller the damping.

7. The human-machine collaborative control method according to claim 6, characterized in that: The admittance parameter change control rule includes changing and controlling the real-time effective virtual damping b(r) and the real-time effective virtual mass m(r), wherein: The rule for the change of real-time effective virtual damping b(r) is as follows: Where z is a constant parameter, which is used to adjust the smoothness of the change of damping due to distance change; b min and b max are the minimum virtual damping and the maximum virtual damping, respectively, which are used to roughly estimate the effective virtual damping range; r(t) is the distance between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body; r0 is the shortest distance between the coordinates of the farthest part of the end workpiece and the key point closest to the key point of the human body, that is, the distance at which the heavy-loaded industrial robot will immediately enter the dangerous space; ξ is a variable parameter used to fine-tune the damping according to changes in human intentions; Take any position of the change curves O0-O3 of the real-time effective virtual mass m(r) and the real-time effective virtual damping b(r) in the farthest stationary or non-contact state in the collaborative space, and the rule of the change of the real-time effective virtual mass m(r) is as follows: Where b' min is a constant parameter, the virtual damping when the collaborative space is at its farthest distance or without contact; m m is a constant parameter, which serves as the virtual mass when the space distance is farthest and the object is stationary or without contact; β is a constant parameter, which is used to adjust the convergence degree of the acceleration curve O0-O2; α is a constant parameter, which is used to adjust the rise and fall degree of the curve O0-O2 under acceleration; τ is a constant parameter, which is used to adjust the smoothness of the change of the curve O0-O1 under deceleration; σ is a constant parameter, which is used to adjust the rise and fall degree of the curve O0-O1 under deceleration.

8. The human-machine collaborative control method according to claim 7, characterized in that: The change of the real-time effective virtual damping b(r) is adjusted by changing the parameter ξ, realizing the analysis of human intention, and the interaction force f is applied. ext 5 speed industrial robot with heavy loads The direction is judged by the following formula: Combining the human intention analyzed above, the setting rule for the change parameter ξ is as follows: Where η is a constant parameter with a value of 0<η<1, which is used to adjust the variable parameter ξ according to human intention; f max is the maximum interaction force, the purpose of which is to limit the interaction force f ext The size of the interaction force f min .

Images