Collaborative robot man-machine interaction and anti-collision control method based on admittance

Abstract

The invention provides a collaborative robot man-machine interaction and anti-collision control method and device based on admittance, and relates to the technical field of robot intelligent control. The method comprises the following steps: acquiring control input force; according to the control input force, constructing a relationship between a state variable and control input in the man-machine interaction system; according to the relation, an expected admittance model and a soft saturation function are adopted, and a reference trajectory of the robot end effector is generated; according to the reference trajectory, an Actor-Critic algorithm is adopted, and a reinforcement learning neural network controller based on admittance control is designed; according to the reinforcement learning neural network controller based on admittance control, a control strategy in the man-machine interaction system is adjusted, and an anti-collision strategy in the man-machine interaction system is obtained. According to the invention, the safety and stability of the man-machine interaction system are improved.

Description

Technical Field

[0001] The present invention relates to the field of robot intelligent control technology, and in particular to a collaborative robot human-machine interaction and anti-collision control method and device based on admittance. Background Art

[0002] With the rapid development of robot technology, the market demand for collaborative robots is expanding. Limited by the current level of technological development, robots cannot make intelligent decisions, integrate perception, and operate flexibly like humans, so the current tasks of industrial robots are still relatively simple. Compared with humans, robots have incomparable advantages in accuracy, speed, and repeatability, so they can be responsible for performing tasks that are more difficult, repetitive, and boring for humans, while humans can play their intelligent advantages in perception and decision-making, focusing on processes in the production chain that require high flexibility and flexibility and are currently difficult for robots to achieve. Considering the current technical level and cost considerations, human-machine collaborative operations have become a powerful way to improve the level of automation. In the tasks of human-machine collaboration, interactive compliance and anti-collision technology are particularly important. Admittance control is currently the main way to achieve compliance control. This method adjusts the robot position and contact force by keeping the robot end position and contact force in the desired admittance relationship, adjusting the ideal admittance coefficient to adjust the robot position and contact force, so that the robot has active flexibility and can effectively limit the interactive force. Compliant interactive robots can better integrate into the human working environment and improve the naturalness and fluency of collaboration. At the same time, anti-collision technology is the key to ensuring the safety of human-machine collaboration. Compared with traditional industrial robots, collaborative robots suitable for human-machine interaction are not only different in appearance and mechanical structure design, but more importantly, how to use control algorithms to enable robots to have safe, natural and reliable human-machine interaction capabilities.

[0003] Ideally, when the external interactive environment information is precisely known, traditional admittance control can achieve relatively ideal control performance. However, in unstructured environments and actual human-machine interaction scenarios, due to the complexity and uncertainty of the environment, it is often difficult to obtain accurate environmental information and state feedback. Fixed admittance control parameters cannot adapt to the dynamic changes of the outside world, resulting in traditional admittance control often showing poor control performance. For example, in tasks such as human-machine collaborative assembly, traditional admittance control has some limitations in its adaptability to unknown constraints. Since assembly tasks usually involve precise component placement and adaptation to unknown environments, traditional admittance control may not be able to effectively handle end-point pose errors and component placement accuracy issues. The above problems may cause the robot to be unable to accurately predict and adapt to environmental changes when performing tasks, thereby affecting the quality and efficiency of task completion. In addition, in real environments, clearly defined position constraint boundaries are necessary to ensure the safety of operators and avoid collisions. Although traditional admittance control performs well in handling high-frequency, small-amplitude noise, it is insufficient in suppressing medium-frequency, high-intensity impacts, which may cause the robot to be unable to simultaneously comply with human traction and resist external collisions during the collaborative process, thereby affecting the fluency and safety of the task. Summary of the invention

[0004] In order to solve the safety problem in the existing human-machine interaction technology in the prior art caused by the end effector of the collaborative robot colliding with surrounding obstacles or human operators, the embodiment of the present invention provides a collaborative robot human-machine interaction and anti-collision control method and device based on admittance. The technical solution is as follows:

[0005] On the one hand, a collaborative robot human-machine interaction and anti-collision control method based on admittance is provided, the method is implemented by a collaborative robot human-machine interaction and anti-collision control device based on admittance, and the method includes:

[0006] S1, obtaining control input force;

[0007] S2. constructing a relationship between a state variable and a control input in a human-computer interaction system according to the control input force;

[0008] S3, generating a reference trajectory of the robot end effector using a desired admittance model and a soft saturation function according to the relationship;

[0009] S4. According to the reference trajectory, an Actor-Critic algorithm is used to design a reinforcement learning neural network controller based on admittance control;

[0010] S5. According to the admittance control-based reinforcement learning neural network controller, the control strategy in the human-computer interaction system is adjusted to obtain an anti-collision strategy in the human-computer interaction system.

[0011] On the other hand, a collaborative robot human-machine interaction and anti-collision control device based on admittance is provided, and the device is applied to a collaborative robot human-machine interaction and anti-collision control method based on admittance, and the device comprises:

[0012] A first acquisition unit, used for acquiring a control input force;

[0013] A construction unit, used to construct a relationship between a state variable and a control input in a human-computer interaction system according to the control input force;

[0014] A generating unit, for generating a reference trajectory of the robot end effector according to the relationship, using a desired admittance model and a soft saturation function;

[0015] A design unit is used to design a reinforcement learning neural network controller based on admittance control by using an Actor-Critic algorithm according to the reference trajectory;

[0016] The second acquisition unit is used to adjust the control strategy in the human-computer interaction system according to the reinforcement learning neural network controller based on admittance control, so as to obtain the anti-collision strategy in the human-computer interaction system.

[0017] The second acquisition unit 350 is used to adjust the control strategy in the human-computer interaction system according to the reinforcement learning neural network controller based on admittance control, so as to obtain the anti-collision strategy in the human-computer interaction system.

[0018] On the other hand, a collaborative robot human-machine interaction and collision avoidance control device based on admittance is provided, and the collaborative robot human-machine interaction and collision avoidance control device based on admittance comprises: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned collaborative robot human-machine interaction and collision avoidance control methods based on admittance is implemented.

[0019] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned admittance-based collaborative robot human-machine interaction and collision avoidance control methods.

[0020] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0021] The embodiment of the present invention first obtains the control input force; according to the control input force, the relationship between the state variable and the control input in the human-computer interaction system is constructed; according to the relationship, the expected admittance model and the soft saturation function are used to generate the reference trajectory of the robot end effector; secondly, according to the reference trajectory, the Actor-Critic algorithm is used to design a reinforcement learning neural network controller based on admittance control; finally, according to the reinforcement learning neural network controller based on admittance control, the control strategy in the human-computer interaction system is adjusted to obtain the anti-collision strategy in the human-computer interaction system. The embodiment of the present invention uses the soft saturation function and the expected admittance model to shape the reference trajectory, which significantly improves the anti-collision ability of the robot in human-computer interaction; when an obstacle is identified, the robot generates a new reference trajectory according to the soft saturation admittance model, which effectively improves the anti-collision effect in the interaction. The embodiment of the present invention defines the Lyapunov function to keep the system output within the constraint, ensure the safe distance between the robot and the obstacle, and improve the safety and stability of the system; based on ROS, a complete robot interaction system platform is built to monitor the state information of each joint of the robot and the information of the robot end effector in real time, and the nodes transmit the relevant data information of the robot through the topic and service communication mechanism. The robot interaction system platform can achieve high-level automation and intelligence, and has good scalability and integration; the embodiment of the present invention takes into account the uncertainty of the human-machine system and proposes a reinforcement learning algorithm to improve the adaptability of the human-machine interaction system. Through a large number of experiments, it is proved that the proposed control method can effectively achieve smooth interaction and collision avoidance in unstructured human-machine interaction scenarios, and has good tracking effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 It is a flow chart of a collaborative robot human-machine interaction and anti-collision control method based on admittance provided by an embodiment of the present invention;

[0024] Figure 2 is a structural diagram of a reinforcement learning framework based on admittance control provided by an embodiment of the present invention;

[0025] Figure 3 It is a schematic diagram of the structure of a human-computer interaction experiment platform provided by an embodiment of the present invention;

[0026] Figure 4 is a human-computer interaction flow chart provided by an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of a non-interactive tracking test provided by an embodiment of the present invention;

[0028] Figure 6 is a performance result diagram of a non-interactive tracking test provided by an embodiment of the present invention;

[0029] Figure 7 is a schematic diagram of a non-interactive anti-collision test provided by an embodiment of the present invention;

[0030] Figure 8 is a schematic diagram of obtaining the obstacle distance on the X-axis provided by an embodiment of the present invention;

[0031] Fig. 9 is a graph showing the performance results of a non-interactive collision avoidance test provided by an embodiment of the present invention;

[0032] Fig.10 is a schematic diagram of a human-machine interaction anti-collision test provided by an embodiment of the present invention;

[0033] Fig.11 is a graph showing the performance results of a human-machine interaction anti-collision test provided by an embodiment of the present invention;

[0034] Fig.12 It is a block diagram of a collaborative robot human-machine interaction and anti-collision control device based on admittance provided by an embodiment of the present invention;

[0035] Fig.13 It is a structural schematic diagram of a collaborative robot human-machine interaction and anti-collision control device based on admittance provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0037] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0038] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0039] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0040] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0041] The embodiment of the present invention provides a collaborative robot human-machine interaction and anti-collision control method based on admittance, which can be implemented by a collaborative robot human-machine interaction and anti-collision control device based on admittance, and the collaborative robot human-machine interaction and anti-collision control device based on admittance can be a terminal or a server. Figure 1 The process flow chart of the human-machine interaction and anti-collision control method of the collaborative robot based on admittance is shown, and the process flow of the method may include the following steps:

[0042] S1. Obtain control input force.

[0043] S2. Based on the control input force, construct the relationship between the state variables and the control input in the human-computer interaction system.

[0044] Optionally, the state variables in the human-computer interaction system include:

[0045] a position of a robot end effector, a velocity of the robot end effector, and an acceleration of the robot end effector;

[0046] The control input in the human-machine interaction system includes: the force detected by the force sensor of the robot's end effector and the control input force.

[0047] Among them, the control input of the human-computer interaction system is the derived control input force, and the control input is converted into torque as input.

[0048] Optionally, the specific implementation of S2 may include S21-S23:

[0049] S21, obtaining the position of the robot end effector, the speed of the robot end effector, and the acceleration of the robot end effector;

[0050] S22, obtaining the force detected by the force sensor of the robot end effector and the control input force;

[0051] S23. Based on the inertia matrix, Coriolis and centripetal matrices and gravity vector of the robot dynamics model, the relationship between the state variables and control inputs in the human-computer interaction system is constructed.

[0052] In a feasible implementation manner, the relationship between the state variables and the control input in the human-computer interaction system can be expressed by the following formula (1):

[0053]

[0054] in, x represents the position of the robot end effector; Indicates the speed of the robot end effector; represents the acceleration of the robot end effector; The inertia matrix representing the robot dynamics model; Coriolis and centripetal matrices representing the robot dynamics model; The gravity vector representing the robot's dynamic model; represents the force detected by the force sensor of the robot end effector; represents the control input force.

[0055] S3. Based on the relationship, the expected admittance model and the soft saturation function are used to generate the reference trajectory of the robot end effector.

[0056] Among them, the expected admittance model describes the dynamic behavior of the robot's end effector when interacting with the environment. It includes parameters such as position, velocity, and acceleration. By adjusting the parameters, the robot's motion trajectory can be optimized, making the robot more compliant and safer when operating at close range with humans.

[0057] Among them, the soft saturation function is an activation function that gradually saturates when the input value is large or small, but the derivative of the soft saturation function will not drop directly to zero like hard saturation, which helps to solve the gradient vanishing problem and allows the gradient to propagate more stably during training.

[0058] Optionally, the specific implementation of S3 may include S31-S32:

[0059] S31, constructing an admittance model of the robot in three dimensions in the task space using an expected admittance model according to the relationship;

[0060] In a feasible implementation, in the task space, the admittance model of the robot in three dimensions can be expressed by the following formula (2):

[0061]

[0062] in, as well as is a positive constant; among them, Indicates an intermediate quantity.

[0063] S32. Based on the admittance model of the robot in three dimensions, a soft saturation function is used to generate a reference trajectory of the robot end effector.

[0064] The soft saturation function can be expressed by the following formula (3):

[0065]

[0066] in, represents the reference trajectory of the robot end effector; δ(0<<δ<1) is a constant that is infinitely close to 1 and satisfies

[0067] Among them, the soft saturation function can ensure that the reference trajectory of the robot end effector has quadratic differentiability and constraint characteristics in the task space, and can ensure that the reference trajectory of the robot end effector will not break through the constraint boundary, so that the robot end effector can achieve collision avoidance.

[0068] S4. According to the reference trajectory, the Actor-Critic algorithm is used to design a reinforcement learning neural network controller based on admittance control.

[0069] Optionally, the Actor-Critic algorithm of S4 includes: evaluating a neural network and executing a neural network; wherein,

[0070] The evaluation neural network includes: an input layer, two hidden layers and an output layer; wherein the activation function Relu is used for calculation between the layers; the evaluation neural network is used to evaluate the state quantity and input quantity of the current human-computer interaction system;

[0071] Among them, the evaluation neural network gives corresponding evaluation according to the state quantity and input quantity of the current human-computer interaction system.

[0072] Execute a neural network, including: input layer, hidden layer, and output layer;

[0073] The execution neural network is used to adjust the weights according to the feedback of the evaluation neural network, and adjust the control strategy through the adjusted weights.

[0074] The input layer is composed of input signals; the hidden layer is composed of radial basis function transformation functions, and the number of nodes can be adjusted according to different task requirements;

[0075] Among them, the executive neural network can adjust the weights from the feedback of the evaluation neural network, and then adjust the control strategy to better approximate the uncertainty in the human-computer interaction system.

[0076] S5. According to the reinforcement learning neural network controller based on admittance control, the control strategy in the human-computer interaction system is adjusted to obtain an anti-collision strategy in the human-computer interaction system.

[0077] Optionally, the specific implementation of S5 may include S51-S53:

[0078] S51. Define the long-term cost function;

[0079] Among them, the long-term cost function can quantitatively reflect the performance of the control strategy, provide evaluation annotations, and guide the neural network to systematically adjust the control strategy and further optimize the performance of the human-computer interaction system.

[0080] In a feasible implementation, a state quantity, an error vector and a virtual control variable are defined; a long-term cost function is defined according to the state quantity, the error vector and the virtual control variable; wherein the long-term cost function can be expressed by the following formula (4):

[0081]

[0082] Among them, J is a constant reflecting future costs; Represents the state quantity; z1=x1-x r ,z2=x2-α represents the error vector; where, Among them, α = [α1, α2, ..., α n ] T is a dummy control variable; ρ(t) is the instantaneous cost function;

[0083] In a feasible implementation, the instantaneous cost function can be expressed by the following formula (5):

[0084]

[0085] in, represents the first positive definite matrix; represents the second positive definite matrix; f represents the derived control input. Since the robot cannot use force as input, the control input is converted into torque as input.

[0086] S52, obtaining the status information of the current human-computer interaction system; inputting the status information of the current human-computer interaction system into the evaluation neural network, evaluating the current strategy through the long-term cost function and the evaluation neural network, and obtaining the evaluation result;

[0087] In a feasible implementation, Figure 2 The schematic diagram of the reinforcement learning framework based on admittance control is shown in the figure; after receiving the state information of the current human-computer interaction system and the action generated by the execution neural network as input, the evaluation neural network outputs a value; wherein the output value reflects the effect of executing the action under a given state; the smaller the output value, the higher the efficiency of the strategy, that is, the better the strategy. This mechanism enables the evaluation neural network to evaluate the current strategy, obtain the evaluation results, and optimize the strategy according to the evaluation results. In the above way, the evaluation neural network can guide the execution neural network to adjust its adaptive rate so as to make better choices in future decisions, thereby improving the overall decision quality.

[0088] The weight update rate of the evaluation neural network can be expressed by the following formula (6):

[0089]

[0090] in, Represents the update rate of the evaluation neural network; γ c >0 indicates the learning rate; where J is a constant used to discount future costs. Uc is the basis function for evaluating the neural network, and Zc is equal to the error vector Z1.

[0091] S53, inputting the state information of the previous human-computer interaction system into the execution neural network, and dynamically adjusting the control input parameters of the execution neural network according to the evaluation result to obtain the anti-collision strategy in the human-computer interaction.

[0092] Among them, the output of the neural network is obtained through nonlinear spatial transformation from the output layer of the neural network to the hidden layer, and linear optimization from the hidden layer to the output layer. The learning speed is accelerated and the local minimum problem of the system is avoided by online adjustment of the connection weights.

[0093] The execution neural network analyzes the state information received from the human-computer interaction system, including the error and the output at the previous moment; based on the current state and the key feedback provided by the evaluation function, the execution neural network dynamically adjusts the control input parameters. The purpose is to continuously optimize the overall performance of the system and ensure that the best working state can be achieved under various operating conditions.

[0094] In a feasible implementation, the initial input force of the human-machine interaction system is obtained according to the robot system model equation, which can be expressed by the following formula (7):

[0095]

[0096] Among them, f1 represents the initial input force of the human-computer interaction system; K2 is the positive definite gain matrix, k c is the distance to the obstacle;

[0097] In a feasible implementation, according to the initial input force of the human-computer interaction system, an execution neural network is used to effectively approximate the uncertainty of the system memory, and the final control input force is obtained by executing the neural network, which can be expressed by the following formula (8):

[0098]

[0099] Where f2 represents the control input force of the human-computer interaction system; is the update rate of the neural network; U a (z a ) represents the basis function vector of the execution network, including represents the estimated value of the execution network weight; W a * Represents the optimal value of the network weights.

[0100] In a feasible implementation, the output result of the evaluation neural network is input into the execution neural network for training. After the training is completed, the weight is output to the execution neural network for updating. The specific process can be expressed by the following formula (9):

[0101]

[0102] Among them, γ a represents the learning rate; Represents the error of executing the neural network estimation; represents the cost estimation error; K P It indicates the impact degree of the evaluation results.

[0103] Optionally, S5 adjusts the control strategy in the human-computer interaction system according to the reinforcement learning neural network controller based on admittance control, obtains the strategy steps for collision avoidance in the human-computer interaction system, and then uses the Lyapunov function to verify the stability of the reinforcement learning neural network controller based on admittance control.

[0104] In a feasible implementation, the Lyapunov function can be expressed by the following formula (10):

[0105]

[0106] in, represents the estimated values ​​of the weights of the neural network, Evaluate the weights of the neural network, represents the independent variable in the variable limit integral function, where a series of derivative changes of the Lyapunov function can be obtained The conclusion can prove that the designed human-computer interaction system is semi-globally consistent and ultimately bounded.

[0107] In a feasible implementation manner, in order to verify the effectiveness and stability of the embodiment of the present invention, a human-computer interaction system platform is built for verification. Figure 3 The figure shows the human-machine interaction experimental platform provided by the present invention. The human-machine interaction platform system consists of a host computer, a Baxter robot and a Kinect camera. The Baxter robot arm adopts a rotating joint to connect the rigid connecting rod, and the joint is connected by an elastic actuator, that is, the load is driven by a motor and a reducer in series with a spring, which plays a role in protecting the person or the robot body under human-machine collaboration or external impact; the Baxter robot head has a 360-degree sonar sensor and a forward camera, and the double mechanical arms each have 7 flexible joints with full degrees of freedom; the Baxter robot has an electric parallel gripper and a mobile base, which can be used as the main experimental platform for completing human-machine interaction control; the Baxter robot adopts an open source robot operating system based on ROS, and runs on the Linux platform. Users can read information or send instructions through the network and the internal computer of the robot. The host computer control system adopts the Ubuntu20.04 system, and the computer control platform and the physical robot are connected to the same IP through USB and router. The entire robot interaction system framework is based on the robot operating system ROS noetic version. The entire algorithm is programmed in Python. ROS processes subtasks in a distributed manner through independently running functional nodes, and completes information transmission between nodes through a variety of communication mechanisms, thereby improving algorithm reusability and writing efficiency.

[0108] In a feasible implementation, the input force f2 is used as the control input to implement three cases to prove the effectiveness, including: a non-interaction tracking test, a non-interaction collision avoidance test, and a human-machine interaction collision avoidance test.

[0109] like Figure 4 The figure shows the human-machine interaction flow chart provided by the embodiment of the present invention; before performing the above three tests, it is necessary to connect the robot to the robot host computer via USB or Ethernet, and start the robot driver to establish a connection between the host computer and the robot body. After the host computer system successfully connects to the robot, Baxter is enabled. After the enabling is successful, the control routine can be run. The Baxter robot cannot directly accept force control input, and the input force f2 is converted into torque input into the 7 joints of Baxter, τ=J T(q)f2, where J(q) is the Jacobian matrix of Baxter obtained through the Pykdl software package in ROS. At the same time, the host computer can also obtain the joint status and real-time position of the robot end effector in real time through ROS.

[0110] In a feasible implementation, the input force f2 is used as the control input, and the starting position of the Baxter robot end effector is set to x(0) = [0.64, -0.85, 0.05] (m). The controller gains are set to k1 = 20, k2 = 18.5, k3 = 28, and K2 = diag [14.5, 24.5, 10.5]. The desired trajectory of the robot end effector in the task space can be expressed by the following formula (11):

[0111]

[0112] in, represents the desired trajectory of the robot end effector in the X direction; represents the desired trajectory of the robot end effector in the Y direction; represents the desired trajectory of the robot end effector in the Z direction;

[0113] In a feasible implementation, Figure 5 Schematic diagram of the non-interaction tracking test of the end effector of the Baxter robot is shown; Figure 6 Shown are the tracking performance and tracking error results for the non-interaction tracking test in the X, Y, and Z axes. Figure 6 It can be seen that the embodiment of the present invention has good tracking performance.

[0114] Among them, Figure 7 The figure shows a schematic diagram of a non-interaction collision test of the Baxter robot end effector; the obstacle distance on the X-axis is Depend on Figure 8 The Kinect camera and quick response code are obtained as shown. Fig. 9 Shown is the performance and tracking error of the non-interaction collision avoidance test in the X, Y, and Z axes, with a collision occurring from t1 to t2. Fig. 9 It can be seen that the embodiment of the present invention has good tracking performance and can avoid collision with obstacles.

[0115] Among them, Fig.10 The anti-collision test of human-machine collaborative cargo handling is shown. The obstacle distance on the X-axis is Depend on Figure 8 The Kinect camera and quick response code are obtained as shown. Fig.11The figure shows the performance and tracking error of the human-machine interaction tracking test on the X, Y, and Z axes, with the human partner intervening in the interaction system from t1 to t2. Fig.11 It can be seen that the embodiment of the present invention has good tracking performance. When the human partner intervenes in the interactive system, the robot can follow the force of the interactor and avoid collision with obstacles. In order to highlight the performance of the present invention, the embodiment of the present invention is compared with the controller design without position constraints and without trajectory shaping under the same conditions. Regardless of whether there is no position constraint or no trajectory shaping, the robot will collide with obstacles when moving toward the boundary.

[0116] The embodiment of the present invention first obtains the control input force; according to the control input force, the relationship between the state variable and the control input in the human-computer interaction system is constructed; according to the relationship, the expected admittance model and the soft saturation function are used to generate the reference trajectory of the robot end effector; secondly, according to the reference trajectory, the Actor-Critic algorithm is used to design a reinforcement learning neural network controller based on admittance control; finally, according to the reinforcement learning neural network controller based on admittance control, the control strategy in the human-computer interaction system is adjusted to obtain the anti-collision strategy in the human-computer interaction system. The embodiment of the present invention uses the soft saturation function and the expected admittance model to shape the reference trajectory, which significantly improves the anti-collision ability of the robot in human-computer interaction; when an obstacle is identified, the robot generates a new reference trajectory according to the soft saturation admittance model, which effectively improves the anti-collision effect in the interaction. The embodiment of the present invention defines the Lyapunov function to keep the system output within the constraint, ensure the safe distance between the robot and the obstacle, and improve the safety and stability of the system; based on ROS, a complete robot interaction system platform is built to monitor the state information of each joint of the robot and the information of the robot end effector in real time, and the nodes transmit the relevant data information of the robot through the topic and service communication mechanism. The robot interaction system platform can achieve high-level automation and intelligence, and has good scalability and integration; the embodiment of the present invention takes into account the uncertainty of the human-machine system and proposes a reinforcement learning algorithm to improve the adaptability of the human-machine interaction system. Through a large number of experiments, it is proved that the proposed control method can effectively achieve smooth interaction and collision avoidance in unstructured human-machine interaction scenarios, and has good tracking effect.

[0117] Fig.12 1 is a block diagram of a collaborative robot human-machine interaction and anti-collision control device based on admittance according to an exemplary embodiment. The device is used in a collaborative robot human-machine interaction and anti-collision control method based on admittance. Fig.12 The device includes a first acquisition unit 310, a construction unit 320, a generation unit 330, a design unit 340 and a second acquisition unit 350. Wherein:

[0118] A first acquisition unit 310, used to acquire a control input force;

[0119] A construction unit 320, configured to construct a relationship between a state variable and a control input in a human-computer interaction system according to the control input force;

[0120] A generating unit 330 is used to generate a reference trajectory of the robot end effector according to the relationship by using a desired admittance model and a soft saturation function;

[0121] A design unit 340 is used to design a reinforcement learning neural network controller based on admittance control using an Actor-Critic algorithm according to the reference trajectory;

[0122] The second acquisition unit 350 is used to adjust the control strategy in the human-computer interaction system according to the reinforcement learning neural network controller based on admittance control, so as to obtain the anti-collision strategy in the human-computer interaction system.

[0123] Optionally, the state variables in the human-computer interaction system include:

[0124] a position of a robot end effector, a velocity of the robot end effector, and an acceleration of the robot end effector;

[0125] The control input in the human-machine interaction system includes: the force detected by the force sensor of the robot end effector and the control input force.

[0126] Optionally, the construction unit 320 is configured to:

[0127] Obtaining the position of the robot end effector, the velocity of the robot end effector, and the acceleration of the robot end effector;

[0128] Obtain the force detected by the force sensor of the robot end effector and the control input force;

[0129] According to the inertia matrix, Coriolis and centripetal matrices and gravity vector of the robot dynamics model, the relationship between the state variables and control inputs in the human-machine interaction system is constructed.

[0130] Optionally, the generating unit 330 is configured to:

[0131] According to the relationship, an expected admittance model is used to construct an admittance model of the robot in three dimensions in the task space;

[0132] According to the admittance model of the robot in three dimensions, a soft saturation function is used to generate the reference trajectory of the robot end effector.

[0133] Optionally, the Actor-Critic algorithm includes: evaluating a neural network and executing a neural network; wherein,

[0134] The evaluation neural network includes: an input layer, two hidden layers and an output layer; wherein the activation function Relu is used for calculation between layers; the evaluation neural network is used to evaluate the state quantity and input quantity of the current human-computer interaction system;

[0135] The execution neural network comprises: an input layer, a hidden layer and an output layer;

[0136] The execution neural network is used to adjust the weights according to the feedback of the evaluation neural network, and adjust the control strategy through the adjusted weights.

[0137] Optionally, the second obtaining unit 350 is configured to:

[0138] Define the long-term cost function;

[0139] Obtain the status information of the current human-computer interaction system; input the status information of the current human-computer interaction system into the evaluation neural network, evaluate the current strategy through the long-term cost function and the evaluation neural network, and obtain the evaluation result;

[0140] The state information of the previous human-computer interaction system is input into the execution neural network. According to the evaluation results, the execution neural network dynamically adjusts the control input parameters to obtain the anti-collision strategy in human-computer interaction.

[0141] Optionally, after adjusting the control strategy in the human-computer interaction system according to the reinforcement learning neural network controller based on admittance control and obtaining the anti-collision strategy steps in the human-computer interaction system, the stability of the reinforcement learning neural network controller based on admittance control is verified using the Lyapunov function.

[0142] The embodiment of the present invention first obtains the control input force; according to the control input force, the relationship between the state variable and the control input in the human-computer interaction system is constructed; according to the relationship, the expected admittance model and the soft saturation function are used to generate the reference trajectory of the robot end effector; secondly, according to the reference trajectory, the Actor-Critic algorithm is used to design a reinforcement learning neural network controller based on admittance control; finally, according to the reinforcement learning neural network controller based on admittance control, the control strategy in the human-computer interaction system is adjusted to obtain the anti-collision strategy in the human-computer interaction system. The embodiment of the present invention uses the soft saturation function and the expected admittance model to shape the reference trajectory, which significantly improves the anti-collision ability of the robot in human-computer interaction; when an obstacle is identified, the robot generates a new reference trajectory according to the soft saturation admittance model, which effectively improves the anti-collision effect in the interaction. The embodiment of the present invention defines the Lyapunov function to keep the system output within the constraint, ensure the safe distance between the robot and the obstacle, and improve the safety and stability of the system; based on ROS, a complete robot interaction system platform is built to monitor the state information of each joint of the robot and the information of the robot end effector in real time, and the nodes transmit the relevant data information of the robot through the topic and service communication mechanism. The robot interaction system platform can achieve high-level automation and intelligence, and has good scalability and integration; the embodiment of the present invention takes into account the uncertainty of the human-machine system and proposes a reinforcement learning algorithm to improve the adaptability of the human-machine interaction system. Through a large number of experiments, it is proved that the proposed control method can effectively achieve smooth interaction and collision avoidance in unstructured human-machine interaction scenarios, and has good tracking effect.

[0143] Fig.13 is a schematic diagram of the structure of a collaborative robot human-machine interaction and anti-collision control device based on admittance provided by an embodiment of the present invention, such as Fig.13 As shown, the human-machine interaction and anti-collision control device of the collaborative robot based on admittance may include the above Fig.12 The admittance-based collaborative robot human-machine interaction and anti-collision control device shown. Optionally, the admittance-based collaborative robot human-machine interaction and anti-collision control device 410 may include a first processor 2001.

[0144] Optionally, the admittance-based collaborative robot human-machine interaction and collision avoidance control device 410 may also include a memory 2002 and a transceiver 2003 .

[0145] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0146] Combine the following Fig.13The components of the admittance-based collaborative robot human-machine interaction and anti-collision control device 410 are specifically introduced:

[0147] The first processor 2001 is the control center of the admittance-based collaborative robot human-machine interaction and anti-collision control device 410, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0148] Optionally, the first processor 2001 can perform various functions of the admittance-based collaborative robot human-machine interaction and collision avoidance control device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0149] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Fig.13 CPU0 and CPU1 are shown in FIG.

[0150] In a specific implementation, as an embodiment, the admittance-based collaborative robot human-machine interaction and collision avoidance control device 410 may also include multiple processors, such as Fig.13 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0151] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0152] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Fig.13 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0153] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0154] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Fig.13 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0155] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and communicate with the anti-collision control device 410 through the interface circuit ( Fig.13 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0156] It should be noted that Fig.13 The structure of the admittance-based collaborative robot human-machine interaction and anti-collision control device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0157] In addition, the technical effects of the admittance-based collaborative robot human-machine interaction and anti-collision control device 410 can refer to the technical effects of the admittance-based collaborative robot human-machine interaction and anti-collision control method described in the above method embodiment, and will not be repeated here.

[0158] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0159] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0160] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0161] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0162] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0163] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0164] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0166] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0169] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0170] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A collaborative robot human-machine interaction and anti-collision control method based on admittance, characterized in that: The method comprises: S1, obtaining control input force; S2. constructing a relationship between a state variable and a control input in a human-computer interaction system according to the control input force; S3, generating a reference trajectory of the robot end effector using a desired admittance model and a soft saturation function according to the relationship; S4. According to the reference trajectory, an Actor-Critic algorithm is used to design a reinforcement learning neural network controller based on admittance control; S5. According to the admittance control-based reinforcement learning neural network controller, the control strategy in the human-computer interaction system is adjusted to obtain an anti-collision strategy in the human-computer interaction system.

2. The method for human-machine interaction and anti-collision control of collaborative robots based on admittance according to claim 1, characterized in that: The state variables in the human-computer interaction system include: a position of a robot end effector, a velocity of the robot end effector, and an acceleration of the robot end effector; The control input in the human-machine interaction system includes: the force detected by the force sensor of the robot end effector and the control input force.

3. The method for human-machine interaction and anti-collision control of collaborative robots based on admittance according to claim 2, characterized in that: The step S2 constructs the relationship between the state variables and the control input in the human-computer interaction system according to the control input force, including: S21, obtaining the position of the robot end effector, the speed of the robot end effector, and the acceleration of the robot end effector; S22, obtaining the force detected by the force sensor of the robot end effector and the control input force; S23. Based on the inertia matrix, Coriolis and centripetal matrices and gravity vector of the robot dynamics model, the relationship between the state variables and control inputs in the human-computer interaction system is constructed.

4. The method for human-machine interaction and anti-collision control of collaborative robots based on admittance according to claim 1, characterized in that: The step S3 generates a reference trajectory of the robot end effector by using an expected admittance model and a soft saturation function according to the relationship, including: S31, constructing an admittance model of the robot in three dimensions in the task space using an expected admittance model according to the relationship; S32. Based on the admittance model of the robot in three dimensions, a soft saturation function is used to generate a reference trajectory of the robot end effector.

5. The method for human-machine interaction and anti-collision control of collaborative robots based on admittance according to claim 1, characterized in that: The Actor-Critic algorithm of S4 includes: evaluating a neural network and executing a neural network; wherein, The evaluation neural network includes: an input layer, two hidden layers and an output layer; wherein the activation function Relu is used for calculation between layers; the evaluation neural network is used to evaluate the state quantity and input quantity of the current human-computer interaction system; The execution neural network comprises: an input layer, a hidden layer and an output layer; The execution neural network is used to adjust the weights according to the feedback of the evaluation neural network, and adjust the control strategy through the adjusted weights.

6. The method for human-machine interaction and anti-collision control of collaborative robots based on admittance according to claim 5, characterized in that: The step S5 adjusts the control strategy in the human-computer interaction system to obtain an anti-collision strategy in the human-computer interaction system, including: S51. Define the long-term cost function; S52, obtaining the status information of the current human-computer interaction system; inputting the status information of the current human-computer interaction system into the evaluation neural network, evaluating the current strategy through the long-term cost function and the evaluation neural network, and obtaining the evaluation result; S53, inputting the state information of the previous human-computer interaction system into the execution neural network, and dynamically adjusting the control input parameters of the execution neural network according to the evaluation result to obtain the anti-collision strategy in the human-computer interaction.

7. The method for human-machine interaction and anti-collision control of collaborative robots based on admittance according to claim 1, characterized in that: In S5, the control strategy in the human-computer interaction system is adjusted according to the reinforcement learning neural network controller based on admittance control, and after obtaining the anti-collision strategy steps in the human-computer interaction system, the stability of the reinforcement learning neural network controller based on admittance control is verified by using the Lyapunov function.

8. A collaborative robot human-machine interaction and anti-collision control device based on admittance, wherein the collaborative robot human-machine interaction and anti-collision control device based on admittance is used to implement the collaborative robot human-machine interaction and anti-collision control method based on admittance as claimed in any one of claims 1 to 7, characterized in that: The device comprises: A first acquisition unit, used for acquiring a control input force; A construction unit, used to construct a relationship between a state variable and a control input in a human-computer interaction system according to the control input force; A generating unit, for generating a reference trajectory of the robot end effector according to the relationship, using a desired admittance model and a soft saturation function; A design unit is used to design a reinforcement learning neural network controller based on admittance control by using an Actor-Critic algorithm according to the reference trajectory; The second acquisition unit is used to adjust the control strategy in the human-computer interaction system according to the reinforcement learning neural network controller based on admittance control to obtain an anti-collision strategy in the human-computer interaction system.

9. A collaborative robot human-machine interaction and anti-collision control device based on admittance, characterized in that: The admittance-based collaborative robot human-machine interaction and anti-collision control device includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images