A Q-learning-based impedance control method for a slag removal robot
Through the reinforcement learning method combined with Q-learning and Maxwell models, the real-time and stability problems of impedance control of the slag-splitting robot are solved, and the rapid estimation of the dynamic model of the slag-splitting robot and the rapid generation of the control torque are realized, which improves the slag-splitting robot. Effect.
Patent Information
- Application Number
- CN202210196429.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-01
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-03-01
AI Technical Summary
During the non-ferrous metal casting process, it is difficult for the slag-removing robot to achieve precise impedance control, resulting in unsatisfactory slag-removing effect. The real-time calculation of the dynamic model of the industrial robot arm is difficult, which affects the stability and real-time nature of impedance control.
Using a reinforcement learning method based on Q-learning, the dynamic model of the slag-slicing robot is quickly estimated, combined with the Maxwell model and time delay estimation, the variable impedance controller is built by online correction of impedance parameters to realize the impedance control of the slag-slicing robot.
The computing efficiency of the slag-stripping robot control is improved, the flexibility of contact force control during movement is ensured, and the rapid estimation of the dynamic model and the rapid generation of control torque are realized.
Smart Images

Figure CN114571444B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of robot control, and in particular to an impedance control method of a slag removal robot based on Q-learning. Background Art
[0002] In the non-ferrous metal casting process, the slag generated by oxidation of the metal surface at high temperature varies in shape and size. Simply using a slag removal robot according to a fixed slag removal process often leads to unsatisfactory slag removal results. At the same time, the slag removal process requires the slag removal tool to move steadily and at a low speed along the surface of the liquid metal. Moving too fast or shaking will cause the liquid metal in the mold to vibrate and easily form flash and ripples. In addition, the depth of the scraper into the liquid metal should not be too deep or too shallow. If it is too deep, it is easy to bring out too much liquid metal, and droplets will be left on the mold or cause the liquid metal to adhere to the side of the mold to form flash. If it is too shallow, the slag removal will be insufficient.
[0003] To solve the above problems, commonly used methods include passive force control method, force / position hybrid control method, impedance control method and force-controlled end effector technology. In order to further improve the humanoid characteristics of the robot arm and improve the control effect of the impedance controller, impedance control based on the dynamic model with torque control as the inner loop and impedance control as the outer loop has become a research hotspot at home and abroad. In order to obtain the torque control quantity of the robot arm, its dynamic model must be calculated. However, due to its higher degree of freedom and more special application background, the dynamic model of the industrial robot arm is difficult to realize real-time calculation. On the other hand, when the industrial robot arm is performing slag removal operations, the end is subjected to time-varying contact force, which places high requirements on the stability and real-time performance of the impedance control model. Summary of the invention
[0004] Since the robot arm model is a highly dynamic, coupled and uncertain nonlinear system, accurate dynamic calculation in the slag removal robot system will obviously consume a lot of resources and it is difficult to ensure the real-time performance of impedance control during movement. The present invention is based on an algorithm that can quickly estimate the dynamic model of the slag removal robot, applies reinforcement learning to the parameter optimization of the inner loop of impedance control, and then realizes the impedance control of the slag removal robot based on the dynamic model, and finally achieves the purpose of using the slag removal robot to complete the slag removal process.
[0005] In order to achieve the above technical purpose, the technical solution of the present invention is:
[0006] A Q-learning-based impedance control method for a slag removal robot comprises the following steps:
[0007] S1: Design an impedance controller based on the expected moving trajectory of the slag removal robot; and collect the actual contact force between the robot arm and the target object;
[0008] S2: Use the Maxwell model to estimate the expected contact force of the slag removal robot, calculate the deviation between the actual contact force and the expected contact force, and input the deviation into the impedance controller of the slag removal robot to realize the correction of the expected trajectory;
[0009] S3: Based on the last and current operating states of each joint of the slag scraping robot, combined with the Q-learning reinforcement learning method, the impedance parameters of the impedance controller in step S1 are corrected online to optimize the control parameters, thereby improving the impedance controller into a variable impedance controller;
[0010] S4: Based on the operating status of each joint of the slag scraping robot at the previous moment, the estimated dynamic model of the slag scraping robot is constructed using time delay estimation, expected speed feedback, and expected position feedback. The angular space trajectory obtained in step S1 is converted into control torque signals of each joint and input into the robot servo controller to realize the slag pushing operation of the slag scraping robot.
[0011] In the impedance control method of a slag scraping robot based on Q-learning, the impedance controller in step S1 is:
[0012] S101: First, a coordinate system is established for the target object and the slag removal robot. The target object coordinate system is established with the geometric center point of the target object in space as the origin. L},by is the pose matrix trajectory of the target object relative to the world coordinate system {O}; the robot coordinate system is the same as the world coordinate system, that is, the center of the robot base coincides with the origin of the world coordinate system;
[0013] S102: The coordinate system established in step S101 is used to calculate the expected trajectory of the center point of the target object moving in Cartesian space. As well as the size L of the target object, the coordinate transformation method is used to obtain the position and posture of the end coordinate system of the slag scraper robot relative to the coordinate system of the target object when the slag scraper robot moves the target object in this posture
[0014]
[0015] where rotx(0)∈R 3×3 Represents a rotation matrix that rotates by 0 angle in the negative direction of the x-axis using the right-hand rule. is the coordinate system of the end of the robot arm A relative to the target object {O L}; use R n Indicates that a vector is an n-dimensional vector, using R n×n Indicates that the matrix is an n×n dimensional matrix;
[0016] S103: obtained from step S102 Using coordinate transformation operations Get the Cartesian space trajectory of the end of the robot relative to the world coordinate system {O} in, Represents the target object coordinate system {O L}Relative to the end coordinate system of the robot arm {O E}, through The matrix is inverted to obtain;
[0017] S104: Using the Rodriguez formula, Transformed into the desired trajectory pose vector representation x d ∈R 3 :
[0018]
[0019] Among them, Rot E ∈R 3×3 is the rotation matrix describing the end posture of the robot arm, P E ∈R 3 is the position vector describing the end position of the robot arm;
[0020] S105: Substitute the result of step S104 and the measured actual contact force into the impedance formula to obtain the impedance controller of the slag scraping robot, which is described as:
[0021]
[0022] in, are the displacement, velocity and acceleration vector of the desired trajectory of the end effector, are the displacement, velocity and acceleration vectors corrected by the impedance controller, They are the displacement deviation, velocity deviation and acceleration deviation that need to be controlled by the impedance controller respectively; M d ∈R 3 is the expected inertia matrix, B d ∈R 3 is the expected damping matrix, K d ∈R 3 is the desired stiffness matrix, F e ∈R 3 and F d ∈R 3 They represent the actual contact force and expected contact force at the end of the robot actuator, respectively.
[0023] In the impedance control method of the slag scraping robot based on Q-learning, in the step S1, the actual contact force is collected by a six-degree-of-freedom force / torque sensor installed at the end of the slag scraping robot.
[0024] The impedance control method of a slag removal robot based on Q-learning is characterized in that step S2 comprises:
[0025] S201: Analyze the surface stress characteristics of molten metal and obtain the stress F based on the Maxwell model d for:
[0026]
[0027] Among them, μ(t) represents the equivalent proportion of the spring model in the Maxwell model, which is called the solidification ratio here, ε represents the total deformation, which is the sum of the deformation of the spring model and the deformation of the damping model, E is the stiffness coefficient of the equivalent spring model of the molten metal, η is the damping coefficient of the equivalent damping model of the molten metal, and d is the derivative symbol; the calculated F d is the expected contact force;
[0028] S202: Shift the phase of the description of the impedance controller to obtain:
[0029]
[0030] in, It is the trajectory acceleration control quantity of the Cartesian space at the end of the slag removal robot corrected by the impedance controller;
[0031] S203: Using the derivative of the robot Jacobi formula, we have:
[0032]
[0033] Among them, J, are the Jacobian matrix given for the robot system and the derivative of the Jacobian matrix, is the joint speed control quantity input to the robot by the system; through the above formula, the angular acceleration control quantity of each joint of the slag scraping robot after the impedance controller is corrected is obtained
[0034] The impedance control method of a slag removal robot based on Q-learning is characterized in that step S3 comprises:
[0035] S301: Defining negative reward function based on reinforcement learning for:
[0036]
[0037] Among them, Q1, Q2, Q3 are positive definite matrices used to define the reward coefficient; u r Defined as control input;
[0038] S302: Define the action value function, i.e., Q function, through the continuous-time Bellman equation and Euler formula. The recursive relation is expressed as:
[0039] Q(X r (k),u r (k)) = r(X r (k),u r (k))·Δt+Q(X r (k+1),h(X r (k+1)))
[0040] Among them, k represents the current state, and k+1 represents the next state; The vector representing the end displacement and velocity deviation, u r =h(X r ) is the control strategy;
[0041] S303: Using the linear discrete time state equation and the value function approximation idea, the Q function is simplified to:
[0042]
[0043] in, Represents tensor operations on vectors. represents tensor product; A represents parameter set, superscript T represents matrix transpose, A=(A xx A xu A uu ) T , where A xx ,A xu ,A uu Respectively The corresponding parameter results;
[0044] S304: construct an error function ef(k) for optimizing parameter set A:
[0045]
[0046] Thus, the optimization problem of A is transformed into the minimum value problem of ef(k);
[0047] S305: Obtain the optimal parameter set A through the solution of the stochastic gradient method * , where the superscript * indicates optimal; at this time, the Q function reaches its minimum and the optimal output By solving the following formula to obtain:
[0048]
[0049] in Indicates the partial derivative, we have:
[0050]
[0051] in, It has the same form as a mechanical spring damping system, namely:
[0052]
[0053] The final optimal parameter set always maintains A xx , A xu and A uu is diagonal, positive definite, and bounded, so that and Positive definite and bounded, so that the optimal parameter set has a spring-damper form and the system remains stable.
[0054] The impedance control method of a slag removal robot based on Q-learning is characterized in that step S4 comprises the following steps:
[0055] S401: The result obtained in step S203 is Integrate to get the corrected robot arm angular velocity control value Compare it with the actual angular velocity value at that moment Difference and multiply by gain factor Γ v ∈R 6×6 That is, the expected speed feedback value is obtained Right now:
[0056]
[0057]
[0058] S402: The result obtained in step S401 is Integrate to get the corrected robot arm angle control value q ideal , and compare it with the actual angle value q at that moment real Difference and multiply by gain factor Γ p ∈R 6×6 Get the expected position feedback value Right now:
[0059]
[0060]
[0061] S403: Assume that the sampling period of the system is L, and for the current time t, record the control torque τ of each joint at the previous sampling time (t-L) And the actual angular acceleration of each joint under the control torque The robot arm dynamics time delay estimation model combining the expected velocity feedback and the expected position feedback is obtained as follows:
[0062]
[0063] in, is the estimated value of the inertia matrix, which is an n-dimensional positive constant diagonal matrix; at the same time:
[0064]
[0065] Thus, the value of the control torque τ of the slag scraping robot is obtained.
[0066] An electronic device, comprising:
[0067] one or more processors;
[0068] a storage device for storing one or more programs,
[0069] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.
[0070] A computer readable medium stores a computer program, which implements the above method when executed by a processor.
[0071] The technical effect of the present invention is that, in view of the working environment and actual needs of the slag scraping robot, an impedance controller based on the outer loop is built by combining the Maxwell model and the Q-learning reinforcement learning method. On this basis, the present invention fully considers the linear and nonlinear characteristics of the dynamic model of the slag scraping robot, estimates the linear part in the dynamic model through time delay estimation, and compensates for the remaining nonlinear part of the dynamic model with the help of expected speed feedback and expected position feedback, thereby obtaining the control torque of each joint, and building a motion controller with dynamic control as the inner loop. The control method combining the two not only ensures the flexibility of contact force control during the movement of the slag scraping robot, but also can realize the rapid estimation of the mechanical arm dynamic model and the rapid generation of control torque, thereby improving the computational efficiency of the slag scraping robot control.
[0072] The present invention will be further described below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] Figure 1 It is a schematic flow chart of the control method of the slag removal robot according to the present invention.
[0074] Figure 2 It is a schematic diagram of the contact force of the end effector of the slag removal robot of the present invention.
[0075] Figure 3a) is a simulation effect diagram of the slag scraping robot of the present invention; b) is a detailed diagram of the simulation trajectory of the slag scraping robot of the present invention.
[0076] Figure 4 a) is a comparison chart of the average motion error of the slag removal robot during its motion process; b) is a comparison chart of the single operation time of the slag removal robot during its motion process.
[0077] Figure 5 The following is a comparison chart of the actual trajectories of the three methods. DETAILED DESCRIPTION
[0078] See attached Figure 1 As shown, with Figure 3 Taking the slag-removing robot system composed of a six-degree-of-freedom Rokae mechanical arm as an example, the impedance control method of the slag-removing robot based on Q-learning described in this example includes the following steps:
[0079] S1: Design an impedance controller based on the expected moving trajectory of the slag removal robot; and collect the actual contact force between the robot arm and the target object.
[0080] The impedance controller is established by the following steps:
[0081] S101: First, a coordinate system is established for the target object and the slag removal robot. The target object coordinate system is established with the geometric center point of the target object in space as the origin. L},by is the pose matrix trajectory of the target object relative to the world coordinate system {O}; the robot coordinate system is the same as the world coordinate system, that is, the center of the robot base coincides with the origin of the world coordinate system;
[0082] S102: The coordinate system established in step S101 is used to calculate the expected trajectory of the center point of the target object moving in Cartesian space. As well as the size L of the target object, the coordinate transformation method is used to obtain the position and posture of the end coordinate system of the slag scraper robot relative to the coordinate system of the target object when the slag scraper robot moves the target object in this posture
[0083]
[0084] where rotx(0)∈R 3×3 Represents a rotation matrix that rotates by 0 angle in the negative direction of the x-axis using the right-hand rule. is the coordinate system of the end of the robot arm A relative to the target object {O L}; use R n Indicates that a vector is an n-dimensional vector, using R n×n Indicates that the matrix is an n×n dimensional matrix;
[0085] S103: obtained from step S102 Using coordinate transformation operations Get the Cartesian space trajectory of the end of the robot relative to the world coordinate system {O} in, Represents the target object coordinate system {O L}Relative to the end coordinate system of the robot arm {O E}, through The matrix is inverted to obtain;
[0086] S104: Using the Rodriguez formula, Transformed into the desired trajectory pose vector representation x d ∈R 3 :
[0087]
[0088] Among them, Rot E ∈R 3×3 is the rotation matrix describing the end posture of the robot arm, P E ∈R 3 is the position vector describing the end position of the robot arm;
[0089] S105: Substitute the result of step S104 and the measured actual contact force into the impedance formula to obtain the impedance controller of the slag scraping robot, which is described as:
[0090]
[0091] in, are the displacement, velocity and acceleration vector of the desired trajectory of the end effector, are the displacement, velocity and acceleration vectors corrected by the impedance controller, They are the displacement deviation, velocity deviation and acceleration deviation that need to be controlled by the impedance controller respectively; M d ∈R 3 is the expected inertia matrix, B d ∈R 3 is the expected damping matrix, K d ∈R 3 is the desired stiffness matrix, F e ∈R 3 and F d ∈R 3 They represent the actual contact force and expected contact force at the end of the robot actuator, respectively.
[0092] S2: The Maxwell model is used to estimate the expected contact force of the slag scraping robot, and the deviation between the actual contact force and the expected contact force is calculated. The deviation is input into the impedance controller of the slag scraping robot to realize the correction of the expected trajectory.
[0093] Specifically, step S2 includes:
[0094] S201: Analyze the surface stress characteristics of molten metal and obtain the stress F based on the Maxwell model d for:
[0095]
[0096] Among them, μ(t) represents the equivalent proportion of the spring model in the Maxwell model, which is called the solidification ratio here, ε represents the total deformation, which is the sum of the deformation of the spring model and the deformation of the damping model, E is the stiffness coefficient of the equivalent spring model of the molten metal, η is the damping coefficient of the equivalent damping model of the molten metal, and d is the derivative symbol; the calculated F d is the expected contact force;
[0097] S202: Shift the phase of the description of the impedance controller to obtain:
[0098]
[0099] in, It is the trajectory acceleration control quantity of the Cartesian space at the end of the slag removal robot corrected by the impedance controller;
[0100] S203: Using the derivative of the robot Jacobi formula, we have:
[0101]
[0102] Among them, J, are the Jacobian matrix given for the robot system and the derivative of the Jacobian matrix, is the joint speed control quantity input to the robot by the system; through the above formula, the angular acceleration control quantity of each joint of the slag scraping robot after the impedance controller is corrected is obtained This enables correction of the desired trajectory.
[0103] S3: Based on the last and current operating states of each joint of the slag scraping robot, combined with the Q-learning reinforcement learning method, the impedance parameters of the impedance controller in step S1 are corrected online to optimize the control parameters, thereby improving the impedance controller into a variable impedance controller;
[0104] Wherein step S3 specifically comprises:
[0105] S301: Defining negative reward function based on reinforcement learning for:
[0106]
[0107] Among them, Q1, Q2, Q3 are positive definite matrices used to define the reward coefficient; u r Defined as control input;
[0108] S302: Define the action value function, i.e., Q function, through the continuous-time Bellman equation and Euler formula. The recursive relation is expressed as:
[0109] Q(X r (k),u r (k)) = r(X r (k),u r (k))·Δt+Q(X r (k+1),h(X r (k+1)))
[0110] Among them, k represents the current state, and k+1 represents the next state; The vector representing the end displacement and velocity deviation, u r =h(X r ) is the control strategy;
[0111] S303: Using the linear discrete time state equation and the value function approximation idea, the Q function is simplified to:
[0112]
[0113] in, Represents tensor operations on vectors. represents tensor product; A represents parameter set, superscript T represents matrix transpose, A=(A xx A xu A uu ) T , where A xx ,A xu ,A uu Respectively The corresponding parameter results;
[0114] S304: construct an error function ef(k) for optimizing parameter set A:
[0115]
[0116] Thus, the optimization problem of A is transformed into the minimum value problem of ef(k);
[0117] S305: Obtain the optimal parameter set A through the solution of the stochastic gradient method * , where the superscript * indicates optimal; at this time, the Q function reaches its minimum and the optimal output By solving the following formula to obtain:
[0118]
[0119] in It means to find partial derivatives, we have:
[0120]
[0121] in, It has the same form as a mechanical spring damping system, namely:
[0122]
[0123] Based on this, in the optimization and design of the parameter set, on the one hand, it is necessary to ensure that the final optimal parameter set has a spring damping form, and on the other hand, it is necessary to ensure the stability of the system. In order to achieve the above purpose, according to the basic theory of impedance control, it is necessary to ensure and Positive definite and bounded. Then further, in order to meet the above conditions and simplify the reinforcement learning parameter set, it is necessary to always keep A xx , A xu and A uu Diagonal, positive definite and bounded. Through this setting method, the process of using Q-learning to complete the optimal online correction of impedance parameters is realized.
[0124] S4: Based on the operating status of each joint of the slag scraping robot at the previous moment, the estimated dynamic model of the slag scraping robot is constructed using time delay estimation, expected speed feedback, and expected position feedback. The angular space trajectory obtained in step S1 is converted into control torque signals of each joint and input into the robot servo controller to realize the slag pushing operation of the slag scraping robot.
[0125] Therefore, step S4 includes:
[0126] S401: The result obtained in step S203 is Integrate to get the corrected robot arm angular velocity control value Compare it with the actual angular velocity value at that moment Difference and multiply by gain factor Γ v ∈R 6×6 (The gain coefficient is set by the user) to get the expected speed feedback value Right now:
[0127]
[0128]
[0129] S402: The result obtained in step S401 is Integrate to get the corrected robot arm angle control value q ideal , and compare it with the actual angle value q at that moment real Difference and multiply by gain factor Γ p ∈R 6×6 Get the expected position feedback value Right now:
[0130]
[0131]
[0132] S403: Assume that the sampling period of the system is L, and for the current time t, record the control torque τ of each joint at the previous sampling time (t-L) And the actual angular acceleration of each joint under the control torque The robot arm dynamics time delay estimation model combining the expected velocity feedback and the expected position feedback is obtained as follows:
[0133]
[0134] in, is the estimated value of the inertia matrix, which is an n-dimensional positive constant diagonal matrix; at the same time:
[0135]
[0136] Thus, the value of the control torque τ of the slag scraping robot is obtained.
[0137] So far, the technical solution of the present invention has been described in conjunction with the accompanying drawings. In this example, the trajectory of the slag removal process is first calculated by the kinematics of the slag removal robot arm, and the information of the actual contact force is sensed by the force sensor. After that, the online optimization of the impedance parameters is completed in real time through the variable impedance controller based on Q-learning. Finally, referring to the expected position feedback and expected speed feedback, and using the time delay estimation model to construct an estimated slag removal robot dynamics model, and based on the comprehensive estimation of the previous content, the control torque corresponding to the slag removal robot reaching the expected posture is estimated.
[0138] According to an embodiment of the present invention, the present invention also provides an electronic device and a computer-readable medium.
[0139] Electronic equipment includes:
[0140] one or more processors;
[0141] a storage device for storing one or more programs,
[0142] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.
[0143] In specific use, users can interact with servers, which are also electronic devices, through electronic devices that are terminal devices and based on the network to achieve functions such as receiving or sending messages. Terminal devices are generally various electronic devices equipped with display devices and used based on human-computer interfaces, including but not limited to smart phones, tablet computers, laptops and desktop computers. Various specific application software can be installed on the terminal devices as needed, including but not limited to web browser software, instant messaging software, social platform software, shopping software, etc. The server is a network service end used to provide various services.
[0144] The method provided in this embodiment is generally executed by a server. In actual use, if necessary conditions are met, the terminal device can also directly execute this method.
[0145] Similarly, the computer-readable medium of the present invention stores a computer program thereon, and when the program is executed by a processor, the method of the embodiment of the present invention is implemented.
Claims
1. A Q-learning-based impedance control method for a slag removal robot. It is characterized in that The following steps are involved: S1: Design an impedance controller based on the expected moving trajectory of the slag removal robot; and collect the actual contact force between the robot arm and the target object; S2: Use the Maxwell model to estimate the expected contact force of the slag removal robot, calculate the deviation between the actual contact force and the expected contact force, and input the deviation into the impedance controller of the slag removal robot to realize the correction of the expected trajectory; S3: Based on the last and current operating states of each joint of the slag scraping robot, combined with the Q-learning reinforcement learning method, the impedance parameters of the impedance controller in step S1 are corrected online to optimize the control parameters, thereby improving the impedance controller into a variable impedance controller; S4: Based on the operation status of each joint of the slag scraping robot at the previous moment, the estimated dynamic model of the slag scraping robot is constructed by using time delay estimation, expected speed feedback, and expected position feedback. The angle space trajectory obtained in step S1 is converted into the control torque signal of each joint and input into the robot servo controller to realize the slag pushing operation of the slag scraping robot. The step S3 comprises: S301: Defining negative reward function based on reinforcement learning for: Among them, Q 1 , Q 2 , Q 3 is a positive definite matrix, used to define the reward coefficient; u r Defined as control input; S302: Define the action value function, i.e., Q function, through the continuous-time Bellman equation and Euler formula. The recursive relation is expressed as: Q(X r (k),u r (k))=r(X r (k),u r (k))·Δt+Q(X r (k+1),h(X r (k+1))) Among them, k represents the current state, and k+1 represents the next state; The vector representing the end displacement and velocity deviation, u r =h(X r ) is the control strategy; S303: Using the linear discrete time state equation and the value function approximation idea, the Q function is simplified to: in, Represents tensor operations on vectors. represents tensor product; A represents parameter set, superscript T represents matrix transpose, A=(A xx A xu A uu ) T , where A xx ,A xu ,A uu Respectively The corresponding parameter results; S304: construct an error function ef(k) for optimizing parameter set A: Thus, the optimization problem of A is transformed into the minimum value problem of ef(k); S305: Obtain the optimal parameter set A through the solution of the stochastic gradient method * , where the superscript * indicates optimal; at this time, the Q function reaches its minimum and the optimal output By solving the following equation to obtain: in It means to find partial derivatives, we have: in, It has the same form as a mechanical spring damping system, namely: The final optimal parameter set always maintains A xx , A xu and A uu is diagonal, positive definite, and bounded, so that and Positive definite and bounded, so that the optimal parameter set has a spring-damper form and the system remains stable.
2. According to the Q-learning-based impedance control method of a slag removal robot according to claim 1, It is characterized in that The impedance controller in step S1 is: S101: First, a coordinate system is established for the target object and the slag removal robot. The target object coordinate system is established with the geometric center point of the target object in space as the origin. L },by is the pose matrix trajectory of the target object relative to the world coordinate system {O}; the robot coordinate system is the same as the world coordinate system, that is, the center of the robot base coincides with the origin of the world coordinate system; S102: The coordinate system established in step S101 is used to calculate the expected trajectory of the center point of the target object moving in Cartesian space. As well as the size L of the target object, the coordinate transformation method is used to obtain the position and posture of the end coordinate system of the slag scraper robot relative to the coordinate system of the target object when the slag scraper robot moves the target object in this posture where rotx(0)∈R 3×3 Represents a rotation matrix that rotates by 0 angle in the negative direction of the x-axis using the right-hand rule. is the coordinate system of the end of the robot arm A relative to the target object {O L }; use R n Indicates that a vector is an n-dimensional vector, using R n×n Indicates that the matrix is an n×n dimensional matrix; S103: obtained from step S102 Using coordinate transformation operations Get the Cartesian space trajectory of the end of the robot relative to the world coordinate system {O} in, Represents the target object coordinate system {O L }Relative to the end coordinate system of the robot arm {O E }, through The matrix is inverted to obtain; S104: Using the Rodriguez formula, Transformed into the desired trajectory pose vector representation x d ∈R 3 : Among them, Rot E ∈R 3×3 is the rotation matrix describing the end posture of the robot arm, P E ∈R 3 is the position vector describing the end position of the robot arm; S105: Substitute the result of step S104 and the measured actual contact force into the impedance formula to obtain the impedance controller of the slag scraping robot, which is described as: in, are the displacement, velocity and acceleration vector of the desired trajectory of the end effector, are the displacement, velocity and acceleration vectors corrected by the impedance controller, They are the displacement deviation, velocity deviation and acceleration deviation that need to be controlled by the impedance controller respectively; M d ∈R 3 is the expected inertia matrix, B d ∈R 3 is the desired damping matrix, K d ∈R 3 is the desired stiffness matrix, F e ∈R 3 and F d ∈R 3 They represent the actual contact force and expected contact force at the end of the robot actuator, respectively.
3. According to the Q-learning-based impedance control method of a slag removal robot according to claim 1, It is characterized in that In the step S1, the actual contact force is collected by a six-degree-of-freedom force / torque sensor installed at the end of the slag scraping robot.
4. According to claim 2, a Q-learning-based impedance control method for a slag removal robot, It is characterized in that Step S2 includes: S201: Analyze the surface stress characteristics of molten metal and obtain the stress F based on the Maxwell model d for: Among them, μ(t) represents the equivalent proportion of the spring model in the Maxwell model, which is called the solidification ratio here, ε represents the total deformation, which is the sum of the deformation of the spring model and the deformation of the damping model, E is the stiffness coefficient of the equivalent spring model of the molten metal, η is the damping coefficient of the equivalent damping model of the molten metal, and d is the derivative symbol; the calculated F d is the expected contact force; S202: Shift the phase of the description of the impedance controller to obtain: in, It is the trajectory acceleration control quantity of the Cartesian space at the end of the slag removal robot corrected by the impedance controller; S203: Using the derivative of the robot Jacobi formula, we have: Among them, J, are the Jacobian matrix given for the robot system and the derivative of the Jacobian matrix, is the joint speed control quantity input to the robot by the system; through the above formula, the angular acceleration control quantity of each joint of the slag scraping robot after the impedance controller is corrected is obtained 5. According to the Q-learning-based impedance control method of a slag removal robot according to claim 1, It is characterized in that The step S4 comprises the following steps: S401: The result obtained in step S203 is Integrate to get the corrected robot arm angular velocity control value Compare it with the actual angular velocity value at that moment Difference and multiply by gain factor Γ v ∈R 6×6 That is, the expected speed feedback value is obtained Right now: S402: The result obtained in step S401 is Integrate to get the corrected robot arm angle control value q ideal , and compare it with the actual angle value q at that moment real Difference and multiply by gain factor Γ p ∈R 6×6 Get the expected position feedback value Right now: S403: Assume that the sampling period of the system is L, and for the current time t, record the control torque τ of each joint at the previous sampling time (t-L) And the actual angular acceleration of each joint under the control torque The robot arm dynamics time delay estimation model combining the expected velocity feedback and the expected position feedback is obtained as follows: in, is the estimated value of the inertia matrix, which is an n-dimensional positive constant diagonal matrix; at the same time: Thus, the value of the control torque τ of the slag scraping robot is obtained.
6. An electronic device, It is characterized in that include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.
7. A computer readable medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
High-compliance method for guiding robot to cooperatively work by people
CN109848983A
Double-arm robot cooperative impedance control method based on estimated dynamics model
CN110421547A
Robot trajectory planning method for dynamic deslagging in ingot casting process
CN111014594A
Mobile mechanical arm position / force active disturbance rejection control method based on adaptive algorithm
CN111319036A
Hexapod robot impedance control method based on reinforcement learning
CN112743540A
