Robot dexterous arm teleoperation sharing control method and related device

Abstract

The invention discloses a robot dexterous arm teleoperation sharing control method and a related device, and relates to the technical field of robotics.The method comprises the steps that smooth expected pose information and expected angle state information are obtained, and pose information of a mechanical arm of a slave end at the current stage and angle state information and contact force information of a dexterous hand are obtained; based on the expected pose information and the joint angle information of the mechanical arm, the inverse kinematics of the mechanical arm is solved through a nonlinear optimization method, and a joint angle instruction of the mechanical arm is obtained; based on the expected angle state information, the angle state information of the dexterous hand and the contact force information, utilizing a fuzzy logic theory and a dynamic compliance primitive model to carry out adaptive variable impedance control on the dexterous hand to obtain a joint torque instruction of the dexterous hand; and feeding back the contact force information of the dexterous hand in the next stage to a main end operator. The success rate and the working efficiency of the teleoperation robot can be improved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a method for remotely controlling a robot's dexterous arm and a related device. Background Art

[0002] With the rapid development of robotics, robotic systems have become indispensable tools for operations in hazardous environments, extreme conditions, or high-precision environments. Despite significant progress in fully autonomous robotics, ensuring safety, accuracy, and stability remains challenging when handling complex, unstructured tasks. In this context, telerobotics has attracted significant attention due to its ability to combine human decision-making with the advantages of machine execution, offering a viable technical solution for hazardous environments and precision operations.

[0003] Existing teleoperation technologies typically employ data gloves and motion capture devices to remotely control dexterous hands and robotic arms. However, these technologies face numerous challenges in their practical application. For dexterous hands, there are issues such as insufficient immersion and poor operational compliance during grasping. For robotic arm motion control, there are issues such as discontinuous motion trajectories and kinematic singularities, which severely impact operational fluidity and safety. These issues not only increase the physical and mental burden on the operator and easily induce operator fatigue, but also significantly reduce the success rate and efficiency of teleoperation. Consequently, these technologies are hindered from widespread application in real-world operational scenarios. Summary of the Invention

[0004] The purpose of this application is to provide a shared control method and related devices for remote operation of a robot's dexterous arm, which can improve the success rate and operating efficiency of the remotely operated robot.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for remotely controlling a dexterous robot arm, comprising:

[0007] Obtaining smoothed desired posture information and desired angle state information; the desired posture information and the desired angle state information are obtained by smoothing the posture information of the arm and the angle state information of the hand of the master-end operator at the current stage; the posture information includes position information and posture information; the angle state information includes angle position and speed;

[0008] Obtaining the current position information of the slave end's robotic arm, as well as the angle state information and contact force information of the dexterous hand; the contact force information is the contact force information between the dexterous hand and the object, and the contact force information is obtained by the force sensor on the dexterous hand;

[0009] Based on the desired posture information and the posture information of the robotic arm, a nonlinear optimization method is used to solve the inverse kinematics of the robotic arm to obtain a joint angle command of the robotic arm; the joint angle command is used to control the movement of the robotic arm so that the error between the posture information of the robotic arm in the next stage and the desired posture information is within a set range;

[0010] Based on the desired angle state information, the angle state information of the dexterous hand, and the contact force information, the dexterous hand is adaptively controlled by variable impedance using fuzzy logic theory and a dynamic compliance primitive model to obtain joint torque instructions for the dexterous hand; the joint torque instructions are used to control the movement of the dexterous hand so that the error between the angle state information of the dexterous hand and the desired angle state information in the next stage is within a set range, and the contact force information of the dexterous hand in the next stage is obtained;

[0011] The contact force information of the dexterous hand in the next stage is fed back to the master-end operator; the contact force information is used to provide force feedback and vibration tactile feedback to the master-end operator.

[0012] Optionally, obtaining smooth desired pose information and desired angle state information specifically includes:

[0013] The sliding window mean filter is used to smooth the arm posture information and hand angle state information of the master operator in the current stage to obtain the smoothed expected posture information and expected angle state information.

[0014] Optionally, based on the desired posture information and the joint angle information of the robotic arm, a nonlinear optimization method is used to solve the inverse kinematics of the robotic arm to obtain the joint angle instructions of the robotic arm, specifically including:

[0015] Determining constraint parameters; the constraint parameters include inequality constraint functions, equality constraint functions, upper and lower limits of manipulator joint angles, upper and lower limits of joint velocities, and singular configuration avoidance;

[0016] Constructing a multi-objective optimization function; the multi-objective optimization function is a weighted sum of multiple normalized objective functions; the objective function includes end effector position matching, end effector posture matching, and trajectory smoothing;

[0017] Based on the desired posture information, the joint angle information of the robotic arm and the constraint parameters, a nonlinear optimization method is used to solve the multi-objective optimization function to obtain the joint angle instructions of the robotic arm.

[0018] Optionally, the objective function of the end effector position matching is to minimize the Euclidean L2 norm error between the desired position information and the end effector position information of the robotic arm; the end effector position information of the robotic arm is calculated using forward kinematics on the joint angle information of the robotic arm;

[0019] The objective function of the end effector posture is to minimize the size of the rotation vector between the desired posture information and the end effector posture information of the manipulator; the end effector posture information of the manipulator is calculated by using forward kinematics on the joint angle information of the manipulator;

[0020] The objective function of the trajectory smoothing motion is to minimize the joint velocity, joint acceleration, joint jerk and the speed of the end effector position space of the manipulator, as well as the preset speed hard constraint condition for the joint velocity limit.

[0021] Optionally, based on the desired angle state information and the angle state information of the dexterous hand, and based on the desired angle state information, the angle state information of the dexterous hand and the contact force information, adaptive variable impedance control is performed on the dexterous hand using fuzzy logic theory and a dynamic compliance primitive model to obtain joint torque instructions of the dexterous hand, specifically including:

[0022] Based on the contact force information, a derivative of the contact force information with respect to time is calculated;

[0023] The contact force information and its derivative with respect to time are calculated using fuzzy logic theory to obtain a maximum reference grasping force and a maximum reference grasping stiffness;

[0024] Processing the maximum reference grasping force and the maximum reference grasping stiffness using a dynamic compliance primitive model to obtain a stiffness coefficient;

[0025] Based on the expected angle state information, the angle state information of the dexterous hand, the stiffness coefficient of the impedance control and the preset damping coefficient, a joint torque instruction of the dexterous hand is obtained.

[0026] Optionally, the calculation formula of the dynamic compliance primitive model is:

[0027]

[0028] Among them, x represents the stiffness change curve; z represents the change speed of the stiffness change curve; x0 represents the initial stiffness value; x t represents the target stiffness value; k s represents the stiffness coefficient; k d represents the damping coefficient; λ represents the velocity modulation parameter; f(s) represents the nonlinear forcing term; Represents the derivative of the stiffness change curve; The derivative of the rate of change of the stiffness change curve.

[0029] Optionally, the nonlinear forcing term is composed of a normalized linear superposition of multiple nonlinear Gaussian basis functions; the nonlinear forcing term is used to learn a variation curve of the grasping stiffness characteristics of the dexterous hand.

[0030] In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any one of the above-described methods for remotely controlling a dexterous robot arm.

[0031] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods for remote-operation shared control of a dexterous robot arm.

[0032] In a fourth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned methods for remote operation and shared control of a dexterous robot arm.

[0033] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0034] This application provides a shared control method and related device for remote operation of a robotic dexterous arm. By smoothing the arm pose information and hand angle state information of the current master operator, this application can more accurately reflect the master operator's intentions, thereby reducing control errors and improving the response accuracy and smoothness of the robotic arm. Furthermore, by solving the inverse kinematics of the robotic arm using a nonlinear optimization method, it can ensure that the robotic arm generates reasonable joint angle commands based on the desired pose information, allowing the robotic arm to accurately reach the desired spatial position and posture. Simultaneously, based on the desired angle state information and the current angle state information of the dexterous hand, fuzzy logic theory and a dynamic compliance primitive model are used to perform adaptive variable impedance control of the dexterous hand. This allows the dexterous hand's joint torque commands to be dynamically adjusted during contact with an object, thereby more accurately controlling the dexterous hand's force and achieving compliance in grasping the object. Furthermore, the contact force information obtained by the dexterous hand through the force sensor can be transmitted to the master operator in real time and transmitted through force feedback and vibrotactile feedback systems, allowing the operator to perceive the contact state and force of the object, enhancing the immersiveness of the operation and the control accuracy. In summary, the present application can effectively improve the success rate and operating efficiency of the remote-controlled robot by improving the control accuracy, smoothness, flexibility and immersion of the robotic arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 This is a diagram of an application environment for a method for remotely controlling a dexterous robot arm in accordance with an embodiment of the present application;

[0037] Figure 2 A flowchart of a method for remotely controlling a robot's dexterous arm provided in accordance with an embodiment of the present application;

[0038] Figure 3 A schematic diagram of a process for solving inverse kinematics of a robotic arm according to an embodiment of the present application;

[0039] Figure 4 A schematic diagram of a flow chart of adaptive variable impedance control provided in one embodiment of the present application;

[0040] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0042] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0043] The robot dexterous arm teleoperation sharing control method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the desired posture information and the desired angle state information, as well as the posture information of the manipulator of the current stage, the angle state information and contact force information of the dexterous hand to the server 104. Based on the desired posture information and the posture information of the manipulator, the server 104 uses a nonlinear optimization method to solve the inverse kinematics of the manipulator to obtain the joint angle instruction of the manipulator; based on the desired angle state information, the angle state information and contact force information of the dexterous hand, the fuzzy logic theory and the dynamic compliance primitive model are used to perform adaptive variable impedance control on the dexterous hand to obtain the joint torque instruction of the dexterous hand; and the contact force information of the dexterous hand in the next stage is fed back to the master operator. The server 104 can feed back the obtained joint angle instruction of the manipulator, joint torque instruction of the dexterous hand and contact force information to the terminal 102.

[0044] The terminal 102 may be, but is not limited to, a robotic arm, a dexterous hand, a motion capture device, or a force feedback glove. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0045] In an exemplary embodiment, Figure 2 As shown, a method for remote control sharing of a robot's dexterous arm is provided. The method is executed by a computer device, specifically a computer device such as a server. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the process, including the following steps 201 to 205.

[0046] Step 201, obtain smooth expected posture information and expected angle state information; the smoothed expected posture information and expected angle state information are obtained by smoothing the posture information of the arm and the angle state information of the hand of the master operator in the current stage; the posture information includes position information and posture information; the angle state information includes angle position and speed.

[0047] Step 202, obtain the posture information of the robot arm at the current stage, as well as the angle state information and contact force information of the dexterous hand; the contact force information is the contact force information between the dexterous hand and the object, and the contact force information is obtained through the force sensor on the dexterous hand.

[0048] In step 203, based on the desired posture information and the posture information of the robot arm, the nonlinear optimization method is used to solve the inverse kinematics of the robot arm to obtain the joint angle command of the robot arm; the joint angle command is used to control the movement of the robot arm so that the error between the posture information of the robot arm in the next stage and the desired posture information is within a set range.

[0049] In step 204, based on the desired angle state information, the angle state information of the dexterous hand, and the contact force information, the fuzzy logic theory and the dynamic compliance primitive model are used to perform adaptive variable impedance control on the dexterous hand to obtain the joint torque instructions of the dexterous hand; the joint torque instructions are used to control the movement of the dexterous hand so that the error between the angle state information of the dexterous hand and the desired angle state information in the next stage is within a set range, and the contact force information of the next-stage dexterous hand is obtained.

[0050] Step 205 : Feedback the contact force information of the dexterous hand in the next stage to the master operator; the contact force information is used to provide force feedback and vibration tactile feedback to the master operator.

[0051] By implementing steps 201 to 205 above, the present application can more accurately reflect the intention of the master operator by smoothing the arm posture information and hand angle state information of the master operator at the current stage, thereby reducing control errors and improving the response accuracy and smoothness of the manipulator. Moreover, by solving the inverse kinematics of the manipulator using a nonlinear optimization method, it can be ensured that the manipulator generates reasonable joint angle commands based on the desired posture information, so that the manipulator can accurately reach the desired spatial position and posture. At the same time, based on the desired angle state information and the current angle state information of the dexterous hand, the fuzzy logic theory and dynamic compliance primitive model are used to perform adaptive variable impedance control on the dexterous hand. The joint torque commands of the dexterous hand can be dynamically adjusted during contact with the object, thereby more accurately controlling the force of the dexterous hand and achieving compliance in grasping the object. In addition, the contact force information obtained by the dexterous hand through the force sensor can be transmitted to the master operator in real time and transmitted through the force feedback and vibration tactile feedback system, so that the operator can perceive the contact state and force of the object, enhancing the immersiveness and control accuracy of the operation. In summary, the present application can effectively improve the success rate and operating efficiency of the remote-controlled robot by improving the control accuracy, smoothness, flexibility and immersion of the robotic arm.

[0052] Furthermore, obtaining smooth desired posture information and desired angle state information in step 201 specifically includes:

[0053] The motion capture device acquires the arm pose information of the master operator in real time, and the force feedback data glove acquires the hand angle state information of the master operator in real time. The arm pose information and hand angle state information are smoothed to obtain smooth desired pose information and desired angle state information, which are then transmitted in incremental form to the slave robot arm and dexterous hand via the TCP / IP protocol in the local area network. The sliding window mean filter is used to smooth the arm pose information and hand angle state information to obtain smooth desired pose information and desired angle state information, specifically:

[0054]

[0055] Among them, q d (k) is the expected posture information and angle state information after filtering at the k-th sampling moment, that is, the smoothed expected posture information and expected angle state information; q initial is the arm posture information and hand angle status information before filtering; h is the data at the current sampling time and the next two sampling times.

[0056] Furthermore, in step 203, based on the desired posture information and the joint angle information of the manipulator, the nonlinear optimization method is used to solve the inverse kinematics of the manipulator to obtain the joint angle instructions of the manipulator, which specifically includes:

[0057] The inverse kinematics solution of the robotic arm is expressed as the following nonlinear optimization problem:

[0058]

[0059] Where Θ is a set of joint angles; st represents the constraint parameters; c in (Θ) is the inequality constraint function; c e (Θ) is the equality constraint function; u i is the upper limit of the i-th robotic arm joint; l i is the lower limit of the i-th manipulator joint; f(Θ) represents the multi-objective optimization function, which is expressed as the weighted sum of multiple normalized objective functions, such as end effector position matching, end effector posture matching, and trajectory smoothing. The specific process is as follows Figure 3 As shown, it is formulated as follows:

[0060]

[0061] Among them, w η represents the weight of the nth target; f η (Θ,Ω η ) represents the objective function after normalization of the nth target; Ω ηis the model parameter used to construct a specific loss function; k represents the total number of targets. In order to combine various objective functions on the same scale, each objective function needs to be normalized. The formula for the normalized objective function is:

[0062]

[0063] Among them, the scalar values ​​n, s, c, r constitute the model parameter set Ω, which serves as the loss function parameter of a single target; n∈{0,1}, 0 means that the Gaussian distribution is positive, 1 means that the Gaussian distribution is negative, the negative Gaussian area is the area of ​​high reward, and the optimization target will be away from the positive Gaussian area with high cost; the parameter s is used to shift the function horizontally; the parameter c is used to adjust the width of the Gaussian reward area; the parameter r is used to adjust the transition slope between the polynomial area and the Gaussian area, a higher value means that the function enters the Gaussian area more steeply, and a lower value means that the function enters the Gaussian area more flatly; the scalar function χ η (Θ) Assigns a numerical value to the current robot arm joint configuration, which will be used as input to the loss function.

[0064] The first goal of solving the inverse kinematics of the manipulator is to match the position of the manipulator's end effector to the desired position. The optimization goal is to minimize the Euclidean L2 norm error between the end effector position and the desired position under a given manipulator joint configuration. The objective function is formulated as follows:

[0065] χ P (Θ)=||p g -FK(Θ)||2;

[0066] Among them, χ P (Θ) is the objective function of the end effector position matching; FK(Θ) represents the end effector position of the manipulator obtained by forward kinematics of the joint angle information of the manipulator given the joint configuration of the manipulator; p g is the desired position.

[0067] The second goal of solving the inverse kinematics of the manipulator is to match the posture of the manipulator's end effector to the desired posture. The optimization goal is to minimize the size of the rotation vector between the posture of the end effector and the desired posture under the given manipulator joint configuration, which is expressed as The formula of the objective function term is as follows:

[0068]

[0069] Among them, χ o (Θ) represents the objective function of the end-effector posture matching; disp(·) represents the rotation interpolation function between two quaternions; q gExpressing an expectation posture; represents the pose of the end effector of the manipulator, calculated by forward kinematics based on the joint angle information of the manipulator, given the joint configuration of the manipulator. q[·] represents the conversion of the rotation matrix into a quaternion; Represents quaternion multiplication operation, log(·) is the logarithmic mapping of quaternion; q1 and q2 represent two quaternions; q1 -1 Represents the inverse of a quaternion.

[0070] The third goal of solving the inverse kinematics of the manipulator is to achieve smooth motion of the manipulator trajectory and avoid discontinuities in the joint space. To achieve this goal, four smoothing objectives and one velocity hard constraint are used. The first three smoothing objectives are to minimize the joint velocity, joint acceleration, and joint jerk, respectively. The formulas are as follows:

[0071]

[0072] Among them, χ v (Θ) represents the objective function of minimizing joint velocity; represents the L2 norm of the joint velocity; χ a (Θ) represents the objective function of minimizing joint acceleration; represents the L2 norm of joint acceleration; χ j (Θ) represents the objective function of minimizing joint acceleration; Represents the L2 norm of the joint acceleration; represents the joint velocity; represents the joint acceleration; Indicates the joint jerk.

[0073] The fourth smoothing objective is to minimize the velocity of the end effector position space of the manipulator. This objective helps reduce the jitter of the end effector when performing fine motion tasks. The formula is as follows:

[0074]

[0075] Among them, χ e (Θ) represents the objective function of the position space velocity of the end effector of the manipulator; Indicates the position space velocity of the robot end effector.

[0076] In addition, a hard constraint of speed limit is added to each joint. The formula of speed hard constraint is as follows:

[0077]

[0078] in, Inequality constraints representing joint velocities; represents the velocity of the i-th joint; v i represents the velocity limit of the i-th joint; g represents the degree of freedom of the robot.

[0079] The constraint parameters also include singular configuration avoidance. The singular configuration avoidance constraint is to exclude inferior configurations whose Jacobi matrix condition number is lower than the preset configuration hard constraint condition; because when the manipulator is singular, the Jacobi matrix is ​​not full rank (the Jacobi matrix is ​​determined based on the joint velocity). By performing singular value decomposition on the Jacobi matrix, the ratio between the minimum singular value and the maximum singular value is calculated, and the ratio is recorded as c = σ min / σ max , this value is called the Jacobi matrix condition number. When the Jacobi matrix condition number is very small, it means that the Jacobi matrix is ​​about to lose its full rank. The Jacobi matrix condition number is used to approximately represent the distance between the current configuration and the singular point. In the preprocessing stage, 500,000 robot configurations are randomly sampled and the mean value of all condition numbers μ is calculated. c and standard deviation std c Based on the assumption that the condition number random variable approximately obeys the normal distribution, hard constraints are set in the optimization process to exclude matrix condition numbers lower than μ c -2*std c Inferior configuration.

[0080] Furthermore, in step 204, based on the desired angle state information, the angle state information of the dexterous hand, and the contact force information, the dexterous hand is adaptively controlled by variable impedance using fuzzy logic theory and a dynamic compliance primitive model to obtain the joint torque instructions of the dexterous hand, which specifically includes:

[0081] Step 2041: Based on the contact force information, the derivative of the contact force information with respect to time is calculated.

[0082] Step 2042 , using fuzzy logic theory to calculate the contact force information and its derivative with respect to time, to obtain the maximum reference grasping force and the maximum reference grasping stiffness;

[0083] Step 2043: Process the maximum reference grasping force and the maximum reference grasping stiffness using a dynamic compliance primitive model to obtain a stiffness coefficient;

[0084] Step 2044: Based on the desired angle state information, the angle state information of the dexterous hand, the stiffness coefficient of the impedance control, and the preset damping coefficient, the joint torque instruction of the dexterous hand is obtained.

[0085] In step 2041, the derivative of the contact force information with respect to time is calculated based on the contact force information, and the formula is as follows:

[0086]

[0087] Among them, F and F n represents the contact force information, i.e., the average force of contact between the dexterous hand and the object, which is the input of the fuzzy logic control system; F σ Indicates the instantaneous force between the dexterous hand and the object; F n+1 It represents the average force of contact between the dexterous hand and the object in the next time period; T represents the time period.

[0088] The fuzzy logic theory module in step 2042 includes three parts: fuzzification, fuzzy reasoning, and defuzzification. The inputs to the fuzzy logic theory module are the contact force between the dexterous hand and the object and the time derivative of the contact force. The outputs are the maximum reference grasping force and the maximum reference grasping stiffness. Both the input and output variables are defined as five fuzzy sets: very large (XL), large (L), medium (M), small (S), and very small (XS). The membership functions for both the input and output use standard triangular membership functions.

[0089] Based on the standard Mamdani reasoning method, the centroid method is used to defuzzify the output of the maximum reference grasping force and the maximum reference grasping stiffness.

[0090]

[0091] Among them, F r is the maximum reference grasping force; K r is the maximum reference grasping stiffness; g m represents the centroid of the result set of the mth triggering rule, and M represents the number of triggering rules; and B j Represent the fuzzy sets of two input variables respectively; and denote the membership functions of the two input variables, α1 and α2, and β1 and β2 denote the coefficients of the input variable scaling, and * denotes the t-norm operator.

[0092] In step 2043, the maximum reference grasping force and the maximum reference grasping stiffness are processed using the dynamic compliance primitive model to obtain the stiffness coefficient of the adaptive variable impedance control. The specific process is as follows: Figure 4 As shown. The formula for adaptive variable impedance control is:

[0093]

[0094] Where, ξ represents the joint torque controlling the dexterous hand; K s and K d denote the stiffness coefficient and damping coefficient of impedance control respectively; ε and Represent the error of angular position and speed respectively; q and represent the actual angular position and velocity of the dexterous hand respectively; q d and denote the desired angular position and velocity of the dexterous hand, respectively.

[0095] The dynamic compliance primitive model consists of a spring damping model and a nonlinear forcing term. The formula of the dynamic compliance primitive model is:

[0096]

[0097] Among them, x represents the stiffness change curve; z represents the change speed of the stiffness change curve; x0 represents the initial stiffness value; x t represents the target stiffness value; k s represents the stiffness coefficient; k d represents the damping coefficient; λ represents the speed modulation parameter, which is a positive value. By adjusting the speed modulation parameter λ, the time required to complete the task can be adjusted; f(s) represents the nonlinear forcing term; Represents the derivative of the stiffness change curve; The derivative of the stiffness change curve.

[0098] The nonlinear forcing term is formed by the normalized linear superposition of multiple nonlinear Gaussian basis functions, which is used to learn the changing curve of the grasping stiffness characteristics of the dexterous hand. Among them, the variable s reflects the state of the first-order dynamic system and has the same velocity modulation parameter λ. The variable s is obtained by the following differential equation:

[0099]

[0100] Among them, α s Is a preset constant, and the domain of s is [0,1]. In this system, regardless of the initial value, as time tends to infinity, s will eventually converge monotonically to the target state, so it can be regarded as a phase parameter. And in the entire domain, s shows a monotonically increasing trend. When s approaches 0, f(s) at the target stiffness position continues to decrease. Due to the characteristics of the regular system, the dynamic compliance primitive model uses the phase parameter s as an independent variable, allowing it to adapt to various scenarios without changing the stiffness curve example. At the same time, the two parameters α s Both λ and λ can affect the convergence rate of the system.

[0101] The formula for the nonlinear forcing term f(s) is:

[0102]

[0103] Among them, φ δ (s) represents the δth normalized Gaussian basis function, each Gaussian basis function is characterized by the center position and height, and is associated with a weight value w δ ; N represents the total number of Gaussian basis functions; the δth normalized Gaussian basis function φδ The formula for (s) is:

[0104]

[0105] Among them, c ε represents the center position of one of the Gaussian basis functions; h ε Represents the height of one of the Gaussian basis functions; h ξ It also represents the height of one of the Gaussian basis functions; c ξ It also represents the center position of one of the Gaussian basis functions.

[0106] Given an example of a stiffness change curve Target nonlinear function f target (s) can be expressed as follows:

[0107]

[0108] Then, the weights of the dynamic compliance primitive model can be calculated by the local weighted regression method, with the goal of minimizing the cost function. The cost function formula is as follows:

[0109] J=∑(f target (s)-f(s)) 2 .

[0110] In order to achieve a smooth and natural stiffness change, we choose the logic function as an example of the stiffness characteristic curve. It can provide a gradual stiffness coefficient adjustment method, which is suitable for the variable impedance control mode. After learning the grasping stiffness characteristic curve, similar stiffness characteristic curves can be reproduced when grasping different objects. The starting point of the grasping stiffness characteristic curve is fixed to the preset value K0, and the end point is the maximum reference grasping stiffness K output by the fuzzy logic module. r , the independent variable R is defined as the proportional coefficient of the contact force, that is, the contact force F between the current five-finger dexterous hand and the grasped object c With its maximum reference force F r The ratio between them is used to output the stiffness coefficient K of the current impedance control in real time. s Among them, the contact force F between the dexterous hand and the grasped object is c The formula is:

[0111]

[0112] This ensures that the contact force does not exceed the maximum reference value, thus preventing potential damage to the grasped object or the dexterous hand. The design of the fuzzy logic theory-dynamic compliance primitive model enables the five-fingered dexterous hand to dynamically output the most appropriate impedance control coefficient during each grasping process. It also enables the five-fingered dexterous hand to perform adaptive variable impedance compliant grasping with similar gripping stiffness characteristic curves when grasping different objects.

[0113] Furthermore, the next stage of the dexterous hand's contact force information is fed back to the master operator. This contact force information is used to provide vibration feedback to the master operator. Specifically, the slave's force sensor acquires contact force information with the object and transmits it to the master operator in real time via the TCP / IP protocol over a local area network. When contact is established between the dexterous hand and the object, the force feedback glove activates vibration feedback, alerting the master operator that the slave's dexterous hand has made contact with the object.

[0114] This application uses a motion capture device to obtain the position information of the master-end operator's arm in real time, and uses a force feedback glove to obtain the angle status information of the master-end operator's hand in real time, and sends it to the slave-end manipulator and dexterous hand through a data link; the problem of solving the inverse kinematics of the manipulator is converted into a weighted and nonlinear optimization problem; according to the fuzzy logic object stiffness characteristic identification result, the slave-end dexterous hand adopts adaptive variable impedance control based on dynamic compliance primitives to achieve adaptive impedance characteristic changes when grasping different objects; the force sensor of the slave-end dexterous hand obtains contact force information with the object, and feeds it back to the master-end operator in real time through a data link. The present application provides a shared control method and related device for remote operation of a robot dexterous hand, which improves the immersion and compliance of the robot dexterous hand during remote operation, improves the smoothness and safety of the manipulator arm during remote operation, and thus improves the success rate and efficiency of remote operation tasks.

[0115] The present application also provides an application scenario, which applies the above-mentioned robot dexterous arm teleoperation sharing control method. Specifically: the robot dexterous arm teleoperation sharing control method provided in this embodiment can be applied in the object grasping scenario. The object grasping scenario includes the joint angle instruction determination link of the robot arm, the joint torque instruction determination link of the dexterous hand and the information feedback link; the joint angle instruction determination link is used to solve the inverse kinematics of the robot arm based on the expected posture information and the joint angle information of the robot arm using a nonlinear optimization method to obtain the joint angle instruction of the robot arm; the joint torque instruction determination link of the dexterous hand is used to perform adaptive variable impedance control on the dexterous hand based on the expected angle state information and the angle state information of the dexterous hand using fuzzy logic theory and dynamic compliance primitive model to obtain the joint torque instruction of the dexterous hand; the information feedback link is used to feed back the contact force information of the dexterous hand in the next stage to the master-end operator. The robot dexterous arm teleoperation sharing control method provided in this embodiment belongs to the joint angle instruction determination link of the robot arm, the joint torque instruction determination link of the dexterous hand and the information feedback link.

[0116] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a shared control method for remote operation of a dexterous robot arm is realized.

[0117] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0118] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0119] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0120] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0122] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0123] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0124] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for remote control of a robot's dexterous arm, characterized in that: include: Obtaining smoothed desired posture information and desired angle state information; the desired posture information and the desired angle state information are obtained by smoothing the posture information of the arm and the angle state information of the hand of the master-end operator at the current stage; the posture information includes position information and posture information; the angle state information includes angle position and speed; Obtaining the current position information of the slave end's robotic arm, as well as the angle state information and contact force information of the dexterous hand; the contact force information is the contact force information between the dexterous hand and the object, and the contact force information is obtained by the force sensor on the dexterous hand; Based on the desired posture information and the posture information of the robotic arm, a nonlinear optimization method is used to solve the inverse kinematics of the robotic arm to obtain a joint angle command of the robotic arm; the joint angle command is used to control the movement of the robotic arm so that the error between the posture information of the robotic arm in the next stage and the desired posture information is within a set range; Based on the desired angle state information, the angle state information of the dexterous hand, and the contact force information, the dexterous hand is adaptively controlled by variable impedance using fuzzy logic theory and a dynamic compliance primitive model to obtain joint torque instructions for the dexterous hand; the joint torque instructions are used to control the movement of the dexterous hand so that the error between the angle state information of the dexterous hand and the desired angle state information in the next stage is within a set range, and the contact force information of the dexterous hand in the next stage is obtained; The contact force information of the dexterous hand in the next stage is fed back to the master-end operator; the contact force information is used to provide force feedback and vibration tactile feedback to the master-end operator.

2. The method for remote control of a dexterous robot arm according to claim 1, characterized in that: Obtain smooth desired pose information and desired angle state information, including: The sliding window mean filter is used to smooth the arm posture information and hand angle state information of the master operator in the current stage to obtain the smoothed expected posture information and expected angle state information.

3. The method for remote control of a dexterous robot arm according to claim 1, characterized in that: Based on the desired posture information and the joint angle information of the manipulator, the inverse kinematics of the manipulator is solved using a nonlinear optimization method to obtain the joint angle instructions of the manipulator, specifically including: Determining constraint parameters; the constraint parameters include inequality constraint functions, equality constraint functions, upper and lower limits of manipulator joint angles, upper and lower limits of joint velocities, and singular configuration avoidance; Constructing a multi-objective optimization function; the multi-objective optimization function is a weighted sum of multiple normalized objective functions; the objective function includes end effector position matching, end effector posture matching, and trajectory smoothing; Based on the desired posture information, the joint angle information of the robotic arm and the constraint parameters, a nonlinear optimization method is used to solve the multi-objective optimization function to obtain the joint angle instructions of the robotic arm.

4. The method for remote control of a dexterous robot arm according to claim 3, characterized in that: The objective function of the end effector position matching is to minimize the Euclidean L2 norm error between the desired position information and the end effector position information of the manipulator; the end effector position information of the manipulator is calculated using forward kinematics on the joint angle information of the manipulator; The objective function of the end effector posture is to minimize the size of the rotation vector between the desired posture information and the end effector posture information of the manipulator; the end effector posture information of the manipulator is calculated by using forward kinematics on the joint angle information of the manipulator; The objective function of the trajectory smoothing motion is to minimize the joint velocity, joint acceleration, joint jerk and the speed of the end effector position space of the manipulator, as well as the preset speed hard constraint condition for the joint velocity limit.

5. The method for remote control of a dexterous robot arm according to claim 1, characterized in that: Based on the desired angle state information, the angle state information of the dexterous hand, and the contact force information, the dexterous hand is adaptively controlled by variable impedance using fuzzy logic theory and a dynamic compliance primitive model to obtain joint torque instructions for the dexterous hand, specifically including: Based on the contact force information, a derivative of the contact force information with respect to time is calculated; The contact force information and its derivative with respect to time are calculated using fuzzy logic theory to obtain a maximum reference grasping force and a maximum reference grasping stiffness; Processing the maximum reference grasping force and the maximum reference grasping stiffness using a dynamic compliance primitive model to obtain a stiffness coefficient; Based on the expected angle state information, the angle state information of the dexterous hand, the stiffness coefficient of the impedance control and the preset damping coefficient, a joint torque instruction of the dexterous hand is obtained.

6. The method for remote control of a dexterous robot arm according to claim 1, characterized in that: The calculation formula of the dynamic compliance primitive model is: Among them, x represents the stiffness change curve; z represents the change speed of the stiffness change curve; x0 represents the initial stiffness value; x t represents the target stiffness value; k s represents the stiffness coefficient; k d represents the damping coefficient; λ represents the velocity modulation parameter; f(s) represents the nonlinear forcing term; Represents the derivative of the stiffness change curve; The derivative of the stiffness change curve.

7. The method for remote control of a dexterous robot arm according to claim 6, characterized in that: The nonlinear forcing term is composed of the normalized linear superposition of multiple nonlinear Gaussian basis functions; the nonlinear forcing term is used to learn the variation curve of the grasping stiffness characteristics of the dexterous hand.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the shared control method for remote operation of a dexterous robot arm according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the robot dexterous arm teleoperation sharing control method according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the robot dexterous arm teleoperation sharing control method according to any one of claims 1 to 7 is implemented.

Images