Human-robot shared autonomy teleoperation method and system based on multiple motor skill priors

Abstract

The application discloses a kind of based on multi-motion skill prior art human-machine shared autonomous teleoperation method and system, comprising the following steps: based on the method of dynamic motion primitive, by teaching learning to different task trajectory, construct motion skill library;Optimal classification in running skill library is obtained in real-time trajectory;Human and robot share control's admittance control model is constructed;Based on the confidence of human and robot, share control weight is obtained, and share control rate arbitration is carried out.The planning control of person and robot is unified under admittance control framework, and arbitration function is designed through the confidence of two, share control weight is distributed, and robot autonomous level is adjusted online, to realize shared control teleoperation;Compared with the off-line trajectory generation method of robot dominance, the flexibility of human intervention operation is increased in the present method, can realize obstacle avoidance, trajectory trimming and other functions, improve the reproduction accuracy, reduce human power consumption and task completion time.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of robot control, and more particularly, relates to a human-robot shared autonomous teleoperation method and system based on multiple motion skill priors. BACKGROUND

[0002] Medical surgery robots have become a new research hotspot in the field of robots due to their advantages of high precision, strong flexibility, less fatigue for doctors, and removal of hand tremor. Due to the complexity and variability of the surgical environment, current artificial intelligence technology is not sufficient to support robots to achieve completely autonomous operation, and robot offline programming technology increases the risk and loses its flexibility. Therefore, shared control in human-computer interaction, which combines human decision intention and robot autonomous planning control strategy, has great application prospects in medical surgery robots. How to balance the variable environment and complex tasks, and integrate human intention and robot autonomous decision planning is a big difficulty of this technology.

[0003] To solve the above problems, patent document CN201811159046.2 discloses a kind of autonomous remote control underwater robot human-machine shared control method, which includes behavior management, basic behavior and behavior synthesis three units, responsible for robot operation mode switching, basic behavior output speed vector calculation and mixing of each basic behavior output respectively, improve the task performance of autonomous remote control underwater robot and reduce the work burden of operator. In addition, patent document CN201410056953.X discloses a kind of robot hand controller shared control method based on force fusion, which is guided by target object, realizes human hand control robot movement through intelligent decision of operator itself; At the same time, a virtual guide force is constructed to make the robot approach the target object autonomously, the virtual force guides the movement of the robot controlled by the hand controller, and the operator's manipulation force and the virtual guide force of the target object are fused, the movement of the hand controller and the robot is controlled through the fused force, realizes the robot approaching the target object under the joint action of machine intelligence and human intelligence, and completes the work process.

[0004] However, the methods of patent document CN201811159046.2 and patent document CN201410056953.X are limited to a single motion skill. Although both methods realize the fusion of robot and human operator control through different principles, they do not consider different task types such as resection, suture and puncture in actual surgical environment. In the process of human-machine shared control, the robot's own control strategy cannot realize specific trajectory for different tasks, and thus cannot effectively plan and arbitrate the sharing of control right for variable scenarios. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a human-robot shared autonomous teleoperation method and system based on multiple motion skill priors, which unifies the planning control of the human and the robot in a mobility control framework, designs an arbitration function through the confidence measures of the human and the robot, allocates a shared control weight value, adjusts the autonomous level of the robot online, and realizes shared control teleoperation; compared with the robot-dominant offline trajectory generation method, the method of the present application increases the flexibility of human intervention operation, can realize obstacle avoidance, trajectory adjustment and other functions, improves the reproduction accuracy, and reduces the human power consumption and task completion time.

[0006] To achieve the above object, according to one aspect of the present application, a human-robot shared autonomous teleoperation method based on multiple motion skill priors is provided, comprising the following steps:

[0007] S100 a method based on dynamic motion primitives, which constructs a motion skill library by teaching and learning different task trajectories;

[0008] S200 obtaining the optimal classification of the real-time trajectory in the running skill library;

[0009] S300 constructing a mobility control model for shared control of the human and the robot;

[0010] S400 obtaining a shared control weight based on the confidence of the human and the robot, and performing shared control rate arbitration.

[0011] Further, the S200 comprises: S210 classifying and summarizing the real-time trajectory:

[0012]

[0013] wherein x 0→T represents the trajectory from the start to the current time; N is the total number of dynamic motion primitives contained in the motion skill library; g represents a given target point; P is the probability; and T is the total time;

[0014] S220 obtaining the similarity relationship between the real-time generated trajectory and the expected trajectory in the motion skill library:

[0015]

[0016] wherein i=1, 2, …m, j=1, 2, …n, m and n respectively represent the total number of path data points of the trajectories x A and x B ; and N is the length of the regular path;

[0017] S230 comparing the real-time generated trajectory with the expected trajectory in the motion skill library one by one to obtain the similarity measure of the known trajectory with respect to each motion skill library skill

[0018] S240 obtains the optimal classification based on the similarity measure and the maximum entropy principle:

[0019]

[0020] where β is an adjustable parameter, and the optimal parameter value is where ε is a positive constant, and ε = 0.001 is taken;

[0021] Further, the S220 comprises:

[0022] S221 defines the similarity between the paths A and B as:

[0023]

[0024] where D represents a distance function; the integral length of the path is used to evaluate the distance between the current trajectory and the standard path in the skill library, and the following constraints are used:

[0025]

[0026] where m and n represent the total number of path data points of trajectories x A and x B , respectively; s is the arc length, S0and S m represent the arc lengths of each segment from the starting point 0 to m, and S0and S n represent the arc lengths of each segment from the starting point 0 to n, respectively;

[0027] S222 calculates the distance using dynamic time warping:

[0028]

[0029] where x A and x B are trajectories;

[0030] S223 constructs a distance matrix D m×n using the Euclidean distance between path points and as the i-th row and j-th column, solves the dynamic programming, and introduces a forgetting factor for correction:

[0031]

[0032] where i = 1, 2, … m, j = 1, 2, … n, m and n represent the total number of path data points of trajectories x A and x B , respectively; and ρ is a forgetting factor;

[0033] S224 takes the average of the distance measurement based on to obtain the trajectory xA and x B The distance between the real-time generated trajectory and the expected trajectory in the motion skill library is the similarity relationship:

[0034]

[0035] Wherein, N is the length of the regular path.

[0036] Further, the S100 specifically includes:

[0037] S101 determines the motion trajectory characteristics in the DMP motion planning framework, and obtains any motion skill representation:

[0038] p=[τ,w1,w2,...w N ] T

[0039] Wherein, τ is the speed factor of task execution; {w i} is the weight learned from human demonstration data; N is the number of Gaussian basis functions, which is set by artificial parameter setting; T is the vector transpose;

[0040] S102 obtains a plurality of groups of p i by demonstrating learning on different task trajectories, and constructs a cluster of dynamic motion primitives library, i.e. motion skill library:

[0041] DMPs={p1,p2,...,p l}.

[0042] Further, the S300 specifically includes:

[0043] S301 obtains the admittance control model of the Cartesian space under the zero stiffness condition of the outer ring:

[0044]

[0045] Wherein, M d and C d are the designed inertia and damping parameters; x r (t) represents the expected trajectory of the Cartesian space; u r (t) and f h (t) represent the input of the force applied by the robot and the human operator respectively, and α and β are the control weights of the robot and the human operator respectively;

[0046] S302 constructs the stiffness-free admittance control based on the dynamic motion primitive method, and obtains the damping parameter and the input of the force applied by the robot:

[0047]

[0048] wherein M d is an inertia matrix; τ is a speed factor of task execution; D is a damping coefficient matrix, K is a stiffness coefficient, g is a target point, x is a current position, x0 is a starting position, s is a phase variable, and the implicit represents time; f(s) is a forcing term;

[0049] S303 inertia matrix M d is set as a diagonal matrix:

[0050] M d = m d I

[0051] wherein I is a unit matrix, m d is an expected inertia amplitude, which can be artificially adjusted according to the force drag sensitivity of the robot.

[0052] Further, the control weights α and β of the robot and the human operator satisfy:

[0053] α+β=1.

[0054] Further, the S400 specifically comprises:

[0055] S401 obtaining the confidence of the robot to its own optimal trajectory DMP * :

[0056] C r = P(DMP * |x 0→T ,g) - P(DMP # |x 0→T ,g)

[0057] wherein P(DMP * |x 0→T ,g) is the probability of optimal classification; P(DMP # |x 0→T ,g) is the probability of suboptimal classification; g is a target point;

[0058] S402 obtaining the confidence of the human to itself:

[0059]

[0060] wherein λ and δ are hyperparameters, which respectively determine the lateral distribution scaling scale and the symmetry axis of the S curve, |f h (t)| is the size of the force applied to the end of the master hand robot;

[0061] S403 constructing an arbitration function:

[0062]

[0063] wherein, ξ, γ, ε, η are hyperparameters, C r is the confidence of the robot; C h is the confidence of the human.

[0064] According to a second aspect of the present application, a human-robot shared autonomy teleoperation system based on multiple motor skill priors is provided, characterized in that it comprises:

[0065] a demonstration learning module for constructing a motor skill library by demonstrating learning on different task trajectories based on a dynamic motor primitive method;

[0066] a classification summary module for obtaining the optimal classification of a real-time trajectory in the running skill library;

[0067] a control system construction module for constructing an admittance control model for shared control of humans and robots;

[0068] a shared arbitration module for obtaining a shared control weight based on the confidence of humans and robots and performing shared control rate arbitration.

[0069] According to a third aspect of the present application, an electronic device is provided, comprising:

[0070] at least one processor, at least one memory, and a communication interface; wherein,

[0071] the processor, memory, and communication interface communicate with each other;

[0072] the memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the method.

[0073] According to a fourth aspect of the present application, a non-transitory computer readable storage medium is provided, which stores computer instructions, and the computer instructions cause the computer to execute the method.

[0074] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0075] 1. The human-robot shared autonomy teleoperation method of the present application unifies the respective planning control of humans and robots under an admittance control framework, and designs an arbitration function through the confidence measurement of both, allocates a shared control weight, and adjusts the autonomy level of the robot online to realize shared control teleoperation; compared with the robot-dominant offline trajectory generation method;

[0076] 2. The human-robot shared autonomy teleoperation method of the present application increases the flexibility of human intervention operation, can realize obstacle avoidance, trajectory tuning, and other functions, improves the reproduction accuracy, and reduces human power consumption and task completion time. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 This is a diagram of the remote operation shared control framework of the present invention;

[0078] Figure 2 Schematic diagram of search constraints for regularized paths of the present invention;

[0079] Figure 3 This is a block diagram of the robot admittance control of the present invention;

[0080] Figure 4 F of the present invention m =10 when human confidence curve;

[0081] Figure 5 A diagram showing a change rate of a shared control arbitration factor according to the present invention;

[0082] Figure 6 This is a flow chart of the human-machine shared autonomous teleoperation method based on multi-motion skill priors of the present invention. DETAILED DESCRIPTION

[0083] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0084] like Figure 1 and Figure 6 As shown, the present invention provides a human-machine shared autonomous teleoperation method based on multi-sport skill priors, comprising the following steps:

[0085] S100 is based on the method of dynamic motion primitives, which builds a motion skill library by teaching and learning different task trajectories;

[0086] S200 obtains the optimal classification of real-time trajectories in the operational skill library;

[0087] S300 builds an admittance control model for shared control between humans and robots;

[0088] S400 obtains the shared control weight based on the confidence of humans and robots and performs shared control rate arbitration.

[0089] The method of the application unifies the planning control of the human and the robot respectively under a mobility control framework, and designs an arbitration function through the confidence metrics of the two, allocates a shared control value, adjusts the robot autonomy level online, and realizes shared control teleoperation; compared with the robot-dominant offline trajectory generation method, the method of the application increases the flexibility of human intervention operation, can realize obstacle avoidance, trajectory adjustment and other functions, improves the reproduction accuracy, and reduces the human power consumption and task completion time.

[0090] Based on the above embodiment, as an optional embodiment, the application provides a human-robot shared autonomy teleoperation method based on multiple motor skill priors, and the S100 comprises:

[0091] S101 determines a motor trajectory feature in a DMP motor planning framework, and obtains any motor skill representation:

[0092] p=[τ,w1,w2,...w N ] T

[0093] Wherein, τ is a speed factor of task execution; {w i} is a weight value learned from human demonstration data; N is the number of Gaussian basis functions, which is set by artificial parameter adjustment; T is;

[0094] S102 obtains a plurality of groups of p i by teaching and learning different task trajectories, and constructs a cluster of dynamic motor primitive libraries, i.e., a motor skill library:

[0095] DMPs={p1,p2,...,p l}.

[0096] Dynamic motor primitive (DMP) is a very representative method, and the DMP method is as follows:

[0097]

[0098] Using DMP as a model of robot autonomous control, through human demonstration, learning the trajectories that different tasks in the surgical scene should have, and encoding. In the dynamic motor primitive (DMP), the speed factor τ of task execution and the weight value {w i} learned from human demonstration data are factors that determine the motor trajectory feature, that is, a group of parameters p=[τ,w1,w2,...w N ] T represent a feature vector of a motor primitive, that is, a specific motor skill. By teaching and learning different task trajectories, a plurality of groups of p

[0099] DMPs = {p1, p2,..., pn} (1) DMPs = {p1, p2,..., pn} (2). l} (2).

[0100] Since there are various motor skills, the known trajectories must be classified online, and the method is carried out on the premise that the target position of the task is known.

[0101] Based on the above embodiment, as an optional embodiment, the application provides a human-machine shared autonomous teleoperation method based on multi-motor skill priori, and the S200 comprises:

[0102] S210 classifies and summarizes real-time trajectories:

[0103]

[0104] Wherein, x 0→T represents the trajectory from the start to the current time; N is the total number of dynamic motor primitives contained in the motor skill library; g represents a given target point;

[0105] S220 obtains the similarity relationship between the real-time generated trajectory and the expected trajectory in the motor skill library:

[0106]

[0107] Wherein, i = 1, 2, … m, j = 1, 2, … n, m, n respectively represent the total number of path data points of trajectories x A and x B ; N is the length of the regular path;

[0108] S230 compares the real-time generated trajectory with the expected trajectory in the motor skill library one by one, and obtains the similarity measurement of the real-time trajectory relative to each motor skill in the motor skill library

[0109] S240 obtains the optimal classification based on the similarity measurement and the maximum entropy principle as the criterion:

[0110]

[0111] Wherein, N is the length of the regular path; is the similarity measurement.

[0112] Based on the above embodiment, as an optional embodiment, the S220 comprises:

[0113] S221 defines the similarity between A and B two paths as:

[0114]

[0115] where D represents the distance function; the distance between the current trajectory and the standard path in the skill library is evaluated using the integral length of the path, with the following constraints:

[0116]

[0117] where m and n represent the total number of path data points of trajectories x A and x B respectively; s is the forgetting factor.

[0118] S222 uses dynamic time warping to calculate the distance:

[0119]

[0120] S223 uses the Euclidean distance between path points and to construct the distance matrix D m×n of the ith row and jth column, solves the dynamic programming, and introduces a forgetting factor for correction:

[0121]

[0122] where i = 1, 2, … m, j = 1, 2, … n, m and n represent the total number of path data points of trajectories x A and x B respectively; p is the forgetting factor.

[0123] S224 measures the distance based on the average of , obtaining the distance between trajectories x A and x B , i.e. the similarity relationship between the real-time generated trajectory and the expected trajectory in the motion skill library:

[0124]

[0125] where N is the length of the normalized path.

[0126] Online trajectory recognition and classification can be represented as:

[0127]

[0128] The distance between the historical path and the standard path in the dynamic motion primitive library is obtained as the index of equation (3), the historical path is generated in real time according to the shared control, and the standard path can be generated offline through each model in the motion skill library. The similarity between paths A and B is defined as:

[0129]

[0130] where D represents the distance function.

[0131] If the distance between the historical path and the standard path is used to represent, since the historical path is generated online, there are some jitter and speed imbalance phenomenon; and the path generated by DMP also implicitly depends on time, directly with the start to end time of the movement as the evaluation range has inaccuracy, therefore using the integral length of the path as the scope of evaluating the distance between the current trajectory and the standard path in the skill library, that is, in (4), there are following constraints:

[0132]

[0133] Wherein, m, n respectively represent the total number of path data points of trajectories x A and x B .

[0134] Because the speed of the two trajectories is different, most of the cases have m≠n, that is, the number of data points of the two paths is different, and the distance between the two cannot be directly measured by subtracting the corresponding time and then integrating. Therefore, by means of the method for measuring the similarity of two different time series in the field of speech recognition, dynamic time warping (DTW) is used to calculate the distance between the two. That is:

[0135]

[0136] Specifically, let the trajectories x A and x B be:

[0137]

[0138] The Euclidean distance between the path points and is denoted as d(i,j)

[0139]

[0140] The distance matrix D is constructed with d(i,j) as the ith row and jth column m×n . DTW is converted to find a path from the lower left corner of D m×n to the upper right corner, so that the cumulative distance is minimized. The search of the path needs to satisfy the rules as shown in Figure 2 , that is: the path point (i,j) must pass through one of (i-1,j-1), (i-1,j) or (i,j-1). It is easy to know that this problem is a classic dynamic programming (DP) problem, and its time complexity is O(m+n), and the recurrence formula is as follows:

[0141]

[0142] In the original DTW recursion formula shown in formula (9), the weight of each point is 1, which means that the influence of each point on classification in the DTW path is the same, which is not conducive to online trajectory modification or correction. Therefore, a forgetting factor is introduced to modify it:

[0143]

[0144] The forgetting factor p is preferably 0.99.

[0145] In addition, since the motion skill library generates multiple expected trajectories at the same time, the number of path points after DTW is not necessarily the same, therefore, the distance metric is averaged on the basis of , that is, formula (6) can be written as:

[0146]

[0147] Where N is the length of the normalized path.

[0148] The real-time generated trajectory is compared with the expected trajectory in the motion skill library one by one to calculate the similarity metric

[0149] Through the similarity metric, the optimal classification is calculated based on the maximum entropy principle:

[0150]

[0151] Where β is an adjustable parameter, and the optimal parameter value is Where ε is a small positive constant, here ε = 0.001. The final DMP * is the best classification of the current trajectory in the skill library.

[0152] Based on the above embodiment, as an optional embodiment, the present application provides a human-machine shared autonomous teleoperation method based on multiple motion skill priors, and the S300 comprises:

[0153] S301 obtains a mobility control model in Cartesian space under the zero stiffness condition of the outer ring:

[0154]

[0155] Where M d and C d are the designed inertia and damping parameters; x r (t) represents the expected trajectory in the Cartesian space; u r (t) and f h(t) represents the force input applied by the robot and the human operator, respectively, and α and β are the control weights of the two respectively;

[0156] like Figure 3 As shown in Figure 2, the robot is placed under admittance control, which is implemented through two loops: an internal position control loop and an external force control loop. Since commercial robots on the market already provide controllers with very precise joint position control, it can be assumed that the system already has a perfect internal position control loop. That is, q(t) = q r (t), where q r (t) is the desired joint angle. q r (t) Generate the admittance control model in Cartesian space under the condition of zero stiffness of the outer loop:

[0157]

[0158] Among them, M d and C d are the designed inertia and damping parameters respectively; x r (t) represents the expected trajectory in Cartesian space; u r (t) and f h (t) represents the force input applied by the robot and the human operator, respectively, and α and β are the control weights of the two.

[0159] The desired trajectory in the joint space is:

[0160]

[0161] S302 constructs a stiffness-free admittance control based on the dynamic motion primitive method to obtain the damping parameters and the input of the force applied by the robot:

[0162]

[0163] u(t) should be determined by DMP * Generate, considering formula (1), the model variables are expressed as position x and its differential:

[0164]

[0165] Move the terms in the above equation and multiply both sides by the desired inertia matrix M. d , thus constructing the stiffness-free admittance control form as shown in formula (13), we have:

[0166]

[0167] From this, the damping term in Equation (13) and the robot input are derived:

[0168]

[0169] f in the formula * (s) represents DMP * Gauss forcing term.

[0170] At this point, the parameters that need to be artificially adjusted in formula (13) are only the inertia matrix M d and the control weights α and β. Among them, the inertia matrix M d can be used as the parameter of the sensitivity in the teleoperation process, and to further reduce the complexity of artificial parameter adjustment, M d is set to a diagonal matrix: M d = m d I, where I is an identity matrix, and m d is the desired inertia amplitude, which can be artificially adjusted according to the force drag sensitivity of the robot.

[0171] S303 sets the inertia matrix M d to a diagonal matrix:

[0172] M d = m d I

[0173] where I is an identity matrix, and m d is the desired inertia amplitude, which can be artificially adjusted according to the force drag sensitivity of the robot.

[0174] At this time, it can be seen from formula (13) that when the control weight α = 0, the control rate becomes the standard admittance control, that is:

[0175]

[0176] And when α = 1, the control rate degenerates into the DMP * motion skill shown in formula (15).

[0177] After the model architecture and related parameters are constructed, the shared control weights α and β need to be arbitrated according to the confidence of the robot on its optimal trajectory DMP * and the confidence of the human operator. In this paper, the sharing control process of the two is regarded as a game process of the two, so there is a constraint α + β = 1, so only the change rate of α needs to be designed.

[0178] Based on the above embodiment, as an optional embodiment, the present application provides a human-machine shared autonomous teleoperation method based on multiple motion skill priors, which comprises the following steps:

[0179] S401 obtains the confidence of the robot on its optimal trajectory DMP * :

[0180] C r = P(DMP * |x 0→T ,g) - P(DMP # |x 0→T ,g)

[0181] where P(DMP * |x 0→T ,g) is the probability of the optimal classification; P(DMP # |x 0→T ,g) is the probability of the suboptimal classification;

[0182] The robot's confidence C r in itself is reflected in the cross-entropy difference between the optimal classification DMP * and the suboptimal classification, whose value should be a small number between 0 and 1. Let DMP * be the suboptimal solution other than DMP # , then:

[0183]

[0184] The robot's confidence is:

[0185] C r = P(DMP * |x 0→T ,g) - P(DMP # |x 0→T ,g) (21)

[0186] Intuitively, the more ambiguous the classification result, the smaller C r , the more clear the classification result, the larger C r .

[0187] S402 obtains the human's confidence in itself:

[0188]

[0189] where λ and δ are hyperparameters that determine the horizontal distribution scaling and the symmetry axis of the S curve, respectively, and |f h (t)| is the magnitude of the force applied to the master end robot end.

[0190] The human's confidence C h in itself is reflected in the magnitude of the force applied to the master end robot end, i.e., |f h (t)|. Since it is desired that C h also be between 0 and 1, a mapping function needs to be performed with |f h (t)| as the independent variable, and here the Sigmoid function is selected:

[0191]

[0192] where λ and δ are hyperparameters that determine the scaling of the lateral distribution and the symmetry axis of the S-curve, respectively, thus adjusting the shape and distribution of the S-curve. Set the maximum threshold F m = |f h (t)| max , then the symmetry axis is At the same time, when |f h (t) | = 0, C h should be close to 0, which is set to 0.01 here, i.e., C h (0) = 0.01, and C m (0) = 0.01 can be solved. Figure 4 The function image when F m = 10 is shown.

[0193] S403 constructs an arbitration function:

[0194]

[0195] where ξ, γ, ε, η are hyperparameters, C r is the confidence of the robot; and C h is the confidence of the human.

[0196] After obtaining the respective confidence of the robot and the human operator, an arbitration function can be constructed using a two-dimensional S-function framework:

[0197]

[0198] where ξ, γ, ε, η are hyperparameters.

[0199] To prevent singularity caused by a denominator of 0, set ε = 0.01; since it is desired that the influence of the human's decision be greater when the confidence of the two is the same, set γ = 2, η = 1, and δ = 0.2; the maximum value of α should be 1, i.e., when C h = 0 and C r = 1, α = 1, therefore, adjust ξ to ensure that α (0, 1) = 1, and solve for ξ = 1.5466. As shown in Figure 5 is a three-dimensional grid chart of α (C h , C r ) after setting the above parameters.

[0200] The implementation basis of each embodiment of the present application is achieved by programmed processing of a device with a central processor function. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the present application can be packaged into various modules. Based on this actual situation, on the basis of each embodiment described above, an embodiment of the present application provides a human-robot shared autonomous teleoperation system based on multiple motor skill priors, which is used to execute the human-robot shared autonomous teleoperation method based on multiple motor skill priors in the method embodiment described above. It includes:

[0201] A demonstration learning module is configured to construct a motor skill library by demonstrating and learning different task trajectories based on a dynamic motor primitive;

[0202] A classification and summary module is configured to obtain an optimal classification of a real-time trajectory in the running skill library;

[0203] A control system construction module is configured to construct a mobility control model shared by humans and robots;

[0204] A sharing arbitration module is configured to obtain a shared control weight based on the confidence of humans and robots, and to arbitrate a shared control rate.

[0205] It should be noted that the device in the device embodiment provided by the present application can be used to implement the methods in the method embodiments described above, and can also be used to implement the methods in other method embodiments provided by the present application. The difference is only that the corresponding functional modules are set, and the principle is basically the same as that of the above-mentioned device embodiments provided by the present application. As long as the person skilled in the art can obtain the corresponding technical means by combining technical features on the basis of the above-mentioned device embodiments, and the technical solutions formed by these technical means, on the premise of ensuring the practicality of the technical solutions, the device in the above-mentioned device embodiment can be improved to obtain the corresponding device class embodiment, which is used to implement the method in other method class embodiments.

[0206] The method of the embodiment of the present application is implemented by relying on an electronic device, so it is necessary to introduce the related electronic device. For this purpose, an embodiment of the present application provides an electronic device, which comprises: at least one central processor, a communications interface, at least one memory and a communications bus, wherein the at least one central processor, the communications interface and the at least one memory complete the communication among each other through the communications bus. The at least one central processor can call the logic instructions in the at least one memory to execute all or part of the steps of the method provided by each method embodiment.

[0207] In addition, the logic instructions in the at least one memory described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various method embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0208] The device embodiments described above are only schematic, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0209] From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0210] The computer program product of the present application can be a computer program product, which is a machine-readable medium (or media) having stored therein some code (i.e., some computer code or software) that, when executed by a machine, causes the machine to perform any of the features or functions of the present application. Note that the computer program product of the present application can be implemented on a computer controlled by a single processor or multiple processors. Note also that the computer program product of the present application can be implemented on a general-purpose computer or a special-purpose computer. Note further that the computer program product of the present application can be implemented on a computer controlled by a single computer or multiple computers.

[0211] In this application, the term "comprise", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0212] Those skilled in the art will readily understand that the above described embodiments of the application are merely intended to be illustrative and not limiting of the scope of the application. Any modification, equivalent replacement or improvement not departing from the spirit and principle of the application should be included in the scope of the protection of the application.

Claims

1. A human-machine shared autonomous teleoperation method based on multi-sport skill priors, characterized in that: The following steps are involved: S100 is based on the method of dynamic motion primitives, which builds a motion skill library by teaching and learning different task trajectories; S200 obtains the optimal classification of real-time trajectories in the operational skill library; S300 builds an admittance control model for shared control between humans and robots; S400 obtains shared control weights based on the confidence of humans and robots and arbitrates the shared control rate; The S200 includes: S210 classifying and summarizing the real-time trajectories: Where x0→T represents the trajectory from the beginning to the current moment; N is the total number of dynamic motion primitives contained in the motion skill library; g represents the given target point; P is the probability; T is the total time; S220 obtains the similarity relationship between the real-time generated trajectory and the expected trajectory in the sports skill library: Among them, i = 1, 2, ... m, j = 1, 2, ... n, m, n represent the trajectory x respectively. A and x B The total number of path data points; N is the length of the regular path; S230 compares the real-time generated trajectory with the expected trajectory in the sports skill library one by one to obtain a similarity measure of the known trajectory relative to each sports skill library skill. S240 is based on similarity measurement and takes the maximum entropy principle as the criterion to obtain the optimal classification: Among them, β is an adjustable parameter, and the optimal parameter value is Where ε is a positive constant, take ε = 0.001; The S220 includes: S221 defines the similarity between paths A and B as: Where D represents the distance function; the integral length of the path is used to evaluate the distance between the current trajectory and the standard path in the skill library, with the following constraints: Among them, m, n represent the trajectory x A and x B The total number of path data points; s is the arc length, S0 and S m Respectively represent the arc lengths of the trajectory starting point 0 to m, S0 and S n Respectively represent the arc lengths of the trajectory starting points 0 to n; S222 uses dynamic time normalization to calculate distance: Among them, x A and x B For the trajectory; S223 with waypoints and The Euclidean distance between them is used as the i-th row and j-th column to construct the distance matrix D m×n , solve the dynamic programming and introduce the forgetting factor correction: Among them, i = 1, 2, ... m, j = 1, 2, ... n, m, n represent the trajectory x respectively. A and x B The total number of path data points; ρ is the forgetting factor; The metric of S224 distance is Take the average based on A and x B The distance is the similarity between the real-time generated trajectory and the expected trajectory in the sports skill library: Where N is the length of the regular path.

2. The method of human-machine shared autonomous teleoperation based on multi-sport skill priors according to claim 1, characterized in that: The S100 specifically includes: S101 determines the motion trajectory characteristics in the DMP motion planning framework and obtains any motion skill representation: p=[τ,w1,w2,...w N ] T Among them, τ is the speed factor of task execution; {w i } is the weight learned from human teaching data; N is the number of Gaussian basis functions, which is set by manual parameter adjustment; T is the vector transpose; S102 obtains several groups of p by teaching different task trajectories i , construct a cluster of dynamic motion primitive libraries, namely the motion skill library: DMPs={p1,p2,...,p l }。 3. The method of human-machine shared autonomous teleoperation based on multi-sport skill priors according to claim 1, characterized in that: The S300 specifically includes: S301 obtains the Cartesian space admittance control model under the condition of zero stiffness of the outer loop: Among them, M d and C d are the designed inertia and damping parameters respectively; x r (t) represents the expected trajectory in Cartesian space; u r (t) and f h (t) represents the force input applied by the robot and human operator, respectively, and α and β are the control weights of the robot and human operator, respectively; S302 constructs a stiffness-free admittance control based on the dynamic motion primitive method to obtain the damping parameters and the input of the force applied by the robot: Among them, M d is the inertia matrix; τ is the speed factor of task execution; D is the damping coefficient matrix, K is the stiffness coefficient, g is the target point, x is the current position, x0 is the starting position, s is the phase variable, implicitly representing time; f(s) is the forcing term; S303 Inertia Matrix M d Set up as a diagonal matrix: M d =m d I Where I is the identity matrix, m d The desired inertia amplitude can be adjusted manually according to the force drag sensitivity of the robot.

4. The method of human-machine shared autonomous teleoperation based on multi-sport skill priors according to claim 3, characterized in that: The control weights α and β of the robot and human operator satisfy: α+β＝1。 5. The method of human-machine shared autonomous teleoperation based on multi-sport skill priors according to claim 1, characterized in that: The S400 specifically includes: S401 obtains the robot's optimal trajectory DMP * Confidence level: C r =P(DMP * |x 0→T ,g)-P(DMP # |x 0→T ,g) Among them, P(DMP * |x 0→T ,g) is the probability of the optimal classification; P(DMP # |x 0→T ,g) is the probability of suboptimal classification; g is the target point; S402 obtains human confidence in themselves: Among them, λ and δ are hyperparameters, which respectively determine the horizontal distribution scaling scale and symmetry axis of the S-curve. h (t)| is the magnitude of the force applied to the end of the robot at the master hand; S403 constructs arbitration function: Among them, ξ, γ, ε, η are hyper parameters, C r is the robot's confidence; C h For human confidence in themselves.

6. A human-machine shared autonomous teleoperation system based on multi-sport skill priors, for executing the method according to any one of claims 1 to 5, characterized in that: include: The teaching and learning module is used to build a motion skill library by teaching and learning different task trajectories based on the dynamic motion primitive method; The classification and summary module is used to obtain the optimal classification of real-time trajectories in the operation skill library; Control system building blocks for constructing admittance control models for shared control between humans and robots; The shared arbitration module is used to obtain the shared control weight based on the confidence of humans and robots and to arbitrate the shared control rate.

7. An electronic device, characterized in that: include: At least one processor, at least one memory and a communication interface; wherein, The processor, memory and communication interface communicate with each other; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to perform the method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions that cause a computer to execute the method of any one of claims 1 to 5.

Images