Teleoperation skill transmission method combined with strategy adjustment

Abstract

The invention discloses a teleoperation skill transmission method and device combined with strategy adjustment, a medium and equipment, and the method comprises the steps: completing an operation task based on a pre-constructed force feedback teleoperation system to collect demonstration data, and determining a main motion direction according to interaction force data and contact force data; dividing each stage of the operation task based on the main motion direction to obtain corresponding demonstration data of each task stage; determining strategy adjustment parameters in each task stage based on the demonstration data; a dynamic motion primitive model is adopted to encode the moving trajectory of the robot in stages, and the mapping relation between the strategy adjustment parameters in the main motion direction and the moving trajectory is calculated through an interpolation algorithm; the target action of the robot is generated based on the pre-constructed motion planner, so that the robot autonomously completes the contact operation, the action planning strategy of the human in the contact interaction task is brought into man-machine skill transmission, and the robot can learn the strategy adjustment ability of the human in the task.

Description

Technical Field

[0001] The present application relates to the field of robot control, and in particular to a remote operation skill transfer method, device, medium and equipment combined with strategy adjustment. Background Art

[0002] At present, robots can replace humans to perform various tasks in industrial fields or dangerous areas. Through force feedback teleoperation, the operator can obtain the interaction status of the robot with the environment or target object through tactile feedback during the remote control of the robot and make real-time decisions and reactions, thereby extending the human's behavioral and perceptual abilities. When interacting with an unknown environment, the human operator can adaptively adjust the motion strategy in the process of interacting with the force field or obstacles according to the task requirements and the perception of the environment, so that the arm can show appropriate stiffness and achieve the limitation and balance of trajectory deviation and contact force in contact. When the robot performs contact operations autonomously, or when the teleoperation system fails, the robot needs to adjust its movements in the process of interacting with the unknown environment and adjust the trajectory planning strategy in real time. Traditional robot trajectory tracking and impedance control have the problems of low flexibility and poor adaptability in such scenarios. Variable impedance control is often used to deal with the scenarios described above, but the design and adjustment of the impedance parameter adjustment strategy still requires a lot of work and a long period of time. Summary of the invention

[0003] The main purpose of this application is to provide a remote operation skill transfer method, device, medium and equipment combined with strategy adjustment, aiming to realize human-to-robot skill transfer by collecting multimodal information analysis at both master and slave ends and obtaining operation skills including strategy adjustment.

[0004] To achieve the above-mentioned purpose, the first aspect of the present application provides a remote operation skill transfer method combined with strategy adjustment, including: completing an operation task based on a pre-built force feedback remote operation system to collect demonstration data, wherein the demonstration data includes interaction force data between the operator and the control device, the operator's electromyography information, the current operation trajectory of the robot, and the contact force data between the robot and the environment; determining the main motion direction according to the interaction force data and the contact force data, and dividing the operation task into various stages based on the main motion direction to obtain corresponding demonstration data for each task stage; determining corresponding strategy adjustment parameters based on the demonstration data of each task stage; using a dynamic motion primitive model to encode the robot's operation trajectory in stages, and calculating the mapping relationship between the strategy adjustment parameters in the main motion direction and the operation trajectory through an interpolation algorithm; planning the robot's target action based on the pre-built motion planner according to the mapping relationship, expected position and the contact force data obtained at the current moment, so that the robot can autonomously complete the contact operation, wherein the expected position is determined based on the current operation trajectory.

[0005] Optionally, the force feedback teleoperation system includes a task end and an operation end, the task end has a robot, and the operation end has a hand controller. Before the pre-built force feedback teleoperation system completes the operation task to collect demonstration data, the teleoperation skill transfer method combined with strategy adjustment also includes: based on the pre-built force feedback teleoperation system, obtaining the terminal posture information and the contact force information of the robot when completing the operation task; sending the terminal posture information and the contact force information to the hand controller; processing the terminal posture information and the contact force information to obtain the terminal posture information of the hand controller and the hand controller interaction force information; fixedly collecting the terminal posture information and the hand controller position change information of the hand controller and sending them to the robot; processing the terminal posture information and the hand controller position change of the hand controller to obtain the position instruction of the robot controller; obtaining the contact force information of the robot, transmitting the contact force information to the hand controller, and processing the contact force information to obtain the force instruction of the hand controller.

[0006] Optionally, determining the main motion direction according to the interaction force data and the contact force data, and dividing the task stages based on the main motion direction, includes: unitizing the operation task based on the preprocessed interaction force data, the current running trajectory, the contact force data and the electromyographic information to obtain corresponding unitized data; processing the interaction force data and the contact force data using a sliding average algorithm to obtain the preprocessed interaction force data and the contact force data; calculating the difference between the preprocessed interaction force data and the contact force data, determining the main motion direction matrix based on the difference and a unit step matrix, and determining the main motion direction according to the main motion direction matrix; dividing the operation task into multiple task stages based on the change of the main motion direction matrix in the demonstration data.

[0007] Optionally, determining corresponding strategy adjustment parameters based on demonstration data of each task stage includes: determining the strategy adjustment parameters of each task stage based on an index of the electromyographic information.

[0008] Optionally, the expression of the dynamic motion primitive model, the expression of the dynamic motion primitive model includes nonlinear terms, and the dynamic motion primitive model is used to encode the robot's running trajectory in stages, and the mapping relationship between the strategy adjustment parameters in the main motion direction and the running trajectory is calculated through an interpolation algorithm, including: determining the nonlinear terms of each task stage based on basis functions, regular system expressions and weights of the basis functions; determining the dynamic motion primitive model of each task stage based on each nonlinear term; and calculating the mapping relationship between the strategy adjustment parameters in the main motion direction and the running trajectory through an interpolation algorithm based on the robot's running trajectory and strategy adjustment parameters in each task stage.

[0009] Optionally, the pre-built motion planner plans the target action of the robot according to the acquired mapping relationship, expected position and contact force data at the current moment, including: acquiring the trajectory components of the current running trajectory of the robot in various directions at the current moment based on the dynamic motion primitive model, and determining the expected running trajectory of the robot based on the trajectory components in various directions; determining the end position deviation at the current moment based on the difference between the expected running trajectory and the current running trajectory; determining the expected running trajectory of the robot at the next moment based on the end position deviation and the contact force data; calculating the step length at the next moment based on a preset step length calculation formula and the mapping relationship; determining the end position instruction of the robot based on the step length at the next moment, the current running trajectory and the expected running trajectory, and obtaining the planned target action of the robot based on the end position instruction.

[0010] Optionally, the expression of the end position instruction is:

[0011]

[0012] Among them, s t+1 Indicates the step size of the next moment, P s,t+1 represents the expected running trajectory, P r,t Indicates the current running track.

[0013] In addition, to achieve the above-mentioned purpose, the second aspect of the present application also provides a remote operation skill transfer device combined with strategy adjustment, including: a data collection module, which is used to complete the operation task based on a pre-built force feedback remote operation system to collect demonstration data, wherein the demonstration data includes the interaction force data between the operator and the control device, the operator's electromyography information, the current operation trajectory of the robot and the contact force data between the robot and the environment; a task division module, which is used to determine the main motion direction according to the interaction force data and the contact force data, and divide the various stages of the operation task based on the main motion direction to obtain the corresponding demonstration data for each task stage; a parameter determination module, which is used to determine the corresponding strategy adjustment parameters based on the demonstration data of each task stage; a mapping module, which is used to encode the robot's operation trajectory in stages using a dynamic motion primitive model, and calculate the mapping relationship between the strategy adjustment parameters in the main motion direction and the operation trajectory through an interpolation algorithm; an action planning module, which is used to plan the target action of the robot based on the mapping relationship, the expected position and the contact force data obtained at the current moment based on the pre-built motion planner, so that the robot can autonomously complete the contact operation, wherein the expected position is determined based on the current operation trajectory.

[0014] To achieve the above-mentioned purpose, the third aspect of the present application also provides a computer-readable storage medium, which includes instructions, which, when executed on a computer, enables the computer to execute the remote operation skill transfer method combined with strategy adjustment provided in the first aspect.

[0015] To achieve the above-mentioned purpose, the fourth aspect of the present application also provides an electronic device, characterized in that the electronic device includes: at least one processor, a memory and an input and output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the remote operation skill transfer method combined with strategy adjustment provided in the first aspect.

[0016] The embodiment of the present application proposes a remote operation skill transfer method, device, medium and equipment combined with strategy adjustment, which completes the operation task based on a pre-built force feedback remote operation system to collect demonstration data, wherein the demonstration data includes the interaction force data between the operator and the control device, the operator's electromyography information, the current operation trajectory of the robot and the contact force data between the robot and the environment; the main motion direction is determined according to the interaction force data and the contact force data, and the task stage is divided based on the main motion direction to obtain the main motion direction corresponding to each task stage; the strategy adjustment parameters in each task stage are determined based on the demonstration data; the mapping relationship between the strategy adjustment parameters in the main motion direction and the current operation trajectory is encoded in stages using a dynamic motion primitive model; based on the pre-built motion planner, the target action of the robot is planned according to the mapping relationship, expected position and contact force data obtained at the current moment, so that the robot can autonomously complete the contact operation. The present application realizes the incorporation of human action planning strategies in contact interaction tasks into human-machine skill transfer, so that the robot can learn the strategy adjustment capabilities of humans in tasks, which helps the robot adopt appropriate strategies in the interaction process with force fields or obstacles to achieve the limitation and balance of trajectory deviation and contact force. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart diagram of an embodiment of a remote operation skill transfer method combined with strategy adjustment provided by the present application;

[0018] Figure 2 A structural block diagram is provided for an embodiment of a remote operation skill transfer device combined with strategy adjustment in the present application.

[0019] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0021] In the prior art, human-machine skill transfer refers to a person transferring his or her skills to a robot in a generalized manner, thereby realizing robot programming. This can replace the traditional robot programming method and is also called programming by demonstration (PbD) or learning from demonstration (LfD). Dynamic motion primitives (DMPs), as a widely used LfD method, have many beneficial properties, such as good versatility and high generalization.

[0022] This application combines demonstration learning with a variable stiffness strategy that mimics human physical interaction behavior. The movement of a robot using a variable stiffness strategy during physical interaction is affected by the operator. By collecting data during teleoperation, the skill transfer algorithm can be used to obtain motion skills and corresponding action planning strategy changes, and appropriate antagonistic or compliant motion can be obtained by reproducing the skills in the scenario of multi-contact tasks.

[0023] The teleoperation skill transfer method combined with strategy adjustment proposed in this application is described in detail below with reference to the embodiments and drawings.

[0024] Reference Figure 1 The teleoperation skill transfer method combined with strategy adjustment provided in the first embodiment of the present application can be executed by a processor on the robot side. The teleoperation skill transfer method combined with strategy adjustment can include:

[0025] S10, completing the operation task based on the pre-built force feedback teleoperation system to collect demonstration data, wherein the demonstration data includes interaction force data between the operator and the control device, the operator's electromyographic information, the current operation trajectory of the robot, and the contact force data between the robot and the environment;

[0026] In this step, the processor uses the force feedback teleoperation system to collect demonstration data. The operator wears a muscle electrical signal sensor and controls the robot through the control device to interact with the environment with unknown mechanical properties and complete the task. During the task, the system continuously records relevant data, including the interaction force between the operator and the control device, the operator's electromyography information, the robot's running trajectory, and the contact force between the robot and the environment.

[0027] S20, determining the main motion direction according to the interaction force data and the contact force data, and dividing the operation task into various stages based on the main motion direction to obtain corresponding demonstration data for each task stage;

[0028] In this step, the processor performs trajectory preprocessing and divides the task into stages. The processor smoothes the collected demonstration data and performs corresponding filtering and noise reduction on the muscle electrical signal sensor. The main motion direction is obtained based on the interaction force data between the operator and the control device and the contact force data between the robot and the environment, and the task stages are divided according to the changes in the main motion direction.

[0029] S30, determining corresponding strategy adjustment parameters based on the demonstration data of each task stage;

[0030] In this step, the processor calculates the strategy adjustment parameters from the demonstration data, analyzes the operator's action strategy at each moment according to the demonstration data, and calculates the strategy adjustment parameters in each task stage.

[0031] S40, using a dynamic motion primitive model to encode the robot's running trajectory in stages, and calculating the mapping relationship between the strategy adjustment parameters in the main motion direction and the running trajectory through an interpolation algorithm;

[0032] In this step, the processor encodes the trajectory and strategy adjustment parameters in stages. For each task stage, the processor uses the dynamic motion primitive model to encode the relationship between the desired trajectory of the robot end and the current running trajectory, and records the relationship between the strategy adjustment parameters and the main motion direction position.

[0033] S50. Based on the pre-built motion planner, the target motion of the robot is planned according to the acquired mapping relationship, expected position and contact force data at the current moment, so that the robot can complete the contact operation autonomously, wherein the expected position is determined based on the current running trajectory.

[0034] In this step, the processor completes the task by reproducing the skills. First, the robot's motion planner is constructed. In the task, the robot obtains the expected trajectory corresponding to the moment and the strategy adjustment parameters corresponding to the position through the dynamic primitive model in step S40. The motion planner follows the current strategy to plan the target action according to the expected position, current position and environmental contact force, so that the robot can complete the contact operation autonomously.

[0035] In summary, the present application proposes a teleoperation skill transfer method combined with strategy adjustment, which completes the operation task based on a pre-built force feedback teleoperation system to collect demonstration data, wherein the demonstration data includes the interaction force data between the operator and the control device, the operator's electromyography information, the current operation trajectory of the robot, and the contact force data between the robot and the environment; the main motion direction is determined according to the interaction force data and the contact force data, and the task stage is divided based on the main motion direction to obtain the main motion direction corresponding to each task stage; the strategy adjustment parameters in each task stage are determined based on the demonstration data; the mapping relationship between the strategy adjustment parameters in the main motion direction and the current operation trajectory is encoded in stages using a dynamic motion primitive model; based on the pre-built motion planner, the target action of the robot is planned according to the mapping relationship, expected position and contact force data obtained at the current moment, so that the robot can autonomously complete the contact operation, wherein the expected position is determined based on the current operation trajectory. The present application incorporates the action planning strategy of humans in contact interaction tasks into the human-machine skill transfer, so that the robot can learn the strategy adjustment ability of humans in the task, which helps the robot to adopt appropriate strategies in the interaction process with the force field or obstacles to achieve the limitation and balance of trajectory deviation and contact force. Based on the modeling and parameterization of the interaction strategy, this application uses dynamic motion primitives to learn the motion trajectory and strategy changes, which can achieve the learning and reproduction of human operation skills and improve the integrity of operation skills. This application divides the task stages in the trajectory data preprocessing stage, and performs segmented skill learning based on the divided stages, and also refers to the main motion direction corresponding to the stage in the action planning, so that the system has better adaptability to different environments and task objectives.

[0036] In an embodiment of the present application, the force feedback teleoperation system includes a task end and an operation end, the task end has a robot, and the operation end has a hand controller. Before completing the operation task based on the pre-built force feedback teleoperation system to collect demonstration data, the teleoperation skill transfer method combined with strategy adjustment also includes:

[0037] Based on the pre-built force feedback teleoperation system, the robot's terminal position information and contact force information are obtained when completing the operation task;

[0038] Sending the end position information and contact force information to the hand controller;

[0039] Process the terminal position information and contact force information to obtain the terminal position information of the hand controller and the interaction force information of the hand controller;

[0040] Fixedly collect the hand controller end position information and hand controller position change information and send them to the robot;

[0041] Process the hand controller end position information and the hand controller position change to obtain the position instruction of the robot controller;

[0042] The contact force information of the robot is obtained, the contact force information is transmitted to the hand controller, and the contact force information is processed to obtain the force instruction of the hand controller.

[0043] Specifically, the specific implementation of the present invention will be described with a specific task example. The robot in the example is UR5, the control device used by the human operator is the omega.6 hand controller of Force Dimension, and the collection of muscle electrical signals is completed by the myoelectric bracelet. The task goal is to move the end of the robot from one point in space to another point in a plane. In this process, it is necessary to conform to the surface of hard obstacles and overcome soft obstacles. The present invention is further described in conjunction with the accompanying drawings:

[0044] The processor implements the construction of the force feedback teleoperation system through the above steps. Further, the construction process of the force feedback teleoperation system may include the following specific execution steps:

[0045] Construct the task end and the operation end, and place a robot with a force sensor at the end of the task end. The end position information of the robot is represented by the homogeneous matrix T, and the contact force obtained by the force sensor is represented by F r It is expressed as shown in formula (1):

[0046]

[0047] In formula (1), R r is the rotation matrix corresponding to the robot terminal posture, P r =[x r y r z r ] T F is the position coordinate of the robot end. r Contains force components in three directions, denoted as F r =[F xr F yr F zr ] T The processor sends the end position information and contact force information to the hand controller at the operating end.

[0048] The hand controller will be placed in force feedback mode. When there is no force command, the operator can freely translate and rotate the hand controller by holding the end of the hand controller. By setting the force command, the force sense can be presented at the end of the hand controller. The processor can read the hand controller end pose information Q and the interaction force F between the operator and the hand controller end. q , as shown below:

[0049]

[0050] In formula (2), R q is the rotation matrix corresponding to the hand controller posture, P q =[x q y q z q ] T F is the position coordinate of the end of the hand controller. q Contains force components in three directions, denoted as F q =[F xq F yq F zq ] T .

[0051] During the initialization of the teleoperation system, the robot end and the hand controller end will be placed in the same posture R 0 , and record the initial positions P of both after completion q0 The hand controller collects the terminal's position data at a fixed frequency and calculates the change in position at time t relative to the initial position ΔP t Transmitted to the robot end, the hand controller position change ΔP t As shown in formula (3)::(3)

[0052] The task-side robot calculates the corresponding command value based on the received posture and position change and inputs it into the robot controller, the force command of the hand controller for:

[0053]

[0054] The task-side robot collects contact force information with the environment during work. r,t And transmitted to the operating end as a force command Input the hand controller, the force command of the hand controller The expression is as follows:

[0055]

[0056] In step S10, the processor uses the force feedback teleoperation system to collect demonstration data. The operator wears the muscle electrical signal sensor and uses the pre-built force feedback teleoperation system control device to control the robot to interact with the environment with unknown mechanical properties and complete the task. During the task, the force feedback teleoperation system continuously records relevant data, including the interaction force F between the operator and the control device. h,t , operator's electromyographic information A t 、Robot's running trajectory P r,t and the contact force F between the robot and the environment r,t , thus obtaining the original demonstration data set L, the original demonstration data set L is shown in formula (6):

[0057] L={P r,t ,A t ,F h,t ,F r,t |t=0,1,2……t end} (5)

[0058] In the formula, the subscript t represents the data corresponding to the data, and the total amount of data points is t end express.

[0059] The processor collects demonstration data by executing step S10, so that the skill transfer algorithm can be used to obtain sports skills and corresponding action planning strategy changes, and appropriate antagonistic movements or compliant movements can be obtained by reproducing the skills in the scenario of multi-contact tasks.

[0060] In an embodiment of the present application, step S20 may include the following execution process:

[0061] S201, unitizing the operation task based on the pre-processed interaction force data, the current operation trajectory, the contact force data and the electromyographic information to obtain corresponding unitized data;

[0062] S202, using a sliding average algorithm to process the interaction force data and the contact force data to obtain pre-processed interaction force data and contact force data;

[0063] S203, calculating the difference between the preprocessed interaction force data and the contact force data, determining the main motion direction matrix based on the difference and the unit step matrix, and determining the main motion direction according to the main motion direction matrix;

[0064] S204: Divide the operation task into multiple task stages based on the change of the main motion direction matrix in the demonstration data.

[0065] The processor implements trajectory preprocessing and divides the task stages through the above steps S201-S204. The present application divides the task stages in the trajectory data preprocessing stage, and performs segmented skill learning based on the divided stages. The main motion direction corresponding to the stage is also referred to in the action planning, so that the system has better adaptability to different environments and task objectives.

[0066] Specifically, the steps of the processor performing trajectory preprocessing and dividing the task phases may include the following execution process:

[0067] Perform trajectory preprocessing and divide the task into stages. The upper right superscript * indicates that it is processed data. The polynomial difference method is used to smooth the running trajectory and preprocess the speed planning to obtain a continuous and smooth demonstration trajectory. The contact force and interaction force in the demonstration data are preprocessed by sliding average. The strength of the electromyographic signal of each channel is weighted and summed, and then processed using sliding average to obtain The results were expressed as * It is expressed as shown in formula (7):

[0068]

[0069] The main motion direction is obtained based on the difference between the interaction force data between the operator and the control device and the contact force data between the robot and the environment. The calculation process is shown in equations (8) and (9):

[0070]

[0071] In formula (7), F d,t =[F xd,t F yd,t F zd,t ] T is the difference between the interaction force data between the operator and the control device and the contact force data between the robot and the environment at time t. In formula (8), D t represents the main motion direction at time t, H(x) is the unit step function, which is defined as shown in formula (10):

[0072]

[0073] According to D t The change of the value in the entire demonstration data can divide the task into different stages, and obtain the main motion direction corresponding to each stage, which is expressed by formula (11):

[0074]

[0075] In an embodiment of the present application, step S30 may include the following execution process:

[0076] The strategy adjustment parameters of each task stage are determined based on the index of electromyographic information.

[0077] In this step, the processor may calculate the policy adjustment parameters from the demonstration data.

[0078] Specifically, the processor can calculate the strategy adjustment parameters in each task stage according to the processed electromyographic information in the demonstration data, as shown in formula (12):

[0079]

[0080] In formula (11), K t is the strategy adjustment parameter at time t, and k and γ are parameters set in the initialization phase based on data characteristics and empirical knowledge.

[0081] In an embodiment of the present application, step S40 may include the following execution process:

[0082] S401, determining nonlinear terms of each task stage based on basis functions, regular system expressions and weights of basis functions;

[0083] S402, determining a dynamic motion primitive model for each task stage based on each nonlinear term;

[0084] S403 . Based on the robot running trajectory and strategy adjustment parameters in each task stage, the mapping relationship between the strategy adjustment parameters and the running trajectory in the main motion direction is calculated by an interpolation algorithm.

[0085] The processor implements encoding of the trajectory and strategy adjustment parameters in stages through the above steps S401 to S403.

[0086] Specifically, the step of the processor encoding the trajectory and strategy adjustment parameters in stages may include the following execution process:

[0087] For each task stage, the dynamic motion primitive is used to encode the relationship between the desired trajectory and time of the robot end. The form of the dynamic motion primitive is shown in formula (13):

[0088]

[0089] Among them, p represents each component of the position information P, that is, for the x, y, z dynamic motion primitives, the same form is satisfied. Replacing p can obtain the specific form of each component. In the formula g is the set target value, τ is the set time scaling constant, α p and β p is a set constant, f is a nonlinear term, and the construction is shown in formula (14).

[0090]

[0091] Where θ comes from the regular system, θ=1 corresponds to the initial state, and the expression of the regular system is shown in formula (15):

[0092]

[0093] In formula (13), p 0 represents the initial state, ω i For different basis functions Ψ i The weight of (θ), N is the number of basis functions, and the basis function Ψ i The expression of (θ) is shown in (16).

[0094]

[0095] In formula (15), c i is the center position of the basis function, which needs to be obtained in the learning. After constructing the dynamic motion primitive, it is necessary to learn through the data obtained in step 4 to obtain various parameters in the model.

[0096] Extract the obtained in step 3 With t 1 express The total duration of this data segment is first set based on experience. p ,α θ ,β p ,N. For the learning of f, the local weighted regression method is used. By transforming formula (12), the f to be learned can be obtained t As shown in formula (17):

[0097]

[0098] In formula (16), p d P r The components of the basis function Evenly distributed over the entire time period, define ξ(t) = θ(t)(gp 0 ), we can get the weight ω of the basis function i , as shown in formula (18):

[0099]

[0100] in:

[0101]

[0102] According to the main motion direction, running trajectory and strategy adjustment parameters in each stage, the mapping relationship between the strategy adjustment parameters and the trajectory is obtained by interpolation, as shown in formula (20):

[0103]

[0104] In formula (19), Obtain the position coordinates of the running trajectory at each moment in the active direction of each task stage, h n () represents the mapping relationship between the strategy adjustment parameters and the running trajectory of the corresponding task in the nth stage.

[0105] In an embodiment of the present application, step S50 may include the following execution process:

[0106] S501, obtaining the trajectory components of the current running trajectory of the robot in various directions at the current moment based on the dynamic motion primitive model, and determining the expected running trajectory of the robot based on the trajectory components in various directions;

[0107] S502: determining the terminal position deviation and strategy adjustment parameters at the current moment based on the difference between the expected running trajectory and the current running trajectory;

[0108] S503, determining the expected running trajectory of the robot at the next moment based on the strategy adjustment parameters, the end position deviation and the contact force data;

[0109] S504, calculating the step length at the next moment based on a preset step length calculation formula and mapping relationship;

[0110] S505 , based on the step size at the next moment, the current running trajectory and the expected running trajectory, determine the end position instruction of the robot, and obtain the planned target action of the robot based on the end position instruction.

[0111] The processor completes the task by reproducing skills through the above steps S501 to S505. The present application uses dynamic motion primitives to learn motion trajectories and strategy changes, which can realize the learning and reproduction of human operating skills and improve the integrity of operating skills.

[0112] Specifically, the steps of completing the task through skill reproduction may include the following execution process:

[0113] Reproduce skills in order to complete the task. In each stage, first obtain the expected trajectory corresponding to that moment through the dynamic primitive model in step 5: determine the target position g according to the task information of that stage r and the time scaling parameter τ of the task. p ,β p ,ω i ,Ψ i ,α θ , combined with formula (21), the components p of the robot end reference trajectory in each direction can be calculated:

[0114]

[0115] The expected trajectory P of the robot is obtained by formula (20): d,t =[P dx,t P dy,t P dz,t ] T , and calculate the end position deviation E at the current moment t The strategy adjustment parameters corresponding to the current position are shown in equations (22) and (23) respectively:

[0116] E t =P d,t -P r,t (twenty one)

[0117]

[0118] The expected trajectory P of the robot end at the next moment s,t+1 It is calculated by combining the position deviation and contact force of the robot end with the strategy adjustment parameters, as shown in formula (24):

[0119]

[0120] In formula (23), F r,t is the contact force measured by the robot end force sensor in each direction at time t, K 0 is a fixed parameter.

[0121] In P s,t+1 ≤P s,min When, step length s t+1 =0; in P s,t+1 >P s,min When, step length s t+1 Calculate according to formula (25):

[0122]

[0123] In formula (24), s m and l These are parameters set in advance according to the task, namely the basic compensation value and the step limit value. t+1 and the desired direction of motion P s,t+1 Can calculate the robot end position command Realize the control of the robot as shown in the formula.

[0124]

[0125] Among them, s t+1 represents the step size of the next moment, P s,t+1 represents the expected running trajectory, P r,t Indicates the current running track.

[0126] refer to Figure 2 Based on the above embodiments, the present application further provides a remote operation skill transfer device combined with strategy adjustment. The remote operation skill transfer device 100 combined with strategy adjustment includes a data collection module 101, a task division module 102, a parameter determination module 103, a mapping module 104 and an action planning module 105, wherein:

[0127] The data collection module 101 is used to complete the operation task based on the pre-built force feedback teleoperation system to collect demonstration data, wherein the demonstration data includes the interaction force data between the operator and the control device, the operator's electromyographic information, the current operation trajectory of the robot, and the contact force data between the robot and the environment;

[0128] The task division module 102 is used to determine the main motion direction according to the interaction force data and the contact force data, and divide the operation task into various stages based on the main motion direction to obtain corresponding demonstration data for each task stage;

[0129] The parameter determination module 103 is used to determine the corresponding strategy adjustment parameters based on the demonstration data of each task stage;

[0130] The mapping module 104 is used to encode the robot's running trajectory in stages using a dynamic motion primitive model, and calculate the mapping relationship between the strategy adjustment parameters in the main motion direction and the running trajectory through an interpolation algorithm;

[0131] The action planning module 105 is used to plan the robot's target action based on the pre-built motion planner according to the mapping relationship, expected position and contact force data at the current moment, so that the robot can autonomously complete the contact operation, wherein the expected position is determined based on the current running trajectory.

[0132] Based on the above embodiments, the present application also provides a computer-readable storage medium, which includes instructions, which, when executed on a computer, enables the computer to implement any of the aforementioned methods to provide a remote operation skill transfer method combined with strategy adjustment.

[0133] Based on the above embodiments, the present application also provides an electronic device, which includes: at least one processor, a memory and an input and output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the remote operation skill transfer method combined with strategy adjustment provided by any of the aforementioned methods.

[0134] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A teleoperation skill transfer method combined with strategy adjustment, characterized in that: include: Complete the operation task based on the pre-built force feedback teleoperation system to collect demonstration data, where the demonstration data includes the interaction force data between the operator and the control device, the operator's electromyography information, the robot's current running trajectory, and the contact force data between the robot and the environment; Determine a main motion direction according to the interaction force data and the contact force data, and divide the operation task into various stages based on the main motion direction to obtain corresponding demonstration data for each task stage; Determine the corresponding strategy adjustment parameters based on the demonstration data of each task stage; The dynamic motion primitive model is used to encode the robot's running trajectory in stages, and the mapping relationship between the strategy adjustment parameters in the main motion direction and the running trajectory is calculated through the interpolation algorithm. Based on the pre-built motion planner, the target action of the robot is planned according to the acquired mapping relationship at the current moment, the expected position and the contact force data, so that the robot can complete the contact operation autonomously, wherein the expected position is determined based on the current operation trajectory.

2. The teleoperation skill transfer method combined with strategy adjustment as claimed in claim 1, characterized in that: The force feedback teleoperation system comprises a task end and an operation end, wherein the task end has a robot, and the operation end has a hand controller. In the pre-built force feedback teleoperation system, an operation task is completed to collect demonstration data. The teleoperation skill transfer method combined with strategy adjustment also includes: Based on the pre-built force feedback teleoperation system, acquiring the terminal position information and the contact force information of the robot when completing the operation task; Sending the end position information and the contact force information to the hand controller; Processing the terminal position information and the contact force information to obtain the hand controller terminal position information and the hand controller interaction force information; Fixedly collecting the hand controller end position information and the hand controller position change information and sending them to the robot; Processing the hand controller end position information and the hand controller position change to obtain the position instruction of the robot controller; The contact force information of the robot is acquired, the contact force information is transmitted to the hand controller, and the contact force information is processed to obtain a force instruction of the hand controller.

3. The teleoperation skill transfer method combined with strategy adjustment as claimed in claim 1, characterized in that: The determining the main motion direction according to the interaction force data and the contact force data, and dividing the task phases based on the main motion direction, includes: Unitize the operation task based on the pre-processed interaction force data, the current operation trajectory, the contact force data and the myoelectric information to obtain corresponding unitized data; Processing the interaction force data and the contact force data using a sliding average algorithm to obtain pre-processed interaction force data and contact force data; Calculating the difference between the preprocessed interaction force data and the contact force data, determining a main motion direction matrix based on the difference and a unit step matrix, and determining the main motion direction according to the main motion direction matrix; Based on the changes of the main motion direction matrix in the demonstration data, the operation task is divided into a plurality of task stages.

4. The teleoperation skill transfer method combined with strategy adjustment as claimed in claim 1, characterized in that: The determining of corresponding strategy adjustment parameters based on the demonstration data of each task stage includes: The strategy adjustment parameter of each task phase is determined based on the index of the electromyographic information.

5. The teleoperation skill transfer method combined with strategy adjustment as claimed in claim 1, characterized in that: The expression of the dynamic motion primitive model includes nonlinear terms, and the dynamic motion primitive model is used to encode the robot's running trajectory in stages, and the mapping relationship between the strategy adjustment parameters in the main motion direction and the running trajectory is calculated by an interpolation algorithm, including: Determining the nonlinear term of each of the task stages based on a basis function, a regular system expression and a weight of the basis function; Determine the dynamic motion primitive model of each task stage based on each nonlinear term; Based on the robot's running trajectory and strategy adjustment parameters in each task stage, the mapping relationship between the strategy adjustment parameters and the running trajectory in the main motion direction is calculated by an interpolation algorithm.

6. The teleoperation skill transfer method combined with strategy adjustment as claimed in claim 1, characterized in that: The pre-built motion planner plans the target motion of the robot according to the acquired mapping relationship at the current moment, the expected position and the contact force data, including: Acquire the trajectory components of the current running trajectory of the robot at the current moment in each direction based on the dynamic motion primitive model, and determine the expected running trajectory of the robot based on the trajectory components in each direction; Determining the terminal position deviation and strategy adjustment parameters at the current moment based on the difference between the expected running trajectory and the current running trajectory; Determining the expected running trajectory of the robot at the next moment based on the strategy adjustment parameter, the end position deviation and the contact force data; Calculating the step length at the next moment based on a preset step length calculation formula and the mapping relationship; Based on the step size at the next moment, the current running trajectory and the expected running trajectory, the end position instruction of the robot is determined, and the planned target action of the robot is obtained based on the end position instruction.

7. The teleoperation skill transfer method combined with strategy adjustment as claimed in claim 6, characterized in that: The expression of the end position instruction is: Among them, s t+1 Indicates the step size of the next moment, P s,t+1 represents the expected running trajectory, P r,t Indicates the current running track.

8. A remote control skill transfer device combined with strategy adjustment, characterized in that: include: A data collection module, used to complete the operation task based on the pre-built force feedback teleoperation system to collect demonstration data, wherein the demonstration data includes the interaction force data between the operator and the control device, the operator's electromyographic information, the current operation trajectory of the robot, and the contact force data between the robot and the environment; A task division module, used for determining a main motion direction according to the interaction force data and the contact force data, and dividing each stage of the operation task based on the main motion direction to obtain corresponding demonstration data for each task stage; A parameter determination module, used to determine corresponding strategy adjustment parameters based on the demonstration data of each task stage; A mapping module is used to encode the robot's running trajectory in stages using a dynamic motion primitive model, and calculate the mapping relationship between the strategy adjustment parameters in the main motion direction and the running trajectory through an interpolation algorithm; The action planning module is used to plan the target action of the robot based on the mapping relationship, the expected position and the contact force data at the current moment acquired by the pre-built motion planner, so that the robot can autonomously complete the contact operation, wherein the expected position is determined based on the current operation trajectory.

9. A computer-readable storage medium, characterized in that: The invention comprises instructions, which, when executed on a computer, enable the computer to execute the remote operation skill transfer method combined with strategy adjustment as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor, memory, and input-output unit; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the remote operation skill transfer method combined with strategy adjustment according to any one of claims 1 to 7.

Images