Adaptive dynamic programming based motor skill teaching control method and system

Abstract

The application relates to the field of robot motion control, and provides a motion skill teaching control method and system based on adaptive dynamic programming, which comprises the following steps: setting a virtual teacher's motion path, constructing a virtual teacher's controller through the motion path, and controlling the virtual teacher to move according to the motion path through the controller; a participant's hand follows the virtual teacher to move, and the virtual teacher calculates the position and speed of the participant's hand and the speed of the virtual teacher in real time; an intention coupling term is calculated through the position and speed of the participant's hand and the speed of the virtual teacher, a progress / regression mathematical model is constructed through the intention coupling term, and the learning state of the participant is judged through the progress / regression mathematical model. The speed and acceleration of the virtual teacher during motion are adjusted in real time through the speed of the participant, so that the motion track and speed of the participant and the virtual teacher have high coincidence, and the learning effect of the participant is optimized.

Description

Technical Field

[0001] This invention relates to the field of robot motion control, and in particular to a motion skill transfer control method and system based on adaptive dynamic programming. Background Technology

[0002] Interpersonal coordination is a natural bridge for individuals to cooperate with others. It originates from a dynamic and complex set of concretization processes involving adaptive nonverbal behaviors, emotions, actions, and feelings. For individuals with social difficulties, in terms of imitative interaction synchronization, they exhibit impaired ability to imitate and synchronize nonverbal behaviors, poor coordination, and clumsy and unnatural movements.

[0003] Currently, there are methods to guide people with social difficulties in social training through motion robots. Existing control methods for motion robots include PID control, iterative learning control, and event-triggered control. These control methods perform well in ensuring the stability, accuracy, and low energy consumption of the robot's motion trajectory. However, because the motion characteristics vary significantly among different individuals, using a single motion pattern to interact with different individuals often fails to achieve the expected results, and these control methods therefore lack flexibility. Summary of the Invention

[0004] To solve the above-mentioned technical problems, the present invention provides a motion skill transfer control method based on adaptive dynamic programming, comprising:

[0005] S1: Set the motion path of the virtual teacher, construct the controller of the virtual teacher through the motion path, and control the virtual teacher to move according to the motion path through the controller;

[0006] S2: The participant's hands follow the virtual teacher's movements, and the virtual teacher calculates and obtains the position and speed of the participant's hands and the speed of the virtual teacher in real time;

[0007] S3: The intention coupling term is obtained by calculating the position and speed of the participants' hands and the speed of the virtual teacher. The progress / regression mathematical model is constructed by the intention coupling term, and the learning status of the participants is judged by the progress / regression mathematical model.

[0008] Preferably, the expression for the movement path of the virtual teacher is:

[0009]

[0010] in, , This represents the amplitude of motion in two orthogonal directions. , These represent fixed motion frequencies in two orthogonal directions. , The initial phases are for two orthogonal directions. The data sampling time for the virtual teacher.

[0011] Preferably, the expression for the virtual teacher's controller is:

[0012]

[0013]

[0014] in, and All are symmetric positive definite matrices of appropriate dimension selected by the controller. The symmetric positive definite matrix selected for the controller The inverse matrix; This is the transpose of the controller's control function; This represents the gradient of the optimal value function with respect to the error vector; Let represent the motion error vector between the virtual teacher and the participant, and its transpose, respectively. These represent the controller's control input vector and its transpose vector, respectively. Let be the cost function.

[0015] Preferably, the formula for calculating the position and speed of the participant's hand is:

[0016]

[0017] in, These represent the participant's velocity and acceleration, respectively. t represents the data sampling time of the virtual teacher; k represents the sampling time, and the value of k is less than t. and This indicates the location of the participant during the two sampling sessions; and These are the speeds calculated from the two samples taken, one before and one after.

[0018] The preferred formula for calculating the speed of the virtual teacher is:

[0019]

[0020] in, and These represent the virtual teacher's position and speed, respectively. Accelerating the development of virtual teachers; t represents the data sampling time of the virtual teacher; k represents the sampling time, and the value of k is less than t.

[0021] Preferably, the formula for calculating the intention coupling term is:

[0022]

[0023] in, These represent the acceleration, velocity, and position of the virtual teacher, respectively. For control parameters, It is the intrinsic frequency of the virtual teacher movement. , These are used to describe the progress coefficient and regression coefficient, respectively, in the learning process. This represents the movement path of the virtual teacher. Indicates the participant's movement speed, This refers to the data sampling time for the virtual teacher. This represents the intended coupling term of the control system.

[0024] The preferred expression for the progress / regression mathematical model is:

[0025]

[0026]

[0027] in, , These are used to describe the progress coefficient and regression coefficient, respectively, in the learning process. , These represent the rate of progress and the rate of regression, respectively. The data sampling time for the virtual teacher.

[0028] A motion skill transfer control system based on adaptive dynamic programming includes:

[0029] The virtual teacher motion module is used to set the motion path of the virtual teacher, build a controller for the virtual teacher through the motion path, and control the virtual teacher to move according to the motion path through the controller;

[0030] The data acquisition module is used to track the movement of the virtual teacher's hands. The virtual teacher calculates and acquires the position and speed of the participants' hands and the speed of the virtual teacher in real time.

[0031] The learning status judgment module is used to calculate the intention coupling term by the position and speed of the participant's hands and the speed of the virtual teacher, construct a progress / regression mathematical model by the intention coupling term, and judge the participant's learning status by the progress / regression mathematical model.

[0032] The present invention has the following beneficial effects:

[0033] 1. The speed and acceleration of the virtual teacher's movement are adjusted in real time based on the speed of the participants, so that the movement trajectory and speed of the participants and the virtual teacher have a high degree of overlap, thus optimizing the learning effect of the participants;

[0034] 2. By adding a progress / regression mathematical model to the control of the virtual teacher, the movement of the virtual teacher can be adaptively changed according to the learning status of the participants, thus optimizing the participants' learning experience. Attached Figure Description

[0035] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of the movement path of the virtual teacher;

[0037] Figure 3 A diagram illustrating the follow-through motion of the participant's hand;

[0038] Figure 4 A schematic diagram illustrating the position and velocity time series of the participants' and virtual teacher's movements;

[0039] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0040] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0041] Reference Figure 1 This invention provides a motion skill transfer control method based on adaptive dynamic programming, comprising:

[0042] S1: Set the motion path of the virtual teacher, construct the controller of the virtual teacher through the motion path, and control the virtual teacher to move according to the motion path through the controller;

[0043] S2: The participant's hands follow the virtual teacher's movements, and the virtual teacher calculates and obtains the position and speed of the participant's hands and the speed of the virtual teacher in real time;

[0044] S3: The intention coupling term is obtained by calculating the position and speed of the participants' hands and the speed of the virtual teacher. The progress / regression mathematical model is constructed by the intention coupling term, and the learning status of the participants is judged by the progress / regression mathematical model.

[0045] In this embodiment, the virtual teacher's movement path is as follows: Figure 2 As shown, this enables the virtual teacher to generate a series of actions to teach human participants smooth motion trajectories, helping participants generate specified motion patterns. The expression for the virtual teacher's motion path is:

[0046]

[0047] in, , This represents the amplitude of motion in two orthogonal directions. , These represent fixed motion frequencies in two orthogonal directions. , The initial phases are for two orthogonal directions. The data sampling time for the virtual teacher.

[0048] In this embodiment, the expression for the virtual teacher's controller is:

[0049]

[0050]

[0051] in, and All are symmetric positive definite matrices of appropriate dimension selected by the controller. The inverse matrix of the symmetric positive definite matrix selected for the controller; This is the transpose of the controller's control function; This represents the gradient of the optimal value function with respect to the error vector; Let represent the motion error vector between the virtual teacher and the participant, and its transpose, respectively. These represent the controller's control input vector and its transpose vector, respectively. Let be the cost function.

[0052] Specifically, the neural network function approximation of the cost function is as follows:

[0053]

[0054] in This is the transpose of the weight coefficient matrix of the neural network. This represents the activation function of a neural network, using the hyperbolic tangent function as the activation function.

[0055] In this embodiment, the participant's hand follows the movement as follows: Figure 3 As shown, the formulas for calculating the position and speed of the participant's hand are:

[0056]

[0057] in, These represent the participant's velocity and acceleration, respectively. t represents the data sampling time of the virtual teacher; k represents the sampling time, and the value of k is less than t. and This indicates the location of the participant during the two sampling sessions; and These are the speeds calculated from the two samples taken, one before and one after.

[0058] In this embodiment, the formula for calculating the speed of the virtual teacher is:

[0059]

[0060] in, and These represent the virtual teacher's position and speed, respectively. Accelerating the development of virtual teachers; t represents the data sampling time of the virtual teacher; k represents the sampling time, and the value of k is less than t.

[0061] In this embodiment, the virtual teacher adjusts its own speed and acceleration according to the participants' speed. The position time series and velocity time series of the participants and the virtual teacher's movements are as follows: Figure 4 As shown;

[0062] The formula for calculating the intention coupling term is:

[0063]

[0064] in, These represent the acceleration, velocity, and position of the virtual teacher, respectively. For control parameters, It is the intrinsic frequency of the virtual teacher movement. , These are used to describe the progress coefficient and regression coefficient, respectively, in the learning process. This represents the movement path of the virtual teacher. Indicates the participant's movement speed, This is the data sampling time for the virtual teacher. This represents the intended coupling term of the control system.

[0065] In this embodiment, the expression for the progress / regression mathematical model is:

[0066]

[0067]

[0068] in, , These are used to describe the progress coefficient and regression coefficient, respectively, in the learning process. , These represent the rate of progress and the rate of regression, respectively. The data sampling time for the virtual teacher.

[0069] Specifically, + =1, when the participant should begin learning , The larger the value, the greater the participant's progress. The larger the value, the greater the regression of the participant.

[0070] Selection of indicators for evaluating participants' learning status:

[0071] After prolonged physical interaction with a virtual teacher, participants learn and perceive the ability to imitate others' behavior—that is, social skills—and master the learned movements. To verify the effectiveness of the learning, this invention designs a series of performance indicators to measure the participants' learning progress.

[0072]

[0073] in, The virtual teacher is in the first The position at the time of the next sampling; It is the first The location of the human participants in the second sampling; This represents the total number of samples taken in the experiment. Used to describe the positional synchronization of two moving parties. The lower the value, the better the synchronization of movement positions.

[0074]

[0075] in, The movement trajectories of human participants and virtual teachers in the 1st... Phase difference at each sampling point; It is the L2 norm of the average value of the entire sample; The phase stability of their trajectories was measured. The higher the value, the more stable the phase difference of the motion is.

[0076]

[0077] in, These represent the kurtosis of the motion velocity distributions for the virtual teacher and the human participant, respectively. These represent the skewness of the motion speed distribution for virtual teachers and human participants, respectively.

[0078] The effectiveness of the virtual teacher controller designed in this invention was verified by comparing the movement trajectories of participants before and after the interactive game.

[0079] This invention provides a motion skill transfer control system based on adaptive dynamic programming, comprising:

[0080] The virtual teacher motion module is used to set the motion path of the virtual teacher, build a controller for the virtual teacher through the motion path, and control the virtual teacher to move according to the motion path through the controller;

[0081] The data acquisition module is used to track the movement of the virtual teacher's hands. The virtual teacher calculates and acquires the position and speed of the participants' hands and the speed of the virtual teacher in real time.

[0082] The learning status judgment module is used to calculate the intention coupling term by the position and speed of the participant's hands and the speed of the virtual teacher, construct a progress / regression mathematical model by the intention coupling term, and judge the participant's learning status by the progress / regression mathematical model.

[0083] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0084] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as identifiers.

[0085] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A motion skill transfer control method based on adaptive dynamic programming, characterized in that, include: S1: Set the motion path of the virtual teacher, construct the controller of the virtual teacher through the motion path, and control the virtual teacher to move according to the motion path through the controller; S2: The participant's hands follow the virtual teacher's movements, and the virtual teacher calculates and obtains the position and speed of the participant's hands and the speed of the virtual teacher in real time; S3: The intention coupling term is obtained by calculating the position and speed of the participants' hands and the speed of the virtual teacher. The progress / regression mathematical model is constructed by the intention coupling term, and the learning status of the participants is judged by the progress / regression mathematical model. The formula for calculating the intention coupling term is: in, These represent the acceleration, velocity, and position of the virtual teacher, respectively. For control parameters, It is the eigenfrequency of the virtual teacher movement. , These are used to describe the progress coefficient and regression coefficient, respectively, in the learning process. This represents the movement path of the virtual teacher. Indicates the participant's movement speed, This refers to the data sampling time for the virtual teacher. Indicates the intended coupling term of the control system; The expression for the progress / regression mathematical model is: in, , These are used to describe the progress coefficient and regression coefficient, respectively, in the learning process. , These represent the rate of progress and the rate of regression, respectively. The data sampling time for the virtual teacher.

2. The motion skill transfer control method based on adaptive dynamic programming according to claim 1, characterized in that, The expression for the movement path of the virtual teacher is: in, , This represents the amplitude of motion in two orthogonal directions. , These represent fixed motion frequencies in two orthogonal directions. , The initial phases are for two orthogonal directions. The data sampling time for the virtual teacher.

3. The motion skill transfer control method based on adaptive dynamic programming according to claim 1, characterized in that, The expression for the virtual teacher's controller is: in, and All are symmetric positive definite matrices of appropriate dimension selected by the controller. The inverse matrix of the symmetric positive definite matrix selected for the controller; This is the transpose of the controller's control function; This represents the gradient of the optimal value function with respect to the error vector; Let represent the motion error vector between the virtual teacher and the participant, and its transpose, respectively. These represent the controller's control input vector and its transpose vector, respectively. Let be the cost function.

4. The motion skill transfer control method based on adaptive dynamic programming according to claim 1, characterized in that, The formula for calculating the position and speed of the participant's hand is: in, These represent the participant's velocity and acceleration, respectively. t represents the data sampling time of the virtual teacher; k represents the sampling time, and the value of k is less than t. and This indicates the location of the participant during the two sampling sessions; and These are the speeds calculated from the two samples taken, one before and one after.

5. The motion skill transfer control method based on adaptive dynamic programming according to claim 1, characterized in that, The formula for calculating the speed of a virtual teacher is: in, and These represent the virtual teacher's position and speed, respectively. Accelerating the development of virtual teachers; t represents the data sampling time of the virtual teacher; k represents the sampling time, and the value of k is less than t.

6. A motion skill transfer control system based on adaptive dynamic programming, characterized in that, To implement the method of any one of claims 1 to 5, comprising: The virtual teacher motion module is used to set the motion path of the virtual teacher, build a controller for the virtual teacher through the motion path, and control the virtual teacher to move according to the motion path through the controller; The data acquisition module is used to track the movement of the virtual teacher's hands. The virtual teacher calculates and acquires the position and speed of the participants' hands and the speed of the virtual teacher in real time. The learning status judgment module is used to calculate the intention coupling term by the position and speed of the participant's hands and the speed of the virtual teacher, construct a progress / regression mathematical model by the intention coupling term, and judge the participant's learning status by the progress / regression mathematical model.

Images