Dynamic motion primitive learning method based on unsupervised trajectory segmentation

Abstract

The invention particularly relates to a dynamic motion primitive learning method based on unsupervised trajectory segmentation. The method comprises the steps that the tail end of the robot is dragged to conduct teaching along an expected track, and observation data of the teaching track are obtained; performing unsupervised learning on the teaching track by adopting a Bayesian nonparametric model, and identifying and segmenting a plurality of track fragments with different dynamic modes by taking observation data of the teaching track as original data; on the basis of the constructed complexity judgment function, complexity calculation is carried out on all the track fragments, and complexity evaluation parameters corresponding to all the track fragments are obtained; dynamically allocating DMP primary functions with different numbers and positions to each track fragment based on the complexity evaluation parameter corresponding to each track fragment; and establishing a dynamic motion primitive model for each track fragment by using the distributed DMP primary function, and completing learning and representation of the teaching track. According to the method, the problem of poor local dynamic fitting of the complex track in the prior art can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot programming and imitation learning, in particular to a dynamic movement primitive learning method based on unsupervised trajectory segmentation. BACKGROUND

[0002] In the field of robot imitation learning, how to enable robots to quickly master complex motor skills through a small amount of demonstration is the key to reducing the programming threshold and improving the intelligence level of robots. Dynamic movement primitives (DMP) as one of the core methods in this field are widely favored because they can stably represent, learn and reproduce complex trajectories.

[0003] The core idea of DMP is to use a dynamic system (usually a set of differential equations) to characterize the trajectory. This system consists of two parts: an "attractor" term that ensures the system converges to the target point, and a "shaping" term that accurately fits the shape of the demonstration trajectory by weighting the basis functions. The basis functions can be understood as a series of "standard strokes" or "templates" with local influence, and by combining them, complex trajectory shapes can be described.

[0004] However, traditional DMP usually treats a complete demonstration trajectory as a single movement primitive. This "holistic" modeling strategy is very effective for simple, single-pattern movements (such as straight-line motion, smooth arcs). But when it comes to complex long trajectories composed of multiple heterogeneous sub-tasks, the "detail loss and oversmoothing problem" defect of DMP is exposed.

[0005] To solve the above problem, the existing technology adopts the method of trajectory segmentation, which decomposes the complex long trajectory into a series of simple, homogeneous sub-trajectory segments according to its movement pattern, and then establishes a simple DMP model for each segment to solve.

[0006] The trajectory segmentation technology of the above dynamic motion primitive has solved the problems of detail loss and over-smoothing to some extent, but still has significant deficiencies. For example, in the existing dynamic motion primitive learning method, the dependence on preset parameters is strong, which specifically manifests that the user needs to specify the number of segmented segments in advance or set a sensitive segmentation threshold. Such parameters are strongly related to specific tasks, and often need to be adjusted in different scenarios, so that the existing dynamic motion primitive learning method is difficult to adapt to diversified unknown tasks. In addition, in the existing dynamic motion primitive learning method, the segmentation basis is one-sided, which is mostly limited to the static geometric features of the trajectory, such as position, curvature, etc., and ignores the dynamic change characteristics of the trajectory (such as the change pattern of kinematics required to generate the trajectory, such as speed, acceleration, force, etc. over time), thereby causing the problems of incorrect segmentation of DMP or inability to segment at key dynamic change points. SUMMARY

[0007] The purpose of the present application is to provide a dynamic motion primitive learning method based on unsupervised trajectory segmentation, which can adaptively segment complex demonstration trajectories without manual preset parameters, greatly reducing the dependence on preset parameters. Moreover, the method can dynamically allocate modeling resources according to the complexity of each sub-segment, solve the problem of poor local dynamic fitting of complex trajectories in the prior art, and significantly improve the imitation accuracy and generalization performance of complex motor skills.

[0008] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0009] In a first aspect, the present application provides a dynamic motion primitive learning method based on unsupervised trajectory segmentation, which comprises:

[0010] S1, dragging the robot end along the intended trajectory for demonstration, obtaining observation data of the demonstration trajectory;

[0011] S2, using a Bayesian non-parametric model to perform unsupervised learning on the demonstration trajectory, taking the observation data of the demonstration trajectory as the original data, identifying and segmenting several trajectory segments with different dynamic modes;

[0012] S3, based on the constructed complexity judgment function, calculating the complexity of all trajectory segments to obtain the complexity evaluation parameters corresponding to each trajectory segment;

[0013] S4, based on the complexity evaluation parameters corresponding to each trajectory segment, dynamically allocating different numbers and positions of DMP basis functions for each trajectory segment;

[0014] S5, using the DMP basis functions allocated in S4 to establish a dynamic motion primitive model for each trajectory segment, completing the learning and representation of the demonstration trajectory.

[0015] As a preferred embodiment of the present application, the Bayesian nonparametric model in S2 is a Beta process autoregressive hidden Markov model.

[0016] As a preferred embodiment of the present application, the complexity judgment function in S3 is a one-dimensional function, including four complexity characteristic indexes of total variation , root mean square acceleration , absolute jerk and segment length .

[0017] As a preferred embodiment of the present application, S4 comprises the following sub-steps:

[0018] S41, dimensionless and weighted sum processing is performed on the four complexity characteristic indexes corresponding to each trajectory segment respectively, to obtain the comprehensive complexity of each trajectory segment in each dimension; S42, based on the proportion of the comprehensive complexity in the total complexity, the number of DMP basis functions that the segment should obtain in the dimension is calculated and allocated by using an integer function.

[0019] As a preferred embodiment of the present application, in S41, the weights corresponding to the four complexity characteristic indexes of total variation , root mean square acceleration , absolute jerk and segment length in the weighted sum operation are 0.2, 0.2, 0.1 and 0.5 respectively.

[0020] As a preferred embodiment of the present application, the specific steps for establishing a dynamic motion primitive model for each trajectory segment in S5 are as follows:

[0021] An independent dynamic motion primitive model is initialized, and Gaussian basis functions of the dynamic motion primitive model are configured according to the number and position of basis functions determined in S4;

[0022] The target forcing term is calculated reversely using the teaching trajectory observation data of the target trajectory segment;

[0023] A set of optimal weight parameters are learned by using a local weighted regression algorithm, so that the error between the weighted combination of the Gaussian basis functions and the target forcing term is minimized.

[0024] In a second aspect, the present application further provides a computer device comprising a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are executed by the processor to implement the steps of the above-mentioned dynamic motion primitive learning method based on unsupervised trajectory segmentation.

[0025] In a third aspect, the present application also provides a computer readable storage medium, wherein the storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement the steps of the above-mentioned dynamic motion primitive learning method based on unsupervised trajectory segmentation.

[0026] In summary, the present application has the following advantages:

[0027] 1. The present application can automatically identify the segments with different dynamic modes in the demonstration trajectory by using the Bayesian non-parametric model for unsupervised trajectory segmentation, without manually presetting the number of segmentation segments or threshold values; the present application reduces the dependence on task-specific parameters, so that the robot can quickly adapt to diverse unknown task scenarios, and significantly improves the generalization ability and ease of use of the system.

[0028] 2. The present application quantifies the complexity of each trajectory segment by introducing a complexity judgment function that fuses geometric, kinematic and time characteristics, and dynamically allocates the number and position of DMP basis functions accordingly; this resource allocation strategy ensures that complex trajectory segments obtain more modeling resources, thereby effectively capturing local details and dynamic changes, and solving the problem of over-smoothing or poor fitting caused by the uniform distribution of basis functions in traditional DMP, while ensuring the learning efficiency, significantly improving the precision of trajectory imitation and the quality of motion skill reproduction. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0030] Figure 1 The method flowchart of the present dynamic motion primitive learning method;

[0031] Figure 2 The example schematic diagram of the two-dimensional trajectory in the embodiment of the present application;

[0032] Figure 3 The segmentation schematic diagram of the two-dimensional trajectory in the embodiment of the present application using Beta process autoregressive hidden Markov; Figure 2

[0033] Figure 4 The basis function distribution diagram after the curve complexity judgment in the embodiment of the present application;

[0034] Figure 5 The reproduction diagram of the two-dimensional trajectory in the embodiment of the present application. DETAILED DESCRIPTION ​

[0035] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that these implementations are discussed solely for the purpose of illustrating aspects of the subject matter described herein and are not a limitation of the scope, applicability, or examples set forth in the claims. Changes in the function and arrangement of elements discussed can be made without departing from the scope of the subject matter described herein. Various examples can omit, substitute, or add various procedures or components as appropriate. For instance, the methods described can be performed in an order different than that described, and / or various steps can be added, omitted, or combined. Also, features described with respect to some examples can be combined in other examples.

[0036] As used herein, the term "includes" and its variants are meant to be interpreted broadly. The term "based on" means "based, at least in part, on." The terms "one embodiment" and "an embodiment" mean "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The terms "a first," "a second," etc. do not require that there be only one of each, but rather, there can be one or more of each. The following detailed description is presented in terms of examples. Further modifications and alternatives can be implemented.

[0037] As Figure 1 shown, the embodiment provides a dynamic motion primitive learning method based on unsupervised trajectory segmentation, the method comprising:

[0038] S1, dragging the robot end along the expected trajectory to teach, obtaining the observation data of the teaching trajectory;

[0039] S2, using a Bayesian non-parametric model to perform unsupervised learning on the teaching trajectory, taking the observation data of the teaching trajectory as the original data, identifying and segmenting several trajectory segments with different dynamic modes;

[0040] S3, based on the constructed complexity judgment function, calculating the complexity of all trajectory segments to obtain the respective complexity evaluation parameters of each trajectory segment;

[0041] S4, based on the respective complexity evaluation parameters of each trajectory segment, dynamically assigning different numbers and positions of DMP basis functions to each trajectory segment;

[0042] S5, using the DMP basis functions assigned by S4 to establish a dynamic motion primitive model for each trajectory segment, completing the learning and representation of the teaching trajectory.

[0043] Firstly, in S1, the trajectory teaching and data collection step is performed. An operator manually drags the end of a robot (e.g. a robotic arm) to move along a desired work trajectory, during which the system collects and records the position sequence (including velocity, acceleration, etc.) of the end effector in the Cartesian space through the sensors (e.g. encoders) built in the robot, forming an observation data set {y t} of the teaching trajectory, where t = 1, 2,..., T, and T is the total number of time steps.

[0044] Then, in S2, the unsupervised trajectory segmentation step based on the Bayesian nonparametric model is performed. This step uses a Beta process autoregressive hidden Markov model (BP-AR-HMM) to perform unsupervised learning and automatic segmentation on the observation data of the teaching trajectory obtained in step S1. As shown in Figure 2 Figure 2 The trajectory is a simulated planar two-dimensional trajectory, and in order to realize adaptive allocation of the basis functions of the dynamic motion primitive, the first task is to segment the teaching trajectory meaningfully. The Beta process autoregressive hidden Markov model used in this embodiment can overcome the limitation of traditional segmentation methods that require pre-defined segment numbers. This model can automatically infer the required number of segments during the segmentation process, and divide the motion into several meaningful sub-motions.

[0045] Specifically, the model assumes that there is a globally shared, potentially infinite "dynamic pattern library". Each pattern (i.e. a hidden state) is defined by its own dynamic parameters , where is a set of autoregressive coefficients, is the noise covariance. A Beta process is used as a prior to define this library, which assigns a global "activation" probability to each dynamic pattern k, as follows:

[0046]

[0047]

[0048] where, is the base measure; is the concentration parameter; is the Dirac function at parameter .

[0049] ​For each individual demonstration trajectory, the model selects a subset of features from the global library through a Bernoulli process, which guarantees that different trajectories can share common motion patterns while preserving their individual uniqueness. Based on the selected feature subset, each trajectory builds a state transition probability distribution that specifies the rules of switching from one motion pattern to another, as follows:

[0050]

[0051] After determining the rules of state transition, the key of the model lies in its autoregressive emission model. When the system is in state at time t, the current observation y t is modeled as a linear function of its previous p observations with a state-specific Gaussian noise, as follows:

[0052]

[0053] When , the linear function of the previous p observations is modeled as .

[0054] As shown in FIG. 3, the Beta process autoregressive hidden Markov model automatically segments the trajectory into three small motion segments, which are represented by different colors in the figure. Figure 3

[0055] Next, in S3, the trajectory segment complexity calculation step is performed.

[0056] Specifically, a complexity judgment function including geometric, kinematic and time features is designed to quantify the complexity of the segmented segments. Since the dynamic motion primitive (DMP) models each dimension of the trajectory separately, and the complexity in each dimension is different, in order to quantify the complexity of each segment, the embodiment designs a one-dimensional complexity judgment function C. The function considers four aspects of features of the segment, including geometry, kinematics, smoothness and time persistence. For the jth motion segment after segmentation, the trajectory in the dth dimension is , where is the time length of the trajectory segment, is the sampling time interval. The complexity judgment function of the curve is as follows:

[0057]

[0058] where, is the total variation; is the root mean square acceleration; is the absolute jerk; ​is the length of the trajectory segment; the set of above functions constitutes the complexity evaluation parameters of the method embodiment.

[0059] Then, the step of building dynamic motion primitive model and learning is executed.

[0060] Specifically, the calculated complexity evaluation parameters are used to guide the number and position distribution of new DMP basis functions; after obtaining the trajectory segment information and the curve complexity evaluation parameters of each trajectory segment, a comprehensive complexity formula is obtained after dimensionless treatment as follows:

[0061]

[0062] wherein, is the weight, In the embodiment, it can be preferably set as .

[0063] Finally, the DMP basis functions allocated to each dimension of each segmented curve are calculated as follows:

[0064]

[0065] wherein, M is the number of trajectory segments; j represents the jth segmented curve; d represents the dth dimension of the jth trajectory segment.

[0066] After obtaining the number of basis functions, they are uniformly distributed in the respective trajectory segments. As shown in Figure 4 , the DMP basis functions of the dynamic motion primitive model of the x, y two coordinate axes are redistributed, and the distribution of each trajectory segment in each dimension is shown in the figure.

[0067] Finally, in S5, the dynamic motion primitive model is established with the new basis functions.

[0068] It is known that the dynamic motion primitive is essentially a second-order, nonlinear spring-damper system, and its internal process is controlled by a monotone decreasing regular system; the phase x gradually converges to 0 with the process, ensuring that the DMP can converge to the target point g, and the forcing term f(x) at a certain moment is used to fit the specific formula of the curve as follows:

[0069]

[0070]

[0071]

[0072]

[0073] wherein, , , respectively represent the position, velocity and acceleration of the system; is the target point of motion; is a selected positive number to ensure the convergence of the system to ; is the learnable weight; N is the total number of basis functions; is the starting point of motion; is the ith Gaussian kernel function; and are the position and width of the basis function; in this embodiment, the new result replaces to reproduce the demonstration trajectory.

[0074] As shown in Figure 5 , the above-mentioned embodiments provided by the present application can better reproduce the demonstration trajectory based on the dynamic motion primitive of unsupervised segmentation.

[0075] In another possible embodiment, a communication device is also provided, which includes a processor and a memory, and the memory stores computer program instructions which, when executed by the processor, perform the steps of the above-mentioned dynamic motion primitive learning method based on unsupervised trajectory segmentation.

[0076] In another possible embodiment, a computer readable storage medium is also provided, which stores computer program instructions, and the computer program instructions, when executed by a processor, perform the steps of the above-mentioned dynamic motion primitive learning method based on unsupervised trajectory segmentation.

[0077] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different than the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or possible.

Claims

1. A dynamic motion primitive learning method based on unsupervised trajectory segmentation, characterized in that the method... The method comprises the following steps: S1, a robot end is dragged along a planned trajectory to demonstrate, and observation data of the demonstration trajectory is obtained; S2, a Bayesian non-parametric model is used to unsupervisedly learn the demonstration trajectory, and a plurality of trajectory segments with different dynamic modes are identified and segmented from the observation data of the demonstration trajectory; S3, a complexity judgment function is constructed, and complexity of all trajectory segments is calculated to obtain complexity evaluation parameters corresponding to each trajectory segment; S4, each trajectory segment is dynamically assigned with different numbers and positions of DMP basis functions based on the complexity evaluation parameters corresponding to each trajectory segment; S5, a dynamic motion primitive model is established for each trajectory segment by using the DMP basis functions assigned in S4, and learning and representation of the demonstration trajectory are completed. 2.The method of claim 1, wherein, The Bayesian non-parametric model in S2 is a Beta process autoregressive hidden Markov model. 3.The method of claim 2, wherein, The complexity judgment function in S3 is a one-dimensional function, including total variation , root mean square acceleration , absolute jerk and segment length four complexity characteristic indexes. 4.The method of claim 3, wherein, S4 comprises the following steps: S41, four complexity characteristic indexes corresponding to each trajectory segment are subjected to dimensionless and weighted summation processing to obtain comprehensive complexity of each trajectory segment in each dimension; S42, the number of DMP basis functions that should be obtained by the segment in the dimension is calculated and assigned by using an integer function based on the proportion of the comprehensive complexity in the total complexity.

5. The method of claim 4, wherein, In S41, in the weighted summation operation, the total variation , the root mean square acceleration , the absolute jerk , and the segment length The respective weights of the four complexity feature indicators are 0.2, 0.2, 0.1, and 0.5, in order. 6.The method of claim 5, wherein, In S5, the specific steps of establishing a dynamic motion primitive model for each trajectory segment are as follows: An independent dynamic motion primitive model is initialized, and Gaussian basis functions of the dynamic motion primitive model are configured according to the number and position of basis functions determined in S4; A target forcing term is reversely calculated by using demonstration trajectory observation data of a target trajectory segment; A set of optimal weight parameters are learned by using a local weighted regression algorithm, so that the error between the weighted combination of the Gaussian basis functions and the target forcing term is minimized.

7. A computer device comprising a processor and a memory, the memory having stored therein computer program instructions, characterized in that, The computer program instructions are executed to implement the steps of the dynamic motion primitive learning method based on unsupervised trajectory segmentation according to any one of claims 1-6.

8. A computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed to implement the steps of the dynamic motion primitive learning method based on unsupervised trajectory segmentation according to any one of claims 1-6.

Images