Dexterous hand dynamics modeling method and equipment

Abstract

The invention provides a dexterous hand dynamics modeling method and equipment. The method comprises the following steps: constructing a data set according to historical state data of the dexterous hand at multiple moments and corresponding historical moments; constructing a linearization model of the dexterous hand according to the dimension raising function; based on the data set, training the linearization model according to a preset loss function to obtain a kinetic model of the dexterous hand; the preset loss function comprises an evolution error term, a reconstruction error term, a controllability discrimination term and a stability constraint term; the evolution error term is used for representing an error between a dimension raising state and a state evolved by a Kupman operator, the reconstruction error term is used for representing an error generated by comparing a result obtained by raising dimension and then reducing dimension of the state with the state before dimension raising, and the controllability judgment term is used for representing a controllability condition of the linearization model; the stability constraint term is used for representing the stability condition of the linearization model; according to the method provided by the invention, the modeling precision and universality can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of humanoid robots, and in particular to a method and device for modeling the dynamics of a dexterous hand. Background Art

[0002] The dynamic model of the dexterous hand of a humanoid robot is a mathematical model that describes the relationship between force and motion during the movement of the dexterous hand.

[0003] In the related art, mechanism equations such as Lagrangian mechanics equations can be used for modeling.

[0004] However, in the process of implementing this application, the inventors found that there are at least the following problems in the prior art: the existing modeling method causes a large error between the constructed model and the real system and has low accuracy. Summary of the invention

[0005] The present application provides a method and device for dynamic modeling of a dexterous hand to improve the modeling accuracy of the dexterous hand.

[0006] In a first aspect, the present application provides a method for modeling the dynamics of a dexterous hand, comprising:

[0007] Construct a data set based on the historical state data of the dexterous hand at multiple moments and the corresponding historical torques;

[0008] Constructing a linearized model of a dexterous hand according to a dimensionality-raising function; the dimensionality-raising function includes a network model for mapping an original state to a high-dimensional observable space;

[0009] Based on the data set, the linearized model is trained according to a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-increased state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error generated by comparing the result of first increasing the dimension of the state and then reducing the dimension of the state with the state before the dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model.

[0010] In a possible design, the expression of the preset loss function is:

[0011]

[0012] Among them, θ is the network model parameter of the dimension-raising function in the training process, is the network model parameter value of the dimension-raising function when the objective function takes the minimum value, that is, the optimal parameter value, Loss 1is the evolution error term, which represents the error between the dimensional state and the state after the Koopman operator evolution. A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, and Loss 2 is the reconstruction error term, which indicates the error between the result obtained by first upgrading the dimension and then reducing the dimension of the current state and the state before upgrading. C represents the state dimensionality reduction matrix, α, β, and are weight coefficients, is the controllability discriminant, is a stability constraint.

[0013] In a possible design, the expression of the evolution error term is:

[0014]

[0015] Among them, Loss1 is the evolution error term, represents the matrix after dimensionality increase, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, and θ represents the network model parameter of the dimension-raising function;

[0016] The expression of the reconstruction error term is:

[0017]

[0018] Among them, Loss2 is the reconstruction error term, represents the matrix after dimension increase, C represents the state dimension reduction matrix, x k Indicates the current state.

[0019] In a possible design, the expression of the controllability discriminant term is:

[0020]

[0021] in, , represents the controllability matrix, is the rank of the matrix, is the dimension of the matrix;

[0022] The expression of the stability constraint term is:

[0023]

[0024] Where P is a symmetric positive definite matrix, represents the Frobenius norm, is the regularization weight coefficient, and Tr() represents the rank of the matrix.

[0025] In a possible design, the expression of the data in the data set is:

[0026]

[0027] in, is the torque input at each moment, is the current state, For data-based The state of the next moment is formed.

[0028] In a possible design, after constructing a data set based on the historical state data of the dexterous hand at multiple moments and the corresponding historical torques, the method further includes:

[0029] The data set is segmented based on the threshold value corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages respectively; the dynamic index includes at least one of the following: joint angular velocity, angular acceleration, control torque, contact force; the multiple task stages include: approaching object stage, grasping object stage and stable grasping stage;

[0030] The method of constructing a linearized model of a dexterous hand according to a dimensionality-raising function comprises:

[0031] Constructing a plurality of linearized models corresponding to the task stages respectively according to the dimensionality-raising function;

[0032] The method of training the linearized model based on the data set according to a preset loss function to obtain a dynamic model of the dexterous hand includes:

[0033] For each task stage, based on the sub-dataset corresponding to the task stage and according to the preset loss function of the task stage, the linearized model of the task stage is trained to obtain the dynamic model of the dexterous hand in the task stage.

[0034] In a possible design, the data set is segmented based on the threshold corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages, including:

[0035] If the joint angular velocity is greater than a first preset angular velocity, the angular acceleration is greater than a preset angular acceleration, and the contact force is less than a first threshold, the corresponding data is determined as a sub-dataset corresponding to the approaching object stage;

[0036] If the contact force is greater than a second threshold, the corresponding data is determined as a sub-data set corresponding to the stage of grasping the object; the second threshold is greater than or equal to the first threshold;

[0037] If the joint angular velocity is less than the second preset angular velocity and the fluctuation value of the control torque is less than the preset torque, the corresponding data is determined as a sub-data set corresponding to the stable grasping stage; the second preset angular velocity is less than the first preset angular velocity;

[0038] If the joint angular velocity is greater than a third preset angular velocity, the corresponding data is determined as a sub-data set corresponding to the moving object stage; the third preset angular velocity is greater than the first preset angular velocity.

[0039] In a possible design, the expression of the data of the sub-dataset is:

[0040]

[0041] Among them, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscripts;

[0042] The expression of the preset loss function of the task stage is:

[0043]

[0044] in, , K is the approximation of the Koopman operator in finite dimensions, represents the state transfer matrix of the dexterous hand in the high-dimensional space at the current stage, represents the input matrix of the current stage, represents the state dimension reduction matrix of the current stage, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage An independent subscript, represents the dimension-upgraded matrix at the next moment in the current stage, represents the dimension-enhanced matrix at the current stage, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, θ τ represents the network model parameters of the dimension-raising function, Represents the loss value.

[0045] In a possible design, constructing a plurality of linearized models corresponding to the task stages respectively according to the dimensionality-raising function includes:

[0046] Determine the linearization model of the current stage;

[0047] Determine the linearization model of the next stage according to the linearization model of the current stage;

[0048] The linearization model of the next stage is determined based on the following expression:

[0049]

[0050] in, represents the control input of the current stage, represents the state transfer matrix of the dexterous hand in the high-dimensional space at the current stage, represents the state transition matrix of the dexterous hand in the next stage of high-dimensional space, represents the input matrix of the current stage, represents the input matrix of the next stage, Represents the state dimension reduction matrix of the current stage, represents the state dimension reduction matrix of the current stage, β τ+1 is the length of the next phase of the time interval, θ τ represents the network model parameters of the dimension-raising function at the current stage, Indicates the next stage of dimensionality increase. express The transpose of Relative to The next moment of the dimensional state, Represents the augmented state-input matrix concatenated with the next stage's dimensionalized state and input. express The transpose of represents the state variable matrix of the next stage, Indicates the dimensionality-raising state of the current stage, express The transpose of represents the identity matrix, Represents the augmented state-input matrix formed by concatenating the current stage's dimensionalized state and input. express The transpose of .

[0051] In a second aspect, the present application provides a dexterous hand dynamics modeling device, comprising:

[0052] A construction module, used to construct a data set according to the historical state data of the dexterous hand at multiple moments and the corresponding historical torques;

[0053] The building module is also used to build a linearized model of the dexterous hand according to the dimension-raising function; the dimension-raising function includes a network model, which is used to map the original state to a high-dimensional observable space;

[0054] A training module is used to train the linearized model based on the data set and according to a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-upgraded state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error generated by comparing the result of first increasing the dimension of the state and then reducing the dimension of the state with the state before the dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model.

[0055] In a third aspect, the present application provides a dexterous hand dynamics modeling device, comprising: at least one processor and a memory;

[0056] The memory stores computer-executable instructions;

[0057] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method described in the first aspect and various possible designs of the first aspect.

[0058] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect is implemented.

[0059] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in the first aspect and various possible designs of the first aspect.

[0060] The present application provides a method and device for modeling the dynamics of a dexterous hand. The method includes constructing a data set based on the historical state data and corresponding historical torque of the dexterous hand at multiple moments, constructing a linearized model of the dexterous hand based on a dimensionality-raising function, the dimensionality-raising function is a deep neural network, and is used to map the original state to a high-dimensional observable space. Based on the data set, the linearized model is trained according to a preset loss function to obtain a dynamic model of the dexterous hand. The preset loss function includes an evolution error term and a reconstruction error term. The evolution error term is used to represent the error between the dimensionality-raising state and the state after the Koopman operator evolves, and the reconstruction error term is used to represent the error generated by comparing the result obtained by first increasing the dimension and then reducing the dimension of the state with the state before the dimensionality-raising. The method provided in the present application collects the state and torque input of each joint of the dexterous hand to establish a data set, and then uses a network model such as a deep neural network to fit the dimensionality-raising function used in the finite-dimensional approximation of the Koopman operator based on the data set. This can improve the accuracy and versatility of modeling, significantly reduce the computational complexity in real-time control scenarios, and ensure the high accuracy and robustness of model prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0062] Figure 1a A schematic diagram of the structure of the index finger of the dexterous hand provided in an embodiment of the present application;

[0063] Figure 1b A schematic diagram of the degrees of freedom coordinates of the index finger of a dexterous hand provided in an embodiment of the present application;

[0064] Figure 2 Schematic diagram 1 of the flow chart of the dexterous hand dynamics modeling method provided in the embodiment of the present application;

[0065] Figure 3 A schematic diagram of the network structure of a network model of a dimensionality-raising function provided in an embodiment of the present application;

[0066] Figure 4 Schematic diagram of the process of the dexterous hand dynamics modeling method provided in the embodiment of the present application Figure 2 ;

[0067] Figure 5 Schematic diagram of the process of the dexterous hand dynamics modeling method provided in the embodiment of the present application Figure 3 ;

[0068] Figure 6 Schematic diagram of the process of the dexterous hand dynamics modeling method provided in the embodiment of the present application Figure 4 ;

[0069] Figure 7A schematic diagram of the structure of a dexterous hand dynamics modeling device provided in an embodiment of the present application;

[0070] Figure 8 Schematic diagram of the hardware structure of the dexterous hand dynamics modeling device provided in an embodiment of the present application.

[0071] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] It should be noted that the dexterous hand dynamics modeling method provided in the present application can be used in the field of humanoid robot technology, and can also be used in any field other than the field of humanoid robot technology. The application field of the dexterous hand dynamics modeling method provided in the present application is not limited.

[0074] The dynamic model of the dexterous hand of a humanoid robot is a mathematical model that describes the relationship between force and motion during the movement of the dexterous hand.

[0075] In the related technology, the establishment of the dexterous hand model can rely on the following three mechanism equations: Newton-Euler equation, Lagrangian mechanics equation and Kane equation. Taking Lagrangian mechanics equation as an example, the dynamic model of each finger subsystem can be established through Lagrangian equation, and the analytical model can be obtained based on the constraint equation of the finger and the grasped object, so as to realize the dynamic modeling of the contact between the dexterous hand and the grasped object.

[0076] Although the mechanism modeling method based on first principles can construct a relatively accurate model, in practical applications, the multiple degrees of freedom and strong nonlinearity of the dexterous hand system often make it difficult for the mechanism equation to accurately describe the dynamic characteristics of the dexterous hand. For example, existing models are usually proposed under the assumption that the fingers are rigid structures and the friction of each component is ignored. Therefore, there will be large errors with the real system.

[0077] In order to solve the above technical problems, the inventors of the present application have found that a data-driven modeling method can be used to achieve efficient modeling and real-time prediction of the dexterous hand model using deep learning technology. Compared with the above-mentioned traditional modeling method, data-driven modeling does not require the manual introduction of complex constraints and auxiliary variables, and can adapt to the dynamic changes of dexterous hands in various operation tasks and scenarios. And it can take into account nonlinear factors such as friction and deformation in actual systems to improve modeling accuracy. Based on this, an embodiment of the present application provides a dexterous hand dynamics modeling method.

[0078] Figure 1a This is a schematic diagram of the structure of the index finger of the dexterous hand provided in an embodiment of the present application. Figure 1b The following is a schematic diagram of the degrees of freedom coordinates of the index finger of the dexterous hand provided in the embodiment of the present application. Figure 1a and Figure 1b As shown in the figure, the dexterous hand includes multiple connection ends (J, K and M) of the index finger and multiple joints, with the joint origins being O, O 1 , O 2 and O 3 , where the joint angle is the passive degree of freedom, and Active degree of freedom.

[0079] In the related technology, the Lagrange equation can be used to establish the nonlinear dynamic model of the index finger as shown in formula (1): (1)

[0080] in, For OO 1 The angle with the OX axis, for The first derivative of for The second-order derivative of O 1 O 2 With O 1 X 1 The angle of the axis, for The first derivative of for The second-order derivative of represents the control torque at joint O, Indicates joint O 1 The control torque at It is an element in the inertia matrix, which includes the inertia of independent axis rotation and the inertial coupling between different axes. yes On the System Generalized Coordinates The rate of change of is used to describe the nonlinear dynamic coupling effect caused by the change of the inertia matrix. However, as can be seen from the above formula, the calculation of nonlinear terms (such as inertial coupling and centrifugal force terms) depends on the real-time joint velocity and generalized coordinates, and the influence of factors such as friction, joint elasticity and material deformation during actual operation is not considered, which will introduce large deviations in the modeling process.

[0081] Therefore, in the specific implementation process of the dexterous hand dynamics modeling method provided in the embodiment of the present application, a data set can be constructed according to the historical state data of the dexterous hand index finger at multiple times and the corresponding historical torque, and a linearized model of the dexterous hand index finger can be constructed according to the dimensionality increase function, wherein the dimensionality increase function includes a network model (such as a deep neural network), which is used to map the original state to a high-dimensional observable space, and then based on the data set, the linearized model is trained according to a preset loss function to obtain a dynamics model of the dexterous hand index finger, and the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term, the evolution error term is used to represent the error between the dimensionality increase state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error generated by comparing the result of first increasing the dimensionality of the state and then reducing the dimensionality with the state before the dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model. The method provided in this embodiment collects the state and torque input of each joint of the index finger of the dexterous hand to establish a data set, and then uses a network model such as a deep neural network to fit the dimensionality-raising function used in the finite-dimensional approximation of the Koopman operator based on the data set. This can improve the accuracy and versatility of modeling, significantly reduce the computational complexity in real-time control scenarios, and ensure the high accuracy and robustness of model prediction. It should be noted that the high dimension mentioned in this application refers to space above three dimensions.

[0082] It should be noted that Figure 1a and Figure 1b The scene diagram of the dexterous hand index finger shown is only an example. The dexterous hand dynamics modeling method and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0083] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0084] Figure 2 Schematic diagram 1 of the flow chart of the dexterous hand dynamics modeling method provided in the embodiment of the present application. Figure 2As shown, the method includes:

[0085] 201. A data set is constructed based on the historical state data of the dexterous hand at multiple moments and the corresponding historical torques.

[0086] The execution subject of this embodiment may be a terminal device or a server.

[0087] Specifically, in order to carry out subsequent model training, a data set can be constructed first. In the process of data collection, taking dexterous hand grasping as an example, the input-state data set of the dexterous hand index finger can be obtained by nonlinear dynamic model simulation calculation or physical collection. Specifically, a large amount of system state data can be obtained by setting a specific collection step, number of sampling points, system initial state and control strategy to construct the corresponding data set.

[0088] Exemplarily, the acquisition step size can be set to 0.01 seconds in order to capture the details of the dynamic changes of the system. The number of sampling points can be determined to be 5000 points. Since the acquisition step size is 0.01 seconds, the entire grasping process lasts about 50 seconds (5000 data points × 0.01 seconds / data point = 50 seconds). In addition, 200 different initial states of the system can be randomly selected, that is, before data acquisition, 200 different initial conditions (such as joint angles, angular velocities, etc.) are randomly set to start the grasping action of the dexterous hand, so that the dynamic characteristics of the system can be fully stimulated and more diverse behaviors can be captured. The control strategy can be open-loop control, that is, during the data acquisition process, the control input will not be adjusted according to the real-time feedback of the system. Instead, a series of control torque inputs can be pre-set, or a fixed torque input can be simply maintained to observe the response of the system under these conditions. Finally, since there are 200 different initial states, 5000 data points are sampled in each initial state, so a total of 200×5000=1000000 sets of data can be obtained. Each set of data includes the state of the dexterous hand (such as the joint angle, angular velocity, angular acceleration of the finger joints) and the control input (such as the control torque at the joint) at a time point.

[0089] In some embodiments, the expression of the data in the data set is:

[0090] (2)

[0091] Wherein, U is the torque input at each moment, X is the state at the current moment, Y is the state at the next moment formed based on the data X, and N is a positive integer greater than 2.

[0092] Specifically, the data set includes N samples, each of which includes the current state , the state x at the next moment Nand the torque input u at the current moment N-1 .

[0093] 202. Construct a linearized model of the dexterous hand according to a dimensionality-raising function; the dimensionality-raising function includes a network model for mapping an original state to a high-dimensional observable space.

[0094] Specifically, Koopman operator theory, as a data-driven method, can elevate the original nonlinear system to a Hilbert space composed of infinite basis functions, and can provide a linear representation of the nonlinear system using only input and output data obtained from the system. The network model can be a deep neural network model.

[0095] Taking the index finger of a dexterous hand as an example, a discrete dynamic model with input can be defined for it. The expression of the model is:

[0096] (3)

[0097] in, are the state variables of the discrete dynamics model (i.e., joint angles and angular velocities), is the control input (i.e. the control torque at the joint), Represents the nonlinear state evolution of the dexterous hand system.

[0098] According to the Koopman operator theory, there is an observable function in the ascending dimensional space for this system. ,satisfy:

[0099] (4)

[0100] in, is the Koopman operator, express and The combination of Represents the extended state variable consisting of state and input.

[0101] In order to transform the state under the observable function space Represented in a form that is easier to calculate, the dimension-raising function dictionary can be designed based on the original state and input. Specifically, the state in the observable function space can be It is represented by the more easily computable form, in which , It usually consists of a set of polynomial functions, radial basis functions or Fourier basis functions.

[0102] When minimizing equation (5), the matrix Forming a state of dimensionality The linear predictor is: .

[0103] (5)

[0104] in, represents the state transition matrix of the dexterous hand in high-dimensional space, represents the input matrix, C represents the state dimension reduction matrix, represents the two-norm.

[0105] For the nonlinear time-varying system of the dexterous hand, it is difficult to accurately construct the observable space by using the traditional extended dynamic mode decomposition. Therefore, a deep learning method can be used to approximate the index finger dynamics model. For example, a deep neural network can be used as a dimensional function. This can more accurately capture the nonlinear characteristics of the system and provide a more precise linear representation.

[0106] like Figure 3 As shown, the dimension-raising function It can be a neural network composed of fully connected layers. The dimension-raising function is used to map the original state to an elevated high-dimensional observable space. The parameters such as network weights are used Indicates that the approximate linear model under this network structure can be expressed as:

[0107] (6)

[0108] in, represents the dimension-upgraded matrix at the current moment, represents the matrix after dimensionality increase at the next moment, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, x k+1 Indicates the state at the next moment, x k Represents the state at the current moment, and θ represents the network model parameter of the dimensionality-raising function.

[0109] 203. Based on the data set, the linearized model is trained according to a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-upgraded state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error between the result of first increasing the dimension of the state and then reducing the dimension and the state before the dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model.

[0110] Specifically, after constructing the data set and linearized model, the model can be trained based on the data set. During the training process, in order to obtain a suitable dimensionality-raising function A loss function including an evolution error term and a reconstruction error term can be set, wherein the evolution error term is used to represent the error between the state after the dimensionality increase and the state after the Koopman operator evolution, and the reconstruction error term is used to represent the error between the result obtained by first increasing the dimensionality of the state and then reducing the dimensionality and the state before the dimensionality increase. In order to ensure that the obtained linearized model meets the requirements of controllability and stability, the preset loss function also includes physical constraint loss terms such as controllability discriminant term and stability constraint term. By setting the two loss terms of controllability discriminant term and stability constraint term, the physical constraints of controllability and stability are respectively converted into optimizable mathematical conditions to ensure that the final model conforms to the dynamic characteristics of the actual system.

[0111] In some embodiments, the expression of the preset loss function is:

[0112] (7)

[0113] Among them, θ is the network model parameter of the dimension-raising function in the training process, is the network model parameter value of the dimension-raising function when the objective function takes the minimum value, that is, the optimal parameter value, Loss 1 is the evolution error term, which represents the error between the upgraded state and the state after Koopman operator evolution. A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, and Loss 2 is the reconstruction error term, which indicates the error between the result obtained by first upgrading the dimension and then reducing the dimension of the current state and the state before upgrading. C represents the state dimensionality reduction matrix, α, β, and are weight coefficients, is the controllability discriminant, is a stability constraint term. The method provided in this embodiment integrates the evolution error term, reconstruction error term, controllability discriminant term and stability constraint term into the loss function. This embodiment can realize the joint optimization of the dimensionality-raising function and the linearized model, while taking into account the multi-dimensional requirements such as dynamic evolution accuracy, state reconstruction fidelity, system controllability and stability. The evolution error term ensures the accuracy of the linear prediction of the Koopman operator, the reconstruction error term maintains the consistency of the dimensionality-reduction mapping, the controllability discriminant term ensures the accessibility of the system by constraining the rank missing amount of the controllability matrix, and the stability constraint term guides the state transfer matrix to meet the negative definite condition based on the Lyapunov equation. This optimization framework that integrates data-driven and physical constraints can embed prior knowledge in data fitting, avoid the traditional method from destroying the physical characteristics of the system due to the simple pursuit of prediction accuracy, and finally obtain a dexterous hand linearization model that conforms to the laws of dynamics and has strong generalization ability.

[0114] In some embodiments, the expression of the evolution error term is:

[0115] (8)

[0116] Among them, Loss1 is the evolution error term, represents the matrix after dimensionality increase, represents the matrix after dimensionality increase at the current moment, represents the matrix after dimensionality increase at the next moment, A represents the state transition matrix of the dexterous hand in the high-dimensional space, B represents the input matrix, x k+1 represents the state at the next moment, x k represents the state at the current moment, u k represents the control input, θ represents the network model parameters of the dimensionality increase function, and N is the number of samples in the dataset;

[0117] The expression of the reconstruction error term is:

[0118] (9)

[0119] Among them, Loss2 is the reconstruction error term, represents the matrix after dimensionality increase, represents the matrix after dimensionality increase at the current moment, C represents the state dimensionality reduction matrix, x k represents the state at the current moment.

[0120] In some embodiments, the expression of the controllability discrimination term can be:

[0121] (10)

[0122] Among them, , Q represents the controllability matrix, is the rank of the matrix, is the dimension of the matrix.

[0123] The stability constraint term can be expressed as:

[0124] (11)

[0125] Among them, P is a symmetric positive definite matrix, represents the Frobenius norm, is the regularization weight coefficient.

[0126] In this embodiment, the first term of is used to approximate a solution that satisfies the inequality by minimizing the difference, that is, taking the Lyapunov equation constraint as a soft constraint to guide to approach O, where O represents the zero matrix, that is, a matrix with all elements being 0. The second term is the trace penalty, which guides the state transition matrix to evolve in the negative definite direction, so as to ensure that the finally obtained state transition matrix satisfies the inequality (12):

[0127] O (12)

[0128] Through the combination of the two, the stability condition is transformed into a differentiable loss term, which penalizes the parameters that do not satisfy the Lyapunov inequality.

[0129] In some embodiments, in order to adapt to the changes in the dynamic characteristics of the dexterous hand in multi-stage tasks, a stage-adaptive loss weight adjustment mechanism can be used to better balance the model's expressiveness and physical consistency. The stage-adaptive weight function expression is:

[0130] (13)

[0131] in, is the weight of the controllability discriminant at time k, is the default base weight, Used to improve controllability constraints during the grasping phase, is the preset maximum adjustable coefficient. Because at the moment of grasping contact, the system faces external disturbances, transient instability and other problems. At this time, it is necessary to significantly strengthen the controllability and stability constraints of the system, while in the approach or stable stage, the relevant restrictions can be appropriately weakened to release the model's expression ability.

[0132] The dexterous hand dynamics modeling method provided in this embodiment collects the states and torque inputs of each joint of the dexterous hand to establish a data set, and then uses network models such as deep neural networks to fit the dimensionality-raising function used in the finite-dimensional approximation of the Koopman operator based on the data set. This can improve the accuracy and versatility of modeling, significantly reduce the computational complexity in real-time control scenarios, and ensure the high accuracy and robustness of model predictions.

[0133] The dexterous hand dynamics model established based on the embodiment of the present application can be subsequently used to control the dynamics of the dexterous hand of a humanoid robot, and can achieve more precise control, such as more precise control of the force and torque of the dexterous hand.

[0134] Figure 4 Schematic diagram of the process of the dexterous hand dynamics modeling method provided in the embodiment of the present application Figure 2 .like Figure 4 As shown, based on the above embodiment, for example Figure 2 On the basis of the embodiment shown, the present embodiment adopts a segmented modeling strategy to adapt to the dynamic characteristics of the dexterous hand in the process of performing tasks, for example, the dynamic characteristics of the index finger of the dexterous hand in the process of grasping will change over time. The method includes:

[0135] 401. Construct a data set based on the historical state data of the dexterous hand at multiple moments and the corresponding historical torques.

[0136] Step 401 in this embodiment is similar to step 201 in the above embodiment and will not be described again here.

[0137] 402. Segment the data set based on a threshold value corresponding to the dynamics index to obtain sub-data sets corresponding to multiple task stages.

[0138] Specifically, since the index finger of the dexterous hand is subjected to different force conditions at different grasping stages, the system dynamics characteristics (such as stiffness and damping coefficient) and external disturbances (such as friction, resistance, etc.) will change over time. Therefore, the existing data set can be divided according to fixed time intervals or thresholds based on dynamic changes, and a segmented modeling method can be used to ensure the robustness of the model when the system characteristics change rapidly. Figure 5 As shown, a data set can be first constructed based on inputs and states such as torque, and then the data in the data set can be preprocessed such as filtering and normalization. Optionally, the data set can be divided into n time intervals to correspond to different task stages. Finally, the Koopman operator and neural network training can be used to obtain a linearized model.

[0139] In some embodiments, the dynamic indicators include at least one of the following: joint angular velocity, angular acceleration, control torque, contact force; the multiple task stages include: approaching object stage, grasping object stage and stable grasping stage.

[0140] Specifically, in the actual grasping task, it can be divided into the following stages according to different models:

[0141] Phase 1: The index finger gradually approaches the target object from the initial position, and the model parameters of the current interval are used to perform real-time state prediction to guide the index finger to move along the planned trajectory;

[0142] Phase 2: When the index finger touches an object, the dynamic characteristics of the dexterous hand change (for example, the influence of contact surface friction, damping, etc.). The model trained in the current phase is combined with model predictive control (MPC) to dynamically adjust the joint torque to ensure that the applied force is stable and accurate.

[0143] Phase 3: After the object is successfully grasped, the system dynamics tend to be stable, but external disturbances (such as object weight or posture changes) still need to be predicted in real time. Based on the updated model parameters , , continuously optimize the control torque, offset external disturbances, and maintain stable contact between the index finger and the object.

[0144] In some embodiments, the data set is segmented based on the threshold value corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages, which may include: if the joint angular velocity is greater than a first preset angular velocity, the angular acceleration is greater than the preset angular acceleration and the contact force is less than the first threshold, the corresponding data is determined as the sub-data set corresponding to the approaching object stage; if the contact force is greater than the second threshold, the corresponding data is determined as the sub-data set corresponding to the grasping object stage; the second threshold is greater than or equal to the first threshold; if the joint angular velocity is less than the second preset angular velocity, and the fluctuation value of the control torque is less than the preset torque, the corresponding data is determined as the sub-data set corresponding to the stable grasping stage; the second preset angular velocity is less than the first preset angular velocity; if the joint angular velocity is greater than the third preset angular velocity, the corresponding data is determined as the sub-data set corresponding to the moving object stage; the third preset angular velocity is greater than the first preset angular velocity.

[0145] In some embodiments, the expression of the data of the sub-dataset is:

[0146] (14)

[0147] Among them, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage An independent subscript.

[0148] 403. Construct a plurality of linearized models corresponding to the task stages respectively according to the dimensionality-raising function.

[0149] Specifically, in the process of constructing corresponding linearized models for different task stages, the method shown in step 202 in the above embodiment may be adopted, and the linearized model of the previous stage may be used to construct the linearized model of the current stage.

[0150] In some embodiments, constructing linearized models corresponding to a plurality of task stages respectively according to the dimensionality-raising function may include:

[0151] After determining the linearization model of the current stage, determining the linearization model of the next stage based on the following expression according to the linearization model of the current stage;

[0152] (15)

[0153] Among them, U τ represents the control input of the current stage, represents the state transfer matrix of the dexterous hand in the high-dimensional space at the current stage, represents the state transition matrix of the dexterous hand in the next stage of high-dimensional space, represents the input matrix of the current stage, represents the input matrix of the next stage, Represents the state dimension reduction matrix of the current stage, represents the state dimension reduction matrix of the current stage, β τ+1 is the length of the next phase of the time interval, θ τ represents the network model parameters of the dimension-raising function at the current stage, Indicates the next stage of dimensionality increase. express The transpose of Relative to The next moment of the dimensional state, Represents the augmented state-input matrix concatenated with the next stage's dimensionalized state and input. express The transpose of represents the state variable matrix of the next stage, Indicates the dimensionality-raising state of the current stage, express The transpose of represents the identity matrix, Represents the augmented state-input matrix formed by concatenating the current stage's dimensionalized state and input. express The transpose of .

[0154] Specifically, in order to speed up the modeling process and avoid repeated calculation of the pseudo-inverse in expression (17), the model approximation within the next time interval can be obtained by numerical solution. τ For full row rank, the model at the next moment can be calculated by expression (15).

[0155] Calculate the next moment After that, similar to the process in the previous time interval, the optimal model parameters in the current time interval can be obtained by minimizing the loss function (18) through the neural network , and then according to formula (15), we can get the matching network after updating , Finally, the dynamic model of the dexterous hand at different time intervals can be obtained. .like Figure 6 As shown, we can first initialize the deep neural network of the dimension-raising function and solve , and then obtain the network parameters by minimizing the loss function , and then the next time interval, i.e., the next task phase, can be calculated based on expression (15). , and then obtain the network parameters by minimizing the loss function , and so on, to obtain linearized models at different stages.

[0156] 404. For each task stage, based on the sub-dataset corresponding to the task stage, the linearized model of the task stage is trained according to a preset loss function of the task stage to obtain a dynamic model of the dexterous hand in the task stage.

[0157] The model training process for each task stage in step 404 in this embodiment is similar to step 203 in the above embodiment and can be used for reference.

[0158] Specifically, for each sub-dataset of the task stage, the loss function L in matrix form is 1 and L 2 The expression is:

[0159] (16)

[0160] In the formula, , , at this time the network Corresponding

[0161] It can be obtained by formula (4.13), where Represent the pseudoinverse of a matrix:

[0162] (17)

[0163] In some embodiments, the expression of the preset loss function of the task stage is:

[0164] (18)

[0165] in, , K is the approximation of the Koopman operator in finite dimensions, represents the state transfer matrix of the dexterous hand in the high-dimensional space at the current stage, represents the input matrix of the current stage, represents the state dimension reduction matrix of the current stage, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscript, Ψ τ represents the dimensionality-raising function at the current stage, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, θ τ The network model parameters representing the dimension-raising function.

[0166] The dexterous hand dynamics modeling method provided in this embodiment adopts a data-driven method to model the dexterous hand dynamics. Compared with the traditional method based on mechanism equations, it can effectively capture nonlinear characteristics such as friction and deformation, and avoid relying on complex auxiliary variables. By combining the Koopman operator and deep neural network, the constructed model can efficiently update and evolve the system state in the prediction stage. The segmented modeling structure significantly improves the adaptability and real-time performance of the model in complex grasping scenarios, and can adapt to hardware configuration changes without sacrificing computing efficiency, providing high-precision and strong robustness technical support for the multi-scenario application of the dexterous hand system.

[0167] Figure 7 This is a schematic diagram of the structure of the dexterous hand dynamics modeling device provided in the embodiment of the present application. Figure 7 As shown, the dexterous hand dynamics modeling device 70 includes: a construction module 701 and a training module 702.

[0168] The construction module 701 is used to construct a data set based on the historical state data of the dexterous hand at multiple moments and the corresponding historical torques.

[0169] The construction module 701 is also used to construct a linearized model of the dexterous hand according to the dimensionality-raising function; the dimensionality-raising function includes a network model, which is used to map the original state to a high-dimensional observable space.

[0170] The training module 702 is used to train the linearized model based on the data set and according to a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-increased state and the state after the Koopman operator evolution, the reconstruction error term is used to represent the error between the result of first increasing the dimension of the state and then reducing the dimension and the state before the dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model.

[0171] The dexterous hand dynamics modeling device provided in the embodiment of the present application collects the states and torque inputs of each joint of the dexterous hand to establish a data set, and then uses network models such as deep neural networks to fit the dimensionality-raising function used in the finite-dimensional approximation of the Koopman operator based on the data set. This can improve the accuracy and versatility of modeling, significantly reduce the computational complexity in real-time control scenarios, and ensure the high accuracy and robustness of model predictions.

[0172] In some embodiments, the expression of the preset loss function is:

[0173]

[0174] Among them, θ is the network model parameter of the dimension-raising function in the training process, is the network model parameter value of the dimension-raising function when the objective function takes the minimum value, that is, the optimal parameter value, Loss 1 is the evolution error term, which represents the error between the dimensional state and the state after the Koopman operator evolution. A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, and Loss 2 is the reconstruction error term, which indicates the error between the result obtained by first upgrading the dimension and then reducing the dimension of the current state and the state before upgrading. C represents the state dimensionality reduction matrix, α, β, and are weight coefficients, is the controllability discriminant, is a stability constraint.

[0175] In some embodiments, the expression of the evolution error term is:

[0176]

[0177] Among them, Loss1 is the evolution error term, represents the matrix after dimensionality increase, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, and θ represents the network model parameter of the dimension-raising function;

[0178] The expression of the reconstruction error term is:

[0179]

[0180] Among them, Loss2 is the reconstruction error term, represents the matrix after dimension increase, C represents the state dimension reduction matrix, x k Indicates the current state.

[0181] In some embodiments, the expression of the controllability discriminant term is:

[0182]

[0183] in, , represents the controllability matrix, is the rank of the matrix, is the dimension of the matrix;

[0184] The expression of the stability constraint term is:

[0185]

[0186] Where P is a symmetric positive definite matrix, represents the Frobenius norm, is the regularization weight coefficient, and Tr() represents the rank of the matrix.

[0187] In some embodiments, the expression of the data in the data set is:

[0188]

[0189] in, is the torque input at each moment, is the current state, For data-based The state of the next moment is formed.

[0190] In some embodiments, the construction module 701 is also used to: divide the data set based on the threshold value corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages; the construction module 701 is specifically used to construct multiple linearized models corresponding to the task stages according to the dimensionality-raising function; the training module 702 is specifically used to: for each task stage, based on the sub-data set corresponding to the task stage, train the linearized model of the task stage according to the preset loss function of the task stage to obtain the dynamic model of the dexterous hand under the task stage.

[0191] In some embodiments, the dynamic indicators include at least one of the following: joint angular velocity, angular acceleration, control torque, contact force; the multiple task stages include: approaching object stage, grasping object stage and stable grasping stage.

[0192] In some embodiments, the construction module 701 is specifically used to: if the joint angular velocity is greater than a first preset angular velocity, the angular acceleration is greater than a preset angular acceleration and the contact force is less than a first threshold, then the corresponding data is determined as a sub-dataset corresponding to the approaching object stage; if the contact force is greater than a second threshold, then the corresponding data is determined as a sub-dataset corresponding to the grasping object stage; the second threshold is greater than or equal to the first threshold; if the joint angular velocity is less than the second preset angular velocity, and the fluctuation value of the control torque is less than the preset torque, then the corresponding data is determined as a sub-dataset corresponding to the stable grasping stage; the second preset angular velocity is less than the first preset angular velocity; if the joint angular velocity is greater than a third preset angular velocity, then the corresponding data is determined as a sub-dataset corresponding to the moving object stage; the third preset angular velocity is greater than the first preset angular velocity.

[0193] In some embodiments, the expression of the data of the sub-dataset is:

[0194]

[0195] Among them, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscripts;

[0196] The expression of the preset loss function of the task stage is:

[0197]

[0198] in, , K is the approximation of the Koopman operator in finite dimensions, represents the state transfer matrix of the dexterous hand in the high-dimensional space at the current stage, represents the input matrix of the current stage, represents the state dimension reduction matrix of the current stage, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage An independent subscript, represents the dimension-upgraded matrix at the next moment in the current stage, represents the dimension-enhanced matrix at the current stage, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, θ τ represents the network model parameters of the dimension-raising function, Represents the loss value.

[0199] In some embodiments, the construction module 701 is specifically used to: determine the linearization model of the current stage;

[0200] Determine the linearization model of the next stage according to the linearization model of the current stage;

[0201] The linearization model of the next stage is determined based on the following expression:

[0202]

[0203] in, represents the control input of the current stage, represents the state transfer matrix of the dexterous hand in the high-dimensional space at the current stage, represents the state transition matrix of the dexterous hand in the next stage of high-dimensional space, represents the input matrix of the current stage, represents the input matrix of the next stage, Represents the state dimension reduction matrix of the current stage, represents the state dimension reduction matrix of the current stage, β τ+1 is the length of the next phase of the time interval, θ τ represents the network model parameters of the dimension-raising function at the current stage, Indicates the next stage of dimensionality increase. express The transpose of Relative to The next moment of the dimensional state, Represents the augmented state-input matrix concatenated with the next stage's dimensionalized state and input. express The transpose of represents the state variable matrix of the next stage, Indicates the dimensionality-raising state of the current stage, express The transpose of represents the identity matrix, Represents the augmented state-input matrix formed by concatenating the current stage's dimensionalized state and input. express The transpose of .

[0204] The dexterous hand dynamics modeling device provided in the embodiment of the present application can be used to execute the above-mentioned method embodiment. Its implementation principle and technical effects are similar, and this embodiment will not be repeated here.

[0205] Figure 8 A schematic diagram of the hardware structure of a dexterous hand dynamics modeling device provided in an embodiment of the present application. The device may be a computer, a message sending and receiving device, a tablet device, a medical device, etc.

[0206] The device 80 may include one or more of the following components: a processing component 801 , a memory 802 , a power component 803 , a multimedia component 804 , an audio component 805 , an input / output (I / O) interface 806 , a sensor component 807 , and a communication component 808 .

[0207] The processing component 801 generally controls the overall operation of the device 80, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 801 may include one or more processors 809 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 801 may include one or more modules to facilitate the interaction between the processing component 801 and other components. For example, the processing component 801 may include a multimedia module to facilitate the interaction between the multimedia component 804 and the processing component 801.

[0208] The memory 802 is configured to store various types of data to support operations on the device 80. Examples of such data include instructions for any application or method operating on the device 80, contact data, phone book data, messages, pictures, videos, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0209] The power supply component 803 provides power to the various components of the device 80. The power supply component 803 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 80.

[0210] The multimedia component 804 includes a screen that provides an output interface between the device 80 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 804 includes a front camera and / or a rear camera. When the device 80 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0211] The audio component 805 is configured to output and / or input audio signals. For example, the audio component 805 includes a microphone (MIC), and when the device 80 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 802 or sent via the communication component 808. In some embodiments, the audio component 805 also includes a speaker for outputting audio signals.

[0212] I / O interface 806 provides an interface between processing component 801 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0213] The sensor assembly 807 includes one or more sensors for providing various aspects of status assessment for the device 80. For example, the sensor assembly 807 can detect the open / closed state of the device 80, the relative positioning of components, such as the display and keypad of the device 80, and the sensor assembly 807 can also detect the position change of the device 80 or a component of the device 80, the presence or absence of user contact with the device 80, the orientation or acceleration / deceleration of the device 80, and the temperature change of the device 80. The sensor assembly 807 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 807 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 807 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0214] The communication component 808 is configured to facilitate wired or wireless communication between the device 80 and other devices. The device 80 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 808 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 808 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0215] In an exemplary embodiment, the device 80 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0216] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 802 including instructions, and the instructions can be executed by a processor 809 of the device 80 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0217] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium may be any available medium that can be accessed by a general or special-purpose computer.

[0218] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0219] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0220] An embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the dexterous hand dynamics modeling method performed by the dexterous hand dynamics modeling device as described above.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for modeling the dynamics of a dexterous hand, characterized in that: include: Construct a data set based on the historical state data of the dexterous hand at multiple moments and the corresponding historical torques; Constructing a linearized model of a dexterous hand according to a dimensionality-raising function; the dimensionality-raising function includes a network model for mapping an original state to a high-dimensional observable space; Based on the data set, the linearized model is trained according to a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-increased state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error generated by comparing the result obtained by first increasing the dimension of the state and then reducing the dimension of the state with the state before the dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model.

2. The method according to claim 1, characterized in that The expression of the preset loss function is: Among them, θ is the network model parameter of the dimension-raising function in the training process, is the network model parameter value of the dimension-raising function when the objective function takes the minimum value, that is, the optimal parameter value, Loss1 is the evolution error term, which represents the error between the dimension-raising state and the state after the Koopman operator evolution, A represents the state transfer matrix of the dexterous hand in the high-dimensional space, B represents the input matrix, Loss2 is the reconstruction error term, which represents the error between the result obtained by first increasing the dimension of the current state and then reducing the dimension and the state before the dimension-raising, C represents the state dimension-reducing matrix, α, β, and are weight coefficients, is the controllability discriminant, is a stability constraint.

3. The method according to claim 2, characterized in that The expression of the evolution error term is: Among them, Loss1 is the evolution error term, represents the matrix after dimensionality increase, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, and θ represents the network model parameter of the dimension-raising function; The expression of the reconstruction error term is: Among them, Loss2 is the reconstruction error term, represents the matrix after dimension increase, C represents the state dimension reduction matrix, x k Indicates the current state.

4. The method according to claim 2, characterized in that: The expression of the controllability discriminant term is: in, , represents the controllability matrix, is the rank of the matrix, is the dimension of the matrix; The expression of the stability constraint term is: Where P is a symmetric positive definite matrix, represents the Frobenius norm, is the regularization weight coefficient, and Tr() represents the rank of the matrix.

5. The method according to claim 1, characterized in that The expression of the data in the data set is: in, is the torque input at each moment, is the current state, For data-based The state of the next moment is formed.

6. The method according to any one of claims 1 to 5, characterized in that: After constructing the data set according to the historical state data of the dexterous hand at multiple moments and the corresponding historical torque, the method further includes: The data set is segmented based on the threshold value corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages respectively; the dynamic index includes at least one of the following: joint angular velocity, angular acceleration, control torque, contact force; the multiple task stages include: approaching object stage, grasping object stage and stable grasping stage; The linearized model of the dexterous hand is constructed according to the dimensionality-raising function, comprising: Constructing a plurality of linearized models corresponding to the task stages respectively according to the dimensionality-raising function; The method of training the linearized model based on the data set according to a preset loss function to obtain a dynamic model of the dexterous hand includes: For each task stage, based on the sub-dataset corresponding to the task stage and according to the preset loss function of the task stage, the linearized model of the task stage is trained to obtain the dynamic model of the dexterous hand in the task stage.

7. The method according to claim 6, characterized in that The data set is segmented based on the threshold value corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages, including: If the joint angular velocity is greater than a first preset angular velocity, the angular acceleration is greater than a preset angular acceleration, and the contact force is less than a first threshold, the corresponding data is determined as a sub-dataset corresponding to the approaching object stage; If the contact force is greater than a second threshold, the corresponding data is determined as a sub-data set corresponding to the stage of grasping the object; the second threshold is greater than or equal to the first threshold; If the joint angular velocity is less than the second preset angular velocity and the fluctuation value of the control torque is less than the preset torque, the corresponding data is determined as a sub-data set corresponding to the stable grasping stage; the second preset angular velocity is less than the first preset angular velocity; If the joint angular velocity is greater than a third preset angular velocity, the corresponding data is determined as a sub-data set corresponding to the moving object stage; the third preset angular velocity is greater than the first preset angular velocity.

8. The method according to claim 6, characterized in that The expression of the data of the sub-dataset is: Among them, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscripts; The expression of the preset loss function of the task stage is: in, , K is the approximation of the Koopman operator in finite dimensions, represents the state transfer matrix of the dexterous hand in the high-dimensional space at the current stage, represents the input matrix of the current stage, represents the state dimension reduction matrix of the current stage, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage An independent subscript, represents the dimension-upgraded matrix at the next moment in the current stage, represents the dimension-enhanced matrix at the current stage, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, θ τ represents the network model parameters of the dimension-raising function, Represents the loss value.

9. The method according to claim 6, characterized in that The step of constructing a plurality of linearized models corresponding to the task stages respectively according to the dimension-raising function includes: Determine the linearization model of the current stage; Determine the linearization model of the next stage according to the linearization model of the current stage; The linearization model of the next stage is determined based on the following expression: in, represents the control input of the current stage, represents the state transfer matrix of the dexterous hand in the high-dimensional space at the current stage, represents the state transition matrix of the dexterous hand in the next stage of high-dimensional space, represents the input matrix of the current stage, represents the input matrix of the next stage, Represents the state dimension reduction matrix of the current stage, represents the state dimension reduction matrix of the current stage, β τ+1 is the length of the next phase of the time interval, θ τ represents the network model parameters of the dimension-raising function at the current stage, Indicates the next stage of dimensionality increase. express The transpose of Relative to The next moment of the dimensional state, Represents the augmented state-input matrix concatenated with the next stage's dimensionalized state and input. express The transpose of represents the state variable matrix of the next stage, Indicates the dimensionality-raising state of the current stage, express The transpose of represents the identity matrix, Represents the augmented state-input matrix formed by concatenating the current stage's dimensionalized state and input. express The transpose of .

10. A dexterous hand dynamics modeling device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the dexterous hand dynamics modeling method according to any one of claims 1 to 9.

Images