Dynamics modeling method and equipment for dexterous hands

Abstract

The present application provides a method and device for modeling the dynamics of a dexterous hand. The method includes: constructing a data set based on the historical state data and corresponding historical torques of the dexterous hand at multiple moments; constructing a linearized model of the dexterous hand based on a dimensionality-raising function; training the linearized model based on the data set according to a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term, and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-raising state and the state after evolution by the Koopman operator; the reconstruction error term is used to represent the error generated by comparing the result obtained by first increasing the dimensionality of the state and then reducing the dimensionality of the state with the state before the dimensionality-raising; the controllability discriminant term is used to represent the controllability condition of the linearized model; and the stability constraint term is used to represent the stability condition of the linearized model. The method provided by the present application can improve the accuracy and versatility of modeling.

Description

Technical Field

[0001] The present application relates to the technical field of humanoid robots, and in particular to a method and device for dynamic modeling of a dexterous hand. Background Art

[0002] The dynamic model of the dexterous hand of a humanoid robot is a mathematical model that describes the relationship between force and motion during the movement of the dexterous hand.

[0003] In related technologies, mechanism equations such as the Lagrangian mechanics equation can be used for modeling.

[0004] However, in the process of realizing this application, the inventors discovered that there are at least the following problems in the prior art: the existing modeling method causes a large error between the constructed model and the real system, and the accuracy is low. Summary of the Invention

[0005] The present application provides a method and device for dynamic modeling of a dexterous hand to improve the modeling accuracy of the dexterous hand.

[0006] In a first aspect, the present application provides a method for modeling the dynamics of a dexterous hand, comprising:

[0007] Construct a dataset based on the historical state data and corresponding historical torque of the dexterous hand at multiple moments;

[0008] Constructing a linearized model of the dexterous hand based on a dimensionality-raising function; the dimensionality-raising function includes a network model for mapping the original state into a high-dimensional observable space;

[0009] Based on the data set, the linearized model is trained according to a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-increased state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error generated by comparing the result of first increasing the dimensionality of the state and then reducing the dimensionality with the state before dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model.

[0010] In a possible design, the expression of the preset loss function is:

[0011]

[0012] Among them, θ is the network model parameter of the dimension-raising function during training, is the network model parameter value of the dimension-raising function when the objective function takes the minimum value, that is, the optimal parameter value, Loss1 is the evolution error term, which represents the error between the dimension-raising state and the state after the Koopman operator evolution, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, Loss2 is the reconstruction error term, which represents the error generated by the result of first dimension-raising and then dimension-reducing the current state compared with the state before dimension-raising, C represents the state dimension-reduction matrix, α, β, and are weight coefficients, is the controllability discriminant, is the stability constraint term, and N is the number of samples.

[0013] In one possible design, the expression of the evolution error term is:

[0014]

[0015] Among them, Loss1 is the evolution error term, represents the matrix after dimensionality increase, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, and θ represents the network model parameter of the dimensionality-raising function;

[0016] The expression of the reconstruction error term is:

[0017]

[0018] Among them, Loss2 is the reconstruction error term, represents the matrix after dimension increase, C represents the state dimension reduction matrix, x k Indicates the current state.

[0019] In a possible design, the expression of the controllability criterion is:

[0020]

[0021] in, , represents the controllability matrix, is the rank of the matrix, is the dimension of the matrix, is the state x based on the current moment k and the dimension-elevated matrix of the network model parameters θ of the dimension-elevating function;

[0022] The expression of the stability constraint term is:

[0023]

[0024] Where P is a symmetric positive definite matrix, represents the Frobenius norm, is the regularization weight coefficient, and Tr() represents the rank of the matrix.

[0025] In a possible design, the expression of the data in the data set is:

[0026]

[0027] in, is the moment input for each sample, is the state of each sample, For data-based The state of the next sample is formed for each sample.

[0028] In a possible design, after constructing a data set based on the historical state data and corresponding historical torques of the dexterous hand at multiple moments, the method further includes:

[0029] The dataset is segmented based on thresholds corresponding to dynamic indicators to obtain sub-datasets corresponding to multiple task phases; the dynamic indicators include at least one of the following: joint angular velocity, angular acceleration, control torque, and contact force; the multiple task phases include: approaching the object phase, grasping the object phase, and stable grasping phase;

[0030] The linearized model of the dexterous hand is constructed according to the dimensionality-raising function, comprising:

[0031] Constructing a plurality of linearized models corresponding to the task stages respectively according to the dimensionality-raising function;

[0032] The linearized model is trained based on the data set according to a preset loss function to obtain a dynamic model of the dexterous hand, including:

[0033] For each task stage, based on the sub-dataset corresponding to the task stage, the linearized model of the task stage is trained according to the preset loss function of the task stage to obtain the dynamic model of the dexterous hand under the task stage.

[0034] In a possible design, the data set is segmented based on the threshold corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages, including:

[0035] If the joint angular velocity is greater than a first preset angular velocity, the angular acceleration is greater than a preset angular acceleration, and the contact force is less than a first threshold, the corresponding data is determined as a sub-dataset corresponding to the approaching object stage;

[0036] If the contact force is greater than a second threshold, the corresponding data is determined as a sub-dataset corresponding to the object grasping stage; the second threshold is greater than or equal to the first threshold;

[0037] If the joint angular velocity is less than a second preset angular velocity and the fluctuation value of the control torque is less than the preset torque, the corresponding data is determined as a sub-dataset corresponding to the stable grasping stage; the second preset angular velocity is less than the first preset angular velocity;

[0038] If the joint angular velocity is greater than the third preset angular velocity, the corresponding data is determined as the sub-dataset corresponding to the moving object stage; the third preset angular velocity is greater than the first preset angular velocity; the time of each sample in the sub-dataset corresponding to the moving object stage is later than the time of each sample in the sub-dataset corresponding to the approaching object stage.

[0039] In a possible design, the expression of the data of the sub-dataset is:

[0040]

[0041] Among them, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscripts;

[0042] The expression of the preset loss function of the task stage is:

[0043]

[0044] in, , K is the approximation of the Koopman operator in finite dimensions, represents the state transition matrix of the dexterous hand in the high-dimensional space at the current stage, Represents the input matrix of the current stage, Represents the state dimensionality reduction matrix of the current stage, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscripts, Represents the dimension-enhanced matrix at the next moment in the current stage, Represents the dimension-enhanced matrix at the current moment in the current stage, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, θ τ represents the network model parameters of the dimension-raising function, Represents the loss value.

[0045] In a possible design, constructing a plurality of linearized models corresponding to the task stages according to the dimensionality-raising function includes:

[0046] Determine the linearization model of the current stage;

[0047] Determining a linearization model for the next stage based on the linearization model of the current stage;

[0048] The linearized model of the next stage is determined based on the following expression:

[0049]

[0050] in, represents the control input of the current stage, represents the state transition matrix of the dexterous hand in the high-dimensional space at the current stage, represents the state transition matrix of the dexterous hand in the next stage of high-dimensional space, represents the input matrix of the current stage, represents the input matrix of the next stage, Represents the state dimensionality reduction matrix of the current stage, represents the next stage state dimension reduction matrix, β τ+1 is the length of the next phase of the time interval, θ τ Represents the network model parameters of the dimensionality-raising function at the current stage, Indicates the next stage of dimensionality upgrade, express The transpose of Relative to The next moment of the dimensional state, Represents the augmented state-input matrix formed by concatenating the dimensionality-increasing state and input of the next stage, express The transpose of represents the state variable matrix of the next stage, Indicates the dimensionality-raising state of the current stage, express The transpose of represents the identity matrix, Represents the augmented state-input matrix formed by concatenating the current stage's dimensionality-increment state and input, express The transpose of .

[0051] In a second aspect, the present application provides a dexterous hand dynamics modeling device, comprising:

[0052] A construction module is used to construct a data set based on the historical state data and corresponding historical torque of the dexterous hand at multiple moments;

[0053] The construction module is further used to construct a linearized model of the dexterous hand according to the dimensionality-raising function; the dimensionality-raising function includes a network model for mapping the original state to a high-dimensional observable space;

[0054] A training module is used to train the linearized model based on the data set and a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-upgraded state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error generated by comparing the result of first increasing the dimensionality of the state and then reducing the dimensionality of the state with the state before dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model.

[0055] In a third aspect, the present application provides a dexterous hand dynamics modeling device, comprising: at least one processor and a memory;

[0056] The memory stores computer-executable instructions;

[0057] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method described in the first aspect and various possible designs of the first aspect.

[0058] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect is implemented.

[0059] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in the first aspect and various possible designs of the first aspect.

[0060] The present application provides a method and device for modeling the dynamics of a dexterous hand. The method includes constructing a data set based on the historical state data and corresponding historical torques of the dexterous hand at multiple moments, constructing a linearized model of the dexterous hand based on a dimensionality-raising function, wherein the dimensionality-raising function is a deep neural network and is used to map the original state to a high-dimensional observable space. Based on the data set, the linearized model is trained according to a preset loss function to obtain a dynamic model of the dexterous hand. The preset loss function includes an evolution error term and a reconstruction error term. The evolution error term is used to represent the error between the dimensionality-raising state and the state after evolution by the Koopman operator, and the reconstruction error term is used to represent the error generated by comparing the result obtained by first increasing the dimensionality of the state and then reducing the dimensionality of the state with the state before the dimensionality-raising. The method provided in the present application collects the state and torque inputs of each joint of the dexterous hand to establish a data set, and then uses a network model such as a deep neural network to fit the dimensionality-raising function used in the finite-dimensional approximation of the Koopman operator based on the data set. This method can improve the accuracy and versatility of modeling, significantly reduce the computational complexity in real-time control scenarios, and ensure the high accuracy and robustness of model prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0062] Figure 1a A schematic diagram of the structure of the index finger of the dexterous hand provided in an embodiment of the present application;

[0063] Figure 1b A schematic diagram of the degree of freedom coordinates of the index finger of the dexterous hand provided in an embodiment of the present application;

[0064] Figure 2 Flowchart 1 of the method for modeling the dynamics of a dexterous hand provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of the network structure of a network model of a dimensionality-raising function provided in an embodiment of the present application;

[0066] Figure 4 Schematic diagram of the process of the dexterous hand dynamics modeling method provided in the embodiment of the present application Figure 2 ;

[0067] Figure 5 Schematic diagram of the process of the dexterous hand dynamics modeling method provided in the embodiment of the present application Figure 3 ;

[0068] Figure 6 Schematic diagram of the process of the dexterous hand dynamics modeling method provided in the embodiment of the present application Figure 4 ;

[0069] Figure 7A schematic diagram of the structure of the dexterous hand dynamics modeling device provided in an embodiment of the present application;

[0070] Figure 8 Schematic diagram of the hardware structure of the dexterous hand dynamics modeling device provided in an embodiment of the present application.

[0071] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0072] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0073] It should be noted that the dexterous hand dynamics modeling method provided in this application can be used in the field of humanoid robot technology, and can also be used in any field other than the field of humanoid robot technology. The application field of the dexterous hand dynamics modeling method provided in this application is not limited.

[0074] The dynamic model of the dexterous hand of a humanoid robot is a mathematical model that describes the relationship between force and motion during the movement of the dexterous hand.

[0075] In related technologies, dexterous hand models can be built using three mechanistic equations: the Newton-Euler equations, the Lagrangian equations, and the Kane equations. Taking the Lagrangian equations as an example, a dynamic model for each finger subsystem can be established using these equations. An analytical model is then derived based on the constraint equations between the fingers and the grasped object, enabling dynamic modeling of the contact between the dexterous hand and the grasped object.

[0076] Although the mechanism modeling method based on first principles can construct a relatively accurate model, in practical applications, the multiple degrees of freedom and strong nonlinearity of the dexterous hand system often make it difficult for the mechanism equation to accurately describe the dynamic characteristics of the dexterous hand. For example, existing models are usually proposed under the assumption that the fingers are rigid structures and the friction of each component is ignored. Therefore, there will be large errors with the real system.

[0077] To address the above technical issues, the inventors of this application have discovered that a data-driven modeling approach can be used to leverage deep learning technology to achieve efficient modeling and real-time prediction of dexterous hand models. Compared to the above-mentioned traditional modeling approach, data-driven modeling eliminates the need to manually introduce complex constraints and auxiliary variables, and can adapt to the dynamic changes of dexterous hands in various operational tasks and scenarios. Furthermore, it can take into account nonlinear factors such as friction and deformation present in actual systems, thereby improving modeling accuracy. Based on this, an embodiment of this application provides a method for modeling the dynamics of dexterous hands.

[0078] Figure 1a This is a schematic diagram of the structure of the index finger of the dexterous hand provided in an embodiment of the present application. Figure 1b This is a schematic diagram of the degree of freedom coordinates of the index finger of the dexterous hand provided in the embodiment of this application. Figure 1a and Figure 1b As shown in the figure, the dexterous hand includes multiple connection ends (J, K and M) of the index finger and multiple joints, with the joint origins being O, O1, O2 and O3, where the joint angles is the passive degree of freedom, and Active degree of freedom.

[0079] In related technologies, the Lagrange equation can be used to establish a nonlinear dynamic model of the index finger as shown in formula (1): (1)

[0080] in, is the angle between OO1 and OX axis, for The first derivative of for The second derivative of is the angle between O1O2 and O1X1 axis, for The first derivative of for The second derivative of represents the control torque at joint O, represents the control torque at joint O1, It is an element in the inertia matrix, which includes the inertia of independent axis rotation and the inertial coupling between different axes. yes On the System Generalized Coordinates The rate of change of is used to describe the nonlinear dynamic coupling effects caused by the change in the inertia matrix. However, as can be seen from the above formula, the calculation of nonlinear terms (such as inertial coupling and centrifugal force) depends on the real-time joint velocity and generalized coordinates. In addition, the influence of factors such as friction, joint elasticity, and material deformation during actual operation is not considered, which can introduce significant deviations in the modeling process.

[0081] Therefore, in the specific implementation process of the dexterous hand dynamics modeling method provided in the embodiment of the present application, a data set can be constructed based on the historical state data and corresponding historical torque of the index finger of the dexterous hand at multiple times, and a linearized model of the index finger of the dexterous hand can be constructed based on the dimensionality-raising function, wherein the dimensionality-raising function includes a network model (such as a deep neural network) for mapping the original state to a high-dimensional observable space, and then based on the data set, the linearized model is trained according to a preset loss function to obtain a dynamic model of the index finger of the dexterous hand. The preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term. The evolution error term is used to represent the error between the dimensionality-raised state and the state after evolution by the Koopman operator. The reconstruction error term is used to represent the error generated by comparing the result of first raising the dimensionality of the state and then reducing the dimensionality with the state before the dimensionality-raising. The controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model. The method provided in this embodiment collects the state and torque inputs of each joint of the dexterous hand's index finger to establish a dataset. Based on this dataset, a network model such as a deep neural network is used to fit the dimensionality-raising function used in the finite-dimensional approximation of the Koopman operator. This method can improve the accuracy and versatility of modeling, significantly reduce the computational complexity in real-time control scenarios, and ensure high accuracy and robustness of model predictions. It should be noted that the high-dimensionality mentioned in this application refers to space with three dimensions or higher.

[0082] It should be noted that Figure 1a and Figure 1b The scene diagram of the dexterous hand index finger shown is only an example. The dexterous hand dynamics modeling method and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0083] The following specific embodiments are used to describe the technical solution of the present application in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0084] Figure 2 Flowchart 1 of the method for modeling the dynamics of a dexterous hand provided in the embodiment of the present application. Figure 2 As shown, the method includes:

[0085] 201. A dataset is constructed based on the historical state data and corresponding historical torque of the dexterous hand at multiple moments.

[0086] The execution subject of this embodiment may be a terminal device or a server.

[0087] Specifically, for subsequent model training, a dataset can be constructed first. During data collection, taking dexterous grasping as an example, the input-state dataset of the dexterous hand's index finger can be obtained through nonlinear dynamics model simulation or physical acquisition. Specifically, a large amount of system state data can be acquired by setting a specific acquisition step size, number of sampling points, system initial state, and control strategy to construct the corresponding dataset.

[0088] For example, the acquisition step size can be set to 0.01 seconds to capture details of the system's dynamic changes. The number of sampling points can be set to 5000. Since the acquisition step size is 0.01 seconds, the entire grasping process lasts approximately 50 seconds (5000 data points × 0.01 seconds / data point = 50 seconds). Furthermore, 200 different system initial states can be randomly selected. That is, before data acquisition, 200 different initial conditions (such as joint angles and angular velocities) are randomly set to initiate the dexterous hand's grasping motion. This fully stimulates the system's dynamic characteristics and captures a wider range of behaviors. The control strategy can be open-loop control, meaning that during data acquisition, the control input is not adjusted based on real-time feedback from the system. Instead, a series of control torque inputs can be pre-set, or a fixed torque input can be simply maintained to observe the system's response under these conditions. Finally, since there are 200 different initial states, and 5,000 data points are sampled in each initial state, a total of 200 × 5,000 = 1,000,000 sets of data are obtained. Each set of data includes the dexterous hand state (such as the joint angle, angular velocity, and angular acceleration of the finger joints) and the control input (such as the control torque at the joint) at a time point.

[0089] In some embodiments, the data in the data set is expressed as:

[0090] (2)

[0091] in, is the moment input for each sample, is the state of each sample, For data-based The state of the next sample of each sample is formed, and N is a positive integer greater than 2.

[0092] Specifically, the data set includes N samples, each of which includes the current state , the state x at the next moment N and the torque input u at the current moment N-1 .

[0093] 202. Construct a linearized model of the dexterous hand based on a dimensionality-raising function; the dimensionality-raising function includes a network model for mapping the original state to a high-dimensional observable space.

[0094] Specifically, Koopman operator theory, as a data-driven method, can elevate the original nonlinear system to a Hilbert space consisting of infinite basis functions, providing a linear representation of the nonlinear system using only input and output data obtained from the system. The network model can be a deep neural network model.

[0095] Taking the index finger of a dexterous hand as an example, a discrete dynamic model with input can be defined for it. The expression of this model is:

[0096] (3)

[0097] in, are the state variables of the discrete dynamics model (i.e., joint angles and angular velocities), is the control input (i.e., the control torque at the joint), Represents the nonlinear state evolution of the dexterous hand system.

[0098] According to Koopman operator theory, there is an observable function in the dimensional space for this system. ,satisfy:

[0099] (4)

[0100] in, is the Koopman operator, express and The combination of Extended state variables representing state and input.

[0101] In order to transform the state under the observable function space Expressed in a form that is easier to calculate, the dimensionality-raising function dictionary can be designed based on the original state and input. Specifically, the state in the observable function space can be Expressed as a more easily calculated form, in which , It usually consists of a set of polynomial functions, radial basis functions or Fourier basis functions.

[0102] When minimizing equation (5), the matrix Forming a state of ascending dimension The linear predictor of : .

[0103] (5)

[0104] in, represents the state transition matrix of the dexterous hand in high-dimensional space, represents the input matrix, C represents the state dimension reduction matrix, represents the two-norm.

[0105] For the nonlinear time-varying system of the dexterous hand, traditional extended dynamic mode decomposition is difficult to accurately construct the observable space. Therefore, deep learning methods can be used to approximate the index finger dynamic model. For example, deep neural networks can be used as dimensionality-raising functions. This can more accurately capture the nonlinear characteristics of the system and provide a more precise linear representation.

[0106] like Figure 3 As shown, the dimension-raising function It can be a neural network composed of fully connected layers. The dimension-raising function is used to map the original state to an elevated high-dimensional observable space. The network weights and other parameters are used to Indicates that the approximate linear model under this network structure can be expressed as:

[0107] (6)

[0108] in, represents the dimension-upgraded matrix at the current moment, represents the matrix after dimensionality increase at the next moment, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, x k+1 Indicates the state at the next moment, x k Represents the state at the current moment, and θ represents the network model parameter of the dimensionality-raising function.

[0109] 203. Based on the data set, the linearized model is trained according to a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-increased state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error generated by comparing the result of first increasing the dimensionality of the state and then reducing the dimensionality of the state with the state before the dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model.

[0110] Specifically, after constructing the data set and linearized model, the model can be trained based on the data set. During the training process, in order to obtain a suitable dimensionality-raising function A loss function can be set up that includes an evolution error term and a reconstruction error term. The evolution error term is used to represent the error between the upgraded state and the state after evolution via the Koopman operator, and the reconstruction error term is used to represent the error resulting from comparing the result obtained by first upgrading the state and then reducing its dimensionality with the state before dimensionality upgrade. To ensure that the resulting linearized model meets requirements such as controllability and stability, the preset loss function also includes physical constraint loss terms such as controllability discriminant terms and stability constraint terms. By setting these two loss terms, the physical constraints of controllability and stability are converted into optimizable mathematical conditions, ensuring that the final model conforms to the dynamic characteristics of the actual system.

[0111] In some embodiments, the expression of the preset loss function is:

[0112] (7)

[0113] Among them, θ is the network model parameter of the dimension-raising function during training, is the network model parameter value of the dimension-raising function when the objective function takes the minimum value, that is, the optimal parameter value. Loss1 is the evolution error term, which represents the error between the dimension-raising state and the state after the Koopman operator evolution. A represents the state transfer matrix of the dexterous hand in high-dimensional space. B represents the input matrix. Loss2 is the reconstruction error term, which represents the error generated by the result of first dimension-raising and then dimension-reducing the current state compared with the state before dimension-raising. C represents the state dimension-reduction matrix. α, β, and are weight coefficients, is the controllability discriminant, is the stability constraint term, and N is the number of samples. The method provided in this embodiment integrates the evolution error term, reconstruction error term, controllability discriminant term and stability constraint term into the loss function. This embodiment can realize the joint optimization of the dimensionality increase function and the linearization model, while taking into account the multi-dimensional requirements such as dynamic evolution accuracy, state reconstruction fidelity, system controllability and stability. The evolution error term ensures the accuracy of the linear prediction of the Koopman operator, the reconstruction error term maintains the consistency of the dimensionality increase-dimensionality reduction mapping, the controllability discriminant term ensures the system accessibility by constraining the rank loss of the controllability matrix, and the stability constraint term guides the state transfer matrix to meet the negative definite condition based on the Lyapunov equation. This optimization framework that integrates data-driven and physical constraints can embed prior knowledge in data fitting, avoid the traditional method from destroying the physical characteristics of the system due to the simple pursuit of prediction accuracy, and finally obtain a dexterous hand linearization model that conforms to the laws of dynamics and has strong generalization ability.

[0114] In some embodiments, the expression of the evolution error term is:

[0115] (8)

[0116] Among them, Loss1 is the evolution error term, represents the matrix after dimension increase, represents the dimension-upgraded matrix at the current moment, represents the matrix after dimensionality increase at the next moment, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, θ represents the network model parameter of the dimension-raising function, and N is the number of samples in the data set;

[0117] The expression of the reconstruction error term is:

[0118] (9)

[0119] Among them, Loss2 is the reconstruction error term, represents the matrix after dimension increase, represents the current moment's dimension-enhanced matrix, C represents the state dimension-reduced matrix, x k Indicates the current state.

[0120] In some embodiments, the expression of the controllability criterion term can be:

[0121] (10)

[0122] in, ,Q represents the controllability matrix, is the rank of the matrix, is the dimension of the matrix, is the state x based on the current moment k And the dimension-elevated matrix of the network model parameters θ of the dimension-elevating function.

[0123] The stability constraint can be expressed as:

[0124] (11)

[0125] Where P is a symmetric positive definite matrix, represents the Frobenius norm, is the regularization weight coefficient.

[0126] In this embodiment, The first term is used to approximate a solution that satisfies the inequality by minimizing the difference, that is, the Lyapunov equation constraint is used as a soft constraint to guide approaches O, where O represents the zero matrix, that is, a matrix with all elements equal to 0. The second term is the trace penalty, which guides the state transfer matrix to evolve in the negative definite direction, thereby ensuring that the final state transfer matrix satisfies inequality (12):

[0127] O (12)

[0128] Through the combination of the two, the stability condition is transformed into a differentiable loss term, which penalizes the parameters that do not satisfy the Lyapunov inequality.

[0129] In some embodiments, in order to adapt to the changes in the dynamic characteristics of the dexterous hand in multi-stage tasks, a stage-adaptive loss weight adjustment mechanism can be used to better balance the model's expressiveness and physical consistency. The stage-adaptive weight function expression is:

[0130] (13)

[0131] in, is the weight of the controllability discriminant at time k, is the default base weight, Used to improve controllability constraints during the grasping phase, is the preset maximum adjustable coefficient. Because the system faces external disturbances and transient instability at the moment of grasping contact, it is necessary to significantly strengthen the system's controllability and stability constraints. However, during the approach or stabilization phase, the relevant constraints can be appropriately weakened to unleash the model's expressive power.

[0132] The dexterous hand dynamics modeling method provided in this embodiment collects the state and torque inputs of each joint of the dexterous hand to establish a data set, and then uses network models such as deep neural networks to fit the dimensionality-raising function used in the finite-dimensional approximation of the Koopman operator based on the data set. This can improve the accuracy and versatility of modeling, significantly reduce the computational complexity in real-time control scenarios, and at the same time ensure the high accuracy and robustness of model predictions.

[0133] The dexterous hand dynamics model established based on the embodiment of the present application can subsequently be used to control the dynamics of the dexterous hand of a humanoid robot, and can achieve more precise control, such as more precise control of the force and torque of the dexterous hand.

[0134] Figure 4 Schematic diagram of the process of the dexterous hand dynamics modeling method provided in the embodiment of the present application Figure 2 .like Figure 4 As shown, based on the above embodiment, for example Figure 2Based on the embodiment shown, this embodiment adopts a segmented modeling strategy to adapt to the dynamic characteristics of the dexterous hand during the execution of tasks, such as the index finger of the dexterous hand during the grasping process, which may change over time. The method includes:

[0135] 401. Construct a data set based on the historical state data of the dexterous hand at multiple moments and the corresponding historical torque.

[0136] Step 401 in this embodiment is similar to step 201 in the above embodiment and will not be described again here.

[0137] 402. Segment the data set based on a threshold value corresponding to the dynamics index to obtain sub-data sets corresponding to multiple task stages.

[0138] Specifically, since the index finger of a dexterous hand is subjected to different force conditions at different grasping stages, the system dynamics (such as stiffness and damping coefficient) and external disturbances (such as friction, resistance, etc.) will change over time. Therefore, the existing data set can be divided according to fixed time intervals or thresholds based on dynamic changes, and a segmented modeling approach can be adopted to ensure the robustness of the model when the system characteristics change rapidly. Figure 5 As shown, a dataset can be constructed based on inputs such as torque and state. The data in the dataset can then be preprocessed, such as filtering and normalization. Optionally, the dataset can be segmented into n time intervals corresponding to different task phases. Finally, a linearized model can be obtained using the Koopman operator and neural network training.

[0139] In some embodiments, the dynamic indicators include at least one of the following: joint angular velocity, angular acceleration, control torque, contact force; the multiple task stages include: approaching the object stage, grasping the object stage and stable grasping stage.

[0140] Specifically, in the actual grasping task, it can be divided into the following stages according to different models:

[0141] Phase 1: The index finger gradually approaches the target object from its initial position, and the model parameters of the current interval are used to perform real-time state prediction to guide the index finger to move along the planned trajectory;

[0142] Phase 2: When the index finger touches an object, the dynamic characteristics of the dexterous hand change (for example, due to the influence of contact surface friction and damping). The model trained in this phase is combined with Model Predictive Control (MPC) to dynamically adjust the joint torque to ensure stable and precise force.

[0143] Phase 3: After the object is successfully grasped, the system dynamics tends to be stable, but external disturbances (such as object weight or posture changes) still need to be predicted in real time. Based on the updated model parameters , , continuously optimize the control torque, offset external disturbances, and maintain stable contact between the index finger and the object.

[0144] In some embodiments, the data set is segmented based on the threshold value corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages, which may include: if the joint angular velocity is greater than a first preset angular velocity, the angular acceleration is greater than the preset angular acceleration and the contact force is less than the first threshold, the corresponding data is determined as the sub-data set corresponding to the approaching object stage; if the contact force is greater than a second threshold, the corresponding data is determined as the sub-data set corresponding to the grasping object stage; the second threshold is greater than or equal to the first threshold; if the joint angular velocity is less than the second preset angular velocity, and the fluctuation value of the control torque is less than the preset torque, the corresponding data is determined as the sub-data set corresponding to the stable grasping stage; the second preset angular velocity is less than the first preset angular velocity; if the joint angular velocity is greater than the third preset angular velocity, the corresponding data is determined as the sub-data set corresponding to the moving object stage; the third preset angular velocity is greater than the first preset angular velocity; the time of each sample in the sub-data set corresponding to the moving object stage is later than the time of each sample in the sub-data set corresponding to the approaching object stage.

[0145] In some embodiments, the data of the sub-dataset is expressed as follows:

[0146] (14)

[0147] Among them, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage An independent subscript.

[0148] 403. Construct a plurality of linearized models corresponding to the task stages according to the dimensionality-raising function.

[0149] Specifically, in the process of constructing corresponding linearized models for different task stages, the method shown in step 202 in the above embodiment can be adopted, and the linearized model of the previous stage can also be used to construct the linearized model of the current stage.

[0150] In some embodiments, constructing linearized models corresponding to the plurality of task stages respectively according to the dimensionality-raising function may include:

[0151] After determining the linearization model of the current stage, determining the linearization model of the next stage based on the following expression according to the linearization model of the current stage;

[0152] (15)

[0153] Among them, U τ represents the control input of the current stage, represents the state transition matrix of the dexterous hand in the high-dimensional space at the current stage, represents the state transition matrix of the dexterous hand in the next stage of high-dimensional space, represents the input matrix of the current stage, represents the input matrix of the next stage, Represents the state dimensionality reduction matrix of the current stage, represents the next stage state dimension reduction matrix, β τ+1 is the length of the next phase of the time interval, θ τ Represents the network model parameters of the dimensionality-raising function at the current stage, Indicates the next stage of dimensionality upgrade, express The transpose of Relative to The next moment of the dimensional state, Represents the augmented state-input matrix formed by concatenating the dimensionality-increasing state and input of the next stage, express The transpose of represents the state variable matrix of the next stage, Indicates the dimensionality-raising state of the current stage, express The transpose of represents the identity matrix, Represents the augmented state-input matrix formed by concatenating the current stage's dimensionality-increment state and input, express The transpose of .

[0154] Specifically, in order to speed up the modeling process and avoid repeated calculation of the pseudo-inverse in expression (17), the model approximation in the next time interval can be obtained by numerical solution. Assuming that Ψ τ For full row rank, the model at the next moment can be calculated by expression (15).

[0155] Calculate the next moment After that, similar to the process in the previous time interval, the optimal model parameters in the current time interval can be obtained by minimizing the loss function (18) through the neural network. , and then according to formula (15) we can get the network that matches the updated network , Finally, the dynamic model of the dexterous hand at different time intervals can be obtained. .like Figure 6 As shown, we can first initialize the deep neural network of the dimensionality-raising function and solve , and then obtain the network parameters by minimizing the loss function , and then the next time interval can be calculated based on expression (15), that is, the next task stage , and then obtain the network parameters by minimizing the loss function , and so on, to obtain linearized models at different stages.

[0156] 404. For each task stage, based on the sub-dataset corresponding to the task stage, the linearized model of the task stage is trained according to the preset loss function of the task stage to obtain the dynamic model of the dexterous hand in the task stage.

[0157] The model training process for each task stage in step 404 in this embodiment is similar to step 203 in the above embodiment and can be used for reference.

[0158] Specifically, for the sub-dataset of each task stage, the expressions of the loss functions L1 and L2 in matrix form are:

[0159] (16)

[0160] Where, , , at this time the network Corresponding

[0161] It can be obtained by formula (17), where Represent the pseudoinverse of a matrix:

[0162] (17)

[0163] In some embodiments, the expression of the preset loss function of the task stage is:

[0164] (18)

[0165] in, , K is the approximation of Koopman operator in finite dimension, represents the state transition matrix of the dexterous hand in the high-dimensional space at the current stage, Represents the input matrix of the current stage, Represents the state dimensionality reduction matrix of the current stage, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscript, Ψ τRepresents the dimensionality-raising function at the current stage, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, θ τ The network model parameters representing the dimension-raising function.

[0166] The dexterous hand dynamics modeling method provided in this embodiment uses a data-driven approach to model dexterous hand dynamics. Compared to traditional methods based on mechanism equations, it can effectively capture nonlinear characteristics such as friction and deformation, and avoid relying on complex auxiliary variables. By combining the Koopman operator and deep neural network, the constructed model can efficiently update and evolve the system state during the prediction phase. The segmented modeling structure significantly improves the model's adaptability and real-time performance in complex grasping scenarios. It can adapt to hardware configuration changes without sacrificing computational efficiency, providing high-precision and strong robustness technical support for the multi-scenario application of dexterous hand systems.

[0167] Figure 7 This is a schematic diagram of the structure of the dexterous hand dynamics modeling device provided in the embodiment of the present application. Figure 7 As shown, the dexterous hand dynamics modeling device 70 includes: a construction module 701 and a training module 702.

[0168] The construction module 701 is used to construct a data set based on the historical state data and corresponding historical torque of the dexterous hand at multiple moments.

[0169] The construction module 701 is further used to construct a linearized model of the dexterous hand according to the dimensionality-raising function; the dimensionality-raising function includes a network model, which is used to map the original state to a high-dimensional observable space.

[0170] Training module 702 is used to train the linearized model based on the data set and a preset loss function to obtain a dynamic model of the dexterous hand; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-increased state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error generated by comparing the result of first increasing the dimension of the state and then reducing the dimension of the state with the state before the dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the stability constraint term is used to represent the stability condition of the linearized model.

[0171] The dexterous hand dynamics modeling device provided in the embodiment of the present application collects the state and torque inputs of each joint of the dexterous hand to establish a data set, and then uses network models such as deep neural networks to fit the dimensionality-raising function used in the finite-dimensional approximation of the Koopman operator based on the data set. This can improve the accuracy and versatility of modeling, significantly reduce the computational complexity in real-time control scenarios, and at the same time ensure the high accuracy and robustness of model predictions.

[0172] In some embodiments, the expression of the preset loss function is:

[0173]

[0174] Among them, θ is the network model parameter of the dimension-raising function during training, is the network model parameter value of the dimension-raising function when the objective function takes the minimum value, that is, the optimal parameter value, Loss1 is the evolution error term, which represents the error between the dimension-raising state and the state after the Koopman operator evolution, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, Loss2 is the reconstruction error term, which represents the error generated by the result of first dimension-raising and then dimension-reducing the current state compared with the state before dimension-raising, C represents the state dimension-reduction matrix, α, β, and are weight coefficients, is the controllability discriminant, is the stability constraint term, and N is the number of samples.

[0175] In some embodiments, the expression of the evolution error term is:

[0176]

[0177] Among them, Loss1 is the evolution error term, represents the matrix after dimensionality increase, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, and θ represents the network model parameter of the dimensionality-raising function;

[0178] The expression of the reconstruction error term is:

[0179]

[0180] Among them, Loss2 is the reconstruction error term, represents the matrix after dimension increase, C represents the state dimension reduction matrix, x k Indicates the current state.

[0181] In some embodiments, the expression of the controllability discriminant term is:

[0182]

[0183] in, , represents the controllability matrix, is the rank of the matrix, is the dimension of the matrix, is the state x based on the current moment k and the dimension-elevated matrix of the network model parameters θ of the dimension-elevating function;

[0184] The expression of the stability constraint term is:

[0185]

[0186] Where P is a symmetric positive definite matrix, represents the Frobenius norm, is the regularization weight coefficient, and Tr() represents the rank of the matrix.

[0187] In some embodiments, the data in the data set is expressed as:

[0188]

[0189] in, is the moment input for each sample, is the state of each sample, For data-based The state of the next sample is formed for each sample.

[0190] In some embodiments, the construction module 701 is also used to: split the data set based on the threshold value corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages; the construction module 701 is specifically used to construct multiple linearized models corresponding to the task stages according to the dimensionality-raising function; the training module 702 is specifically used to: for each task stage, based on the sub-data set corresponding to the task stage, train the linearized model of the task stage according to the preset loss function of the task stage to obtain the dynamic model of the dexterous hand under the task stage.

[0191] In some embodiments, the dynamic indicators include at least one of the following: joint angular velocity, angular acceleration, control torque, contact force; the multiple task stages include: approaching the object stage, grasping the object stage and stable grasping stage.

[0192] In some embodiments, the construction module 701 is specifically used to: if the joint angular velocity is greater than a first preset angular velocity, the angular acceleration is greater than a preset angular acceleration and the contact force is less than a first threshold, then the corresponding data is determined as a sub-dataset corresponding to the approaching object stage; if the contact force is greater than a second threshold, then the corresponding data is determined as a sub-dataset corresponding to the grasping object stage; the second threshold is greater than or equal to the first threshold; if the joint angular velocity is less than the second preset angular velocity, and the fluctuation value of the control torque is less than the preset torque, then the corresponding data is determined as a sub-dataset corresponding to the stable grasping stage; the second preset angular velocity is less than the first preset angular velocity; if the joint angular velocity is greater than a third preset angular velocity, then the corresponding data is determined as a sub-dataset corresponding to the moving object stage; the third preset angular velocity is greater than the first preset angular velocity; the time of each sample in the sub-dataset corresponding to the moving object stage is later than the time of each sample in the sub-dataset corresponding to the approaching object stage.

[0193] In some embodiments, the data of the sub-dataset is expressed as follows:

[0194]

[0195] Among them, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscripts;

[0196] The expression of the preset loss function of the task stage is:

[0197]

[0198] in, , K is the approximation of the Koopman operator in finite dimensions, represents the state transition matrix of the dexterous hand in the high-dimensional space at the current stage, Represents the input matrix of the current stage, Represents the state dimensionality reduction matrix of the current stage, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscripts, Represents the dimension-enhanced matrix at the next moment in the current stage, Represents the dimension-enhanced matrix at the current moment in the current stage, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, θ τ represents the network model parameters of the dimension-raising function, Represents the loss value.

[0199] In some embodiments, the construction module 701 is specifically used to: determine the linearization model of the current stage;

[0200] Determining a linearization model for the next stage based on the linearization model of the current stage;

[0201] The linearized model of the next stage is determined based on the following expression:

[0202]

[0203] in, represents the control input of the current stage, represents the state transition matrix of the dexterous hand in the high-dimensional space at the current stage, represents the state transition matrix of the dexterous hand in the next stage of high-dimensional space, represents the input matrix of the current stage, represents the input matrix of the next stage, Represents the state dimensionality reduction matrix of the current stage, represents the next stage state dimension reduction matrix, β τ+1 is the length of the next phase of the time interval, θ τ Represents the network model parameters of the dimensionality-raising function at the current stage, Indicates the next stage of dimensionality upgrade, express The transpose of Relative to The next moment of the dimensional state, Represents the augmented state-input matrix formed by concatenating the dimensionality-increasing state and input of the next stage, express The transpose of represents the state variable matrix of the next stage, Indicates the dimensionality-raising state of the current stage, express The transpose of represents the identity matrix, Represents the augmented state-input matrix formed by concatenating the current stage's dimensionality-increment state and input, express The transpose of .

[0204] The dexterous hand dynamics modeling device provided in the embodiment of the present application can be used to execute the above-mentioned method embodiment. Its implementation principle and technical effects are similar, and will not be repeated here in this embodiment.

[0205] Figure 8Schematic diagram of the hardware structure of the dexterous hand dynamics modeling device provided in an embodiment of the present application. The device can be a computer, a message sending and receiving device, a tablet device, a medical device, etc.

[0206] Device 80 may include one or more of the following components: a processing component 801 , a memory 802 , a power component 803 , a multimedia component 804 , an audio component 805 , an input / output (I / O) interface 806 , a sensor component 807 , and a communication component 808 .

[0207] The processing component 801 generally controls the overall operation of the device 80, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 801 may include one or more processors 809 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 801 may include one or more modules to facilitate interaction between the processing component 801 and other components. For example, the processing component 801 may include a multimedia module to facilitate interaction between the multimedia component 804 and the processing component 801.

[0208] The memory 802 is configured to store various types of data to support operations on the device 80. Examples of such data include instructions for any application or method operating on the device 80, contact data, phone book data, messages, pictures, videos, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0209] The power supply component 803 provides power to the various components of the device 80. The power supply component 803 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 80.

[0210] The multimedia component 804 includes a screen that provides an output interface between the device 80 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensors can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, the multimedia component 804 includes a front-facing camera and / or a rear-facing camera. When the device 80 is in an operating mode, such as a capture mode or a video mode, the front-facing camera and / or the rear-facing camera can receive external multimedia data. Each front-facing camera and the rear-facing camera can have a fixed optical lens system or have focal length and optical zoom capabilities.

[0211] The audio component 805 is configured to output and / or input audio signals. For example, the audio component 805 includes a microphone (MIC) that is configured to receive external audio signals when the device 80 is in an operating mode, such as a call mode, a recording mode, or a voice recognition mode. The received audio signals may be further stored in the memory 802 or transmitted via the communication component 808. In some embodiments, the audio component 805 also includes a speaker for outputting audio signals.

[0212] The I / O interface 806 provides an interface between the processing component 801 and peripheral interface modules, such as a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0213] The sensor assembly 807 includes one or more sensors for providing various aspects of the status assessment of the device 80. For example, the sensor assembly 807 can detect the open / closed state of the device 80, the relative positioning of components, such as the display and keypad of the device 80. The sensor assembly 807 can also detect changes in the position of the device 80 or a component of the device 80, the presence or absence of user contact with the device 80, the orientation or acceleration / deceleration of the device 80, and temperature changes of the device 80. The sensor assembly 807 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 807 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 807 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0214] The communication component 808 is configured to facilitate wired or wireless communication between the device 80 and other devices. The device 80 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 808 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 808 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0215] In an exemplary embodiment, the device 80 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0216] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 802 including instructions. The instructions can be executed by the processor 809 of the device 80 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0217] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0218] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0219] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0220] An embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the dexterous hand dynamics modeling method performed by the dexterous hand dynamics modeling device as described above.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for modeling the dynamics of a dexterous hand, characterized in that: include: Construct a dataset based on the historical state data and corresponding historical torque of the dexterous hand at multiple moments; Segmenting the data set based on thresholds corresponding to the dynamics indicators to obtain sub-data sets corresponding to multiple task stages; The dynamic indicators include: contact force; the multiple task stages include: approaching the object stage, grasping the object stage and stable grasping stage; the contact force in the grasping object stage is greater than the contact force in the approaching object stage; Constructing a plurality of linearized models corresponding to the task stages respectively according to a dimensionality-raising function; the dimensionality-raising function includes a network model for mapping the original state into a high-dimensional observable space; For each task stage, based on the sub-data set corresponding to the task stage, the linearized model of the task stage is trained according to the preset loss function of the task stage to obtain the dynamic model of the dexterous hand in the task stage; the preset loss function includes an evolution error term, a reconstruction error term, a controllability discriminant term and a stability constraint term; the evolution error term is used to represent the error between the dimensionality-upgraded state and the state after evolution by the Koopman operator, the reconstruction error term is used to represent the error generated by comparing the result obtained by first increasing the dimension of the state and then reducing the dimension with the state before the dimensionality increase, the controllability discriminant term is used to represent the controllability condition of the linearized model, and the system is ensured to be reachable by constraining the rank missing amount of the controllability matrix obtained by the state transfer matrix and the input matrix of the dexterous hand in high-dimensional space, and the stability constraint term is used to represent the stability condition of the linearized model, and the state transfer matrix is guided to satisfy the negative definite condition based on the Lyapunov equation; The stability constraint term includes a first term and a second term; the first term is used to convert the Lyapunov equation constraint into a soft constraint by minimizing the difference; the second term is used to guide the state transfer matrix of the dexterous hand in a high-dimensional observable space to evolve in a negative definite direction.

2. The method according to claim 1, characterized in that The expression of the preset loss function is: Among them, θ is the network model parameter of the dimension-raising function during training, is the network model parameter value of the dimension-raising function when the objective function takes the minimum value, that is, the optimal parameter value, Loss1 is the evolution error term, which represents the error between the dimension-raising state and the state after the Koopman operator evolution, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, Loss2 is the reconstruction error term, which represents the error generated by the result of first dimension-raising and then dimension-reducing the current state compared with the state before dimension-raising, C represents the state dimension-reduction matrix, α, β, and are weight coefficients, is the controllability discriminant, is the stability constraint term, and N is the number of samples.

3. The method according to claim 2, characterized in that The expression of the evolution error term is: Among them, Loss1 is the evolution error term, represents the matrix after dimensionality increase, A represents the state transfer matrix of the dexterous hand in high-dimensional space, B represents the input matrix, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, and θ represents the network model parameter of the dimensionality-raising function; The expression of the reconstruction error term is: Among them, Loss2 is the reconstruction error term, represents the matrix after dimension increase, C represents the state dimension reduction matrix, x k Indicates the current state.

4. The method according to claim 2, characterized in that The expression of the controllability discriminant is: in, , represents the controllability matrix, is the rank of the matrix, is the dimension of the matrix, is the state x based on the current moment k and the dimension-elevated matrix of the network model parameters θ of the dimension-elevating function; The expression of the stability constraint term is: Where P is a symmetric positive definite matrix, represents the Frobenius norm, is the regularization weight coefficient, and Tr() represents the rank of the matrix.

5. The method according to claim 1, wherein The expression of the data in the dataset is: in, is the moment input for each sample, is the state of each sample, For data-based The state of the next sample is formed for each sample.

6. The method according to any one of claims 1 to 5, characterized in that The dynamic index also includes at least one of the following: joint angular velocity, angular acceleration, and control torque.

7. The method according to claim 6, characterized in that The data set is segmented based on the threshold corresponding to the dynamic index to obtain sub-data sets corresponding to multiple task stages, including: If the joint angular velocity is greater than a first preset angular velocity, the angular acceleration is greater than a preset angular acceleration, and the contact force is less than a first threshold, the corresponding data is determined as a sub-dataset corresponding to the approaching object stage; If the contact force is greater than a second threshold, the corresponding data is determined as a sub-dataset corresponding to the object grasping stage; the second threshold is greater than or equal to the first threshold; If the joint angular velocity is less than a second preset angular velocity and the fluctuation value of the control torque is less than the preset torque, the corresponding data is determined as a sub-dataset corresponding to the stable grasping stage; the second preset angular velocity is less than the first preset angular velocity; If the joint angular velocity is greater than the third preset angular velocity, the corresponding data is determined as the sub-dataset corresponding to the moving object stage; the third preset angular velocity is greater than the first preset angular velocity; the time of each sample in the sub-dataset corresponding to the moving object stage is later than the time of each sample in the sub-dataset corresponding to the approaching object stage.

8. The method according to claim 6, characterized in that The expression of the data of the sub-dataset is: Among them, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscripts; The expression of the preset loss function of the task stage is: in, , K is the approximation of the Koopman operator in finite dimensions, represents the state transition matrix of the dexterous hand in the high-dimensional space at the current stage, Represents the input matrix of the current stage, Represents the state dimensionality reduction matrix of the current stage, β τ is the length of the time interval of the current stage, k τ The time interval of the current stage Independent subscripts, Represents the dimension-enhanced matrix at the next moment in the current stage, Represents the dimension-enhanced matrix at the current moment in the current stage, x k+1 Indicates the state at the next moment, x k Indicates the current state, u k represents the control input, θ τ represents the network model parameters of the dimension-raising function, Represents the loss value.

9. The method according to claim 6, characterized in that The step of constructing a plurality of linearized models corresponding to the task stages according to the dimensionality-raising function includes: Determine the linearization model of the current stage; Determining a linearization model for the next stage based on the linearization model of the current stage; The linearized model of the next stage is determined based on the following expression: in, represents the control input of the current stage, represents the state transition matrix of the dexterous hand in the high-dimensional space at the current stage, represents the state transition matrix of the dexterous hand in the next stage of high-dimensional space, represents the input matrix of the current stage, represents the input matrix of the next stage, Represents the state dimensionality reduction matrix of the current stage, represents the next stage state dimension reduction matrix, β τ+1 is the length of the next phase of the time interval, θ τ Represents the network model parameters of the dimensionality-raising function at the current stage, Indicates the next stage of dimensionality upgrade, express The transpose of Relative to The next moment of the dimensional state, Represents the augmented state-input matrix formed by concatenating the dimensionality-increasing state and input of the next stage, express The transpose of represents the state variable matrix of the next stage, Indicates the dimensionality-raising state of the current stage, express The transpose of represents the identity matrix, Represents the augmented state-input matrix formed by concatenating the current stage's dimensionality-increment state and input, express The transpose of .

10. A dexterous hand dynamics modeling device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the dexterous hand dynamics modeling method according to any one of claims 1 to 9.

Images