Multi-degree-of-freedom bionic dexterous hand control method and system

Abstract

The invention provides a multi-degree-of-freedom bionic dexterous hand control method and system. According to the method, joint state information of the bionic dexterous hand in the operation process is obtained, then the joint state information is input into a potential space representation model to obtain potential variables representing the motion mode of the bionic dexterous hand, and corresponding control signals are generated according to the corresponding relation between the potential variables and expected control instructions; and then the bionic dexterous hand is driven to execute the expected target action corresponding to the expected control instruction according to the control signal, so that natural coordination control of the multi-degree-of-freedom bionic dexterous hand in a task is realized, and the problems of difficult control, response delay and unnatural action of a high-dimensional nonlinear system are effectively solved.

Description

Technical Field

[0001] This application relates to robot control technology, and more particularly to a multi-degree-of-freedom bionic dexterous hand control method and system. Background Technology

[0002] With the rapid development of robotics and artificial intelligence, bionic dexterous hands with anthropomorphic manipulation capabilities have gradually become an important research direction in fields such as service robots, medical rehabilitation, industrial assembly, and remote control. Bionic dexterous hands typically mimic the structure of a human hand, including multiple fingers and palm joints. Each finger often has three to five independent degrees of freedom, and the entire hand may contain dozens of degrees of freedom. Compared to traditional mechanical grippers, bionic dexterous hands can achieve complex grasping, pinching, sliding, rotating, and dynamic adjustment movements, thus more closely resembling the functional performance of a human hand.

[0003] However, due to the numerous degrees of freedom in bionic dexterous hands, strong coupling and nonlinear correlations exist between joint movements, requiring the control system to coordinate and control multiple channels simultaneously, including various dynamic parameters such as angle, torque, velocity, and contact force. Existing multi-degree-of-freedom bionic dexterous hands primarily learn the "hand state-control signal" mapping relationship through end-to-end deep networks. However, existing models directly compress the high-dimensional input state data into low-dimensional control signals within the network, which easily leads to problems such as potential spatial instability, abrupt changes in movement, and abnormal force feedback. It is also difficult to interpret the physical meaning between internal states and outputs, resulting in difficulties in controlling high-dimensional nonlinear systems, response lag, and unnatural movements. In particular, its generalization performance is low when faced with new tasks or unknown movement conditions. Summary of the Invention

[0004] This application provides a control method and system for a multi-degree-of-freedom bionic dexterous hand, which solves the problems of difficult control, sluggish response, and unnatural movements in high-dimensional nonlinear systems corresponding to multi-degree-of-freedom bionic dexterous hands.

[0005] Firstly, this application provides a method for controlling a multi-degree-of-freedom bionic dexterous hand, including: The joint state information of the bionic dexterous hand during operation is obtained, including the joint angle, torque and end-effector pose information of each joint; The joint state information is input into the latent space representation model to obtain latent variables that characterize the movement pattern of the bionic dexterous hand, and corresponding control signals are generated according to the correspondence between the latent variables and the desired control commands. The bionic dexterous hand is driven by the control signal to perform the desired target action corresponding to the desired control command.

[0006] Optionally, inputting the joint state information into a latent spatial representation model to obtain latent variables characterizing the movement pattern of the bionic dexterous hand includes: The joint state information is normalized and time-series encoded to align the angles, torques and end poses of different joints under a unified dimension and time reference to generate a multidimensional state vector. The multidimensional state vector is then input into the latent space representation model to extract the latent variables.

[0007] Optionally, the latent space representation model includes an encoder and a decoder, wherein the encoder is used to map the high-dimensional joint state information into low-dimensional latent variables, and the decoder is used to generate corresponding joint control signals based on the latent variables.

[0008] Optionally, the encoder includes an input layer, a temporal convolutional layer, a bidirectional recurrent neural network layer, and a latent vector extraction layer; The temporal convolutional layer is used to extract spatial dependency features between different joints from the normalized joint state information; the bidirectional recurrent neural network layer is used to extract the temporal sequence correlation of the joint state information; and the latent vector extraction layer is used to map the output of the bidirectional recurrent neural network layer to the latent variable. The decoder includes an input layer, a feature fusion layer, and a joint signal prediction layer. The input layer receives the joint input of the latent variables and the desired control command. The feature fusion layer performs nonlinear feature mapping and fusion on the joint input. The joint signal prediction layer outputs multi-channel control signals to drive each joint to perform the corresponding desired target action based on the fusion result.

[0009] Optionally, before inputting the joint state information into the latent spatial representation model, the method further includes: Collect state data such as joint angles, torques, and end-effector poses of the bionic dexterous hand under different operational tasks, along with corresponding end-effector mechanical response data, to form a multi-dimensional temporal task sample set; The task sample set is preprocessed by normalization, time synchronization, and noise filtering. The latent spatial representation model is trained in stages with the goal of minimizing joint state reconstruction error and end-effector mechanical response error. The staged training includes an encoder pre-training stage, a joint training stage of the encoder and decoder, and a model optimization and freezing stage.

[0010] Optionally, the step of training the latent spatial representation model in stages with the objective of minimizing joint state reconstruction error and end-effector mechanical response error includes: During the encoder pre-training phase, only the encoder is trained to learn the mapping relationship between the joint state information and the latent variables; During the joint training phase of the encoder and decoder, both the encoder and the decoder are optimized simultaneously. During the model optimization freezing phase, once the model converges, the encoder parameters are frozen to obtain the latent space representation model.

[0011] Optionally, driving the bionic dexterous hand to execute the desired target action corresponding to the desired control command according to the control signal includes: The multi-channel control signals output by the latent space representation model are decoded and mapped by channel to correspond to the target angle, torque or driving force of each joint of the bionic dexterous hand. The control signal is used to perform closed-loop position and force control on each joint actuator. The closed-loop position and force control is achieved by dynamically comparing the feedback values ​​of the joint position sensor and force sensor with the desired target action in real time.

[0012] Secondly, this application provides a multi-degree-of-freedom bionic dexterous hand control system, comprising: The acquisition module is used to acquire the joint state information of the bionic dexterous hand during the operation process. The joint state information includes the angle, torque and end-effector pose information of each joint. The processing module is used to input the joint state information into the latent space representation model to obtain latent variables that characterize the movement pattern of the bionic dexterous hand, and generate corresponding control signals according to the correspondence between the latent variables and the desired control commands. An execution module is used to drive the bionic dexterous hand to perform the desired target action corresponding to the desired control command according to the control signal.

[0013] Thirdly, this application provides an electronic device, comprising: Processor; and, Memory for storing the executable instructions of the processor; The processor is configured to perform any of the possible methods described in the first aspect by executing the executable instructions.

[0014] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement any of the possible methods described in the first aspect.

[0015] The multi-degree-of-freedom bionic dexterous hand control method and system provided in this application acquires the joint state information of the bionic dexterous hand during operation, and then inputs the joint state information into a latent space representation model to obtain latent variables representing the motion mode of the bionic dexterous hand. Based on the correspondence between the latent variables and the desired control command, corresponding control signals are generated. Then, based on the control signals, the bionic dexterous hand is driven to execute the desired target action corresponding to the desired control command. This realizes the natural and coordinated control of the multi-degree-of-freedom bionic dexterous hand in the task, and effectively solves the problems of difficult control, response lag and unnatural movement of high-dimensional nonlinear systems. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] Figure 1 This is a flowchart illustrating a multi-degree-of-freedom bionic dexterous hand control method according to an example embodiment of this application; Figure 2 This is a flowchart illustrating a latent space representation model training method according to an example embodiment of this application; Figure 3 This is a schematic diagram of the structure of a multi-degree-of-freedom bionic dexterous hand control system according to an example embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application.

[0018] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0020] Figure 1 This is a flowchart illustrating a multi-degree-of-freedom bionic dexterous hand control method according to an example embodiment of this application. Figure 1 As shown, the multi-degree-of-freedom bionic dexterous hand control method provided in this embodiment includes: S110. Obtain joint state information of the bionic dexterous hand during operation.

[0021] In this step, the joint state information of the bionic dexterous hand during operation can be obtained. The joint state information includes the angle, torque and end-effector pose information of each joint.

[0022] In practical applications, a high-speed sampling module can be used to collect state data of all joints in a bionic dexterous hand at fixed intervals, and the time axis can be aligned using a unified data synchronization protocol. Then, a normalization preprocessing method is used to convert signals of different dimensions (angle, torque, pose) into dimensionless feature vectors, allowing each signal to participate in modeling within the same space. That is, the joint state information is normalized and time-series encoded to align the angle, torque, and end-effector pose information of different joints under a unified dimension and temporal reference, generating a multidimensional state vector. This multidimensional state vector is then input into the latent space representation model to extract latent variables.

[0023] S120. Determine the potential variables based on the joint status information, and generate the corresponding control signals based on the correspondence between the potential variables and the desired control commands.

[0024] In this step, joint state information can be input into a latent spatial representation model to obtain latent variables that characterize the movement pattern of the bionic dexterous hand, and corresponding control signals can be generated based on the correspondence between the latent variables and the desired control commands.

[0025] It is worth noting that in actual operation, the movement of a multi-degree-of-freedom dexterous hand is a nonlinear, multivariable coupled system with a large number of implicit cooperative relationships, such as the coordinated contraction trend when multiple fingers grasp simultaneously. These implicit patterns are difficult to express explicitly through analytical equations by traditional algorithms.

[0026] To address this, the aforementioned latent space representation model maps high-dimensional joint state vectors to a low-dimensional latent space. Within this space, instead of directly processing dozens of joint dimensions, it describes the core kinematic features of dexterous hand movements (such as "grip strength," "rotational intention," and "coordination direction") through several latent variables. Through latent space representation, the system can extract physically meaningful low-dimensional motion coordination patterns from high-dimensional states, enabling control commands to be efficiently mapped to multi-joint coordination signals in the low-dimensional space. Compared to traditional algorithms using latent patterns, the mapping learned by this system exhibits better generalization ability, maintaining control stability and natural movement under environmental changes and new tasks.

[0027] Optionally, the model may employ a deep autoencoder structure, including an encoder and a decoder, wherein the encoder is used to map high-dimensional joint state information into low-dimensional latent variables, and the decoder is used to generate corresponding joint control signals based on the latent variables.

[0028] Specifically, the encoder includes an input layer, a temporal convolutional layer, a bidirectional recurrent neural network layer, and a latent vector extraction layer. The temporal convolutional layer extracts spatial dependency features between different joints from the normalized joint state information; the bidirectional recurrent neural network layer extracts the temporal sequence correlations of the joint state information; and the latent vector extraction layer maps the output of the bidirectional recurrent neural network layer to latent variables.

[0029] The decoder described above includes an input layer, a feature fusion layer, and a joint signal prediction layer. The input layer receives a joint input of latent variables and desired control commands. The feature fusion layer performs nonlinear feature mapping and fusion on the joint input. The joint signal prediction layer outputs multi-channel control signals based on the fusion results to drive each joint to perform the corresponding desired target action.

[0030] In multi-DOF bionic dexterous hand systems, the joints of different fingers simultaneously involve posture adjustment and force feedback control when performing tasks. With dozens of degrees of freedom, the number of control signals is extremely large, and complex nonlinear relationships exist between each control command. For example, the angle scaling of the thumb and index finger when pinching is coupled with the force output, and this coupling relationship dynamically changes with the task. Traditional control strategies employ explicit dynamic equations or empirical mappings, but these models are prone to failure when the task environment changes or the object characteristics differ significantly, resulting in unnatural control actions, sluggish responses, and difficulty in generalization.

[0031] Therefore, by training the latent space representation model as described above, we can achieve global collection and learning of the state data of each task of the bionic dexterous hand. This enables the model to automatically learn the implicit nonlinear mapping relationship between joint states, end-effector mechanical responses, and cooperative action patterns, thus achieving generalized control of the dexterous hand across multiple tasks.

[0032] Figure 2 This is a flowchart illustrating a latent space representation model training method according to an example embodiment of this application. Figure 2 As shown, training methods for latent space representation models can include: S121. Collect state data such as joint angles, torques, and end-effector poses of the bionic dexterous hand under different operational tasks, along with corresponding end-effector mechanical response data, to form a multi-dimensional temporal task sample set.

[0033] In existing technologies, single control samples or static operation data are insufficient to describe the dynamic changes of a bionic hand under different tasks, resulting in control models that can only cope with fixed tasks or specific postures and are difficult to handle highly complex operations (such as grasping, rotating, and classifying operations).

[0034] Therefore, in this step, the bionic dexterous hand is driven to perform different operations using a predefined set of tasks (e.g., grasping hard objects, pinching flexible materials, rotating thin sheet structures, etc.). The system synchronously records the joint angles, torques, and end-effector poses at a frequency of 1kHz through a data acquisition module, while also recording the end-effector contact force and object reaction force feedback. The collected data is categorized and stored according to task number, forming a time-series task sample set. Specifically, this can be achieved by deploying multiple types of sensors (such as multi-axis force sensors, position encoders, and inertial measurement units) on the bionic dexterous hand to collect joint angles, torques, and three-dimensional end-effector pose signals, and synchronously recording these signals along with the end-effector mechanical response (e.g., contact force distribution, friction, or object stability indicators) for the corresponding task, thus constructing the aforementioned multi-dimensional time-series task data. Through this step, the training data can cover different task conditions and operation modes, establishing a database of hand cooperative motion laws under real task environments.

[0035] S122. Normalize, synchronize time, and preprocess the task sample set by noise filtering.

[0036] The raw sensor data collected suffers from several issues, including dimensional inconsistencies (angle and force units are different), temporal asynchrony (different sensor sampling frequencies are inconsistent), and noise interference (mechanical vibration or electromagnetic noise, etc.). Directly inputting these data into the model can lead to training convergence difficulties or distorted learning results. Therefore, data normalization, synchronization, and filtering are essential to ensure that data from different dimensions accurately represent hand dynamics on a unified scale and time frame. After normalization and synchronization, the angle and torque characteristics of different joints are comparable in dimensionality and strictly aligned on the time axis. Noise filtering improves the stability and convergence speed of model training, enabling the latent space to more accurately reflect the true collaborative movement patterns of dexterous hands.

[0037] S123. The latent space representation model is trained in stages with the goal of minimizing the joint state reconstruction error and the end-effector mechanical response error.

[0038] Traditional model training often focuses only on the quality of input state reconstruction, neglecting the dynamic consistency of the hand's mechanical response during task execution. Models trained in this way fail to accurately reflect the subtle operational differences of the bionic hand when applying contact forces, resulting in control signals that are accurate only at the attitude level but cannot maintain mechanical stability. To address this, this step employs a multi-stage optimization strategy to progressively stabilize the latent space model: first, in the encoder stage, a state compression mapping is learned; then, in the encoder-decoder joint stage, a multi-task mapping from state to control signals and end-effector responses is learned; finally, in the model freezing stage, the decoder is optimized to ensure that the output response range is consistent with the actual hardware driver.

[0039] Specifically, the phased training includes an encoder pre-training phase, a joint encoder and decoder training phase, and a model optimization and freezing phase, so that the latent variables of the latent space representation model obtained after training can simultaneously represent the mapping relationship between the cooperative motion mode of the bionic dexterous hand and the corresponding control signal.

[0040] In other words, traditional model training processes typically focus on the accuracy of input state reconstruction, optimizing only the model's ability to reproduce posture, without simultaneously considering the dynamic consistency of the bionic dexterous hand's mechanical response during actual task execution. This type of training fails to effectively capture the subtle physical changes in the hand when applying contact forces or performing interactive operations. As a result, while the model's output control signals may be accurate in geometric posture, they exhibit deviations in mechanical stability, making it difficult to support flexible adaptive control for complex tasks. Therefore, the above steps are to gradually build the hierarchical representation capability of the model through phased training: First, through the encoder pre-training stage, the nonlinear mapping relationship between high-dimensional joint state information and low-dimensional latent space is learned, realizing state compression and initial stabilization of the latent space structure; Second, in the encoder and decoder joint training stage, a weighted joint optimization of joint state reconstruction error and end-effector mechanical response error is introduced, so that the latent variables simultaneously represent posture characteristics and mechanical properties, realizing multi-task mapping learning from motion mode to mechanical response; Finally, in the model optimization and freezing stage, after the joint loss function converges, the encoder parameters are frozen, and only the decoder output range and response curve are fine-tuned to ensure that the numerical range of the model output signal is consistent with the physical execution characteristics of the bionic hand hardware drive system, thereby achieving the response stability and execution safety of the model in actual control.

[0041] To make it easier to understand, a traditional bionic hand control model can be understood as a system that can only reproduce the "shape of hand movements." It can imitate the posture of movements, but it hasn't learned "how much force to apply or how to adjust the force." In actual operation, such as squeezing an egg or holding a piece of rubber, it may move very accurately, but the force is not well controlled; it's either too loose to hold and too tight to crush it.

[0042] To ensure the bionic hand can perform both correct movements and apply the correct force, the above steps outline three stages in the training process: The first stage, namely the encoder pre-training stage, teaches the bionic hand how to compress complex hand information into a concise and stable internal representation, thus avoiding erratic output.

[0043] The second stage is the joint training stage of the encoder and decoder, which teaches it both movement and force, so that the bionic hand can learn to adjust the contact force according to the task while learning the posture.

[0044] The third stage: model optimization and freezing stage. Once the bionic hand has learned to coordinate movement and strength, the front part is fixed, and the details are adjusted according to the actual hardware to ensure that the hand can operate stably and safely in a real environment.

[0045] The bionic hand trained in this way can move more naturally and smoothly when performing various tasks, and can also adjust itself when encountering external interference.

[0046] Specifically, in the encoder pre-training phase, only the encoder is trained to learn the mapping relationship between joint state information and latent variables, and a preliminary stable latent space representation is obtained by minimizing the reconstruction error between the input state and the reconstructed state. It is worth noting that in a multi-DOF bionic hand system, direct end-to-end training is difficult due to the extremely high dimensionality of the input data (dozens of joint angles and torques). This makes it hard to find a stable state compression mapping in the early stages of model learning, resulting in a lack of physical continuity in the latent space variables and fluctuations or distortions in the output control signal. Therefore, the encoder needs to be trained separately at the beginning to stably learn the mapping relationship from high-dimensional states to low-dimensional latent space representations, thereby obtaining a physically reliable and interpretable internal dynamic structure. This allows subsequent joint training to learn the mechanical response mapping on a stable latent space. After encoder pre-training, the latent space possesses good physical continuity and stability, enabling smooth representation of the posture differences between adjacent time steps, even when the fingers begin or stop moving, without abrupt changes in latent variables.

[0047] During the joint training phase of the encoder and decoder, both are optimized simultaneously. The joint loss function is the weighted sum of reconstruction error and end-effector mechanical response error. This allows for end-to-end joint training of the latent space representation model, enabling latent variables to simultaneously reflect the motion patterns and end-effector mechanical response characteristics of the bionic dexterous hand. It's worth noting that encoder training alone can only reconstruct posture but does not learn the characteristics of the end-effector mechanical response. Once the hand is involved in manipulating an object, its contact force distribution, friction, or stress changes are not expressed in the latent space, leading to unstable control signal responses to force feedback. To achieve natural and robust motion control, the model needs to learn both posture and mechanical response mappings simultaneously, ensuring the latent space reflects both the finger coordination structure and the end-effector force state. After joint training, the latent space is no longer just a compressed vector of geometric posture but a construction that integrates motion meaning and force response characteristics. Thus, when performing complex dynamic tasks such as grasping flexible objects, the model can adjust the output torque based on minute changes in latent variables, achieving sensitive and smooth manipulation.

[0048] During the model optimization freeze phase, once the joint loss function converges, the encoder parameters are frozen, and only the decoder's output range and response characteristics are fine-tuned to obtain a finalized latent space representation model for practical control. It's worth noting that after training, continuing to adjust all network parameters may lead to overfitting the training data or a decrease in latent space stability. Especially in the actual deployment of the bionic hand, where the encoder input comes from real-time sensors, it's crucial to ensure the latent space representation remains stable during online control; otherwise, even small noises will cause control jitter. Therefore, a model freeze phase is needed to solidify the encoder parameters, ensuring stable latent space mapping, and only fine-tuning the decoder's output range and response curve to match the actual drive system of the bionic hand, guaranteeing the model's controllability and safety at the physical execution level.

[0049] Furthermore, it is worth noting that the above training process can be implemented based on mainstream machine learning and simulation platforms, such as TensorFlow and PyTorch.

[0050] Furthermore, in the control system of a multi-degree-of-freedom bionic dexterous hand, the system needs to extract a low-dimensional latent space from a large amount of sensor data (such as joint angles, torques, postures, and contact force signals). The number of dimensions in this latent space directly determines the quality of the model's representation of hand movement features. In other words, the number of latent variables determines the information throughput that the encoder can represent. When the dimension is insufficient, the information bottleneck of the latent space is too narrow; when the dimension is too high, information redundancy leads to instability in the gradient descent path. That is, if the dimension is set too low, the latent variables will not be able to fully express all movement information, key details will be lost, and thus the control movement will be inaccurate. However, if the dimension is set too high, it will lead to unstable model training, increased computational burden, easy overfitting, and difficulty in the latent variables converging to a physically meaningful subspace. For ease of understanding, the latent space can be understood as a movement map within the system, with each dimension like a road. If there are too few roads (too low dimension), the map is too coarse, and some alleys (fine movements) will be ignored, resulting in the hand being unable to complete fine tasks. If there are too many roads (too high a dimension), the map becomes too complex, making it not only difficult to draw, but also making it easy to get lost during the driving (training) process and increasing the time spent.

[0051] In this regard, the aforementioned latent space representation model may further include a dimension adaptive adjustment module for adaptively determining the latent space dimensions of the model. Specifically, the dimension adaptive adjustment module determines the importance weights of each latent variable based on the mutual information between each latent variable and the input joint state information during model training, and performs dimension pruning on latent variables below a preset threshold to obtain an adaptive optimal latent dimension configuration.

[0052] The aforementioned adaptive adjustment module is integrated within the latent space representation model. It dynamically determines the optimal latent dimension configuration based on the effective information content of latent variables during model training. Its core principle is to evaluate the latent variables' representational ability of the input state by statistically analyzing the mutual information between each latent variable and the input joint state information. Dimension pruning is performed on latent variables below a set threshold, ensuring the model retains the latent variables that contribute most to the output action. This module works in conjunction with the encoder and decoder to adaptively adjust the structure of the latent space during model training, matching the fitted latent variables with the finger movement patterns of the bionic dexterous hand. By adaptively determining the latent space dimension, the model's complexity is matched to the complexity of the hand task in real time, avoiding overfitting or missing representation of control signals caused by manually specifying dimensions. Through a mutual information-based weight evaluation mechanism, the model can achieve data-driven structural optimization, automatically adjusting the dimension scale across different tasks (e.g., grasping, rotation, or compliant force application) to achieve the optimal sparse distribution of the latent space.

[0053] Furthermore, in the process of constructing the latent space model of a multi-degree-of-freedom bionic dexterous hand, if the true contribution of each latent variable to the input joint state information cannot be effectively determined, uneven distribution of latent variable information will occur. This leads to high correlation or chaotic expression between different latent dimensions, making it difficult for the model to accurately generate control signals corresponding to each joint. The underlying reason is that the joint motion-related structures that can be represented by the latent variables have significant differences, and traditional methods of evaluating variable importance through loss functions or gradient norms cannot accurately characterize this nonlinear relationship.

[0054] Therefore, a specific implementation for determining the importance weights of each latent variable based on the mutual information between each latent variable and the input joint state information can be to construct a mutual information estimation module to receive the latent variables output by the encoder and the corresponding input joint state information. Then, the mutual information estimation module calculates the mutual information between each latent variable and the input joint state information based on a neural mutual information estimation network. Next, the obtained mutual information is converted into importance weights of the latent variables through normalization.

[0055] The above steps involve constructing a mutual information estimation module to receive the latent variables output by the encoder and the corresponding input joint state information, and to calculate the mutual information between them based on a neural mutual information estimation network. The mutual information reflects the statistical dependency between the latent variables and the joint state signals; a higher value indicates a stronger representational ability of the latent variable for the input state features. The technical principle is that the mutual information calculation module determines the degree of statistical dependency based on the log-likelihood difference between the joint distribution and the marginal independent distribution of the samples, and converts the mutual information of all latent variables into importance weights after normalization. This allows for dynamic evaluation of the relative effectiveness of latent variables based on the weight distribution, providing a data foundation for subsequent dimensionality pruning. Through the mutual information estimation mechanism, the latent variables can be effectively ranked for the motion information of the bionic dexterous hand, thus prioritizing the preservation of important feature dimensions in the latent space.

[0056] Furthermore, during model training, if the latent space structure lacks dynamic adjustability, the number of latent dimensions becomes fixed in the early stages of training. This prevents the model from adaptively adjusting to the characteristics of the task, resulting in insufficient representation of complex tasks or dimensional redundancy. The underlying cause of this technical problem is that gradient propagation in latent space feature extraction does not distinguish between the magnitude of information contribution, leading to invalid dimensions continuously consuming computational resources and reducing model training efficiency and stability.

[0057] Therefore, the dimensionality adaptive adjustment module can be configured to work in conjunction with the encoder. By configuring a dimensionality weight update operator at the output of the latent vector extraction layer, the distribution of the importance of latent variables in each iteration cycle is statistically updated. When the importance weight of the target latent is lower than a preset threshold for several consecutive iteration cycles, the gradient propagation of the target latent is automatically frozen, and the target latent dimension is removed in subsequent training to achieve progressive self-optimization of the latent space structure.

[0058] By incorporating a dimension weight update operator and a dimension pruning mechanism within the latent space representation model, working in conjunction with the encoder, progressive self-optimization of the model structure is achieved. The core principle lies in the dimension weight update operator receiving the latent variable weight distribution output by the mutual information calculation module and statistically updating this distribution in each iteration cycle. If the weight of a latent variable is detected to be below a preset threshold for several consecutive iteration cycles, the gradient propagation corresponding to that latent variable is automatically frozen, and the dimension is removed from the latent vector in subsequent iterations.

[0059] Through the above mechanism, the model dynamically filters and eliminates latent variables during the training phase. The dimensional adaptive adjustment module can gradually optimize the latent space structure during training, ensuring that the final model retains only the latent dimensions that contribute most to the control of the bionic dexterous hand. This module works synergistically with the encoder at the output of the latent vector extraction layer, enabling the feature representation output by the encoder to continuously match the current optimal latent dimensional structure, thereby improving the overall quality of control signal generation.

[0060] S130. Drive the bionic dexterous hand to execute the desired target action corresponding to the desired control command according to the control signal.

[0061] In this step, the multi-channel control signals output from the latent space representation model can be decoded and mapped to correspond to the target angles, torques, or driving forces of each joint in the bionic dexterous hand. Then, closed-loop position and force control is performed on each joint actuator based on the control signals. This closed-loop position and force control is achieved by dynamically comparing the feedback values ​​from the joint position and force sensors with the desired target motion in real time.

[0062] Specifically, the system distributes the multi-channel signals output from the decoder to each joint actuator via the channel decoding module. Each signal corresponds to a target angle or torque setpoint. Sensors at each joint provide real-time feedback on current position and torque information, continuously comparing it with the desired target. The drive commands are adjusted in a closed-loop manner using PID or adaptive fuzzy control to ensure synchronized and smooth execution of movements. Through a closed-loop control mechanism based on latent spatial signals, multi-joint coordinated and stable control can be achieved in the high-degree-of-freedom structure of the bionic dexterous hand, reducing vibration and hysteresis during execution and achieving response characteristics close to the natural movements of a human hand.

[0063] In this embodiment, firstly, the angles, torques, and end-effector poses of each joint of the bionic dexterous hand are acquired in real time through a multi-sensor fusion module. The multi-dimensional state signals are then normalized and time-synchronized to achieve feature alignment of different joint data under a unified dimension and time reference. Next, the preprocessed state vector is input into a latent spatial representation model. The encoder of this model extracts spatial features through a temporal convolutional layer and temporally dependent features through a bidirectional recurrent network, thereby obtaining low-dimensional latent variables. These latent variables can be considered a compact expression representing the cooperative motion pattern of the dexterous hand. The decoder then generates multi-channel control signals based on these latent variables and external control commands to drive each joint to perform the corresponding target action. During execution, the system also performs closed-loop adjustments through sensor feedback to ensure accurate motion reproduction and stable dynamic response. Through this method, the system achieves natural coordinated control of a multi-degree-of-freedom bionic dexterous hand in tasks, effectively solving the problems of difficult control, sluggish response, and unnatural movements in high-dimensional nonlinear systems.

[0064] Specifically, the system first normalizes and extracts temporal features from the collected joint angles, torques, and end-effector pose information. These features are then jointly encoded by a temporal convolutional layer and a bidirectional recurrent neural network layer and mapped to low-dimensional latent variables to represent the coordinated motion patterns of each joint in the latent space. Subsequently, these latent variables are fused with the desired control commands, and the decoder outputs multi-channel control signals to drive each joint, thereby ensuring the overall coordination and smooth transition of hand movements. During execution, the system continuously collects feedback signals from the joint position and force sensors, performing closed-loop adjustments to the output control signals to maintain dynamic consistency between the actual movement and the desired target movement.

[0065] Figure 3 This is a schematic diagram of the structure of a multi-degree-of-freedom bionic dexterous hand control system according to an example embodiment of this application. Figure 3 As shown, the multi-degree-of-freedom bionic dexterous hand control system 300 provided in this embodiment includes: The acquisition module 310 is used to acquire joint state information of the bionic dexterous hand during operation. The joint state information includes the angle, torque and end pose information of each joint. The processing module 320 is used to input the joint state information into the latent space representation model to obtain latent variables that characterize the movement pattern of the bionic dexterous hand, and generate corresponding control signals according to the correspondence between the latent variables and the desired control commands. The execution module 330 is used to drive the bionic dexterous hand to perform the desired target action corresponding to the desired control command according to the control signal.

[0066] Optionally, the acquisition module 310 is specifically used for: The joint state information is normalized and time-series encoded to align the angles, torques and end poses of different joints under a unified dimension and time reference to generate a multidimensional state vector. The multidimensional state vector is then input into the latent space representation model to extract the latent variables.

[0067] Optionally, the latent space representation model includes an encoder and a decoder, wherein the encoder is used to map the high-dimensional joint state information into low-dimensional latent variables, and the decoder is used to generate corresponding joint control signals based on the latent variables.

[0068] Optionally, the encoder includes an input layer, a temporal convolutional layer, a bidirectional recurrent neural network layer, and a latent vector extraction layer; The temporal convolutional layer is used to extract spatial dependency features between different joints from the normalized joint state information; the bidirectional recurrent neural network layer is used to extract the temporal sequence correlation of the joint state information; and the latent vector extraction layer is used to map the output of the bidirectional recurrent neural network layer to the latent variable. The decoder includes an input layer, a feature fusion layer, and a joint signal prediction layer. The input layer receives the joint input of the latent variables and the desired control command. The feature fusion layer performs nonlinear feature mapping and fusion on the joint input. The joint signal prediction layer outputs multi-channel control signals to drive each joint to perform the corresponding desired target action based on the fusion result.

[0069] Optionally, the processing module 320 is specifically used for: Collect state data such as joint angles, torques, and end-effector poses of the bionic dexterous hand under different operational tasks, along with corresponding end-effector mechanical response data, to form a multi-dimensional temporal task sample set; The task sample set is preprocessed by normalization, time synchronization, and noise filtering. The latent spatial representation model is trained in stages with the goal of minimizing joint state reconstruction error and end-effector mechanical response error. The staged training includes an encoder pre-training stage, a joint training stage of the encoder and decoder, and a model optimization and freezing stage.

[0070] Optionally, the processing module 320 is specifically used for: During the encoder pre-training phase, only the encoder is trained to learn the mapping relationship between the joint state information and the latent variables, and a preliminary stable latent space representation is obtained by minimizing the reconstruction error between the input state and the reconstructed state. During the joint training phase of the encoder and decoder, the encoder and decoder are optimized simultaneously, and the latent space representation model is jointly trained end-to-end using the weighted sum of the reconstruction error and the end-mechanical response error as the joint loss function. During the model optimization freezing phase, once the joint loss function converges, the encoder parameters are frozen to obtain the latent space representation model.

[0071] Optionally, the execution module 330 is specifically used for: The multi-channel control signals output by the latent space representation model are decoded and mapped by channel to correspond to the target angle, torque or driving force of each joint of the bionic dexterous hand. The control signal is used to perform closed-loop position and force control on each joint actuator. The closed-loop position and force control is achieved by dynamically comparing the feedback values ​​of the joint position sensor and force sensor with the desired target action in real time.

[0072] Figure 4 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application. For example... Figure 4 As shown, the electronic device 400 provided in this embodiment includes: a processor 401 and a memory 402; wherein: Memory 402 is used to store computer programs, and the memory may also be flash memory.

[0073] Processor 401 is used to execute the execution instructions stored in the memory to implement the various steps in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0074] Alternatively, the memory 402 can be either standalone or integrated with the processor 401.

[0075] When the memory 402 is a device independent of the processor 401, the electronic device 400 may further include: Bus 403 is used to connect the memory 402 and the processor 401.

[0076] This embodiment also provides a readable storage medium storing a computer program, which, when executed by at least one processor of an electronic device, enables the electronic device to perform the methods provided in the various embodiments described above.

[0077] This embodiment also provides a program product including a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the methods provided in the various embodiments described above.

[0078] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0079] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A multi-degree-of-freedom biomimetic dexterous hand control method, characterized by, The method comprises the following steps: acquiring joint state information of a bionic dexterous hand during operation, the joint state information including joint angles, torques, and end pose information; inputting the joint state information into a latent space representation model to obtain latent variables representing motion patterns of the bionic dexterous hand, and generating corresponding control signals according to a correspondence between the latent variables and a desired control instruction; driving the bionic dexterous hand to perform a desired target action corresponding to the desired control instruction according to the control signals.

2. The method of claim 1, wherein, The inputting of the joint state information into the latent space representation model to obtain the latent variables representing the motion patterns of the bionic dexterous hand comprises: normalizing and time series encoding the joint state information to align the angle, torque, and end pose information of different joints in a unified dimension and time sequence reference to generate a multi-dimensional state vector; and inputting the multi-dimensional state vector into the latent space representation model to extract the latent variables.

3. The method of claim 1, wherein, The latent space representation model comprises an encoder and a decoder, the encoder being configured to map high-dimensional joint state information to low-dimensional latent variables, and the decoder being configured to generate corresponding joint control signals according to the latent variables.

4. The method of claim 3, wherein, The encoder comprises an input layer, a time convolution layer, a bidirectional recurrent neural network layer, and a latent vector extraction layer; wherein the time convolution layer is configured to extract spatial dependency features between different joints from the normalized joint state information, the bidirectional recurrent neural network layer is configured to extract time sequence correlation of the joint state information, and the latent vector extraction layer is configured to map an output of the bidirectional recurrent neural network layer to the latent variables; the decoder comprises an input layer, a feature fusion layer, and a joint signal prediction layer, the input layer receiving joint input of the latent variables and the desired control instruction, the feature fusion layer performing nonlinear feature mapping and fusion on the joint input, and the joint signal prediction layer outputting multi-channel control signals for driving each joint to perform a corresponding desired target action according to the fusion result.

5. The method of claim 3, wherein, Before the inputting of the joint state information into the latent space representation model, the method further comprises: collecting joint angle, torque, end pose, and other state data and corresponding end mechanical response data of the bionic dexterous hand under different operation tasks to form a multi-dimensional time sequence task sample set; normalizing, time synchronizing, and noise filtering the task sample set for pretreatment; training the latent space representation model in stages with the objective of minimizing joint state reconstruction error and end mechanical response error, wherein the training in stages comprises an encoder pre-training stage, a joint training stage of the encoder and the decoder, and a model optimization freezing stage.

6. The method of claim 5, wherein, The training of the latent space representation model in stages with the objective of minimizing joint state reconstruction error and end mechanical response error comprises: in the encoder pre-training stage, only training the encoder to learn a mapping relationship between the joint state information and the latent variables; In the joint training phase of the encoder and the decoder, the encoder and the decoder are simultaneously optimized; In the model optimization freezing phase, when the model converges, the encoder parameters are frozen to obtain the latent space representation model.

7. The method according to any one of claims 1 to 6, characterized in that, The driving of the bionic dexterous hand according to the control signal to perform the expected target action corresponding to the expected control instruction comprises: Channel decoding and instruction mapping are performed on the multi-channel control signal output by the latent space representation model to correspond to the target angle, torque or driving force of each joint of the bionic dexterous hand respectively; According to the control signal, the closed-loop position and force control of each joint execution unit is performed, and the closed-loop position and force control is realized by dynamically comparing the feedback values of the joint position sensor and the force sensor with the expected target action.

8. A multi-degree-of-freedom bionic dexterous hand control system, characterized in that, It comprises: An acquisition module is configured to acquire joint state information of the bionic dexterous hand during operation, wherein the joint state information comprises joint angle, torque and end pose information; A processing module is configured to input the joint state information into a latent space representation model to obtain latent variables representing the motion mode of the bionic dexterous hand, and generate a corresponding control signal according to the corresponding relationship between the latent variables and the expected control instruction; An execution module is configured to drive the bionic dexterous hand to perform an expected target action corresponding to the expected control instruction according to the control signal.

9. An electronic device, comprising: It comprises: A processor; And A memory for storing executable instructions of the processor; The processor is configured to execute the executable instructions to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method of any one of claims 1 to 7.

Images