Active and passive combined type tendon-like driving bionic dexterous hand control system and method

Abstract

The invention relates to an active and passive combined type tendon-like driving bionic dexterous hand control system and method, and belongs to the field of dexterous hand control. A dexterous hand grabbing control system integrates a target detection algorithm and a posture estimation algorithm based on vision, and is matched with a posture and touch sensor of a dexterous hand; accurate grabbing of objects with different hardness and shapes and at any position in a space is achieved. The main process comprises the steps that an object to be grabbed is recognized and subjected to hardness classification, point cloud information of the dexterous hand and the object to be grabbed is extracted, a grabbing path is planned, in order to prevent the grabbing process from failing, an upper computer rapidly collects feedback information of all joints of the dexterous hand through a CANOpen protocol, and the feedback information of all joints of the dexterous hand is transmitted to the upper computer. And in combination with multi-mode sensing signals of vision, angle, pressure and current sensors, the dexterous hand can quickly recognize and accurately grab an object, and the adaptability and the working efficiency of the dexterous hand in a complex environment are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more particularly to a grasping control system and method for a combined active and passive tendon-driven bionic dexterous hand. Background Technology

[0002] Currently, dexterous hands are an important functional execution component of humanoid robots. Because dexterous hands can simulate the dexterous activities of human hands, they have broad application prospects.

[0003] Currently, lever-driven dexterous hands can analyze the joint angle information during the grasping process by building models. However, it is difficult to build accurate models for dexterous hands driven by non-lever mechanisms. For example, patent CN111390892A uses pneumatic muscles to pull tendon ropes to drive finger joint movement. However, such a design cannot accurately predict joint angle information directly through the model during actual grasping, resulting in a slow and unsmooth grasping process.

[0004] Meanwhile, most existing dexterous hand grasping solutions rely heavily on pure tactile sensors. For example, patent CN119388448A calculates the target position of the servo motor based on contact data from a three-dimensional force tactile sensor and feedback data from the actuator module.

[0005] However, existing non-linkage driven dexterous hands struggle to construct accurate models, and pneumatic muscle-driven methods cannot accurately predict joint angle information, resulting in slow and choppy grasping processes. Pure tactile sensor solutions are susceptible to environmental factors and cannot adapt to complex work scenarios. In practical work, due to the uncertainty of the dexterous hand's working environment, the information collected by the sensors is affected by electromagnetic fields, temperature, and the material of the object being grasped. Therefore, tactile sensor-only solutions are not entirely suitable for real-world work scenarios. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide a combined active and passive tendon-driven bionic dexterous hand control system with stable grasping effect and high precision.

[0007] To address the above problems, this invention provides a control system for various non-linkage driven dexterous hands. The control system is implemented through corresponding control modules, which include: a motor drive module for controlling the bending, extension, and lateral movements of each finger; a signal acquisition module for acquiring information such as joint angles and fingertip force during finger movement; a communication module for exchanging sensor and controller information to achieve closed-loop grasping control; a central processing module for running motor control algorithms and calculating various sensor information; and a vision system for recognizing images and acquiring point cloud information, working in conjunction with the dexterous hand's built-in sensors to achieve more stable and precise grasping movements.

[0008] The system is based on multimodal sensor fusion data to perform grasping control of a dexterous hand, and the multimodal sensor fusion data refers to the fusion data of vision, touch, joint angle and current, etc.

[0009] Furthermore, the motor drive module includes a brushless DC motor driver, a stepper motor driver, and a current detection module. After the host computer sends control information to the central processing module, the designated brushless DC motor is driven through the FOC algorithm, thereby controlling the bending and straightening of each finger. The lateral movement of the dexterous hand is achieved by using a micro stepper motor to drive a linkage mechanism to realize a ±10° lateral movement of the fingers.

[0010] Furthermore, the signal acquisition module collects the angle information of the proximal and middle phalanges of each finger based on software simulation of hardware IIC, and at the same time collects the tactile perception information at the fingertip through the main control 12-bit ADC. The two types of sensors will feed back the finger posture state to the host computer. The host computer dynamically adjusts the acquisition weight according to the distance of the finger to the object to be grasped, thereby realizing a relatively smooth trajectory movement and grasping action.

[0011] Furthermore, the communication module employs the CAN_Open protocol, using different data structures for angle encoder, current information, and tactile sensor information to facilitate parsing by the host computer. A unified signal upload frequency is set according to the acquisition frequency of different signals to ensure the timeliness of various data types.

[0012] Furthermore, the central processing module receives control commands from the host computer via the communication module, parses them, and performs FOC algorithm calculations according to the command requirements, thereby controlling the motors driving the tendon ropes corresponding to each finger of the dexterous hand. Simultaneously, it integrates and processes the data collected by various sensors, and transmits angle, current, and pressure information during the grasping control process back via the communication module.

[0013] Furthermore, after the dexterous hand is initialized, the visual perception system first identifies the object to be grasped and classifies it according to a defined hardness classification table. After determining the object information, it scans the point cloud information of both the dexterous hand and the object, and plans the grasping trajectory based on their position information. As the dexterous hand approaches the object, the weights of the magnetic encoder and tactile sensor are dynamically adjusted, and collision detection of the point cloud information is performed simultaneously to ensure a smooth grasping process.

[0014] Furthermore, the motor drive module adopts the FOC vector control algorithm, which decouples the voltage vector through Clark transformation (Equation 1) and Park transformation (Equation 2), and combines it with the SVPWM algorithm (Equations 3-4) to improve voltage utilization.

[0015]

[0016] Furthermore, when grasping fragile objects, the motor drive module adopts a dual-motor cooperative drive strategy, wherein the DC brushless motor achieves six-step commutation control through a Hall sensor, combined with the following safety protection mechanisms:

[0017] When the phase current collected by the current detection module exceeds the threshold, the overcurrent protection is triggered and the drive signal is automatically cut off.

[0018] Rotor position compensation is performed based on the sector signal output by the Hall sensor (Formula 5), ​​and the compensation amount Δθ is calculated by the following formula:

[0019] Δθ=(θ 实际 -θ 理论) / K d

[0020] Where θ 理论 To calculate the target angle using the inverse Park transformation (Equation 2), K d The damping coefficient;

[0021] The SVPWM algorithm (Formula 3-4) is used to dynamically adjust the PWM duty cycle so that the motor output torque matches the preset gripping force curve.

[0022] The present invention also provides a method for controlling a bionic dexterous hand based on the aforementioned system, comprising:

[0023] S1. Object recognition and classification steps: The vision system acquires images of the objects to be grasped, classifies the objects based on a preset hardness classification table, and generates hardness classification codes which are then sent to the central processing module.

[0024] S2. Point cloud acquisition and trajectory planning steps: Generate 3D point cloud of the object to be grasped based on NeRF neural network, and plan the dexterous hand grasping trajectory by combining Lagrange dynamics modeling;

[0025] S3. Multimodal fusion control steps: fuse visual point cloud data, tactile data, joint angle data and current data, and control the motor drive module to perform grasping actions;

[0026] S4. Closed-loop feedback and execution steps: Transmit data through the communication module, adjust the grabbing action according to the feedback information, and handle exceptions.

[0027] Furthermore, the NeRF neural network encodes distance information into a high-dimensional vector through position encoding (Equation 8), and combines multi-head attention mechanism to fuse multi-view features (Equations 9-11) to improve the accuracy of point cloud reconstruction.

[0028] Compared with the prior art, the active-passive composite tendon-driven bionic dexterous hand control system of the present invention has at least the following beneficial effects:

[0029] It boasts enhanced compatibility and stability. Existing technical solutions rely on linkage mechanisms capable of precise modeling, achieving most grasping actions solely through tactile sensors. However, this is not fully compatible with non-linkage-driven dexterous hands, such as those using tendon ligaments or pneumatic mechanisms, which cannot accurately establish kinematic models. Relying solely on the dexterous hand's own sensors cannot perfectly realize the grasping process. The visual acquisition system incorporated in this invention, linked with the dexterous hand's own sensors, significantly mitigates the impact of factors such as changes in tendon ligament tension and limitations in its own sensing capabilities during the grasping process. Through multi-sensor fusion and advanced control algorithms, this system successfully solves the modeling challenges of non-linkage-driven dexterous hands, achieving a 98% grasping success rate in industrial sorting scenarios.

[0030] The following description, in conjunction with the accompanying drawings, further illustrates the active-passive composite tendon-driven bionic dexterous hand control system of the present invention. Attached Figure Description

[0031] Figure 1 A system framework diagram of an active-passive composite tendon-driven bionic dexterous hand control system of the present invention;

[0032] Figure 2 The software flowchart of the bottom control board of the active-passive composite tendon-driven bionic dexterous hand control system of the present invention.

[0033] Figure 3 This is a schematic diagram of the grasping process of a composite active and passive tendon-driven bionic dexterous hand according to the present invention;

[0034] Figure 4 This is a schematic diagram of a combined active and passive tendon-driven bionic dexterous hand for grasping, according to the present invention.

[0035] Figure 5 This invention discloses a flowchart of point cloud information acquisition for a combined active and passive tendon-driven bionic dexterous hand control system. Detailed Implementation

[0036] like Figure 1 - Figure 4 The figures shown are, respectively, a system framework diagram of an active-passive composite tendon-driven bionic dexterous hand control system of the present invention, a software flowchart of the underlying control board, a dexterous hand grasping process diagram, and a dexterous hand gripping diagram.

[0037] The dexterous hand control board of this invention is connected to the PC via a CAN-TTL module. The host computer sends control information through the CAN_Open protocol and receives information from each sensor periodically.

[0038] On the ROS host computer, a camera with depth information acquisition capability first identifies the category of the object to be grasped. Following manual calibration, the required grasping force information is sent to the slave computer. After confirming the information transmission is complete, the host computer begins acquiring point cloud information of the dexterous hand and the object to be grasped. This point cloud information is constructed using the NeRF (Neural Radiation Field) method. After obtaining the pose information of the dexterous hand and the object, the robotic arm is driven to bring the dexterous hand close to the object, simultaneously determining the appropriate grasping method. Once the dexterous hand is close to the object, the host computer receives information from various sensors via the communication module. Collision detection using the joint angle, fingertip tactile sensors, and point cloud information from the vision module is used to jointly determine whether the grasping was successful.

[0039] Specifically, when the dexterous hand is activated, the microcontroller first initializes the peripherals, and the dexterous hand resets, with each joint returning to its initial position. At this time, the host computer sends a communication test signal to the microcontroller. If communication fails, it will attempt to re-establish communication and print an error message to the operator for troubleshooting. After successful communication, the microcontroller begins executing the main program, waiting to begin the grasping process.

[0040] Furthermore, after the operation process on the host computer is completed, the corresponding control signal is sent to the dexterous hand control board, and the program enters the instruction parsing process. The control signal is parsed according to the preset CAN_Open protocol to determine the actions required for the dexterous hand to grasp, such as pinching, gripping, or clamping.

[0041] Furthermore, after determining the grasping action, the desired angle information is then allocated to the brushless DC motor to control the bending and straightening of the fingers, and to the stepper motor to control the lateral movement of the fingers. The brushless DC motor control process employs a three-loop PID control system consisting of a current loop, a speed loop, and a position loop. The current loop collects the phase current at a frequency of 15kHz and performs inverse Clark and inverse Park transforms to obtain the D-axis and Q-axis information.

[0042] Specifically, the Clark transform converts the voltage signals in the three-phase stationary coordinate system (Ua, Ub, Uc) into signals in the two-phase stationary coordinate system. This transformation decouples the complex three-phase system into two independent components: the direct-axis component Uα and the quadrature-axis component Uβ, thereby improving the efficiency of motor control and laying the foundation for the FOC algorithm.

[0043]

[0044] In the Field-Oriented Control (FOC) algorithm, the Parker transform is a crucial step in achieving high-performance motor control. Its core function is to convert physical quantities from a stationary coordinate system to a rotating coordinate system. Specifically, the Parker transform maps the voltage vector in the α-β coordinate system to the dq coordinate system via a rotation angle θ. In this coordinate system, the stator current is decomposed into the d-axis (excitation component), used to control the motor's magnetic field strength, and the q-axis (torque component), used to directly control the magnitude of the motor torque. By tracking the rotor position θ in real time, the Parker transform ensures that the d-axis direction is always aligned with the rotor direction, achieving the core objective of field-oriented control: maximizing the utilization of stator current to generate torque, and significantly improving motor efficiency, dynamic response, and speed range.

[0045]

[0046] Furthermore, based on the preset d and q axis information, PID control is used followed by inverse Park transformation. To improve the overall gripping ability of the dexterous hand, the SVPWM algorithm is adopted after the inverse Park transformation to improve the overall voltage utilization, enabling the brushless DC motor to obtain stronger driving force, thus improving the overall control response of the dexterous hand.

[0047] Specifically, the SVPWM control strategy controls the converter based on the switching of the inverter's space voltage vector. It uses the switching of the inverter's space voltage vector to obtain a quasi-circular magnetic field, thereby achieving better control performance for the AC motor at a relatively low switching frequency. After obtaining Uα and Uβ through the Park inverse transform, the system determines the sector where the target voltage vector is located. It generates a three-bit binary number based on the symbol combination of A, B, and C, converts it to decimal, and assigns the corresponding sector number (1-6). It then calculates the duration of action of two adjacent effective vectors and the zero vector, allocating them to the three-phase bridge arms to synthesize the target voltage vector.

[0048]

[0049] Where Ts is the corresponding PWM output period, and T1 and T2 are the two adjacent effective vectors mentioned above. It is important to note that if T1 + T2 > Ts, normalization is required to ensure physical realizability.

[0050] Furthermore, during the FOC algorithm operation, the inverse Park transform also needs to obtain the rotor angle position. To reduce the complexity of the entire system and improve anti-interference capabilities, three Hall sensors installed with a 120° position difference are used to estimate the electrical angle through an interpolation algorithm. A single transition signal can process angle information more quickly, and the quasi-electrical angle information is forced at each sector transition to ensure the smooth operation of the motor.

[0051] Specifically, when the motor rotor rotates, the Hall sensors output square wave signals based on changes in the rotor's magnetic field. The rising and falling edges of these signals correspond to specific positions in the rotor's magnetic field. By detecting and processing these signals, the position information of the motor rotor can be obtained. Based on different combinations of the output signals from the three Hall sensors, the location of the motor rotor in six sectors (each sector being a 60° electrical angle) can be determined. For example, when Hall sensors A, B, and C output 100, it indicates that the rotor is in sector 1.

[0052] Taking sector 1 as an example, when the state of the Hall sensor changes from 000 to 100, the time t0 is recorded. When the state changes from 100 to 110, the time t1 is recorded. The electrical angle θ can be estimated using the following linear interpolation formula.

[0053]

[0054] The 0 represents the exact angle at the last level trigger. For example, when the Hall sensor state changes to 110, it means that the rotor angle has entered the second sector, which is 60°.

[0055] Furthermore, when a dexterous hand is required to perform a gripping operation, the number of pulses needed to reach the preset angle is calculated through the mapping relationship between the step angle and the slide displacement. The dexterous hand control board then controls the micro stepper motor slide module to achieve the lateral movement of the fingers based on the number of pulses.

[0056] like Figure 5 As shown, it is a flowchart of point cloud information acquisition.

[0057] Specifically, when the system starts running, it will first identify the object to be grasped based on the ROI area selected by the user. After the object is identified, it will send the corresponding grasping force data to the dexterous hand control board according to the hardness classification preset by the program.

[0058] Furthermore, after the objects to be grasped are classified, the system uses a NeRF neural network to collect point clouds of both the objects to be grasped and the dexterous hand itself, for use in subsequent collision detection.

[0059] Specifically, the multi-view images I_i (i = 1, 2, ..., n, where n is the number of views) of the object to be grasped are input into the 2D backbone network to obtain 2D features. This step typically depends on the network architecture used. For example, for a simple convolutional layer, its output feature map F OUT The calculation formula is:

[0060]

[0061] Where F in It is the input feature map, C inis the input channel, k is half the kernel size, w is the kernel weight, and b is the bias.

[0062] Next, spatial coordinate sampling is performed, uniformly sampling the spatial coordinates throughout the entire scene to obtain the original sampling points P = {p1, p2, ..., p...}. n}, where n is the number of sampling points. Each sampling point has coordinates in 3D space. After sampling is complete, overall fusion begins, projecting the sampling points Pi onto 2D features. In this process, bilinear interpolation is used to sample features in two-dimensional space to obtain the multi-view feature volume V. m For the i-th viewpoint, sampling point p i The projected coordinates on the 2D feature map are (Ui, Vi), and the eigenvalue V is obtained by bilinear interpolation. m (i,p i )for:

[0063]

[0064] Where ω jk These are the weights of the bilinear interpolation, based on (u i ,v i The decimal part is calculated from the decimal part of the result.

[0065] Furthermore, obtain each sampling point p i To the camera's position z i The distance z is obtained by a sine encoder. i Encoded as e(z) i The following formula is typically used:

[0066]

[0067] Where ω1 and ω2 are frequency parameters of different frequencies, used to encode the distance into a high-dimensional vector.

[0068] Furthermore, the feature volume V of the same point from multiple perspectives m (i,p i splicing e(z) to this sampling point i ), to obtain the feature V′ of each viewpoint. m (i,p i )=[V m (i,p i ),e(z i The features V′ of each viewpoint m All layers pass through a linear MLP to obtain the multi-head weights ω(i,p) for each viewpoint. i ) and multi-headed feature V” m (i,p i Assuming the parameter of the MLP is θ, then:

[0069] [ω(i,p i ),V″ m (i,p i )]=MLP θ (V′m(i,p i ))·············(9)

[0070] Long-term weight ω(i,p) i After passing through the softmax function, it is combined with the multi-head feature V” m (i,p i Multiplying these components yields the fused multi-view feature V. f (p i ):

[0071]

[0072] Finally, the features of all the sampling points are combined to obtain the spatial features:

[0073] V o ={V f (p1),V f (p2),…,V f (p N )}··············(11)

[0074] Furthermore, after obtaining the point cloud information of both, the system first uses dynamic modeling analysis to obtain the joint angle information during the grasping process. The dynamic modeling method used in this system is the Lagrange functional equilibrium method. First, a finger kinematic model is established using the DH method to obtain the transformation matrix T of each finger joint. i Then, the dynamic model of the finger is established using the Lagrange equation method:

[0075]

[0076] In the formula, the Lagrange function L = QW, where Q is the total kinetic energy of the finger, W is the total potential energy of the finger, and θ is the total potential energy of the finger. i Let τ be the joint variable and τ be the generalized driving torque.

[0077] The total kinetic energy and total potential energy of the fingers are respectively:

[0078]

[0079] In the formula I i T is the pseudo-inertia matrix. i Let m be the transformation matrix. i Let K be the mass of link i. i is the spring constant of the torsion spring.

[0080] Substituting this into the Lagrange equation and simplifying, we obtain the dynamic equation of the finger as follows:

[0081]

[0082] In the formula D ij For inertial force, D ijk For the centripetal force and Coriolis force terms, D i These are the gravitational potential energy and elastic potential energy terms.

[0083]

[0084] Based on geometric conditions, the generalized driving torque is calculated as follows:

[0085] τ i =F i l i ……………………………(16)

[0086] In the formula F i For the driving force of each joint, l i For each joint, there is a driving force arm.

[0087] Finally, by substituting formulas (15) and (16) into (14), the relationship between the driving force and the joint angles can be obtained. After obtaining the desired angle information of the dexterous hand grasping the object, the data is split according to the set CAN_Open protocol and sent to the underlying dexterous hand control board to execute the grasping program.

[0088] As the dexterous hand approaches the object to be grasped, the dexterous hand control panel provides real-time feedback on the angle information of each joint. The system then adjusts the desired angle based on this feedback. Once the finger is about to reach the object, it begins collecting information from the fingertip pressure sensor, reducing the frequency of angle information collection from each finger joint. This information is then combined with collision detection from the point cloud data to jointly determine the grasping status, ensuring the stability and accuracy of the grasp.

[0089] Table 1 compares the performance advantages of the present invention with those of conventional solutions:

[0090] Table 1 Comparison of System Performance Indicators

[0091] index Traditional solution (CN111390892A) This system Increase Scraping success rate 82％ 96％ +14％↑ Response delay 120ms 45ms 62.5％↓ Modeling accuracy ±3mm (purely tactile) ±0.5mm (multimodal) 83％↑ Energy efficiency 85％ 92% (SVPWM optimized) 8％↑

[0092] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A combined active and passive tendon-driven bionic dexterous hand control system, characterized in that, include: Motor drive module, used to drive and control the movement of the dexterous hand; The signal acquisition module is used to collect data on the angle of dexterous hand joints, tactile pressure, and current. A vision system is used to construct a 3D point cloud of a dexterous hand and the object to be grasped, and to identify object hardness classification and perform collision detection; The central processing module and communication module are used to realize data interaction between the host computer and the underlying control board; The system uses multimodal sensor fusion data to perform grasping control of the dexterous hand.

2. The active-passive composite tendon-driven bionic dexterous hand control system according to claim 1, characterized in that, The motor drive module includes an FOC driver board, a stepper motor driver board, and a current detection module for driving the movement of the dexterous hand. The module is controlled by a central processing unit to control a DC brushless motor and a micro stepper motor to enable the dexterous hand to bend and deflect, and complete finger movements.

3. The active-passive composite tendon-driven bionic dexterous hand control system according to claim 1, characterized in that, The signal acquisition module integrates the signal acquisition functions of sensors including tactile sensors and magnetic encoders, and acquires key parameters of the dexterous hand, including joint movement status, tactile perception and system energy consumption. It acquires multi-source data through the IIC bus, and the acquired information includes: Joint angle data: The angle information of each finger joint of the dexterous hand is collected by a magnetic encoder; Tactile pressure data: Analog pressure information at the contact point is obtained through fingertip tactile sensors; Operating status data: Collects analog current information of each component (such as drive motor) during operation to monitor the system's operating status.

4. The active-passive composite tendon-driven bionic dexterous hand control system according to claim 1, characterized in that, The communication module packages and sends various information, including pose information, current information, and tactile information, during the grasping process to the ROS host computer via the CAN_Open protocol, achieving smooth grasping by relying on the speed and stability of the CAN_Open protocol.

5. The active-passive composite tendon-driven bionic dexterous hand control system according to claim 1, characterized in that, After receiving the control signal from the host computer, the central processing module distributes it to the corresponding drive module. At the same time, it processes the collected signals and sends them to the ROS host computer, realizing closed-loop control of "instruction issuance - execution feedback - status correction".

6. The active-passive composite tendon-driven bionic dexterous hand control system according to claim 1, characterized in that, The vision system is divided into two parts. The first part is used to identify the object to be grasped. After identifying the object, it is classified according to the preset hardness information. After the identification and classification are completed, the second part collects the point cloud information of the dexterous hand and the object to be grasped. Based on the pose information, the system plans the trajectory. After the dexterous hand approaches the object to be grasped, it performs collision detection and determines whether the grasping is successful based on the pressure sensor of the fingertip and the current information of the motor.

7. The active-passive composite tendon-driven bionic dexterous hand control system according to claim 1, characterized in that, The motor drive module adopts the FOC vector control algorithm, which decouples the voltage vector through Clark transformation (Equation 1) and Park transformation (Equation 2), and improves voltage utilization by combining it with the SVPWM algorithm (Equations 3-4).

8. The active-passive composite tendon-driven bionic dexterous hand control system according to claim 1, characterized in that, When grasping fragile objects, the motor drive module adopts a dual-motor cooperative drive strategy, in which the DC brushless motor achieves six-step commutation control through a Hall sensor, combined with the following safety protection mechanisms: When the phase current collected by the current detection module exceeds the threshold, the overcurrent protection is triggered and the drive signal is automatically cut off. Rotor position compensation is performed based on the sector signal output by the Hall sensor (Formula 5), ​​and the compensation amount Δθ is calculated by the following formula: Δθ=(θ 实际 -θ 理论) / K d Where θ 理论 To calculate the target angle using the inverse Park transformation (Equation 2), K d The damping coefficient; The SVPWM algorithm (Formula 3-4) is used to dynamically adjust the PWM duty cycle so that the motor output torque matches the preset gripping force curve.

9. A method for controlling a combined active and passive tendon-driven bionic dexterous hand based on the system described in any one of claims 1-8, characterized in that, include: S1. Object recognition and classification steps: The vision system acquires images of the objects to be grasped, classifies the objects based on a preset hardness classification table, and generates hardness classification codes which are then sent to the central processing module. S2. Point cloud acquisition and trajectory planning steps: Generate 3D point cloud of the object to be grasped based on NeRF neural network, and plan the dexterous hand grasping trajectory by combining Lagrange dynamics modeling; S3. Multimodal fusion control steps: fuse visual point cloud data, tactile data, joint angle data and current data, and control the motor drive module to perform grasping actions; S4. Closed-loop feedback and execution steps: Transmit data through the communication module, adjust the grabbing action according to the feedback information, and handle exceptions.

10. The active-passive composite tendon-driven bionic dexterous hand control method according to claim 9, characterized in that, The NeRF neural network encodes distance information into a high-dimensional vector through position encoding (Equation 8), and combines multi-head attention mechanism to fuse multi-view features (Equations 9-11) to improve the accuracy of point cloud reconstruction.

Images