Bionic Dexterous Hand Control System and Method Based on Electromyography Signals and Deep Learning

Abstract

This invention discloses a bionic dexterous hand control system and method based on electromyography (EMG) signals and deep learning. The system places a drive motor at the rear of the forearm strap, transmitting power to a front-mounted bionic hand module via a flexible shaft to control the movement of the corresponding fingers. This eliminates the need for any drive components within the hand module, resulting in a significantly reduced size. The method includes: acquiring arm EMG signals; filtering and extracting features, then inputting the signals into a deep learning model composed of a residual module, a long short-term memory network, and a multi-channel multilayer perceptron to independently identify the movement intentions of each finger; controlling the rear-mounted motor based on the identification results, and achieving closed-loop control by combining an encoder and a current sensor. This invention, through its innovative split architecture, solves the problems of traditional dexterous hands being bulky and lacking concealment, providing a functional and aesthetically pleasing alternative for patients with partial hand and finger loss.

Description

Technical Field

[0001] This invention relates to the technical fields of bionic robots, medical devices, and biosignal control, particularly to the technical field of bionic dexterous hand control systems and methods based on electromyographic signals and deep learning. Background Technology

[0002] Surface electromyography (sEMG) control technology provides an intuitive way to control prostheses for people with upper limb disabilities. However, most existing sEMG-controlled dexterous hands integrate drive motors, reducers, and other components inside the hand or forearm prosthesis, resulting in a bulky and obtrusive appearance that severely impacts user experience and concealment during daily wear, making them particularly difficult to fit for patients with partial hand loss. Although existing research has focused on improving recognition accuracy through algorithms or optimizing mechanical structures, it has not fundamentally resolved the core contradiction between the space occupied by the drive unit and the need for high conformity of the hand area.

[0003] Patent document CN113349995A discloses a mechanical finger prosthesis, but it relies on the movement of the human wrist joint for actuation, resulting in limited functionality. Analysis revealed that to achieve a highly biomimetic appearance, a disruptive hardware architecture is necessary to separate the actuation system from the hand, laying the foundation for a highly biomimetic look. Simultaneously, a powerful intelligent algorithm is required to achieve precise control on this new architecture. Therefore, there is an urgent need in the field for a novel solution that deeply integrates innovative hardware with intelligent software. Summary of the Invention

[0004] The purpose of this invention is to solve the problems of large size, lack of concealment and aesthetics caused by the built-in drive unit in existing bionic dexterous hands. It proposes a bionic dexterous hand control system and method based on electromyography signals and deep learning. By placing the drive motor in the forearm, using flexible soft shaft transmission, and adopting a split bionic hand module, the invention achieves extreme miniaturization of the hand part, creating conditions for external bionic skin coverage, and ultimately achieving a good concealment effect. Furthermore, it uses advanced deep learning algorithms to achieve independent, precise, and real-time control of the movement of multiple fingers.

[0005] To achieve the above objectives, this invention proposes a bionic dexterous hand control system based on electromyography (EMG) signals and deep learning, mainly comprising a rear-drive module, a signal acquisition module, a control circuit module, a bionic hand module, and a flexible transmission module. The rear-drive module is mounted on the user's forearm or wrist via an adjustable strap and includes a motor, a motor drive circuit, a slide for fixing the motor, and a sensor for control feedback. The signal acquisition module collects surface EMG signals from the user's arm and sends them to the control circuit module, including a surface EMG signal acquisition sensor and a flexible strap for fixing the sensor. The control circuit module includes a deduction circuit for receiving surface EMG signals and performing deep learning deduction, and a drive circuit for receiving the deduction results and controlling different motors. The deduction circuit receives surface EMG signals in real time, deduces the movement intentions of each finger, and then sends the deduction results to the motor drive circuit for independent control of different motors. The bionic hand module is worn on the end of a user's residual limb. The module contains no internal motor or driver; it includes a palm base shell and a personalized connection base that matches the shape of the user's residual limb. The palm base shell and the personalized connection base are detachably connected. The flexible transmission module includes a flexible shaft. One end of the flexible shaft is connected to the output of the motor in the rear-mounted drive module, and the other end extends into the bionic hand module, where it is connected to the mechanical fingers of the bionic hand module via a traction cable or connecting rod. The output of the control circuit module is communicatively connected to the microprocessor of the rear-mounted drive module. The microprocessor is configured to control the movement of the motor based on the recognized movement intention. This core architecture completely solves the problem of bulky hand components.

[0006] Preferably, the motor in the rear drive module is mounted via a sliding mechanism. This sliding mechanism includes a linear slide and a slide rail mounted on the strap. The motor is fixed to the linear slide, and the base of the linear slide engages with the slide rail, allowing the motor to move axially along the flexible shaft and slide along the slide rail as a whole with the linear slide, passively compensating for changes in the transmission path length caused by wrist flexion. This preferred solution passively compensates for path changes caused by wrist flexion, improving transmission reliability.

[0007] Preferably, the sensors for control feedback include a position sensor for feedback of motor rotation angle and a current detection circuit for monitoring motor operating current. This preferred solution achieves dual closed-loop control of position and force, ensuring operational accuracy and providing overload protection.

[0008] Preferably, the deep learning model used in the control circuit module consists of two layers of residual blocks, a Long Short-Term Memory (LSTM) network, and a multi-layer perceptron (MLP) network, capable of independently outputting the motion state of each finger. The MLP network has independent output channels corresponding to the number of fingers to be controlled, with each channel independently outputting the corresponding finger's motion state category, including outward opening, inward flexion, and relaxation. The residual blocks use convolutional kernels of different sizes to process input features in parallel and then superimpose the processing results to form residual connections. The LSTM network processes data in a time-series manner and includes forget gates, input gates, and output gates. This preferred solution eliminates the reliance on predefined gestures and achieves parallel and continuous control of each finger. The residual blocks extract features through multi-branch convolutions, and the LSTM network processes temporal information. This preferred solution combines the advantages of multiple deep learning architectures, ensuring recognition accuracy and robustness.

[0009] Preferably, the signal acquisition module includes an electromyography (EMG) signal acquisition device, employing four dry electrodes respectively positioned on the elbow side, wrist side, elbow side, and wrist side of the finger flexors, finger extensors, and finger extensors of the user's arm. The acquired EMG signals are filtered by a 50Hz band-stop filter to remove power frequency interference and then processed using a sliding window with a window length of 75ms and an overlap rate of 50%. The processed data is used as input to the deep learning model. Alternatively, the EMG signal acquisition device can be multiple ring electrodes or multiple high-density electrode arrays. This preferred scheme ensures signal quality and provides a reliable data foundation for high-precision recognition. Alternatively, a high-density electrode array can be used to acquire richer muscle spatial information. This preferred scheme provides flexible configuration for signal acquisition, and the high-density approach provides a foundation for improving recognition performance.

[0010] Preferably, the microprocessor of the rear drive module executes closed-loop control including angle and current thresholds based on the recognition result: when the movement intention is "opening outwards," the motor is controlled to rotate forward until the angle fed back by the position sensor reaches the set threshold; when the movement intention is "bending inwards," the motor is controlled to rotate in reverse until the current value detected by the current detection circuit reaches the set threshold; when the movement intention is "relaxing," the motor is controlled to stop rotating. This preferred scheme ensures the accuracy and safety of the action execution.

[0011] Preferably, the control circuit module and the microprocessor of the rear drive module interact via wired or wireless communication; the deduction circuit and its power supply battery in the control circuit module are configured as detachable independent modules. This reduces the wiring constraints between modules and improves the flexibility and convenience of wearing the device.

[0012] Preferably, the mechanical fingers within the bionic hand module are driven by a linkage mechanism; the end of the flexible transmission module is connected to a belt drive mechanism or a gear drive mechanism to convert the rotational motion of the motor into linear or rotational motion driving the linkage mechanism. This preferred solution provides an alternative power transmission path to cable traction and may be suitable for applications requiring higher rigidity or greater load capacity.

[0013] Preferably, the bionic hand module is externally covered with a bionic skin material. This preferred solution is suitable for applications where aesthetics are a high priority.

[0014] This invention also provides a bionic dexterous hand control method based on electromyographic signals and deep learning, which is implemented using any of the aforementioned control systems, including: The flexible transmission module transmits the power generated by the rear drive module to the bionic hand module. The signal acquisition module acquires surface electromyographic signals from the user's arm and sends them to the control circuit module. The control circuit module receives the surface electromyography signals and performs deep learning inference to identify the user's movement intentions for each finger. The microprocessor in the rear-mounted drive module generates control commands based on the identified motion intention, driving the motor to move, which in turn drives the mechanical fingers in the bionic hand module to complete the corresponding actions via the flexible transmission module. This method represents a complete process that integrates the aforementioned hardware system innovation with intelligent algorithms.

[0015] The beneficial effects of this invention are: 1. Innovative Hardware Architecture for Miniaturization and High Biomimetic Performance: This invention proposes a hardware architecture of "rear-mounted drive + flexible soft shaft transmission + split biomimetic hand". This architecture eliminates the need for drive components inside the biomimetic hand module, reducing its size and thickness, creating conditions for external biomimetic skin coverage, and improving concealment.

[0016] 2. Improved motion adaptability and reliability: The rear drive module adopts a sliding design, which allows the motor to passively move to compensate for changes in the transmission path when the wrist bends, reducing the internal stress of the transmission system and improving the reliability and comfort of the device.

[0017] 3. High precision in intelligent recognition and control: The deep learning model used in this invention can output the movement intention of each finger in parallel and independently, eliminating the dependence on predefined gestures and making control more intuitive.

[0018] 4. High system integration and comprehensive control and protection mechanisms: This invention highly integrates functional modules such as electromyography signal acquisition, intelligent recognition, motor drive, position and current detection to form a complete closed-loop control system, ensuring motion accuracy and system safety.

[0019] 5. High practicality: The system and method provided by this invention can be manufactured and used. It not only solves the problem of functional substitution, but its highly biomimetic characteristics also meet the psychological needs of users and have application potential.

[0020] The features and advantages of the present invention will be described in detail through embodiments and in conjunction with the accompanying drawings. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the three-dimensional structure of the bionic dexterous hand of the present invention. Figure 1 ; Figure 2 This is a schematic diagram of the three-dimensional structure of the bionic dexterous hand of the present invention. Figure 2 ; Figure 3 This is a schematic diagram of the structure of the bionic dexterous hand of the present invention in use; Figure 4 This is a schematic diagram of the bionic hand module and flexible transmission module of the present invention. Figure 5 This is a diagram of the neural network structure for decoding finger motion state based on residual network and LSTM of the present invention; Figure 6 This is a detailed structural diagram of the residual module, LSTM module and MLP module of the present invention; Figure 7 This is a block diagram of the actuator real-time control system based on surface electromyography signals of the present invention.

[0022] In the diagram: 100-arm, 1-rear drive module, 2-bionic hand module, 3-flexible transmission module, 4-control circuit module, 5-signal acquisition module, 6-flexible strap, 11-linear slide, 21-palm heel shell, 22-personalized connection base, 31-flexible shaft. Detailed Implementation

[0023] See Figures 1-7 The control system of the present invention includes: The rear drive module, which is set on the user's forearm or wrist via an adjustable strap, includes a motor, a motor drive circuit, a slide for fixing the motor, and sensors for control feedback. The signal acquisition module is used to acquire surface electromyography (EMG) signals from the user's arm and send them to the control circuit module. It includes a surface EMG signal acquisition sensor and a flexible strap for fixing the sensor. The control circuit module includes a deduction circuit for receiving surface electromyography signals and performing deep learning deduction, and a drive circuit for receiving deduction results and controlling different motors. The deduction circuit receives surface electromyography signals in real time, deduces the movement intention of each finger, and then sends the deduction results to the motor drive circuit for independent control of different motors. A bionic hand module for wearing on the end of a user's residual limb. The bionic hand module has no motor or driver inside. It includes a palm base shell and a personalized connection base that matches the shape of the user's residual limb end. The palm base shell and the personalized connection base are detachably connected. The flexible transmission module includes a flexible shaft, one end of which is connected to the output end of the motor in the rear drive module, and the other end extends into the interior of the bionic hand module and is connected to the mechanical fingers of the bionic hand module via a traction cable or connecting rod.

[0024] The methods corresponding to the above control system include: The flexible transmission module transmits the power generated by the rear drive module to the bionic hand module. The signal acquisition module acquires surface electromyographic signals from the user's arm and sends them to the control circuit module. The control circuit module receives the surface electromyography signals and performs deep learning inference to identify the user's movement intentions for each finger. The microprocessor of the rear drive module generates control commands based on the identified motion intention, drives the motor to move, and then drives the mechanical fingers in the bionic hand module to complete the corresponding actions through the flexible transmission module.

[0025] The working process of this invention: The present invention relates to a bionic dexterous hand control system and method based on electromyography signals and deep learning, which is described in conjunction with the accompanying drawings during operation.

[0026] Example 1: As attached Figure 1-7 As shown, the control system of the present invention consists of a rear drive module 1, a bionic hand module 2, a flexible transmission module 3, a signal acquisition module 5, and a control circuit module 4.

[0027] The rear drive module 1 is secured to the user's forearm via an adjustable, flexible strap 6, similar to a watch band. A slide rail runs along the forearm direction on the outer side of the strap. A linear slide 11 is mounted to this rail, allowing it to slide along the rail. The motor is fixed to the linear slide 11 via a mounting bracket. This design allows the motor to move on the linear slide 11 and further slide relative to the arm as a whole with the linear slide 11, passively adapting to changes in the transmission path caused by wrist flexion. An encoder is installed at the motor's tail to detect speed and angle. A microcontroller unit (MCU) and motor drive circuitry are also integrated within this module. The MCU is connected to a current detection circuit to monitor the motor's operating current.

[0028] The bionic hand module 2 is worn on the end of a user's residual limb and includes a palm base shell 21 and a personalized connecting base 22. The personalized connecting base 22 is a uniquely shaped curved structure manufactured using 3D scanning and printing technology, precisely matching the anatomical structure of the user's residual limb, allowing it to fit comfortably and securely onto the limb. The palm base shell 21 and the personalized connecting base 22 are detachably connected and fixed via snap-fit ​​structures or locking screws located on both sides of the web between the thumb and forefinger. The entire assembled bionic hand module 2 is covered with a soft, medical-grade silicone bionic skin layer. This silicone layer not only provides a realistic skin texture and appearance but also completely conceals the internal mechanical structure. Internally, it contains mechanical fingers, driven by cable traction. To achieve automatic extension and repositioning of the fingers, elastic repositioning components, such as torsion springs, are provided at the finger joints.

[0029] The flexible transmission module 3 is a flexible shaft 31. The flexible shaft 31 extends from the rear drive module 1, runs along the outer side of the forearm and wrist, and finally enters the palm base housing 21 through the wrist entrance. Inside the palm, the end of the flexible shaft 31 is connected to a winding reel. The traction cable extends from the winding reel, passes through guide holes on each finger bone, and is finally fixed to the fingertip.

[0030] The workflow of the signal acquisition module 5 is as follows: The electromyography (EMG) signal acquisition device (using four dry electrode sensors, respectively arranged on the elbow side, wrist side, elbow side, and wrist side of the flexor digitorum muscles) acquires raw sEMG signals at a sampling frequency of 2000 times per second. After the signal is filtered (50Hz band-stop and band-pass filtering) by the signal processing unit, it is sent to the control circuit module 4. The sliding window processing uses a window length of 75ms and an overlap rate of 50%.

[0031] The workflow of the control circuit module 4 is as follows: it receives feature data from the signal acquisition module 5, which is then deduced by the deep learning recognition unit.

[0032] The model running by the deep learning recognition unit consists of two layers of residual blocks, a long short-term memory (LSTM) network, and a multi-layer perceptron (MLP).

[0033] The structure of the residual module is as follows: the input features are first processed in parallel by two convolutional kernels of different sizes (such as 1x1 and 3x1) to extract features at different time scales. Then, the processing results of the two branches are superimposed to form a residual connection, and finally output through an activation function.

[0034] The structure of the LSTM is as follows: it processes data in time series (t1 to tn), and contains a forget gate, an input gate and an output gate to effectively memorize information over long time intervals and output time series features.

[0035] The structure of the multi-channel MLP is as follows: it consists of multiple linear layers and nonlinear activation functions. In this embodiment, it has three independent output channels, corresponding to the index finger, middle finger, and ring finger, respectively. Each channel independently outputs the probability of the finger being in the three states of "open", "flexed", and "relaxed".

[0036] The recognition results are packaged into control commands and sent back to the main control MCU in the rear driver module 1 via serial port.

[0037] To train the deep learning model, sEMG signals need to be collected when the user performs different finger movements. For each movement category (e.g., single finger open, fist clenched, etc.), the state should be maintained and signal data should be collected continuously for more than 20 seconds to ensure sufficient training samples. Longer collection time generally helps improve the model's generalization ability.

[0038] The MCU control unit constitutes a real-time control system, which receives serial port commands and performs the following operations: Motor control system: The MCU sends PWM signals to the DC motor driver according to the instructions to drive the motor to perform forward rotation, reverse rotation or stop.

[0039] Encoder motor position reading system: The MCU acquires the digital signal fed back by the encoder through the external interrupt interface, obtains the motor rotation angle in real time, and uses it for closed-loop control of the "opening out" action.

[0040] Current acquisition system: The MCU reads the analog signal fed back by the current monitoring chip through the ADC interface, monitors the motor operating current in real time, and uses it for limit and overcurrent protection of the "bending inward" action.

[0041] The MCU controls the motor's movement according to instructions and performs closed-loop control by reading the encoder's count value and current detection value in real time. The entire system is powered by a lithium battery.

[0042] Specifically, when performing an "inward flexion" action and the current reaches a set threshold, the MCU synchronously records the angle value fed back by the encoder at this time. This angle value reflects the actual range of motion of the finger in the current grasping task. The system uses this angle value to dynamically calibrate and update the stopping threshold of the "outward opening" action, so that the limit position of the finger opening can adapt to different grasping objects and wear conditions, realizing the self-learning and optimization of control parameters.

[0043] Example 2: Based on Embodiment 1, the bionic hand module 2 can be replaced with a version that drives five fingers, and its internal transmission mechanism can use a linkage mechanism instead of cable traction. Correspondingly, the multi-channel MLP output layer of the deep learning recognition unit is expanded to five independent channels, each corresponding to one of the five fingers. The rest is the same as in Embodiment 1. This embodiment provides a more complete alternative solution for hand functions.

[0044] Example 3: Based on Embodiment 1, the connection and control methods of this system are modified. Both the control circuit module and the MCU in the rear drive module 1 integrate Bluetooth communication chips, and control commands and feedback data between them interact via Bluetooth wireless transmission. Simultaneously, the power supply battery of the rear drive module 1 can be placed in the user's pocket and connected to the rear drive module 1 via a cable. The rest is the same as in Embodiment 1. This embodiment reduces the wiring constraints between modules, improving the flexibility and convenience of wearing the device.

[0045] Example 4: Based on Embodiment 1, the transmission mechanism inside the bionic hand module 2 is modified. The mechanical finger is driven by a linkage mechanism. Correspondingly, the end of the flexible shaft 31 is connected to a belt drive mechanism or gearbox to convert the rotational motion of the motor into the motion required to drive the linkage mechanism. The rest is the same as in Embodiment 1. This embodiment provides an alternative transmission option besides cable traction, which may be suitable for scenarios requiring greater gripping force.

[0046] The above embodiments are illustrative of the present invention and are not intended to limit the present invention. Any simple modifications to the present invention are within the scope of protection of the present invention.

Claims

1. A biomimetic dexterous hand control system based on electromyographic signals and deep learning, characterized in that, include: The rear drive module, which is attached to the user's forearm or wrist via an adjustable strap, includes a motor, a motor drive circuit, a slide for fixing the motor, and a sensor for control feedback. The motor in the rear drive module is mounted via a sliding mechanism, which includes a linear slide and a slide rail mounted on the strap. The motor is fixed to the linear slide, and the base of the linear slide cooperates with the slide rail, allowing the motor to move axially along the flexible shaft and slide along the slide rail as a whole with the linear slide, thereby passively compensating for changes in the transmission path length caused by wrist flexion. A signal acquisition module is used to acquire surface electromyography (EMG) signals from the user's arm and send them to the control circuit module. The signal acquisition module includes a surface EMG signal acquisition sensor and a flexible strap for fixing the sensor. The control circuit module includes a deduction circuit for receiving surface electromyography signals and performing deep learning deduction, and a drive circuit for receiving deduction results and controlling different motors. The deduction circuit receives surface electromyography signals in real time, deduces the movement intention of each finger, and then sends the deduction results to the motor drive circuit for independent control of different motors. A bionic hand module for wearing on the end of a user's residual limb. The bionic hand module has no motor or driver inside. It includes a palm base shell and a personalized connection base that matches the shape of the user's residual limb end. The palm base shell and the personalized connection base are detachably connected. The flexible transmission module includes a flexible shaft, one end of which is connected to the output end of the motor in the rear drive module, and the other end extends into the interior of the bionic hand module and is connected to the mechanical fingers of the bionic hand module via a traction cable or connecting rod.

2. The control system according to claim 1, characterized in that, The sensors used for control feedback include a position sensor for feedback of motor rotation angle and a current detection circuit for monitoring motor operating current.

3. The control system according to claim 1, characterized in that, The control circuit module employs a deep learning model consisting of two residual modules, a long short-term memory network, and a multi-channel multilayer perceptron network. The multi-channel multilayer perceptron network has independent output channels corresponding to the number of fingers to be controlled, with each channel independently outputting the corresponding finger's motion state category, which includes outward opening, inward flexion, and relaxation. The residual modules use convolutional kernels of different sizes to process input features in parallel and then superimpose the processing results to form residual connections. The long short-term memory network processes data in a time-series manner and includes a forget gate, an input gate, and an output gate.

4. The control system according to claim 1, characterized in that, The signal acquisition module includes an electromyography (EMG) signal acquisition device, which consists of four dry electrodes respectively positioned on the elbow side, wrist side, elbow side, and wrist side of the flexor digitorum muscles of the user's arm. The acquired EMG signals are filtered by a 50Hz band-stop filter to remove power frequency interference and then processed using a sliding window with a window length of 75ms and an overlap rate of 50%. The processed data is used as the input to the deep learning model in the control circuit module. Alternatively, the EMG signal acquisition device may consist of multiple ring electrodes or multiple high-density electrode arrays.

5. The control system according to claim 1, characterized in that, The microprocessor of the rear drive module performs closed-loop control: when the movement intention is "opening outward", the motor is controlled to rotate forward until the angle fed back by the position sensor reaches the set threshold; when the movement intention is "bending inward", the motor is controlled to rotate in reverse until the current value detected by the current detection circuit reaches the set threshold; when the movement intention is "relaxing", the motor is controlled to stop rotating.

6. The control system according to claim 1, characterized in that, The control circuit module and the microprocessor of the rear drive module interact with each other via wired or wireless communication; the deduction circuit and its power supply battery in the control circuit module are set as detachable independent modules.

7. The control system according to claim 1, characterized in that, The bionic hand module is covered with bionic skin material.

Images