A myoelectric signal-based robot dexterous hand control system

Abstract

The application discloses a kind of robot dexterous hand control system based on myoelectric signal, comprising: signal acquisition module, signal processing module, grasping force estimation module and grasping force control module;Signal acquisition module is used to collect the myoelectric signal of human forearm and the real-time grasping force of robot dexterous hand;Signal processing module is used to pre-process myoelectric signal and obtain processed myoelectric signal, signal processing module is also used to pre-process real-time grasping force and obtain processed real-time grasping force;Grasping force estimation module is used to generate expected grasping force based on processed myoelectric signal and processed real-time grasping force;Grasping force control module is used to real-time control the grasping force of robot dexterous hand based on processed real-time grasping force and expected grasping force.The application adopts the multimodal teleoperation method of robot dexterous hand driven by myoelectric signal, overcomes the problem that traditional robot dexterous hand operation flexibility is poor, and the compliance is low, helps to improve robot dexterous hand grasping efficiency and quality.

Description

Technical Field

[0001] This invention belongs to the field of robot dexterous hand control technology, specifically relating to a robot dexterous hand control system based on electromyographic signals. Background Technology

[0002] With the continuous advancement of science and technology and the increasing maturity of artificial intelligence, robotics has become a focal point in modern scientific research and industry. Particularly in the area of ​​high-precision and highly flexible robotic dexterous hand grasping technology, its application prospects and research value are increasingly prominent. However, current robotic dexterous hand grasping technologies often rely on prior knowledge of object properties, which can lead to difficulties when dealing with objects with unknown or variable properties. Therefore, it is necessary to explore new methods to guide the grasping force of robotic dexterous hands by capturing and analyzing human biosignals, thereby achieving more flexible and intelligent grasping control.

[0003] Electromyography (EMG) signals are electrical signals generated by human muscle activity. They can be acquired through non-invasive wearable devices (such as EMG wristbands) to obtain real-time information about the contraction and force of human arm muscles, which can be used to predict and control the grasping force of robotic dexterity hands. However, processing and interpreting EMG signals is a challenging task because they are multivariate time-series data containing complex time dependencies and nonlinear relationships.

[0004] On the other hand, remote control refers to the technology of remotely controlling and operating robots. For environments where humans have difficulty or are unable to enter, robots can be remotely controlled to perform tasks. However, for fragile or variable objects, existing robotic dexterous hand grasping schemes pose a potential risk of damage. Position-based grasping methods are not very flexible because they can only control the position of the end effector, not its force or torque. This limits their application to tasks requiring precise force control. Furthermore, contact-based methods require prior knowledge input, making it challenging to apply these detection methods to unknown objects.

[0005] In summary, although some research has been conducted on robotic dexterous hand grasping based on electromyographic (EMG) signals, certain limitations and challenges remain, particularly in real-time prediction of grasping force and achieving flexible grasping. Therefore, developing a multimodal telemanipulation method and system for robotic dexterous hands based on EMG signals is essential. This would not only improve the grasping efficiency and accuracy of robotic dexterous hands but also provide strong support for the further development and application of robotics technology. Summary of the Invention

[0006] The present invention aims to address the shortcomings of the prior art by proposing a robot dexterous hand control system based on electromyographic signals. This system does not require any prior knowledge of the properties of objects (such as shape, weight, and surface friction), thereby enabling the robot dexterous hand to flexibly and adaptively grasp various objects.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A robot dexterous hand control system based on electromyography signals includes: a signal acquisition module, a signal processing module, a grasping force estimation module, and a grasping force control module;

[0009] The signal acquisition module is used to acquire electromyographic signals of the human forearm and the real-time grasping force of the robot's dexterous hand;

[0010] The signal processing module is used to preprocess the electromyographic signal to obtain a processed electromyographic signal. The signal processing module is also used to preprocess the real-time grasping force to obtain a processed real-time grasping force.

[0011] The grasping force estimation module is used to generate an expected grasping force based on the processed electromyographic signal and the processed real-time grasping force;

[0012] The grasping force control module is used to adjust the grasping force of the robot's dexterous hand in real time based on the processed real-time grasping force and the expected grasping force.

[0013] Preferably, the signal acquisition module includes: an electromyography signal acquisition unit and a grasping force acquisition unit;

[0014] The electromyography (EMG) signal acquisition unit is used to acquire the EMG signals of the human forearm;

[0015] The grasping force acquisition unit is installed on the robot's dexterous hand, and the grasping force acquisition unit is used to acquire the real-time grasping force.

[0016] Preferably, the signal processing module includes: an electromyography signal processing unit and a grasping force processing unit;

[0017] The electromyography signal processing unit uses fast Fourier transform and Butterworth filtering to denoise the electromyography signal to obtain the processed electromyography signal.

[0018] The grasping force processing unit uses Fast Fourier Transform and Butterworth filtering to denoise the real-time grasping force, and uses Savitzky-Golay filter to smooth the processed data and eliminate outliers to obtain the processed real-time grasping force.

[0019] Preferably, the electromyography signal processing unit includes: a first denoising subunit and a first filtering subunit;

[0020] The first denoising subunit is used to process the electromyographic signal using fast Fourier transform, capture the power spectral density of the electromyographic signal during the process, and thus perform denoising to obtain the first data;

[0021]

[0022] Where X(k) is the amplitude of the k-th frequency component, x(n) is the n-th sample in the time domain, and N is the number of samples;

[0023] The first filtering subunit uses a Butterworth filter to perform low-pass filtering on the first data to obtain the processed electromyographic signal.

[0024] Preferably, the gripping force processing unit includes: a second noise reduction subunit, a second filtering subunit, and an anomaly elimination subunit;

[0025] The second denoising subunit is used to process the real-time grasping force using a fast Fourier transform to obtain the second data:

[0026]

[0027] Where X(k) is the amplitude of the k-th frequency component, x(n) is the n-th sample in the time domain, and N is the number of samples;

[0028] The second filtering subunit uses a Butterworth filter to perform low-pass filtering on the second data to obtain the processed data;

[0029] The anomaly removal subunit uses a SaVitzky-Golay filter to smooth the processed data and remove outliers, resulting in the processed real-time grasping force.

[0030]

[0031] Among them, y i For the smoothed data points, x i+j For the original data points, c j is the polynomial coefficient, and m is the window coefficient.

[0032] Preferably, the grasping force estimation module includes: a network construction unit, a network training unit, and an estimation unit;

[0033] The network building unit is used to build a deep learning network, which includes: one input layer, two long short-term memory network layers, two regularization layers, one fully connected layer, and one output layer.

[0034] The network training unit is used to train the deep learning network to obtain the trained network;

[0035] The estimation unit is used to generate the expected grasping force using the trained network based on the processed electromyographic signal and the processed real-time grasping force.

[0036] Preferably, the training process includes:

[0037] The expected grasping force is calculated through forward propagation of the deep learning network and compared with the processed real-time grasping force to calculate the mean squared error loss and R0. 2 Coefficient of determination;

[0038] The method for calculating the mean squared error loss includes:

[0039]

[0040] Among them, y i For real-time grasping power, To predict gripping force, N is the number of samples;

[0041] The R 2 The methods for calculating the coefficient of determination include:

[0042]

[0043] Where M is the sample size. y is the average predictive power of all samples. i The actual force value of the i-th sample Let be the predictive power value of the i-th sample.

[0044] Preferably, the workflow of the grasping force control module includes: acquiring the difference between the processed real-time grasping force and the expected grasping force, and adjusting the grasping force of the robot's dexterous hand based on a proportional-integral-derivative controller strategy.

[0045] Preferably, the controller strategy includes:

[0046]

[0047] P des (n)=P des (N-1)+ΔP des (N)

[0048] Where, e = F ref -F real Force error in units of voltage, F ref For the predicted force value, F real P represents the true force value.des K is sent to the required location in the internal position loop. p K is the proportionality coefficient. i K is the integral coefficient. d is the differential coefficient.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] This invention employs multimodal teleoperation of a robot dexterous hand driven by electromyography (EMG) signals. It converts EMG signals from the human forearm into precise grasping force control of the robot dexterous hand, achieving efficient and harmonious human-machine collaborative operation. It is suitable for precise, flexible, and compliant grasping operations, overcoming the problems of poor operational flexibility and low compliance of traditional robot dexterous hands, and helping to improve the grasping efficiency and quality of robot dexterous hands. Attached Figure Description

[0051] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the system structure according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the structure of a deep learning network according to an embodiment of the present invention;

[0054] Figure 3 This is a flowchart illustrating the prediction of grasping force according to an embodiment of the present invention;

[0055] Figure 4 This is a control block diagram of the gripping force control module according to an embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of the experimental results of an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Example 1

[0060] In this embodiment, as Figure 1 As shown, a robot dexterous hand control system based on electromyography signals includes: a signal acquisition module, a signal processing module, a grasping force estimation module, and a grasping force control module.

[0061] The signal acquisition module is used to acquire electromyographic signals from the human forearm and the real-time grasping force of the robotic dexterous hand. The signal acquisition module includes an electromyographic signal acquisition unit and a grasping force acquisition unit.

[0062] An electromyography (EMG) signal acquisition unit is used to collect EMG signals from the human forearm. In this embodiment, EMG signals from the human forearm are collected invasively using an EMG wristband. This wristband has high sensitivity and accuracy, capable of capturing and transmitting minute electrical signals from the forearm muscles in real time. These signals are synchronously transmitted and received via Bluetooth communication implemented on the Ubuntu 16.04 system, ensuring efficient and accurate data transmission. It can simultaneously collect 16 channels of EMG signals, providing rich and high-quality raw EMG signal data for subsequent data processing and analysis.

[0063] The grasping force acquisition unit is installed on the robot's dexterous hand, and it is used to collect real-time grasping force. The real-time grasping force is obtained through a thin-film pressure sensor and a signal amplification device installed on the robot's dexterous hand. The specific system configuration uses an Arduino development board, which is connected to the thin-film pressure sensor and the signal amplification module to obtain the real-time grasping force of the robot's dexterous hand on the object.

[0064] The signal processing module is used to preprocess the electromyographic (EMG) signals to obtain processed EMG signals. It also preprocesses the real-time grasping force to obtain processed real-time grasping force. The signal processing module includes an EMG signal processing unit and a grasping force processing unit.

[0065] The electromyography (EMG) signal processing unit uses Fast Fourier Transform (FFT) and Butterworth filtering to denoise the EMG signal, obtaining a processed EMG signal. The EMG signal processing unit includes a first denoising subunit and a first filtering subunit. The first denoising subunit processes the EMG signal using FFT, capturing the power spectral density of the EMG signal during the process, thereby denoising and obtaining the first data.

[0066]

[0067] Where X(k) is the amplitude of the k-th frequency component, x(n) is the n-th sample in the time domain, and N is the number of samples.

[0068] The first filtering subunit uses a Butterworth filter to perform low-pass filtering on the first data to obtain the processed electromyography (EMG) signal. In the frequency domain, since most of the energy in the system is concentrated between 5Hz and 492Hz, in this embodiment, the signal after 492Hz is regarded as high-frequency low-power noise, and a second-order Butterworth filter (cutoff frequency 492Hz) is used to perform low-pass filtering on each channel of the sEMG signal to eliminate the DC component, further improving the quality and accuracy of the data, and obtaining the processed EMG signal.

[0069] The grasping force processing unit uses Fast Fourier Transform and Butterworth filtering to denoise the real-time grasping force, and then uses Savitzky-Golay filtering to smooth the denoised data and eliminate outliers, resulting in the processed real-time grasping force. The grasping force processing unit includes: a second denoising subunit, a second filtering subunit, and an outlier elimination subunit;

[0070] The second denoising subunit processes the real-time grasping force using a Fast Fourier Transform to obtain the second data:

[0071]

[0072] Where X(k) is the amplitude of the k-th frequency component, x(n) is the n-th sample in the time domain, and N is the number of samples; the second filtering subunit uses a Butterworth filter to perform low-pass filtering on the first data to obtain denoised data; the anomaly elimination subunit uses a Savitzky-Golay filter to smooth the denoised data and eliminate outliers to obtain the processed real-time grasping force.

[0073]

[0074] Among them, y i For the smoothed data points, x i+j For the original data points, c j denoted as polynomial coefficients, and m as window coefficients.

[0075] The grasping force estimation module is used to generate the expected grasping force based on the processed electromyographic signal and the processed real-time grasping force. The grasping force estimation module includes: a network construction unit, a network training unit, and an estimation unit.

[0076] Network building blocks are used to build deep learning networks, such as... Figure 2 As shown, it includes: 1 input layer, 2 long short-term memory network layers, 2 regularization layers, 1 fully connected layer and 1 output layer; the network training unit is used to train the deep learning network to obtain the trained network; the estimation unit is used to generate the expected grasping force based on the processed electromyographic signal and the processed real-time grasping force using the trained network.

[0077] In this embodiment, the neural network structure takes electromyography (EMG) signal data and real-time grasping force data generated by the robot's dexterous hand as input, and then flows through two consecutive long short-term memory (LSTM) network layers. This network layer structure can capture long-term dependencies in time-series data, which is particularly important for processing continuous time-series data such as EMG signals. After each LSM network layer, a regularization layer that randomly discards some neurons is designed to prevent the model from overfitting. Finally, the predicted grasping force value is output through a fully connected layer.

[0078] The training process includes: calculating the expected grasping force through forward propagation of the deep learning network, comparing it with the processed real-time grasping force, and calculating the mean squared error loss and R0. 2 The coefficient of determination; methods for calculating the mean squared error loss include:

[0079]

[0080] Among them, y i For real-time grasping power, To predict gripping force, N is the number of samples; R 2 The methods for calculating the coefficient of determination include:

[0081]

[0082] Where M is the sample size. y is the average predictive power of all samples. i The actual force value of the i-th sample Let be the predictive power value of the i-th sample.

[0083] Then, using the backpropagation algorithm, the weights and biases in the network are adjusted based on the loss value to minimize the prediction error. After multiple iterations of training, when the mean squared error loss value and R... 2 The training of the neural network ends when the values ​​of the determination coefficients all converge to a small, stable value.

[0084] In this embodiment, the process of predicting the gripping force is as follows: Figure 3As shown, the system first initializes a 16-channel electromyography (EMG) signal list, a predicted force list, and an empty prediction list, the size of which is determined by the time step and the number of features. When ROS (Robot Operating System) communication is not closed, the system checks if the EMG signal list is full. If not, new EMG signal data is added to the head of the list. If full, preprocessing of the data in the list begins, and the processed EMG signals are added to the predicted EMG signal list. The system then checks if the predicted EMG signal list is full. If full, the system uses the trained neural network model to predict the grasping force and adds the predicted force to the head of the predicted force list. If the predicted EMG signal list is not full, the system continues execution until the list is full. This algorithm can predict grip force references in real time, providing an efficient and accurate force control scheme for robots and other automated systems, contributing to more precise and flexible operation.

[0085] The grasping force control module is used to adjust the grasping force of the robot's dexterous hand in real time based on the processed real-time grasping force and the expected grasping force.

[0086] The workflow of the gripping force control module includes: Figure 4 As shown, the difference between the real-time grasping force after processing and the expected grasping force is obtained, and the grasping force of the robot's dexterous hand is adjusted based on the proportional-integral-derivative controller strategy.

[0087] Controller policies include:

[0088]

[0089] P des (n)=P des (N-1)+ΔP des (N)

[0090] Where, e = F ref -F real Force error in units of voltage, F ref For the predicted force value, F real P represents the true force value. des K is sent to the required location in the internal position loop. p K is the proportionality coefficient. i K is the integral coefficient. d is the differential coefficient.

[0091] In this embodiment, a visual feedback module can also be configured, which can use a camera device to provide real-time position feedback and precisely control the robot's dexterous hand to perform grasping operations.

[0092] Example 2

[0093] In this embodiment, the effectiveness and feasibility of the present invention will be verified through experiments.

[0094] By using an electromyography (EMG) wristband and an Arduino development board, we successfully collected EMG signals from the human forearm and real-time grasping force data from a robotic dexterity hand. This data, after preprocessing, was used to train a deep learning neural network model. Model training and validation were performed offline, while the predicted grasping force reference signal was generated in real-time.

[0095] Experimental results are as follows Figure 5 The results show that the neural network model achieves R-values ​​on the test set. 2 The coefficient of determination and mean squared error loss were 0.9638 and 0.1813, respectively, indicating that the model has high predictive accuracy and generalization ability. Furthermore, by using a grasping force control system based on PID and admittance control algorithms, the robot dexterous hand can precisely adjust its position and grasping force according to the difference between the predicted reference force signal and the actual measured force signal. This series of experiments verifies the effectiveness and feasibility of the invention in practical applications, demonstrating that the system can achieve flexible and compliant grasping of various objects by the robot dexterous hand.

[0096] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A robot dexterous hand control system based on electromyographic signals, characterized in that, include: Signal acquisition module, signal processing module, grasping force estimation module, and grasping force control module; The signal acquisition module is used to acquire electromyographic signals of the human forearm and the real-time grasping force of the robot's dexterous hand; The signal processing module is used to preprocess the electromyographic signal to obtain a processed electromyographic signal. The signal processing module is also used to preprocess the real-time grasping force to obtain a processed real-time grasping force. The grasping force estimation module is used to generate an expected grasping force based on the processed electromyographic signal and the processed real-time grasping force; The grasping force control module is used to adjust the grasping force of the robot's dexterous hand in real time based on the processed real-time grasping force and the expected grasping force. The workflow of the grasping force control module includes: acquiring the difference between the processed real-time grasping force and the expected grasping force, and adjusting the grasping force of the robot's dexterous hand based on a proportional-integral-derivative controller strategy; The controller strategy includes: in, The force error is expressed in units of voltage. For the expected grasping power, For real-time grasping power, To send to the required location in the internal location loop, This is the proportionality coefficient. The integral coefficient is... is the differential coefficient.

2. The robot dexterous hand control system based on electromyography signals according to claim 1, characterized in that, The signal acquisition module includes: an electromyography signal acquisition unit and a grasping force acquisition unit; The electromyography (EMG) signal acquisition unit is used to acquire the EMG signals of the human forearm; The grasping force acquisition unit is installed on the robot's dexterous hand, and the grasping force acquisition unit is used to acquire the real-time grasping force.

3. The robot dexterous hand control system based on electromyography signals according to claim 1, characterized in that, The signal processing module includes: an electromyography signal processing unit and a grasping force processing unit; The electromyography signal processing unit uses fast Fourier transform and Butterworth filtering to denoise the electromyography signal to obtain the processed electromyography signal. The grasping force processing unit uses Fast Fourier Transform and Butterworth filtering to denoise the real-time grasping force, and uses Savitzky-Golay filter to smooth the processed data and eliminate outliers to obtain the processed real-time grasping force.

4. The robot dexterous hand control system based on electromyography signals according to claim 3, characterized in that, The electromyography signal processing unit includes: a first noise reduction subunit and a first filtering subunit; The first denoising subunit is used to process the electromyographic signal using fast Fourier transform, capture the power spectral density of the electromyographic signal during the process, and thus perform denoising to obtain the first data; in, For the first k The amplitude of each frequency component, For the first in the time domain n One sample, N The number of samples; The first filtering subunit uses a Butterworth filter to perform low-pass filtering on the first data to obtain the processed electromyographic signal.

5. The robot dexterous hand control system based on electromyography signals according to claim 3, characterized in that, The gripping force processing unit includes: a second noise reduction subunit, a second filtering subunit, and an anomaly elimination subunit; The second denoising subunit is used to process the real-time grasping force using a fast Fourier transform to obtain the second data: in, For the first k The amplitude of each frequency component, For the first in the time domain n One sample, N The number of samples; The second filtering subunit uses a Butterworth filter to perform low-pass filtering on the second data to obtain the processed data; The anomaly removal subunit uses a Savitzky-Golay filter to smooth the processed data and remove outliers, resulting in the processed real-time grasping force. in, For the smoothed data points, For the original data points, For polynomial coefficients, m This is the window coefficient.

6. The robot dexterous hand control system based on electromyography signals according to claim 1, characterized in that, The grasping force estimation module includes: a network construction unit, a network training unit, and an estimation unit; The network building unit is used to build a deep learning network, which includes: one input layer, two long short-term memory network layers, two regularization layers, one fully connected layer, and one output layer. The network training unit is used to train the deep learning network to obtain the trained network; The estimation unit is used to generate the expected grasping force using the trained network based on the processed electromyographic signal and the processed real-time grasping force.

7. The robot dexterous hand control system based on electromyographic signals according to claim 6, characterized in that, The training process includes: The expected grasping force is calculated through forward propagation of the deep learning network and compared with the processed real-time grasping force to calculate the mean squared error loss. Coefficient of determination; The method for calculating the mean squared error loss includes: in, For real-time grasping power, For the expected grasping power, N It is the sample size; The The methods for calculating the coefficient of determination include: Where M is the sample size. This represents the average expected grasping force across all samples. No. i Real-time capture capability of individual samples For the first i Expected grasping power of each sample.

Images