A method, device, and medium for estimating the stiffness of the distal end of the arm based on electromyography signals.

Abstract

This invention relates to a method, device, and medium for estimating distal arm stiffness based on electromyography (EMG) signals. The method includes the following steps: assembling a six-dimensional force sensor at the distal end of a robotic arm; setting the control mode and stiffness coefficient of the robotic arm; randomly positioning the distal end of the robotic arm within a certain range from a predetermined origin; having an operator wear an EMG sensor on their arm and drag the distal end of the robotic arm to the origin position, while real-time acquisition and recording of the distal end position information, six-dimensional force information, and EMG signals; calculating the true value of the distal arm stiffness based on the distal end position information and six-dimensional force information; training an distal arm stiffness mapping network based on the EMG signals and the true value of the distal arm stiffness; acquiring the subject's EMG signals and inputting them into the trained distal arm stiffness mapping network to estimate the subject's distal arm stiffness. Compared with existing technologies, this invention reduces the zero bias of the acquired data, resulting in higher accuracy in distal arm stiffness estimation.

Description

Technical Field

[0001] This invention relates to the field of variable stiffness control, and in particular to a method for estimating the stiffness of the distal end of the arm based on electromyographic signals. Background Technology

[0002] With the rapid development of modern industrial automation and intelligent manufacturing, human-computer interaction (HCI) scenarios are increasingly being applied in real life, and their development has attracted widespread attention. In HCI tasks conducted through impedance control, how to enable the robotic arm to understand human intentions and achieve compliant HCI interaction is a current research focus.

[0003] Among existing methods for understanding human intent, estimating the stiffness of the distal arm based on electromyography (sEMG) signals is a common approach. However, traditional methods for obtaining the true value of distal arm stiffness mostly employ perturbation techniques, which often result in zero bias due to the instability of the human arm, leading to inaccurate true stiffness values.

[0004] The arm stiffness estimation scheme based on sEMG driven by the Chinese patent application CN202410136155.1 requires first estimating the arm force through an LSTM network, and then converting the force into end-effector stiffness through a model. The performance of this method not only depends on the accuracy of the LSTM network in estimating muscle force, but also on the physical consistency and modeling accuracy of the selected force-stiffness model, making it difficult to guarantee accuracy. Summary of the Invention

[0005] The purpose of this invention is to overcome the zero-bias defect in obtaining the true value of the arm end stiffness by perturbation and to provide a method, device and medium for estimating the arm end stiffness based on electromyographic signals.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] As a first aspect of the present invention, a method for estimating the stiffness of the distal end of the arm based on electromyographic signals is provided, the steps of which include:

[0008] A six-dimensional force sensor is installed at the end of the robotic arm to set the control mode and stiffness coefficient of the robotic arm;

[0009] The end effector of the robotic arm is randomly positioned within a certain range from the set origin. The operator wears an electromyography (EMG) sensor on their arm and drags the end effector of the robotic arm to the origin position. The position information, six-dimensional force information and EMG signals of the end effector of the robotic arm are collected and recorded in real time.

[0010] Based on the collected end-effector position information and six-dimensional force information data, the true value of the end-effector stiffness is calculated.

[0011] Based on the collected electromyographic signals and the calculated true values ​​of the end-effector stiffness, an end-effector stiffness mapping network is trained, which takes the electromyographic signals as input and the end-effector stiffness values ​​as output.

[0012] Electromyographic signals from the subjects were collected and input into a trained arm end-effector stiffness mapping network to estimate the arm end-effector stiffness.

[0013] As a preferred technical solution, the end of the robotic arm is equipped with a drag handle.

[0014] As a preferred technical solution, the robotic arm is configured in admittance control mode, and the admittance control equation of the robotic arm is as follows:

[0015]

[0016] Among them, F ext This indicates the actual force applied at the end effector of the robotic arm; F d X represents the desired contact force; c , These represent the actual position, velocity, and acceleration of the robotic arm's end effector, respectively; X d , M represents the desired position, velocity, and acceleration of the end effector, respectively; d d d and K d These represent the required virtual mass matrix, damping matrix, and stiffness matrix, respectively.

[0017] As a preferred technical solution, the method also constructs a real-time display interface for the three-dimensional coordinates of the robotic arm, sets the origin position of the robotic arm end point, calculates and displays the three-dimensional coordinate difference between the robotic arm end point and the origin in real time based on the movement of the robotic arm, and visualizes it on the terminal as visual feedback on the position deviation of the robotic arm end point.

[0018] As a preferred technical solution, the actual value of the end-effector stiffness is calculated based on the collected end-effector position information and six-dimensional force information data, as follows:

[0019] For the starting position X of the current round of data r The operator then drags the robotic arm back to the origin point X. org Under steady-state conditions, the external force F acting on the extremities of the human hand hext Based on the collected end-effector position information and six-dimensional force information data, the true value of the end-effector stiffness F is calculated. ext Conversely; the actual speed of the robotic arm's end effector Since the actual velocity of the human hand's end effector is 0, based on the impedance model of the robotic arm and the arm itself, the true value of the end effector stiffness is:

[0020]

[0021] Among them, X he X ce These represent the positions of the arm's end effector and the robotic arm's end effector in steady state, respectively; X c The initial position of the robotic arm's end effector; X hv K represents the virtual origin position of the human arm. d This is the stiffness matrix of the robotic arm's end effector.

[0022] As a preferred technical solution, the method abstracts the human arm as a spring-damped system and simplifies the acceleration and feedforward force terms. The external force F acting on the end of the human hand... hext Represented as:

[0023]

[0024] Among them, X hv X is the virtual origin position of the human arm. h and These represent the actual position and velocity of the human hand's extremities, F. hext D represents the external force acting on the extremities of the human hand. h and K h These represent the virtual damping and virtual stiffness of the arm, respectively.

[0025] As a preferred technical solution, the collected electromyographic signals are preprocessed and then input into the end-to-end arm stiffness estimation model. The preprocessing steps include: flipping the electromyographic signals, then performing bandpass filtering on the electromyographic data, then extracting the moving average envelope, and finally windowing.

[0026] As a preferred technical solution, the end-to-end arm end-effector stiffness estimation model based on the fusion of long short-term memory is as follows:

[0027] The preprocessed electromyography (EMG) signal is fed into the arm stiffness estimation network and then into an LSTM layer for learning. The LSTM layer consists of multiple LSTM blocks that process time-step data sequentially. The first LSTM block receives the initial state and calculates the next state based on the data from the first time step. Each LSTM block processes the data from subsequent time steps and transmits updated hidden states and unit states. After each LSTM block, a random deactivation layer is passed through, which causes each neuron to stop working with a certain probability during the forward propagation of the network. As the time steps progress, the LSTM layer extracts the time-frequency domain features of the output EMG signal.

[0028] The output of the LSTM layer is dimensionality reduced by compressing the multidimensional tensor into a one-dimensional tensor, which is then input into the fully connected layer for further feature learning. An activation function is used in the output layer to restrict the network output to between 0 and 1, and then multiplied by the maximum stiffness of the arm to obtain the stiffness value of the arm end.

[0029] As a second aspect of the present invention, an electronic device is provided, comprising:

[0030] One or more processors;

[0031] Memory, used to store one or more programs;

[0032] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for estimating the stiffness of the distal end of the arm based on electromyography signals as described above.

[0033] As a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the arm end-effector stiffness estimation method based on electromyographic signals as described above.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The method for estimating the stiffness of the distal arm based on electromyography signals proposed in this invention reduces the zero bias of the acquired data, resulting in higher accuracy in estimating the stiffness of the distal arm. At the same time, the stiffness estimation is performed end-to-end through a network, eliminating the need to introduce other models and reducing the complexity of the system. Attached Figure Description

[0036] Figure 1 This is a flowchart of the arm end-effector stiffness estimation method based on electromyography signals of the present invention;

[0037] Figure 2 This is a schematic diagram of the process for acquiring arm end-effector stiffness data using the arm end-effector stiffness estimation method of the present invention;

[0038] Figure 3 This is a schematic diagram of the end-to-end arm stiffness estimation model based on the fusion of long short-term memory according to the present invention. Detailed Implementation

[0039] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0040] Example 1

[0041] like Figure 1 As shown, this embodiment provides a method for estimating the stiffness of the distal end of the arm based on electromyography signals, including the following steps:

[0042] Step 1) Install a six-dimensional force sensor and a drag handle at the end of the robotic arm, set the robotic arm to admittance control mode, and set an appropriate stiffness coefficient;

[0043] Step 1) includes the following steps:

[0044] Step 1-1) Construct a drag handle at the end of the robotic arm using 3D printing, and assemble a six-dimensional force sensor and drag handle at the end of the robotic arm.

[0045] Steps 1-2) Construct the admittance control equations for the robotic arm:

[0046]

[0047] Among them, F ext F represents the actual applied force. d X represents the desired contact force. c , These represent the actual position, velocity, and acceleration of the robotic arm's end effector, respectively. X d , These represent the desired position, velocity, and acceleration of the end effector, respectively. M d D d and K d These represent the required virtual mass matrix, damping matrix, and stiffness matrix, respectively.

[0048] Step 2) Construct a real-time display interface for the three-dimensional coordinates of the robotic arm, set the origin position of the robotic arm end point, and display the three-dimensional coordinate difference between the current position of the robotic arm end point and the origin position in real time on the host computer according to the movement of the robotic arm.

[0049] Step 2-1) Select the origin position of the robotic arm's end effector:

[0050] X org =[x o ,y o ,z o ] T

[0051] Where, x o ,y o ,z o These are the three-dimensional coordinates of the robotic arm's end effector. The current position of the robotic arm's end effector is set as follows:

[0052] X = [x, y, z] T

[0053] Where x, y, z are the current three-dimensional coordinates of the robotic arm's end effector.

[0054] Step 2-2) Calculate the deviation between the current position of the robotic arm's end effector and the origin position:

[0055] X e =[xx o yy o ,zz o ] T

[0056] Real-time calculation of X e Furthermore, the location deviation is visualized on the terminal as visual feedback.

[0057] Step 3) Wear an electromyography (EMG) sensor on the arm. The robotic arm is randomly placed within a certain range of the set origin. Based on visual feedback, the robotic arm is dragged back to the origin position. The position information of the robotic arm end, six-dimensional force information and EMG signals are recorded in real time.

[0058] Step 3-1) Allow the robotic arm to randomly select an initial position, i.e.:

[0059] X r =[x r ,y r ,z r ] T

[0060] Where, x r ,y r ,z r These are the three-dimensional coordinates of the robotic arm's random initial position. Here, we choose the random initial coordinates at the X... org Within ±20cm, that is:

[0061]

[0062] Step 3-2) Wear an electromyography (EMG) sensor on the arm. Based on the visual feedback, drag the end of the robotic arm to the vicinity of the origin and begin collecting six-dimensional force and EMG signal data.

[0063] Step 4) Calculate the true value of stiffness using the collected data, preprocess the electromyographic signals, and train the arm end-stiffness network.

[0064] Step 4-1) First, abstract the human arm as a spring-damped system, and simplify the acceleration and feedforward force terms, i.e.:

[0065]

[0066] Among them, X hv X is the virtual origin position of the human arm. h and These represent the actual position and velocity of the human hand's extremities, F. hext D represents the external force acting on the extremities of the human hand. h and K h These represent the virtual damping and virtual stiffness of the arm, respectively.

[0067] Step 4-2) Starting position X of the current round data r The robotic arm under test was dragged back to the origin X. org Under steady-state conditions, then F hext =-F ext F ext This represents the force acting on the end effector of the robotic arm. Based on the impedance model of the robotic arm and the robotic arm, we can obtain:

[0068] K h (X he -X hv ) = K d (X ce -X r )

[0069] Among them, X he X ce These represent the positions of the arm's end effector and the robotic arm's end effector in steady state, respectively, i.e., X. he =X ce =X ora Based on this formula, the true value of the arm stiffness can be obtained as follows:

[0070]

[0071] The corresponding stiffness value is then recorded as the subject's maximum arm stiffness, denoted as K. PM Then K can be h Represented as:

[0072]

[0073] Among them, K d =δK PM δ is a coefficient and 0 < δ ≤ 1. Δx, Δy, and Δz represent the magnitudes of the random offsets of each axis of the robotic arm, respectively.

[0074] Step 4-3) Preprocess the acquired electromyography (EMG) signals. First, the signals are flipped, then a 20-50Hz bandpass filter is applied to the EMG signal, followed by moving average envelope extraction, and finally, a window is added. The window length is 100ms, and the window overlap rate is 10ms.

[0075] Step 4-4) Due to the strong nonlinearity between end-effector stiffness and muscle activation, a deep neural network is used to establish the mapping relationship between these variables. Specifically, an end-effector stiffness prediction network based on long short-term memory is used, such as... Figure 2 As shown, the network was trained to construct a mapping model from electromyographic signals to the stiffness of the end of the arm.

[0076] Stiffness trajectories based on fully connected neural networks often require time-frequency domain feature extraction from electromyography (EMG) signals. However, this feature extraction process loses a significant amount of information from the EMG signals, especially fine-grained dynamic features and frequency domain characteristics, which may affect the comprehensive capture of nonlinear relationships. To overcome this problem, an end-to-end arm distal stiffness estimation model based on the fusion of long short-term memory is further proposed.

[0077] Terminal stiffness prediction networks based on long short-term memory, such as Figure 3 As shown, electromyography (EMG) signals from the arm surface are first acquired and preprocessed using bandpass filtering, moving average envelope, and windowing. The preprocessed signals are then fed into the arm stiffness estimation network. The network first passes through LSTM layers for learning. The first LSTM block receives the initial states h0 and c0 and calculates the outputs h1 and c1 based on the data from the first time step. As time steps progress, the LSTM blocks progressively process the data from subsequent time steps and transmit updated hidden states and unit states. Each layer of the LSTM generates features at different time steps. These features are further abstracted by the network, revealing the pattern of sequence features changing over time.

[0078] After each LSTM layer, a Dropout layer follows. When the neural network is large with many parameters but insufficient training samples, it is prone to overfitting. This problem can be mitigated by adding a Dropout layer after each network layer. The Dropout layer allows each neuron to stop working with a certain probability during forward propagation, thus avoiding over-reliance on certain features and reducing the risk of overfitting.

[0079] Next, the LSTM output is dimensionality-reduced, compressing the multidimensional tensor into a one-dimensional tensor for input into the fully connected layer to further learn the features. The sigmoid function is used at the output layer to constrain the network's output between 0 and 1. Finally, the output is multiplied by the maximum arm stiffness K. PM The stiffness value of the arm's end is obtained.

[0080] Example 2

[0081] As a second aspect of the present invention, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method for estimating the stiffness of the distal end of the arm based on electromyography signals. In addition to the processors, memory, and interfaces described above, any data processing device in the embodiments may also include other hardware depending on the actual function of the data processing device, which will not be elaborated further.

[0082] Example 3

[0083] As a third aspect of the present invention, this application also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the above-described method for estimating the stiffness of the distal end of the arm based on electromyography signals. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0084] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for estimating the stiffness of the distal end of the arm based on electromyographic signals, characterized by the following steps: include: A six-dimensional force sensor is installed at the end of the robotic arm to set the control mode and stiffness coefficient of the robotic arm; The end effector of the robotic arm is randomly positioned within a certain range from the set origin. The operator wears an electromyography (EMG) sensor on their arm and drags the end effector back to the origin position. The position information, six-dimensional force information, and EMG signals of the end effector are collected and recorded. Based on the collected end-effector position information and six-dimensional force information data, the true value of the end-effector stiffness is calculated as follows: The starting position of the current round of data The operator then drags the robotic arm back to its original position. ; Under steady-state conditions, the external force acting on the extremities of the human hand Based on the collected end-effector position information and six-dimensional force information data, the true value of the end-effector stiffness is calculated. Conversely; the actual speed of the robotic arm's end effector Since the actual velocity of the human hand's end effector is 0, based on the impedance model of the robotic arm and the arm itself, the true value of the end effector stiffness is: in, , These represent the positions of the arm's end effector and the robotic arm's end effector in steady state, respectively. This is the initial position of the robotic arm's end effector; This represents the virtual origin point of the human arm. The stiffness matrix of the end effector of the robotic arm; Based on the acquired electromyography (EMG) signals and the calculated true values ​​of distal arm stiffness, an end-to-end distal arm stiffness estimation model based on the fusion of long short-term memory is trained, taking EMG signals as input and distal arm stiffness as output. The specific details of the end-to-end distal arm stiffness estimation model based on the fusion of long short-term memory are as follows: The preprocessed electromyography (EMG) signal is fed into the arm stiffness estimation network and then into an LSTM layer for learning. The LSTM layer consists of multiple LSTM blocks that process time-step data sequentially. The first LSTM block receives the initial state and calculates the next state based on the data from the first time step. Each LSTM block processes the data from subsequent time steps and transmits updated hidden states and unit states. After each LSTM block, a random deactivation layer is passed through, which causes each neuron to stop working with a certain probability during the forward propagation of the network. As the time steps progress, the LSTM layer extracts the time-frequency domain features of the output EMG signal. The output of the LSTM layer is dimensionality reduced by compressing the multidimensional tensor into a one-dimensional tensor, and then input into the fully connected layer to further learn the features; an activation function is used in the output layer to restrict the network output to between 0 and 1, and then multiplied by the maximum stiffness of the arm to obtain the stiffness value of the arm end. Electromyography (EMG) signals were collected from the subjects and input into the end-to-end arm end-stiffness estimation model that had completed training, and the end-stiffness of the subjects' arms was estimated.

2. The method for estimating the stiffness of the distal end of the arm based on electromyography signals according to claim 1, characterized in that, The robotic arm is equipped with a drag handle at its end.

3. The method for estimating the stiffness of the distal end of the arm based on electromyography signals according to claim 1, characterized in that, The robotic arm is configured in admittance control mode, and the admittance control equation of the robotic arm is as follows: in, This indicates the actual force applied at the end of the robotic arm; Indicates the desired contact force; , , These represent the actual position, velocity, and acceleration of the robotic arm's end effector, respectively. , , These represent the desired position, velocity, and acceleration of the end effector, respectively. , and These represent the required virtual mass matrix, damping matrix, and stiffness matrix, respectively.

4. The method for estimating the stiffness of the distal end of the arm based on electromyography signals according to claim 1, characterized in that, The method also constructs a real-time display interface for the three-dimensional coordinates of the robotic arm, sets the origin position of the robotic arm end point, calculates and displays the three-dimensional coordinate difference between the robotic arm end point and the origin in real time based on the movement of the robotic arm, and visualizes it on the terminal as visual feedback on the position deviation of the robotic arm end point.

5. The method for estimating the stiffness of the distal end of the arm based on electromyography signals according to claim 1, characterized in that, The method abstracts the human arm as a spring-damped system and simplifies the acceleration and feedforward force terms, representing the external forces acting on the end of the human hand. Represented as: in, The virtual origin position of the human arm is set. and These represent the actual position and velocity of the human hand's extremities. The external force experienced by the extremities of the human hand. and These represent the virtual damping and virtual stiffness of the arm, respectively.

6. The method for estimating the stiffness of the distal end of the arm based on electromyography signals according to claim 1, characterized in that, The collected electromyographic signals are preprocessed and then input into the end-to-end arm stiffness estimation model. The preprocessing steps include: flipping the electromyographic signals, then performing bandpass filtering on the electromyographic data, then extracting the moving average envelope, and finally windowing.

7. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for estimating the stiffness of the distal end of the arm based on electromyography signals as described in any one of claims 1-6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for estimating the stiffness of the distal end of the arm based on electromyography signals as described in any one of claims 1-6.

Images