Arm tail end rigidity estimation method and device based on electromyographic signals and medium

Abstract

The invention relates to an arm tail end rigidity estimation method and device based on electromyographic signals and a medium. The method comprises the steps that a six-dimensional force sensor is assembled at the tail end of a mechanical arm, and the control mode and the rigidity coefficient of the mechanical arm are set; the tail end of the mechanical arm is randomly arranged within a certain range of a set original point, an operator wears an electromyographic signal sensor on the arm, the tail end of the mechanical arm is dragged to the original point position, and position information, six-dimensional force information and electromyographic signals of the tail end of the mechanical arm are collected and recorded in real time; calculating an arm tail end rigidity true value based on the mechanical arm tail end position information and the six-dimensional force information data; training an arm tail end rigidity mapping network based on the electromyographic signals and the arm tail end rigidity true value; electromyographic signals of the subject are collected and input into the trained arm tail end rigidity mapping network, and the arm tail end rigidity of the subject is obtained through estimation. Compared with the prior art, zero offset of collected data is weakened, and the accuracy of arm tail end rigidity estimation is higher.

Description

Technical Field

[0001] The invention relates to the field of variable stiffness control, and in particular to a method for estimating arm end stiffness based on electromyographic signals. Background Art

[0002] With the rapid development of modern industrial automation and intelligent manufacturing, human-computer interaction scenarios have been increasingly used in real life, and their development has also received widespread attention. In human-computer interaction tasks performed through impedance control, how to enable the robot arm to understand human intentions and achieve compliant human-computer interaction is currently being studied.

[0003] Among the existing methods of understanding human intention, estimating the stiffness of the end of the arm based on electromyographic signals (sEMG) is a common method. However, the traditional method of obtaining the true value of the stiffness of the end of the arm mostly adopts the perturbation method, but this method usually causes zero bias due to the instability of the human arm, so the obtained true value of the stiffness is inaccurate.

[0004] The arm stiffness estimation scheme based on sEMG drive adopted in the scheme disclosed in Chinese patent application CN202410136155.1 needs to estimate the arm force through the LSTM network first, and then convert the force into the end stiffness through the model. The performance of this method not only depends on the accuracy of the LSTM network's muscle force estimation, but also highly depends on the physical consistency and modeling accuracy of the selected force-stiffness model. The accuracy is difficult to guarantee. Summary of the invention

[0005] The purpose of the present invention is to overcome the zero bias defect of obtaining the true value of the arm end stiffness by perturbation and to provide an arm end stiffness estimation method, device and medium based on electromyographic signals.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] As a first aspect of the present invention, a method for estimating arm end stiffness based on electromyographic signals is provided, the steps comprising:

[0008] Install a six-dimensional force sensor at the end of the robotic arm and set the control mode and stiffness coefficient of the robotic arm;

[0009] The end of the robot arm is randomly set within a certain range of the set origin. The operator wears an electromyographic signal sensor on his arm and drags the end of the robot arm to the origin position. The position information of the end of the robot arm, six-dimensional force information and electromyographic signal are collected and recorded in real time.

[0010] Based on the collected position information and six-dimensional force information of the end of the robot arm, the true value of the stiffness of the end of the arm is calculated;

[0011] Based on the collected electromyographic signal and the calculated true value of the arm end stiffness, a arm end stiffness mapping network is trained, which takes the electromyographic signal as input and the arm end stiffness value as output;

[0012] The electromyographic signals of the subjects are collected and input into the trained arm end stiffness mapping network to estimate the arm end stiffness of the subjects.

[0013] As a preferred technical solution, a drag handle is provided at the end of the mechanical arm.

[0014] As a preferred technical solution, the robotic arm is set to an admittance control mode, and the admittance control equation of the robotic arm is as follows:

[0015]

[0016] Among them, F ext Indicates the actual force applied by the end of the robotic arm; F d represents the expected contact force; X c , Respectively represent the actual position, velocity and acceleration of the end effector of the robot arm; X d , They represent the desired position, velocity and acceleration of the end effector respectively; M d d d and K d represent the required virtual mass matrix, damping matrix and stiffness matrix respectively.

[0017] As a preferred technical solution, the method also constructs a real-time display interface for the three-dimensional coordinates of the robotic arm, sets the origin position of the robotic arm end, calculates and displays the three-dimensional coordinate difference between the robotic arm end and the origin in real time based on the movement of the robotic arm, and displays it visually on the terminal as visual feedback of the position deviation of the robotic arm end.

[0018] As a preferred technical solution, the real value of the stiffness of the end of the arm is calculated based on the collected position information of the end of the manipulator and the six-dimensional force information data, as follows:

[0019] For the starting position X of the current round of data r , the operator drags the test robot back to the origin X org ; Under steady-state conditions, the external force F on the end of the human hand is hext Based on the collected end position information and six-dimensional force information data of the robotic arm, the true value of the end stiffness F of the arm is calculated. ext On the contrary; the actual speed of the robot end effector The actual speed of the end of the human hand is 0. According to the impedance model of the robot and the arm, the actual value of the stiffness of the end of the arm is:

[0020]

[0021] Among them, X he , X ce are the positions of the end of the arm and the end of the robot in steady state; X c is the initial position of the end effector of the robot arm; X hv is the virtual origin position of the human arm; K d is the stiffness matrix of the end effector of the robot arm.

[0022] As a preferred technical solution, the method abstracts the human arm into a spring damping system and simplifies the acceleration term and the feedforward force term. The external force F hext It is expressed as:

[0023]

[0024] Among them, X hv is the virtual origin position of the human arm, X h and are the actual position and speed of the human hand end, F hext is the external force on the end of the human hand, D h and K h are the virtual damping and virtual stiffness of the arm respectively.

[0025] As an optimal technical solution, the collected electromyographic signal is preprocessed and then input into the end-to-end arm end stiffness estimation model. The preprocessing step includes: flipping the electromyographic signal, then bandpass filtering the electromyogram, then performing moving average envelope extraction, and finally windowing.

[0026] As a preferred technical solution, the end-to-end arm end stiffness estimation model based on fusion of long short-term memory is specifically as follows:

[0027] The preprocessed electromyographic signal is passed into the arm stiffness estimation network and input into the LSTM layer for learning. The LSTM layer includes multiple LSTM blocks that process time step data in sequence. The first LSTM block receives the initial state and calculates and outputs the next state in combination with the data of the first time step. Each LSTM block gradually processes the data of subsequent time steps and transmits the updated hidden state and unit state. After each LSTM block, it will pass through a random deactivation layer. When the network is forward propagating, each neuron will stop working with a certain probability. As the time step progresses, the LSTM layer extracts the time-frequency domain features of the output electromyographic signal.

[0028] The output of the LSTM layer is processed by dimensionality reduction to compress the multi-dimensional tensor into a one-dimensional tensor, and then input into the fully connected layer for further feature learning; an activation function is used in the output layer to limit the output of the network between 0 and 1 and then multiply it by the maximum stiffness of the arm to obtain the stiffness value of the end of the arm.

[0029] As a second aspect of the present invention, there is provided an electronic device, comprising:

[0030] one or more processors;

[0031] A memory for storing one or more programs;

[0032] When the one or more programs are executed by the one or more processors, the one or more processors implement the arm end stiffness estimation method based on electromyography signals as described above.

[0033] As a third aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and wherein when the computer program is executed by a processor, the steps of the arm end stiffness estimation method based on electromyography signals as described above are implemented.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The arm end stiffness estimation method based on electromyographic signals proposed in the present invention weakens the zero bias of the collected data, and the accuracy of the arm end stiffness estimation is higher. At the same time, the stiffness estimation is performed end-to-end through the network without the introduction of other models, thereby reducing the complexity of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The flowchart of the method for estimating arm end stiffness based on electromyographic signals of the present invention is as follows;

[0037] Figure 2 The figure is a schematic diagram of the process of collecting arm end stiffness data using the arm end stiffness estimation method of the present invention;

[0038] Figure 3 It is a schematic diagram of the end-to-end arm end stiffness estimation model based on the fusion of long short-term memory using the present invention. DETAILED DESCRIPTION

[0039] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0040] Example 1

[0041] like Figure 1 As shown, this embodiment provides a method for estimating arm end stiffness based on electromyographic signals, comprising the following steps:

[0042] Step 1) Install a six-dimensional force sensor and a drag handle at the end of the robotic arm, set the robotic arm to the admittance control mode, and set a suitable stiffness coefficient;

[0043] Step 1) comprises the following steps:

[0044] Step 1-1) Build a drag handle at the end of the robotic arm through 3D printing, and assemble a six-dimensional force sensor and a drag handle at the end of the robotic arm.

[0045] Step 1-2) Construct the admittance control equation of the robot:

[0046]

[0047] Among them, F ext Indicates the actual applied force, F d represents the expected contact force, X c , Respectively represent the actual position, velocity and acceleration of the end effector of the robot arm. d , M represent the desired position, velocity and acceleration of the end effector respectively. d , D d and K d represent the required virtual mass matrix, damping matrix and stiffness matrix respectively.

[0048] Step 2) construct a real-time display interface for the three-dimensional coordinates of the robotic arm, set the origin position of the end of the robotic arm, and display the three-dimensional coordinate difference between the current position of the end of the robotic arm and the origin position in real time on the host computer according to the movement of the robotic arm;

[0049] Step 2-1) Select the origin position of the end of the robot arm:

[0050] X org =[x o ,y o ,z o ] T

[0051] Among them, x o ,y o ,z o are the three-dimensional coordinates of the end of the robot arm. The current position of the end of the robot arm is set to:

[0052] X=[x,y,z] T

[0053] Among them, x, y, z are the current three-dimensional coordinates of the end of the robot arm.

[0054] Step 2-2) Calculate the deviation between the current position of the end of the robot and the origin position:

[0055] X e =[xx o ,yy o ,zz o ] T

[0056] Real-time calculation X e And it is visualized at the terminal as visual feedback of the position deviation.

[0057] Step 3) Wear an electromyographic signal sensor on the arm, place the robotic arm randomly within a certain range of the set origin, drag the robotic arm back to the origin according to visual feedback, and record the end position information of the robotic arm, six-dimensional force information and electromyographic signal in real time;

[0058] Step 3-1) Let the robot arm select a random initial position, that is:

[0059] X r =[x r ,y r ,z r ] T

[0060] Among them, x r ,y r ,z r is the three-dimensional coordinate of the random initial position of the robot arm. Here, the random initial coordinate is selected at X org Within the range of ±20cm, that is:

[0061]

[0062] Step 3-2) Wear an electromyography sensor on the arm, drag the end of the robotic arm to the vicinity of the origin according to the visual feedback information, and start collecting six-dimensional force and electromyography data.

[0063] Step 4) Calculate the true value of stiffness using the collected data, pre-process the electromyographic signal, and train the stiffness network at the end of the arm.

[0064] Step 4-1) First, abstract the human arm into a spring damping system and simplify the acceleration term and feedforward force term, namely:

[0065]

[0066] Among them, X hv is the virtual origin position of the human arm, X h and are the actual position and speed of the human hand end, F hext is the external force on the end of the human hand, D h and K h are the virtual damping and virtual stiffness of the arm respectively.

[0067] Step 4-2) The starting position X of the current round of data r , drag the tested robot back to the origin X org , under steady-state conditions, then F hext =-F ext , where F ext represents the force acting on the end of the robot arm, According to the impedance model of the robot and arm, we can get:

[0068] K h (X he -X hv )=K d (X ce -X r )

[0069] Among them, X he , X ce are the positions of the end of the arm and the end of the robot in steady state, that is, X he =X ce =X ora According to this formula, the true value of the arm stiffness can be obtained as:

[0070]

[0071] The corresponding stiffness value was then recorded as the subject's maximum arm stiffness, denoted as K PM , then K h It is expressed as:

[0072]

[0073] Among them, K d =δK PM , δ is a coefficient and 0<δ≤1. Δx, Δy, Δz represent the magnitude of the random offset of each axis of the robot arm;

[0074] Step 4-3) Preprocess the collected electromyographic signal by first flipping the signal, then performing 20-50 Hz bandpass filtering on the electromyogram, then extracting the moving average envelope, and finally adding a window, wherein the window length is 100 ms and the window overlap rate is 10 ms.

[0075] Step 4-4) Since there is a strong nonlinearity between the arm end stiffness and muscle activation, a deep neural network is used to establish the mapping relationship between these variables. Specifically, the terminal stiffness prediction network based on long short-term memory is as follows Figure 2 The network is trained to build a mapping model from EMG signals to the stiffness of the end of the arm.

[0076] The stiffness trajectory based on the fully connected neural network often requires the time-frequency domain feature extraction of the electromyographic signal. However, the feature extraction process will lose a lot of information in the electromyographic signal, especially the fine-grained dynamic features and frequency domain characteristics in the signal, which may affect the comprehensive capture of nonlinear relationships. In order to overcome this problem, an end-to-end arm end stiffness estimation model based on the fusion of long short-term memory is further proposed.

[0077] The terminal stiffness prediction network based on long short-term memory is as follows Figure 3 As shown in the figure, the arm surface electromyographic signal is first obtained, and preprocessing operations such as bandpass filtering, moving average envelope and windowing are performed. The preprocessed signal is passed to the arm stiffness estimation network, and first passes through the LSTM layer for learning. The first LSTM Block receives the initial states h0 and c0, and calculates the outputs h1 and c1 based on the data of the first time step. As the time step advances, the LSTM Block gradually processes the data of the subsequent time steps and transmits the updated hidden state and unit state. The output of each layer of LSTM generates features at different time steps. These features are further abstracted through the network, showing the pattern of how the sequence features change over time.

[0078] After each LSTM layer, it will pass through the Dropout layer. When the neural network is large in scale, has many parameters, and the number of training samples is insufficient, the neural network is prone to overfitting. To address this problem, you can add a Dropout layer after each layer of the network to alleviate it. The Dropout layer can stop each neuron from working with a certain probability when the network is forward propagating, thereby avoiding over-reliance on certain features and reducing the risk of overfitting.

[0079] Next, the output of LSTM is processed by dimensionality reduction, and the multi-dimensional tensor is compressed into a one-dimensional tensor so that it can be input into the fully connected layer for further feature learning. The Sigmoid function is used in the output layer to limit the output of the network between 0 and 1, and finally multiplied by the maximum stiffness K of the arm. PM Get the stiffness value of the end of the arm.

[0080] Example 2

[0081] As a second aspect of the present invention, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the arm end stiffness estimation method based on electromyographic signals as described above. In addition to the above-mentioned processor, memory and interface, any device with data processing capability in which the device in the embodiment is located may also include other hardware according to the actual function of the device with data processing capability, which will not be described in detail.

[0082] Example 3

[0083] As a third aspect of the present invention, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by the processor, the arm end stiffness estimation method based on electromyographic signals as described above is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (FlashCard), etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0084] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for estimating arm end stiffness based on electromyographic signals, characterized in that the steps include: Install a six-dimensional force sensor at the end of the robotic arm and set the control mode and stiffness coefficient of the robotic arm; The end of the robot arm is randomly set within a certain range of the set origin. The operator wears an electromyographic signal sensor on his arm and drags the end of the robot arm back to the origin. Collect and record the end position information, six-dimensional force information and electromyographic signals of the robotic arm; Based on the collected position information and six-dimensional force information of the end of the robot arm, the true value of the stiffness of the end of the arm is calculated; Based on the collected electromyographic signals and the calculated true value of the end-of-arm stiffness, an end-to-end arm stiffness estimation model based on the fusion of long short-term memory is trained, which takes the electromyographic signals as input and the end-of-arm stiffness value as output; The electromyographic signal of the subject is collected and input into the trained end-to-end arm end stiffness estimation model to estimate the stiffness of the subject's arm end.

2. The method for estimating arm end stiffness based on electromyographic signals according to claim 1, characterized in that: The end of the mechanical arm is equipped with a drag handle.

3. The method for estimating arm end stiffness based on electromyographic signals according to claim 1, characterized in that: The robotic arm is set to the admittance control mode, and the admittance control equation of the robotic arm is as follows: Among them, F ext Indicates the actual force applied by the end of the robotic arm; F d represents the expected contact force; X c , Respectively represent the actual position, velocity and acceleration of the end effector of the robot arm; X d , They represent the desired position, velocity and acceleration of the end effector respectively; M d , D d and K d represent the required virtual mass matrix, damping matrix and stiffness matrix respectively.

4. The method for estimating arm end stiffness based on electromyographic signals according to claim 1, characterized in that: The method also constructs a real-time display interface for the three-dimensional coordinates of the robotic arm, sets the origin position of the end of the robotic arm, calculates and displays the three-dimensional coordinate difference between the end of the robotic arm and the origin in real time according to the movement of the robotic arm, and displays it visually on the terminal as visual feedback of the position deviation of the end of the robotic arm.

5. The method for estimating arm end stiffness based on electromyographic signals according to claim 1, characterized in that: The actual value of the stiffness of the end of the arm is calculated based on the collected position information of the end of the manipulator and the six-dimensional force information data, as follows: For the starting position X of the current round of data r , the operator drags the test robot back to the origin X org ; Under steady-state conditions, the external force F on the end of the human hand is hext Based on the collected end position information and six-dimensional force information data of the robotic arm, the true value of the end stiffness F of the arm is calculated. ext On the contrary; the actual speed of the robot end effector The actual speed of the end of the human hand is 0. According to the impedance model of the robot and the arm, the actual value of the stiffness of the end of the arm is: Among them, X he , X ce are the positions of the end of the arm and the end of the robot in steady state; X c is the initial position of the end effector of the robot arm; X hv is the virtual origin position of the human arm; K d is the stiffness matrix of the end effector of the robot arm.

6. The method for estimating arm end stiffness based on electromyographic signals according to claim 5, characterized in that: The method abstracts the human arm as a spring damping system and simplifies the acceleration term and the feedforward force term. The external force F hext It is expressed as: Among them, X hv is the virtual origin position of the human arm, X h and are the actual position and speed of the human hand end, F hex is the external force on the end of the human hand, D h and K h are the virtual damping and virtual stiffness of the arm respectively.

7. The method for estimating arm end stiffness based on electromyographic signals according to claim 1, characterized in that: The collected electromyographic signals are preprocessed and then input into the end-to-end arm end stiffness estimation model. The preprocessing steps include: flipping the electromyographic signals, performing bandpass filtering on the electromyogram, performing moving average envelope extraction, and finally performing windowing.

8. The method for estimating arm end stiffness based on electromyographic signals according to claim 1, characterized in that: The end-to-end arm end stiffness estimation model based on fusion of long short-term memory is specifically as follows: The preprocessed electromyographic signal is passed into the arm stiffness estimation network and input into the LSTM layer for learning. The LSTM layer includes multiple LSTM blocks that process time step data in sequence. The first LSTM block receives the initial state and calculates and outputs the next state in combination with the data of the first time step. Each LSTM block gradually processes the data of subsequent time steps and transmits the updated hidden state and unit state. After each LSTM block, it will pass through a random deactivation layer. When the network is forward propagating, each neuron will stop working with a certain probability. As the time step progresses, the LSTM layer extracts the time-frequency domain features of the output electromyographic signal. The output of the LSTM layer is processed by dimensionality reduction to compress the multi-dimensional tensor into a one-dimensional tensor, and then input into the fully connected layer to further learn the features; An activation function is used in the output layer to limit the output of the network between 0 and 1 and then multiply it by the maximum stiffness of the arm to obtain the stiffness value at the end of the arm.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the arm end stiffness estimation method based on electromyography signals as described in any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for estimating arm end stiffness based on electromyographic signals as described in any one of claims 1 to 8 are implemented.

Images