A simulation robot arm control method based on multi-channel electromyography signals

Abstract

The present invention discloses a simulation robot arm control method based on multi-channel electromyographic signals, which belongs to the technical field of electromyographic signals, and includes the following steps: electromyographic acquisition: collecting human electromyographic signals through multi-channel muscle electric sensors, and transmitting the electromyographic signals to a computer after amplification and filtering; data display: visualizing data and waveforms of multi-channel electromyographic signals collected by multi-channel muscle electric sensors; simulation robot arm control: defining different operations of the robot arm in different muscle states of different channels, and controlling the robot arm in real time according to the collected multi-channel electromyographic signals; multi-channel collaborative control: collaborative control of the robot arm by multi-channel electromyographic signals. The present invention collects electromyographic signals of contraction or relaxation of multi-channel human muscles, performs conditional judgment and controls the simulation robot arm, and can freely define different operations of the robot arm in different muscle states of different channels, such as the opening and closing of the robot claw and the torsion of different nodes of multiple robot arms.

Description

Technical Field

[0001] The invention relates to the technical field of electromyographic signals, and in particular to a simulation mechanical arm control method based on multi-channel electromyographic signals. Background Art

[0002] Human-in-the-loop teleoperation is the mainstream technical method for manipulator control at this stage, that is, it relies on the operator to participate and give control instructions to ensure the smooth completion of the task. The teleoperation system has great research significance and application value. When transmitting signals over long distances, signal delay or time delay problems are inevitable. In the robot control loop, the robot control command issued by the operator needs to be sent from the operator's end to the remote robot, and the relevant information showing the operation status needs to be sent back to the controller's end. The interference of communication delay and signal delay will not only affect real-time operation, cause image interference and robot instrument overshoot or oscillation, but also destroy the stability of the robot system. These deviations bring unpredictability, inefficiency and inaccuracy to the originally controllable and safe operating environment.

[0003] With the development of advanced signal processing technology and predictive control algorithms, bioelectric signal acquisition systems can be used as a type of human-computer interaction equipment for teleoperation systems. Among them, the control method based on bioelectric signals can directly decode the intention of the action, eliminating the time from the generation of consciousness to the execution of the action, accelerating the rapid execution capability of the controlled robotic arm, and shortening the delay time of the execution of teleoperation tasks. By utilizing the time advance of bioelectric signals relative to control decisions, the teleoperation loop time can be further shortened, the risk can be reduced, and the robotic arm can be facilitated to perform complex operation tasks. In the future, it will be an important trend in this field to reduce the impact of delays while improving the autonomy of teleoperation and enhance the reliability and flexibility of complex teleoperation interactions.

[0004] At present, motion encoding and online learning methods such as Hidden Markov Model or clustered Gaussian Mixture Model can enable remote robotic arms to regenerate motion based on intermittent and sparse data streams, effectively improving the flexibility and robustness of teleoperation, and attracting widespread attention from researchers in this field. However, this type of method still does not improve the latency itself, and requires a complex control interface and high professionalism of the operator, which is not conducive to direct human control of the robot for training or intervention. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a simulation robot arm control method based on multi-channel electromyographic signals. By collecting multi-channel electromyographic signals of contraction or relaxation of human muscles, conditional judgment is performed and the simulation robot arm is controlled. Different operations of the robot arm in different muscle states of different channels can be freely defined, such as the opening and closing of the robot claws and the twisting of different nodes of the multiple robot arms.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A simulation robot arm control method based on multi-channel electromyographic signals comprises the following steps:

[0008] S1. Myoelectric acquisition: collect human electromyographic signals through multi-channel electromyographic sensors, and transmit the electromyographic signals to the computer after amplification and filtering;

[0009] S2. Data display: Visualizing the data and waveform of multi-channel electromyographic signals collected by multi-channel electromyographic sensors;

[0010] S3, simulation robot arm control: define different operations of the robot arm in different channels and muscle states, and control the robot arm in real time according to the collected multi-channel electromyographic signals;

[0011] S4. Multi-channel collaborative control: Collaborative control of the robotic arm using multi-channel electromyographic signals.

[0012] A further improvement of the technical solution of the present invention is that the multi-channel muscle electrical sensor is a six-lead muscle electrical sensor, which is provided with 1 to 6 channels, and the six-lead muscle electrical sensor is used to collect the analog circuit of a maximum of 6 channels at the front end.

[0013] A further improvement of the technical solution of the present invention is that: S1 specifically includes the following steps:

[0014] S11, connecting one end of the three-lead wire to the multi-channel muscle electrical sensor, and the other end to the muscle to be measured, the first electrode connected to the three-lead wire is used as a reference electrode to connect to an area without muscle activity, and the second electrode and the third electrode are connected to the muscle to be measured;

[0015] S12, amplifying, filtering and decoding the electromyographic signals collected by the multi-channel electromyographic sensor, and transmitting them to the computer.

[0016] A further improvement of the technical solution of the present invention is that: S3 specifically includes the following steps:

[0017] S31. Open the robot simulation platform, add a robotic arm and a robotic claw, bind the robotic claw to the end of the robotic arm, and add a script to control opening and closing;

[0018] S32, setting the electromyographic signal potential threshold and operation delay determination rule;

[0019] S33. Define different operations of the robot arm in different muscle states according to the potential threshold of the electromyographic signal of each channel.

[0020] A further improvement of the technical solution of the present invention is that the electromyographic signal potential threshold is set to 600 μV.

[0021] A further improvement of the technical solution of the present invention is that the operation delay determination rule is to perform a delay threshold determination, and the delay threshold is 50ms.

[0022] A further improvement of the technical solution of the present invention is that: S4 specifically includes the following steps:

[0023] S4.1 Directly apply multi-channel electromyographic signals to coordinate control of the robot arm;

[0024] S4.2 establishes a database according to the multi-channel electromyographic signals of the corresponding actions, and performs specified control or predictive control on the robot arm according to the database.

[0025] A further improvement of the technical solution of the present invention is that: S4.2 specifically includes the following steps:

[0026] S4.2.1 Connect multiple three-lead wires to the surface of various parts of the arm, collect multi-channel electromyographic signals under various basic movements, convert them into robot arm control signals, and establish a database;

[0027] S4.2.2 When the robot arm needs to complete a specified action, the robot arm control signal of the corresponding action in the database is called to complete the coordinated control;

[0028] S4.2.3 When the hand makes a corresponding action, the program will display the ongoing action of the arm according to the database content, call the database to complete the subsequent action, and directly control it through the arm action by clicking the joint part of the robot arm that needs to be controlled.

[0029] Due to the adoption of the above technical solution, the technical progress achieved by the present invention is:

[0030] 1. To address the time delay problem of remote operation, the present invention utilizes the advanced characteristics of bioelectric information decision-making compared to traditional behavioral decision-making, and proposes a remote operation method based on direct control of a simulated robotic arm using electromyographic signals, which effectively shortens the information transmission path and reduces the impact of time delay on remote operation.

[0031] 2. The present invention collects electromyographic signals through multiple channels and sets delay judgment rules for operation, thereby realizing the diversification, real-time, simplification and popularization of the control method of the simulated robotic arm.

[0032] 3. The present invention supports real-time control of simulated robotic arms by electromyographic signals, and supports customization according to needs, including but not limited to judgment thresholds, functions of different electromyographic channels, opening and closing of simulated robotic claws, movement direction and degree of simulated robotic arm joints, etc., which reduces the operating threshold and enhances scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the process of the present invention;

[0034] Figure 2 It is the wiring diagram of the muscle electrical sensor;

[0035] Figure 3 The UR3 robot arm and RG2 robot claw are used in the simulation robot arm platform;

[0036] Figure 4 This is a schematic diagram of a single-channel electromyographic signal controlling a simulated robotic arm in real time. When the muscle is relaxed, the robotic claw closes.

[0037] Figure 5 This is a schematic diagram of a simulated robotic arm controlled in real time by a single-channel electromyographic signal. When the muscle exerts force, the robotic claw opens. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below with reference to the accompanying drawings and embodiments:

[0039] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", etc. may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "several" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0040] like Figure 1 As shown, a simulation robot arm control method based on multi-channel electromyographic signals includes the following steps:

[0041] S1. Myoelectric acquisition: collect human electromyographic signals through multi-channel electromyographic sensors, and transmit the electromyographic signals to the computer after amplification and filtering;

[0042] In this embodiment, the multi-channel muscle electric sensor is a six-lead muscle electric sensor, which is set with 1 to 6 channels. The six-lead muscle electric sensor can monitor the activity of muscle tissue. The six-lead muscle electric sensor is used to collect the analog circuit of up to 6 channels at the front end. Paste the electrode patch on the muscle to be tested, and connect the port of the three-lead line to the electrode patch. The front-end acquisition circuit collects the electromyographic signal of the human arm or leg through 1 to 6 channels, and after a series of amplification and filtering, the electromyographic signal is transmitted to the computer in real time through the USB port, and the data can be displayed and processed by the post-program.

[0043] like Figure 2 As shown, S1 specifically includes the following steps:

[0044] S11, connecting one end of the three-lead wire to the multi-channel muscle electrical sensor, and the other end to the muscle to be measured, the first electrode connected to the three-lead wire is used as a reference electrode to connect to an area without muscle activity, and the second electrode and the third electrode are connected to the muscle to be measured;

[0045] In this embodiment, the outside of the first electrode is set to red, the outside of the second electrode is set to yellow, and the outside of the third electrode is set to green for distinction; the position of the reference electrode can be selected at the elbow.

[0046] S12, amplifying, filtering and decoding the electromyographic signals collected by the multi-channel electromyographic sensor, and transmitting them to the computer.

[0047] In this embodiment, an ARDUINO development board is used to amplify and filter the electromyographic signal, and is connected to a computer via a USB port, and the port serial number is confirmed in the ports (COM and LPT) of the device manager of the computer.

[0048] S2. Data display: Visualizing the data and waveform of multi-channel electromyographic signals collected by multi-channel electromyographic sensors;

[0049] In this embodiment, the serial numbers of the USB input ports of the multi-channel electromyographic signals are set respectively, and the program is run. It can be seen that the electromyographic signal graph can display the size and waveform of the collected electromyographic signals in real time.

[0050] S3. Simulation of robotic arm control: define different operations of the robotic arm in different channels and muscle states, and control the robotic arm in real time based on the collected multi-channel electromyographic signals.

[0051] The present invention can freely define different operations of the robot arm in different channels and muscle states, and can perform condition judgment and directly control the simulation robot arm by collecting the myoelectric signals of the contraction or relaxation of human muscles;

[0052] In this embodiment, the simulation robot arm module is mainly based on the CoppeliaSim software platform, and the simulation robot arm model UR3 and the mechanical claw model RG2 are added to this platform. The electromyographic signal control program written independently can realize the real-time control of the simulation robot arm and the mechanical claw by the electromyographic signal at the input end. In addition, different operations of the robot arm in different channels and different muscle states can be freely defined, such as one channel electromyographic signal corresponding to the opening and closing of the mechanical claw, and multi-channel electromyographic signals corresponding to the torsion of different nodes of the robot arm.

[0053] like Figure 3 As shown, S3 specifically includes the following steps:

[0054] S31. Open the robot simulation platform, add a robotic arm and a robotic claw, bind the robotic claw to the end of the robotic arm, and add a script to control opening and closing;

[0055] In this embodiment, open the Coppeliasim robot simulation platform, add the UR3 robot arm and the RG2 robot claw, bind the robot claw to the end of the robot arm and add a script to control the opening and closing of the robot claw;

[0056] S32, setting the electromyographic signal potential threshold and operation delay determination rule;

[0057] In this embodiment, the program initially sets the electromyographic signal potential threshold to 600 μV. When the potential value exceeds 600 μV, the mechanical claw is judged to be open. Each channel can freely adjust the electromyographic signal potential threshold according to control requirements.

[0058] When the program is running, the electromyographic signal is collected in real time and a continuous waveform is generated. If the electromyographic signal value is determined continuously, it will inevitably take up a lot of time and storage space. Therefore, the operation delay determination rule is set to prevent the waveform display delay caused by the too-fast execution of the robot arm opening and closing command, which will cause the actual arm force, the waveform and the simulated robot arm opening and closing to not match. The operation delay determination rule is to perform a delay threshold determination, and the delay threshold is 50ms, that is, a determination is made every 50ms. The determination conditions such as delay time and threshold size can be modified later according to the program operation status.

[0059] S33. Define different operations of the robot arm in different muscle states according to the potential threshold of the electromyographic signal of each channel.

[0060] like Figure 4 , Figure 5 As shown, after the settings are completed, you can see that the program can run normally. When the muscle is relaxed, the mechanical claw is in the closed state, and when the muscle is exerted, the mechanical claw is in the open state.

[0061] The program operation logic is: when the potential value of the collected electromyographic signal is less than the electromyographic signal potential threshold, the mechanical claw operation is executed to close, and when the potential value of the collected electromyographic signal is greater than the electromyographic signal potential threshold, the mechanical claw operation is executed to open. In addition to the opening and closing of the mechanical claw, the rotation of the mechanical arm joint can also be realized.

[0062] In this embodiment, in addition to controlling the opening and closing of the robot claw, the rotation of the robot arm can also be controlled. In the main program, add robot.rotateCertainAnglePositive(x, angle) or robot.rotateCertainAngleNegative(x, angle) code to control the forward or reverse rotation of the robot arm. x is 0 to 5, corresponding to the different degrees of freedom of the robot arm; angle is any angle that the robot arm is expected to rotate at each time.

[0063] S4, multi-channel collaborative control: collaborative control of the robotic arm using multi-channel electromyographic signals;

[0064] S4.1 Directly apply multi-channel electromyographic signals to coordinate control of the robot arm;

[0065] By connecting multiple three-lead wires to the surface of various parts of the arm and collecting multi-channel EMG signals, coordinated control of complex operations on multiple parts of the arm can be achieved. The program can also show the size and waveform of the collected multi-channel EMG signals in real time on the EMG signal graph, with the waveforms of different channels marked in different colors.

[0066] S4.2 establishes a database according to the multi-channel electromyographic signals of the corresponding actions, and performs specified control or predictive control on the robot arm according to the database; specifically includes the following steps:

[0067] S4.2.1 Connect multiple three-lead wires to the surface of various parts of the arm, collect multi-channel electromyographic signals under various basic movements, convert them into robot arm control signals, and establish a database; the basic movements include clockwise wrist rotation, counterclockwise wrist rotation, arm flexion, arm extension, fist clenching, fist loosening, and stillness;

[0068] S4.2.2 When the robot arm needs to complete a specified action, the robot arm control signal of the corresponding action in the database is called to complete the coordinated control;

[0069] S4.2.3 When the hand makes a corresponding action, the program will display the ongoing action of the arm according to the database content, call the database to complete the subsequent actions, and directly control it through the arm action by clicking the joint part of the robotic arm that needs to be controlled. For example, when the wrist rotates clockwise, the joint also rotates clockwise; select the gripper, when you make a fist, the robotic gripper also clamps accordingly, etc., to achieve control that conforms to human intuitive intentions, which can reduce reaction time and thus reduce latency.

[0070] In summary, the present invention collects multi-channel electromyographic signals of contraction or relaxation of human muscles to perform conditional judgment and control the simulated robotic arm, and can freely define different operations of the robotic arm in different channels and different muscle states, such as the opening and closing of the robotic claws, the twisting of different nodes of multiple robotic arms, and coordinately control the robotic arm through multi-channel electromyographic signals.

Claims

1. A simulation robot arm control method based on multi-channel electromyographic signals, characterized in that: The following steps are involved: S1. Myoelectric acquisition: collect human electromyographic signals through multi-channel electromyographic sensors, and transmit the electromyographic signals to the computer after amplification and filtering; S1 specifically includes the following steps: S11, connecting one end of the three-lead wire to the multi-channel muscle electrical sensor, and the other end to the muscle to be measured, the first electrode connected to the three-lead wire is used as a reference electrode to connect to an area without muscle activity, and the second electrode and the third electrode are connected to the muscle to be measured; S12, amplifying, filtering and decoding the electromyographic signals collected by the multi-channel electromyographic sensor, and transmitting them to the computer; S2. Data display: Visualizing the data and waveform of multi-channel electromyographic signals collected by multi-channel electromyographic sensors; S3, simulation robot arm control: define different operations of the robot arm in different channels and muscle states, and control the robot arm in real time according to the collected multi-channel electromyographic signals; S3 specifically includes the following steps: S31. Open the robot simulation platform, add a robotic arm and a robotic claw, bind the robotic claw to the end of the robotic arm, and add a script to control opening and closing; S32, setting the electromyographic signal potential threshold and operation delay determination rule; The electromyographic signal potential threshold is set to 600 μV; when the program is running, the electromyographic signal is collected in real time and a continuous waveform is generated. If the electromyographic signal value is determined continuously, it will inevitably take up a lot of time and storage space. Therefore, an operation delay determination rule is set to prevent the waveform display delay caused by the too-fast execution of the mechanical arm opening and closing instruction, resulting in the actual arm force, the waveform and the simulated mechanical arm opening and closing conditions not matching each other; the operation delay determination rule is to perform a delay threshold determination, and the delay threshold is 50 ms, that is, a determination is performed every 50 ms; the delay time and threshold value determination conditions are subsequently modified according to the program running conditions; S33, defining different operations of the robot arm in different muscle states according to the potential threshold of the electromyographic signal of each channel; S4, multi-channel collaborative control: collaborative control of the robotic arm using multi-channel electromyographic signals; S4 specifically includes the following steps: S4.1 Directly apply multi-channel electromyographic signals to coordinate control of the robot arm; S4.2 establishes a database according to the multi-channel electromyographic signals of the corresponding actions, and performs specified control or predictive control on the robot arm according to the database; S4.2 specifically includes the following steps: S4.2.1 Connect multiple three-lead wires to the surface of various parts of the arm, collect multi-channel electromyographic signals under various basic movements, convert them into robot arm control signals, and establish a database; S4.2.2 When the robot arm needs to complete a specified action, the robot arm control signal of the corresponding action in the database is called to complete the coordinated control; S4.2.3 When the hand makes a corresponding action, the program will display the ongoing action of the arm according to the database content, call the database to complete the subsequent action, and directly control it through the arm action by clicking the joint part of the robot arm that needs to be controlled.

2. The method for controlling a simulated mechanical arm based on multi-channel electromyographic signals according to claim 1, characterized in that: The multi-channel muscle electrical sensor is a six-lead muscle electrical sensor, which is provided with 1 to 6 channels. The six-lead muscle electrical sensor is used to collect data of analog circuits of a maximum of 6 channels at the front end.

Images