A method, system and terminal for identifying motion intention

By identifying the user's eye movements and gesture signals, and controlling the movement of the rehabilitation robot in real time, the problem of a single training mode of traditional rehabilitation robot is solved, and user participation and rehabilitation effect are improved.

CN118737421BActive Publication Date: 2024-11-22深圳开鸿数字产业发展有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411223478.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-22
Estimated Expiration
2044-09-03

AI Technical Summary

Technical Problem

The training mode provided by traditional rehabilitation robots is single, and cannot accurately obtain the user's exercise intentions, resulting in low user participation and lack of protection mechanisms.

Method used

By obtaining electro-ophthalmic and electromyography signals, using SVM and CNN classifiers to identify user eye movements and gesture information, generate control instructions, and smoothly control the robot's robot's robotic arm movement to achieve real-time control of user intentions.

Benefits of technology

It improves users' participation and sense of security in rehabilitation training, and enhances the rehabilitation effect by adjusting the robot's movement direction and speed in real time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118737421B_ABST
    Figure CN118737421B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, system and terminal for identifying movement intention, the method comprising: obtaining EOG signals of the target user's head collected by an electrooculogram acquisition headband, preprocessing the EOG signals to obtain target EOG signals, inputting the EOG signals into an SVM classifier for eye movement information recognition, and outputting eye movement prediction results; obtaining sEMG signals of the target user's upper limbs collected by an electromyography acquisition armband, preprocessing the sEMG signals to obtain target sEMG signals, inputting the sEMG signals into a CNN classification model for gesture information recognition, and outputting gesture prediction results; generating control instructions according to the eye movement prediction results and the gesture prediction results, and controlling the movement of the mechanical arm of the upper limb rehabilitation robot through a compliant control strategy according to the control instructions. The present invention uses sEMG and EOG signals to control the movement direction and speed of the rehabilitation robot in real time, combines the user's movement intention with the robot's movement control, and effectively improves the user's participation in the rehabilitation process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of user intention recognition, and in particular to a motion intention recognition method, system, terminal and computer-readable storage medium. Background Art

[0002] As an important branch of medical robots, rehabilitation robots have been widely used in rehabilitation nursing, prosthesis and rehabilitation therapy.

[0003] Rehabilitation training can promote the regeneration and reconstruction of damaged nerves, which can effectively help users recover their limb movement ability and daily living ability. At present, the use of rehabilitation robots in the field of rehabilitation therapy to assist users in completing training can achieve better rehabilitation effects.

[0004] However, traditional rehabilitation robots can generally only provide a single training mode. Usually, users can only passively complete training movements guided by the robot. The robot cannot accurately obtain the user's movement intentions, resulting in low user participation and the system lacks certain protection mechanisms.

[0005] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0006] The main purpose of the present invention is to provide a method, system, terminal and computer-readable storage medium for identifying motion intention, aiming to solve the problem that the traditional rehabilitation robots in the prior art provide a single training mode and cannot accurately obtain the user's motion intention. The user can only passively complete the training movements guided by the robot, resulting in low user participation.

[0007] To achieve the above object, the present invention provides a method for identifying movement intention, the method comprising the following steps:

[0008] Acquire the EOG signal of the target user's head collected by the electrooculogram collection headband, pre-process the EOG signal to obtain the target EOG signal, input the target EOG signal into a pre-trained SVM classifier to perform eye movement information recognition, and output the eye movement prediction result;

[0009] Acquire the sEMG signal of the upper limb of the target user collected by the electromyography acquisition armband, pre-process the sEMG signal to obtain the target sEMG signal, input the target sEMG signal into the pre-trained CNN classification model for gesture information recognition, and output the gesture prediction result;

[0010] A control instruction is generated according to the eye movement prediction result and the gesture prediction result, and according to the control instruction, the movement of the mechanical arm of the upper limb rehabilitation robot is controlled through a compliant control strategy.

[0011] Optionally, the method for identifying movement intention, wherein the step of acquiring an EOG signal of the target user's head collected by an electrooculogram collection headband and preprocessing the EOG signal to obtain a target EOG signal, specifically includes:

[0012] Obtain the EOG signal of the target user's head collected in real time by the TD05 electrooculogram acquisition headband, and perform a preset frequency power frequency filter and a Butterworth low-pass filter on the EOG signal to obtain an effective frequency band signal;

[0013] The effective frequency band signal is normalized to obtain a target EOG signal.

[0014] Optionally, in the motion intention recognition method, the eye movement prediction results include: a single simultaneous blink, a double blink, a triple blink, a single left eye blink and a single right eye blink.

[0015] Optionally, the method for identifying movement intention, wherein the step of acquiring the sEMG signal of the upper limb of the target user collected by the electromyography collection armband and preprocessing the sEMG signal to obtain the target sEMG signal, specifically comprises:

[0016] Acquire the sEMG signal of the upper limb of the target user collected in real time by the MYO electromyography acquisition armband, denoise the sEMG signal by improving the denoising algorithm of modal decomposition, and obtain different levels of modalities, wherein the MYO electromyography acquisition armband is an electromyography acquisition armband with a power frequency filtering function;

[0017] According to the different levels of the modalities, the modalities are reconstructed accordingly, and the reconstructed modalities are fused to obtain the target sEMG signal.

[0018] Optionally, in the method for identifying movement intention, the gesture prediction results include: relaxation, clenching a fist, opening the hand, wrist adduction, wrist abduction, radial flexion and ulnar flexion.

[0019] Optionally, in the method for identifying movement intention, the control instruction includes a first control instruction and a second control instruction;

[0020] Wherein, the first control instruction includes: a stop instruction, a start instruction, an acceleration instruction and a deceleration instruction;

[0021] The second control instructions include: a fist clenching instruction, a hand opening instruction, a wrist adduction instruction, a wrist abduction instruction, a radial flexion instruction and an ulnar flexion instruction.

[0022] Optionally, the method for identifying movement intention, wherein the step of generating a control instruction according to the eye movement prediction result and the gesture prediction result, and controlling the movement of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction, specifically comprises:

[0023] Obtaining the eye movement prediction result and the gesture prediction result;

[0024] If the eye movement prediction result is a double blink, generating the stop instruction, and stopping the movement of the robotic arm according to the stop instruction;

[0025] If the eye movement prediction result is three blinks, generating the start instruction, and starting the movement of the robotic arm according to the start instruction;

[0026] If the eye movement prediction result is a single left eye blink, generating the acceleration instruction, and increasing the movement speed of the robotic arm according to the acceleration instruction;

[0027] If the eye movement prediction result is a single right eye blink, generating the deceleration instruction, and reducing the movement speed of the robotic arm according to the deceleration instruction;

[0028] If the eye movement prediction result is a single simultaneous blink, no instruction is generated;

[0029] If the gesture prediction result is a fist, the fist instruction is generated, and the end of the robotic arm is controlled to move vertically downward along the Z axis according to the fist instruction;

[0030] If the gesture prediction result is an open hand, the open hand instruction is generated, and the end of the robotic arm is controlled to move vertically upward along the Z axis according to the open hand instruction;

[0031] If the gesture prediction result is wrist adduction, the wrist adduction instruction is generated, and the end of the robotic arm is controlled to move along the negative direction of the Y axis in the horizontal plane according to the wrist adduction instruction;

[0032] If the gesture prediction result is wrist abduction, the wrist abduction instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the Y axis in the horizontal plane according to the wrist abduction instruction;

[0033] If the gesture prediction result is radial flexion, the radial flexion instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the X-axis in the horizontal plane according to the radial flexion instruction;

[0034] If the gesture prediction result is ulnar flexion, the ulnar flexion instruction is generated, and the robotic arm is controlled to move along the negative direction of the X-axis in the horizontal plane according to the ulnar flexion instruction;

[0035] If the gesture prediction result is relaxation, no instruction is generated.

[0036] In addition, to achieve the above-mentioned purpose, the present invention further provides a motion intention recognition system, wherein the motion intention recognition system comprises:

[0037] An eye movement result prediction module is used to obtain the EOG signal of the target user's head collected by the electrooculogram collection headband, pre-process the EOG signal to obtain a target EOG signal, input the target EOG signal into a pre-trained SVM classifier to perform eye movement information recognition, and output an eye movement prediction result;

[0038] A gesture result prediction module is used to obtain the sEMG signal of the upper limb of the target user collected by the electromyography collection armband, pre-process the sEMG signal to obtain a target sEMG signal, input the target sEMG signal into a pre-trained CNN classification model for gesture information recognition, and output a gesture prediction result;

[0039] A control instruction generation module is used to generate a control instruction according to the eye movement prediction result and the gesture prediction result, and control the movement of the mechanical arm of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction.

[0040] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a motion intention recognition program stored in the memory and executable on the processor, and when the motion intention recognition program is executed by the processor, the steps of the motion intention recognition method described above are implemented.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a motion intention recognition program, and when the motion intention recognition program is executed by a processor, the steps of the motion intention recognition method as described above are implemented.

[0042] In the present invention, the EOG signal of the target user's head collected by the electrooculogram acquisition headband is obtained, the EOG signal is preprocessed to obtain the target EOG signal, the EOG signal is input into the SVM classifier for eye movement information recognition, and the eye movement prediction result is output; the sEMG signal of the target user's upper limb collected by the electromyography acquisition armband is obtained, the sEMG signal is preprocessed to obtain the target sEMG signal, the sEMG signal is input into the CNN classification model for gesture information recognition, and the gesture prediction result is output; the control instruction is generated according to the eye movement prediction result and the gesture prediction result, and the mechanical arm movement of the upper limb rehabilitation robot is controlled by the compliant control strategy according to the control instruction. The present invention combines the user's movement intention with the robot's movement control, and can use the EOG signal and the sEMG signal to control the movement direction and speed of the rehabilitation robot in real time, effectively improving the user's participation in the rehabilitation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of a preferred embodiment of the method for identifying movement intention of the present invention;

[0044] Figure 2 It is a CNN architecture diagram based on ICEEMDAN and slow fusion model in the motion intention recognition method of the present invention;

[0045] Figure 3 It is a schematic diagram of the ICEEMDAN decomposition result of sEMG data of a window in the motion intention recognition method of the present invention;

[0046] Figure 4 is a structural diagram of a preferred embodiment of the motion intention recognition system of the present invention;

[0047] Figure 5 Schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0048] The present application provides a method for identifying motion intention and related equipment. To make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0050] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0051] The motion intention recognition method described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the motion intention recognition method includes the following steps:

[0052] Step S10, obtaining the EOG signal of the target user's head collected by the electrooculogram collection headband, preprocessing the EOG signal to obtain a target EOG signal, inputting the target EOG signal into a pre-trained SVM classifier to perform eye movement information recognition, and outputting an eye movement prediction result.

[0053] The step of acquiring the EOG signal of the target user's head collected by the electrooculogram collection head ring and preprocessing the EOG signal to obtain the target EOG signal specifically includes:

[0054] Obtain the EOG signal of the target user's head collected in real time by the TD05 electrooculogram acquisition headband, and perform a preset frequency power frequency filter and a Butterworth low-pass filter on the EOG signal to obtain an effective frequency band signal;

[0055] The effective frequency band signal is normalized to obtain a target EOG signal.

[0056] In this embodiment, the electrooculography headband adopts the TD05 model electrooculography headband, and collects EOG (electrooculography) signals through the electrode in the middle of the TD05 model electrooculography headband. The TD05 electrooculography headband is a single-channel, 250 Hz sampling frequency EOG signal portable acquisition device, weighing only 50 g. At the same time, the Type-C interface and Bluetooth 5.0 transmission protocol are adopted. The TD05 headband has a total of 3 electrodes, which are located on three separate blue acquisition modules. Among them, the middle electrode is used to collect EOG signals, and the electrodes on both sides are the ground electrode GND and the reference electrode REF. In terms of electrode materials, the TD05 headband uses hydrogel electrodes, which have good conductivity and corrosion resistance, and the hydrogel fits the human skin better, which can bring higher comfort.

[0057] The EOG signal is subjected to power frequency filtering at a preset frequency (50 Hz), and then passed through a Butterworth low-pass filter to obtain an effective frequency band signal. Taking 50 Hz power frequency filtering as an example, a signal with an effective frequency band of 0.53-15 Hz can be obtained.

[0058] Furthermore, for subsequent feature extraction and classification recognition, the effective frequency band signal obtained after filtering is first normalized to obtain the target EOG signal. , expressed as:

[0059] ;

[0060] in, represents the effective frequency band signal, represents the signal after translation, and Respectively The maximum and minimum values ​​in . Therefore, the difference between the amplitudes of any two points is uniformly constrained to the range of [0, 1], and the amplitude of the signal is not affected by the number of measurements and the different testers.

[0061] Furthermore, the target EOG signal is input into a pre-trained SVM classifier to perform eye movement information recognition, and an eye movement prediction result is output.

[0062] It is understandable that the waveform amplitude of the EOG of a single blink in a normal state begins to increase when the subject begins to close his eyelids. The maximum value is reached at any moment. It indicates the moment when the subject completely closes his eyelids, and then decreases when he begins to open his eyelids. Therefore, the signal amplitude peak is a very effective feature for blink recognition. However, it is not comprehensive to only consider the peak value of the blink signal, and the peak values ​​of each blink of different subjects may be different. Therefore, in this embodiment, in addition to extracting the signal amplitude peak of the EOG signal as a feature for eye movement recognition, the correlation between the EOG signal and the template signal is calculated as another important feature, and the signal amplitude peak of the EOG signal and the correlation between the EOG signal and the template signal are input into a pre-trained SVM classifier for eye movement information recognition, and the eye movement prediction result is output.

[0063] If the signal amplitude peak and correlation coefficient in the time period are both higher than a certain threshold, it can be considered that a blink activity has occurred in this time period. If the time interval between two blinks is less than a certain threshold, it is considered that a double blink activity has occurred. According to experimental findings, the best effect is achieved when the time interval between two blinks is 500ms. Therefore, in the following classification experiment, the blink time interval standard is set to 500ms. In the SVM classifier, the Gaussian kernel is selected as the sum function, and the extracted features are used as input to train a classifier that can distinguish single blinks from non-blinks, and then the time interval can be used to distinguish double blinks.

[0064] The eye movement prediction results include: single simultaneous blink, double blink, triple blink, single left eye blink and single right eye blink.

[0065] Step S20, obtaining the sEMG signal of the upper limb of the target user collected by the electromyography collection armband, preprocessing the sEMG signal to obtain a target sEMG signal, inputting the target sEMG signal into a pre-trained CNN classification model for gesture information recognition, and outputting a gesture prediction result.

[0066] The step of acquiring the sEMG signal of the upper limb of the target user collected by the electromyography collection armband and preprocessing the sEMG signal to obtain the target sEMG signal specifically includes:

[0067] Acquire the sEMG signal of the upper limb of the target user collected in real time by the MYO electromyography acquisition armband, denoise the sEMG signal by improving the denoising algorithm of modal decomposition, and obtain different levels of modalities, wherein the MYO electromyography acquisition armband is an electromyography acquisition armband with a power frequency filtering function;

[0068] According to the different levels of the modalities, the modalities are reconstructed accordingly, and the reconstructed modalities are fused to obtain the target sEMG signal.

[0069] In this embodiment, the myoelectric acquisition armband uses the MYO (myocardial) myoelectric armband. The MYO myoelectric armband is mainly composed of 8 bioelectric sensor units of different sizes and thicknesses. Each sensor unit contains 3 electrodes, a total of 24 electrodes, which can capture the bioelectric changes generated by the user's arm muscle activity in real time and collect the sEMG (surface electromyography) signal of the user's upper limb. The electrodes used by MYO are dry electrodes, which do not require the use of conductive paste, and different electrodes are connected by elastic materials, which is very convenient to wear. The maximum sampling frequency of the MYO device is 200 Hz. At the same time, MYO uses an ARM processor (the ARM processor is the first low-power RISC microprocessor designed by Acorn Ltd. in the UK) to process the collected data and transmit data with the computer via Bluetooth 4.0.

[0070] It can be understood that the MYO electromyography acquisition armband itself has been subjected to industrial frequency filtering. Furthermore, the present invention uses a denoising algorithm based on improved modal decomposition to denoise the sEMG signal to obtain modalities of different levels. According to the different levels of the modalities, the modalities are reconstructed accordingly, and the reconstructed modalities are fused, which can effectively improve the signal-to-noise ratio and correlation coefficient of the signal.

[0071] Furthermore, the target sEMG signal is input into a pre-trained CNN (Convolutional Neural Networks) classification model for gesture information recognition, and a gesture prediction result is output, wherein the gesture prediction result includes: relaxation, fist, open hand, wrist adduction, wrist abduction, radial flexion and ulnar flexion.

[0072] It can be understood that, in this embodiment, the CNN classification model is a convolutional neural network based on ICEEMDAN (Intrinsic Computing Expressive Empirical Mode Decomposition With Adaptive Noise) and a slow fusion model. The architecture of the model is as follows: Figure 2 As shown. When the sEMG signal is modally decomposed, the data is reconstructed in a frequency × time manner. When the signals come from an electrode array, these signals can be naturally reconstructed into a space × frequency × time pattern. Therefore, the motivation for using the slow fusion model in this embodiment is based on the similarity of the data structure between the video information and the decomposed sEMG signal.

[0073] like Figure 3As shown in the figure, for each window size (260ms) of sEMG data, the ICEEMDAN (Intrinsic Computing Expressive Empirical Mode Decomposition With Adaptive Noise) algorithm will output a 7×52 tensor after processing. The decomposed sEMG signal has higher resolution in the time domain and frequency domain. Then downsampling is performed with a factor of 0.25, and 0-order spline interpolation is used to reduce the computational load and overfitting risk of CNN during training and inference. The sEMG signals of the 8 channels of the MYO armband are processed in the same way, and finally a tensor of size 8×7×12 is obtained. In order to obtain an even number of time columns, the data in the last column is removed for the subsequent slow fusion process. At the same time, in order to make the data structure consistent with the video data, the scale is exchanged, and finally a tensor of 12×8×7 (time×channel×frequency) is obtained.

[0074] Step S30, generating a control instruction according to the eye movement prediction result and the gesture prediction result, and controlling the movement of the mechanical arm of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction.

[0075] Specifically, the control instructions include a first control instruction and a second control instruction; wherein the first control instruction includes: a stop instruction, a start instruction, an acceleration instruction and a deceleration instruction; the second control instruction includes: a fist clenching instruction, a hand opening instruction, a wrist adduction instruction, a wrist abduction instruction, a radial flexion instruction and an ulnar flexion instruction.

[0076] The step of generating a control instruction according to the eye movement prediction result and the gesture prediction result, and controlling the movement of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction specifically includes:

[0077] Acquire the eye movement prediction result and the gesture prediction result; if the eye movement prediction result is a double blink, generate the stop instruction, and stop the movement of the robotic arm according to the stop instruction; if the eye movement prediction result is a triple blink, generate the start instruction, and start the movement of the robotic arm according to the start instruction; if the eye movement prediction result is a single left eye blink, generate the acceleration instruction, and increase the movement speed of the robotic arm according to the acceleration instruction; if the eye movement prediction result is a single right eye blink, generate the deceleration instruction, and reduce the movement speed of the robotic arm according to the deceleration instruction; if the eye movement prediction result is a single simultaneous blink, do not generate an instruction.

[0078] If the gesture prediction result is a fist, the fist instruction is generated, and the end of the robotic arm is controlled to move vertically downward along the Z axis according to the fist instruction; if the gesture prediction result is an open hand, the open hand instruction is generated, and the end of the robotic arm is controlled to move vertically upward along the Z axis according to the open hand instruction; if the gesture prediction result is wrist adduction, the wrist adduction instruction is generated, and the end of the robotic arm is controlled to move along the negative direction of the Y axis in the horizontal plane according to the wrist adduction instruction; if the gesture prediction result is wrist abduction, the wrist abduction instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the Y axis in the horizontal plane according to the wrist abduction instruction; if the gesture prediction result is radial flexion, the radial flexion instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the X axis in the horizontal plane according to the radial flexion instruction; if the gesture prediction result is ulnar flexion, the ulnar flexion instruction is generated, and the robotic arm is controlled to move along the negative direction of the X axis in the horizontal plane according to the ulnar flexion instruction; if the gesture prediction result is relaxation, no instruction is generated.

[0079] It can be understood that when the eye movement prediction result is a single simultaneous blink, since single blinking is a daily physiological activity, no control instruction is generated in this embodiment; similarly, when the gesture prediction result is relaxation, since the hand is in a relaxed state is a basic daily gesture, no control instruction is generated in this embodiment.

[0080] Furthermore, the compliant control strategy is a general term for a class of technologies, which refers here to force-based impedance control. Force-based impedance control is essentially a force controller based on position feedback, which can freely adjust the dynamic relationship between position and interaction force by impedance control. The key is the design of the force controller. Generally, the expected dynamic relationship between the end interaction force and motion of the robot arm can be equivalent to a second-order mass-damper-spring system. In the upper limb rehabilitation robot system, the size of the interaction force between the user and the robot arm needs to be effectively limited to a certain range to achieve the tracking effect of the contact force.

[0081] It can be seen that the present invention has designed a method for identifying movement intentions, which can be used for rehabilitation training of users with unilateral upper limb motor dysfunction. This method combines the user's movement intention with the robot's motion control through sEMG-based gesture recognition and EOG-based blink recognition algorithm, effectively improving the user's participation in the rehabilitation process. The user only needs to wear a convenient and lightweight headband and armband to use electrooculogram and electromyography signals to control the movement direction and speed of the rehabilitation robot in real time, thereby efficiently participating in upper limb rehabilitation training.

[0082] Furthermore, if Figure 4As shown, based on the above-mentioned motion intention recognition method, the present invention also provides a motion intention recognition system accordingly, wherein the motion intention recognition system includes:

[0083] The eye movement result prediction module 41 is used to obtain the EOG signal of the target user's head collected by the electrooculogram collection headband, pre-process the EOG signal to obtain the target EOG signal, input the target EOG signal into a pre-trained SVM classifier to perform eye movement information recognition, and output the eye movement prediction result;

[0084] The gesture result prediction module 42 is used to obtain the sEMG signal of the upper limb of the target user collected by the electromyography collection armband, pre-process the sEMG signal to obtain the target sEMG signal, input the target sEMG signal into the pre-trained CNN classification model to perform gesture information recognition, and output the gesture prediction result;

[0085] The control instruction generation module 43 is used to generate a control instruction according to the eye movement prediction result and the gesture prediction result, and control the movement of the mechanical arm of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction.

[0086] Furthermore, if Figure 5 As shown, based on the above-mentioned motion intention recognition method and system, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 5 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0087] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a motion intention recognition program 40 is stored on the memory 20, and the motion intention recognition program 40 can be executed by the processor 10, thereby realizing the motion intention recognition method in the present application.

[0088] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the motion intention recognition method.

[0089] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0090] In one embodiment, when the processor 10 executes the motion intention recognition program 40 in the memory 20, the following steps are implemented:

[0091] Acquire the EOG signal of the target user's head collected by the electrooculogram collection headband, pre-process the EOG signal to obtain the target EOG signal, input the target EOG signal into a pre-trained SVM classifier to perform eye movement information recognition, and output the eye movement prediction result;

[0092] Acquire the sEMG signal of the upper limb of the target user collected by the electromyography acquisition armband, pre-process the sEMG signal to obtain the target sEMG signal, input the target sEMG signal into the pre-trained CNN classification model for gesture information recognition, and output the gesture prediction result;

[0093] A control instruction is generated according to the eye movement prediction result and the gesture prediction result, and according to the control instruction, the movement of the mechanical arm of the upper limb rehabilitation robot is controlled through a compliant control strategy.

[0094] The step of acquiring the EOG signal of the target user's head collected by the electrooculogram collection headband and preprocessing the EOG signal to obtain the target EOG signal specifically includes:

[0095] Obtain the EOG signal of the target user's head collected in real time by the TD05 electrooculogram acquisition headband, and perform a preset frequency power frequency filter and a Butterworth low-pass filter on the EOG signal to obtain an effective frequency band signal;

[0096] The effective frequency band signal is normalized to obtain a target EOG signal.

[0097] The eye movement prediction results include: single simultaneous blink, double blink, triple blink, single left eye blink and single right eye blink.

[0098] The step of acquiring the sEMG signal of the upper limb of the target user collected by the electromyography collection armband and preprocessing the sEMG signal to obtain the target sEMG signal specifically includes:

[0099] Acquire the sEMG signal of the upper limb of the target user collected in real time by the MYO electromyography acquisition armband, denoise the sEMG signal by improving the denoising algorithm of modal decomposition, and obtain different levels of modalities, wherein the MYO electromyography acquisition armband is an electromyography acquisition armband with a power frequency filtering function;

[0100] According to the different levels of the modalities, the modalities are reconstructed accordingly, and the reconstructed modalities are fused to obtain the target sEMG signal.

[0101] The gesture prediction results include: relaxation, clenched fist, open hand, wrist adduction, wrist abduction, radial flexion and ulnar flexion.

[0102] Wherein, the control instruction includes a first control instruction and a second control instruction;

[0103] Wherein, the first control instruction includes: a stop instruction, a start instruction, an acceleration instruction and a deceleration instruction;

[0104] The second control instructions include: a fist clenching instruction, a hand opening instruction, a wrist adduction instruction, a wrist abduction instruction, a radial flexion instruction and an ulnar flexion instruction.

[0105] The step of generating a control instruction according to the eye movement prediction result and the gesture prediction result, and controlling the movement of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction specifically includes:

[0106] Obtaining the eye movement prediction result and the gesture prediction result;

[0107] If the eye movement prediction result is a double blink, generating the stop instruction, and stopping the movement of the robotic arm according to the stop instruction;

[0108] If the eye movement prediction result is three blinks, generating the start instruction, and starting the movement of the robotic arm according to the start instruction;

[0109] If the eye movement prediction result is a single left eye blink, generating the acceleration instruction, and increasing the movement speed of the robotic arm according to the acceleration instruction;

[0110] If the eye movement prediction result is a single right eye blink, generating the deceleration instruction, and reducing the movement speed of the robotic arm according to the deceleration instruction;

[0111] If the eye movement prediction result is a single simultaneous blink, no instruction is generated;

[0112] If the gesture prediction result is a fist, the fist instruction is generated, and the end of the robotic arm is controlled to move vertically downward along the Z axis according to the fist instruction;

[0113] If the gesture prediction result is an open hand, the open hand instruction is generated, and the end of the robotic arm is controlled to move vertically upward along the Z axis according to the open hand instruction;

[0114] If the gesture prediction result is wrist adduction, the wrist adduction instruction is generated, and the end of the robotic arm is controlled to move along the negative direction of the Y axis in the horizontal plane according to the wrist adduction instruction;

[0115] If the gesture prediction result is wrist abduction, the wrist abduction instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the Y axis in the horizontal plane according to the wrist abduction instruction;

[0116] If the gesture prediction result is radial flexion, the radial flexion instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the X-axis in the horizontal plane according to the radial flexion instruction;

[0117] If the gesture prediction result is ulnar flexion, the ulnar flexion instruction is generated, and the robotic arm is controlled to move along the negative direction of the X-axis in the horizontal plane according to the ulnar flexion instruction;

[0118] If the gesture prediction result is relaxation, no instruction is generated.

[0119] In summary, the present invention provides a method, system and terminal for identifying movement intentions, the method comprising: obtaining EOG signals of the target user's head collected by an electrooculogram acquisition headband, preprocessing the EOG signals to obtain target EOG signals, inputting the EOG signals into an SVM classifier for eye movement information recognition, and outputting eye movement prediction results; obtaining sEMG signals of the target user's upper limbs collected by an electromyography acquisition armband, preprocessing the sEMG signals to obtain target sEMG signals, inputting the sEMG signals into a CNN classification model for gesture information recognition, and outputting gesture prediction results; generating control instructions according to the eye movement prediction results and the gesture prediction results, and controlling the movement of the mechanical arm of the upper limb rehabilitation robot through a compliant control strategy according to the control instructions. The present invention uses sEMG and EOG signals to control the movement direction and speed of the rehabilitation robot in real time, combines the user's movement intention with the robot's movement control, and effectively improves the user's participation in the rehabilitation process.

[0120] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0121] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0122] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for identifying motion intention, characterized in that: The motion intention recognition method comprises: Acquire the EOG signal of the target user's head collected by the electrooculogram collection headband, pre-process the EOG signal to obtain the target EOG signal, input the target EOG signal into a pre-trained SVM classifier to perform eye movement information recognition, and output the eye movement prediction result; Acquire the sEMG signal of the upper limb of the target user collected by the electromyography acquisition armband, pre-process the sEMG signal to obtain the target sEMG signal, input the target sEMG signal into the pre-trained CNN classification model for gesture information recognition, and output the gesture prediction result; generating a control instruction according to the eye movement prediction result and the gesture prediction result, and controlling the movement of the mechanical arm of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction; The gesture prediction results include: relaxation, clenched fist, open hand, wrist adduction, wrist abduction, radial flexion and ulnar flexion; The control instruction includes a first control instruction and a second control instruction; Wherein, the first control instruction includes: a stop instruction, a start instruction, an acceleration instruction and a deceleration instruction; The second control instructions include: a fist clenching instruction, a hand opening instruction, a wrist adduction instruction, a wrist abduction instruction, a radial flexion instruction, and an ulnar flexion instruction; The step of generating a control instruction according to the eye movement prediction result and the gesture prediction result, and controlling the movement of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction specifically includes: Obtaining the eye movement prediction result and the gesture prediction result; If the eye movement prediction result is a double blink, generating the stop instruction, and stopping the movement of the robotic arm according to the stop instruction; If the eye movement prediction result is three blinks, generating the start instruction, and starting the movement of the robotic arm according to the start instruction; If the eye movement prediction result is a single left eye blink, generating the acceleration instruction, and increasing the movement speed of the robotic arm according to the acceleration instruction; If the eye movement prediction result is a single right eye blink, generating the deceleration instruction, and reducing the movement speed of the robotic arm according to the deceleration instruction; If the eye movement prediction result is a single simultaneous blink, no instruction is generated; If the gesture prediction result is a fist, the fist instruction is generated, and the end of the robotic arm is controlled to move vertically downward along the Z axis according to the fist instruction; If the gesture prediction result is an open hand, the open hand instruction is generated, and the end of the robotic arm is controlled to move vertically upward along the Z axis according to the open hand instruction; If the gesture prediction result is wrist adduction, the wrist adduction instruction is generated, and the end of the robotic arm is controlled to move along the negative direction of the Y axis in the horizontal plane according to the wrist adduction instruction; If the gesture prediction result is wrist abduction, the wrist abduction instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the Y axis in the horizontal plane according to the wrist abduction instruction; If the gesture prediction result is radial flexion, the radial flexion instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the X-axis in the horizontal plane according to the radial flexion instruction; If the gesture prediction result is ulnar flexion, the ulnar flexion instruction is generated, and the robotic arm is controlled to move along the negative direction of the X-axis in the horizontal plane according to the ulnar flexion instruction; If the gesture prediction result is relaxation, no instruction is generated.

2. The method for identifying movement intention according to claim 1, characterized in that: The step of acquiring the EOG signal of the target user's head collected by the electrooculogram collection head ring and preprocessing the EOG signal to obtain the target EOG signal specifically includes: Obtain the EOG signal of the target user's head collected in real time by the TD05 electrooculogram acquisition headband, and perform a preset frequency power frequency filter and a Butterworth low-pass filter on the EOG signal to obtain an effective frequency band signal; The effective frequency band signal is normalized to obtain a target EOG signal.

3. The method for identifying movement intention according to claim 1, characterized in that: The eye movement prediction results include: a single simultaneous blink, a double blink, a triple blink, a single left eye blink and a single right eye blink.

4. The method for identifying movement intention according to claim 3, characterized in that: The step of acquiring the sEMG signal of the upper limb of the target user collected by the electromyography collection armband and preprocessing the sEMG signal to obtain the target sEMG signal specifically includes: Acquire the sEMG signal of the upper limb of the target user collected in real time by the MYO electromyography acquisition armband, denoise the sEMG signal by improving the denoising algorithm of modal decomposition, and obtain different levels of modalities, wherein the MYO electromyography acquisition armband is an electromyography acquisition armband with a power frequency filtering function; According to the different levels of the modalities, the modalities are reconstructed accordingly, and the reconstructed modalities are fused to obtain the target sEMG signal.

5. A motion intention recognition system, characterized in that: The motion intention recognition system comprises: An eye movement result prediction module is used to obtain the EOG signal of the target user's head collected by the electrooculogram collection headband, pre-process the EOG signal to obtain a target EOG signal, input the target EOG signal into a pre-trained SVM classifier to perform eye movement information recognition, and output an eye movement prediction result; A gesture result prediction module is used to obtain the sEMG signal of the upper limb of the target user collected by the electromyography collection armband, pre-process the sEMG signal to obtain a target sEMG signal, input the target sEMG signal into a pre-trained CNN classification model for gesture information recognition, and output a gesture prediction result; A control instruction generation module, used to generate a control instruction according to the eye movement prediction result and the gesture prediction result, and control the movement of the mechanical arm of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction; The gesture prediction results include: relaxation, clenched fist, open hand, wrist adduction, wrist abduction, radial flexion and ulnar flexion; The control instruction includes a first control instruction and a second control instruction; Wherein, the first control instruction includes: a stop instruction, a start instruction, an acceleration instruction and a deceleration instruction; The second control instructions include: a fist clenching instruction, a hand opening instruction, a wrist adduction instruction, a wrist abduction instruction, a radial flexion instruction, and an ulnar flexion instruction; The step of generating a control instruction according to the eye movement prediction result and the gesture prediction result, and controlling the movement of the upper limb rehabilitation robot through a compliant control strategy according to the control instruction specifically includes: Obtaining the eye movement prediction result and the gesture prediction result; If the eye movement prediction result is a double blink, generating the stop instruction, and stopping the movement of the robotic arm according to the stop instruction; If the eye movement prediction result is three blinks, generating the start instruction, and starting the movement of the robotic arm according to the start instruction; If the eye movement prediction result is a single left eye blink, generating the acceleration instruction, and increasing the movement speed of the robotic arm according to the acceleration instruction; If the eye movement prediction result is a single right eye blink, generating the deceleration instruction, and reducing the movement speed of the robotic arm according to the deceleration instruction; If the eye movement prediction result is a single simultaneous blink, no instruction is generated; If the gesture prediction result is a fist, the fist instruction is generated, and the end of the robotic arm is controlled to move vertically downward along the Z axis according to the fist instruction; If the gesture prediction result is an open hand, the open hand instruction is generated, and the end of the robotic arm is controlled to move vertically upward along the Z axis according to the open hand instruction; If the gesture prediction result is wrist adduction, the wrist adduction instruction is generated, and the end of the robotic arm is controlled to move along the negative direction of the Y axis in the horizontal plane according to the wrist adduction instruction; If the gesture prediction result is wrist abduction, the wrist abduction instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the Y axis in the horizontal plane according to the wrist abduction instruction; If the gesture prediction result is radial flexion, the radial flexion instruction is generated, and the end of the robotic arm is controlled to move along the positive direction of the X-axis in the horizontal plane according to the radial flexion instruction; If the gesture prediction result is ulnar flexion, the ulnar flexion instruction is generated, and the robotic arm is controlled to move along the negative direction of the X-axis in the horizontal plane according to the ulnar flexion instruction; If the gesture prediction result is relaxation, no instruction is generated.

6. A terminal, characterized in that: The terminal includes: a memory, a processor, and a motion intention recognition program stored in the memory and executable on the processor. When the motion intention recognition program is executed by the processor, the steps of the motion intention recognition method as described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a motion intention recognition program, and when the motion intention recognition program is executed by a processor, the steps of the motion intention recognition method according to any one of claims 1 to 4 are implemented.