Hand exoskeleton device and system and control method thereof

Abstract

The invention discloses a hand exoskeleton device and system and a control method thereof, and belongs to the technical field of rehabilitation training devices.The hand exoskeleton device comprises an exoskeleton and an embedded control assembly, and the exoskeleton comprises a connecting rod mechanism, a linear driving assembly and a base; the linear driving assembly is installed on the base and is in signal connection with the embedded control assembly, the embedded control assembly is further in signal connection with the signal collecting device used for collecting brain signals of a user, and one end of the connecting rod mechanism is hinged to the telescopic end of the linear driving assembly. The middle of the connecting rod mechanism is hinged to the base through a matching structure of an arc-shaped long-strip-shaped hole and a pin shaft, the tail end, away from the linear driving assembly, of the connecting rod mechanism is used for being fixed to fingers of a user, and the base is used for covering the hand back of the user. The device can perform hand rehabilitation training according to the consciousness of the user, and compared with a rehabilitation training device which can only be implemented by relying on other people, the device can reduce the requirement for auxiliary personnel, and the user can conveniently perform autonomous rehabilitation training.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation training devices, and in particular to a hand exoskeleton device, system and control method thereof. Background Art

[0002] Brain injury, degenerative diseases and various neurological diseases may cause hand motor dysfunction. Compared with the extensive movements of large limbs, hand movements mainly involve fine movements of small muscle groups in the hands, fingers, wrists and other parts, which strongly rely on the participation and cooperation of multiple cognitive functions such as perception, attention, and visual motor integration. The neural circuits and physiological mechanisms behind them are more complex. Traditional hand movement rehabilitation methods are mainly based on limb physical rehabilitation assisted by rehabilitation therapists. The treatment cost is high and it is not suitable for the current situation where there are a large number of patients with hand movement disorders and many needs. At the same time, rehabilitation therapists mainly help patients complete the required hand movements, and carry out passive rehabilitation training, which is difficult to induce neural plasticity and achieve effective recovery of brain-related motor circuits. Therefore, the rehabilitation process is long and the rehabilitation effect is limited.

[0003] In recent years, with the development of robotics technology, wearable exoskeleton rehabilitation devices can drive hand joint movements through specific mechanical structures, and to a certain extent can assist or replace rehabilitation therapists in providing rehabilitation treatment for patients with hand movement disorders. Most of the hand exoskeleton devices currently on the market do not have controllable and autonomous control solutions, so patients cannot achieve autonomous rehabilitation training and can only rely on physiotherapists or family members to assist in operation, and the degree of autonomous rehabilitation is insufficient.

[0004] Furthermore, by combining brain-computer interface technology with exoskeleton rehabilitation devices, the patient's brain signals can be decoded and converted into control signals for the exoskeleton device, so that the patient can control the hand exoskeleton device to complete the desired movement movements by actively adjusting his own neural signals. This not only helps to improve the patient's sense of participation and enthusiasm for rehabilitation training, but also helps to promote the reconstruction and enhancement of sensory-motor related neural circuits and improve rehabilitation efficiency.

[0005] In addition, most of the existing exoskeleton rehabilitation systems based on brain-computer interface technology collect scalp electroencephalogram (EEG) signals during user rehabilitation training. The acquisition operation is relatively complicated, and the signals are easily disturbed by motion artifacts, which limits their clinical application to a certain extent. Therefore, there is an urgent need for an exoskeleton rehabilitation system that is more convenient to collect and less susceptible to interference from the user's limb and head movements. Summary of the invention

[0006] The purpose of the present invention is to provide a hand exoskeleton device, system and control method thereof to address the defects and shortcomings in the prior art, so as to achieve the technical effect of allowing users to train their fingers according to their own wishes and perform rehabilitation treatment.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] In a first aspect, the present invention provides a hand exoskeleton device, comprising an exoskeleton and an embedded control component, wherein the exoskeleton comprises a connecting rod mechanism, a linear drive component and a base;

[0009] The linear drive component is installed on the base and is connected to the embedded control component by signal. The embedded control component is also connected to the signal acquisition device for acquiring user brain signals. One end of the connecting rod mechanism is hinged to the telescopic end of the linear drive component. The middle part of the connecting rod mechanism is hinged to the base through the matching structure of the arc-shaped long strip hole and the pin shaft. The end of the connecting rod mechanism away from the linear drive component is used to be fixed on the user's finger, and the base is used to cover the back of the user's hand.

[0010] Preferably, the linear drive assembly is a linear motor.

[0011] Preferably, a first bracket and a second bracket are provided on the base, and the linear drive assembly is detachably connected to the first bracket;

[0012] An arc-shaped long hole is provided in the middle of the connecting rod mechanism, and the arc-shaped long hole extends along the length direction of the connecting rod. The distance between the centers of circles where the two ends of the arc-shaped long hole are located is adapted to the length required for curling and relaxing the fingers, and the second bracket is hinged to the arc-shaped long hole.

[0013] Preferably, the number of the connecting rod mechanisms is any one from one to five, and the number of the linear drive components and the number of the first brackets and the second brackets on the base are all adapted to the number of the connecting rod mechanisms.

[0014] Preferably, the end of the connecting rod mechanism away from the linear drive assembly is mounted on the user's finger through a first connecting assembly, and the first connecting assembly is an easily tearable adhesive device or a detachable strap;

[0015] The base is mounted on the back of the user's hand via a second connecting component, and the second connecting component is also an easily tearable adhesive device or a strap that is easy to disassemble.

[0016] Preferably, a soft protective layer is provided on a side of the base close to the back of the hand.

[0017] In a second aspect, the present invention further provides a hand exoskeleton system, including a signal acquisition module, a signal processing module, a motion execution module and a human-computer interaction module;

[0018] The human-computer interaction module is used to: send a collection instruction to the signal collection module when the user performs a motor imagery task;

[0019] The signal acquisition module is used to: after receiving the acquisition instruction, acquire the target fNIRS signal of the cerebral cortex of the user when performing the motor imagery task;

[0020] The signal processing module is used to determine the motion control instruction according to the target fNIRS signal;

[0021] The motion execution module is used to control the hand exoskeleton device described in the first aspect to execute the action corresponding to the motion control instruction according to the motion control instruction.

[0022] Preferably, the hand exoskeleton system includes an online control submodule; the online control submodule is used to:

[0023] Preprocessing the target fNIRS signal to obtain a target time series signal; the target time series signal includes a time series signal of oxygenated hemoglobin and a time series signal of deoxygenated hemoglobin;

[0024] Extracting features of the target time series signal to obtain target neural signal features;

[0025] Inputting the target neural signal feature into a trained movement intention recognition model to obtain a movement intention recognition result;

[0026] The motion intention recognition result is converted into a motion control instruction, and the motion control instruction is sent to the motion execution module.

[0027] Preferably, the hand exoskeleton system further includes an offline training submodule; the offline training submodule is used to:

[0028] Acquire a sample data set; the sample data set includes a plurality of sample fNIRS signals and an action label corresponding to each of the sample fNIRS signals;

[0029] Preprocessing the sample fNIRS signal to obtain a sample timing signal; the sample timing signal includes a timing signal of oxygenated hemoglobin and a timing signal of deoxygenated hemoglobin;

[0030] Extracting features from the sample time series signal to obtain sample neural signal features;

[0031] The sample neural signal features and the action labels corresponding to the sample neural signal features are used to train the movement intention recognition model to obtain a trained movement intention recognition model.

[0032] In a third aspect, the present invention further provides a method for controlling the hand exoskeleton system according to the second aspect, the control method comprising:

[0033] Acquire a sample data set; the sample data set includes a plurality of sample fNIRS signals and an action label corresponding to each of the sample fNIRS signals;

[0034] Using the sample data set to train the motion intention recognition model to obtain a trained motion intention recognition model;

[0035] Collecting a target fNIRS signal of the cerebral cortex of the user during motor imagery;

[0036] determining a movement control instruction according to the target fNIRS signal;

[0037] According to the motion control instruction, the hand exoskeleton device described in the first aspect is controlled to perform an action corresponding to the motion control instruction.

[0038] Compared with the prior art, the present invention has achieved the following beneficial effects:

[0039] 1. The present invention sets an exoskeleton and an embedded control component. The exoskeleton includes a connecting rod mechanism, a linear drive component and a base. The linear drive component is signal-connected to the embedded control component. The embedded control component is also signal-connected to a signal acquisition device for acquiring user brain signals. The linear drive component is installed on the base and is hinged to one end of the connecting rod mechanism. The middle part of the connecting rod mechanism is hinged to the base. The end of the connecting rod mechanism away from the linear drive component is installed on the finger, and the base is installed on the back of the hand. The embedded control component can control the movement of the telescopic end of the linear drive component according to the collected brain signal, and then control the movement of the connecting rod mechanism. The fingers are flexed and extended under the drive of the connecting rod mechanism, which is convenient for helping users to perform autonomous rehabilitation training.

[0040] Other technical solutions of the present invention have achieved the following beneficial effects compared with the prior art:

[0041] 2. The present invention uses functional near-infrared spectroscopy (fNIRS) to obtain fNIRS signals of the cerebral cortex when the user is performing a motor imagery task. Functional near-infrared spectroscopy is a non-invasive optical brain imaging technology that uses the main components of blood to have good scattering properties for 600-900nm near-infrared light, and measures the concentration changes of oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) in the cerebral cortex tissue to reflect the activation of brain function. Compared with the need to apply physiological saline or conductive paste to collect EEG signals, fNIRS signals are easy to collect and are not easily disturbed by the user's limbs and head movements. The fNIRS-based brain-computer interface technology combined with a lightweight hand exoskeleton device allows users to actively participate in the rehabilitation process by controlling the hand exoskeleton device through motor imagery (MI), which helps to rebuild and rehabilitate the brain's motor neural circuits, improve the rehabilitation effect of hand motor function, and provide patients with hand movement disorders with a more active and convenient rehabilitation training program. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a schematic diagram of the overall structure of a hand exoskeleton device disclosed in a specific embodiment of the present invention;

[0044] Figure 2 for Figure 1 Schematic diagram of the structure of the exoskeleton;

[0045] Figure 3 for Figure 1 Schematic diagram of the structure of the middle base;

[0046] Figure 4 is a schematic diagram of the structure of an embedded control component;

[0047] Figure 5 A schematic diagram of the overall structure of a hand exoskeleton system disclosed in a specific embodiment of the present invention;

[0048] Figure 6 The probe arrangement position of the near-infrared spectral imaging device disclosed in a specific embodiment of the present invention;

[0049] Figure 7 The arrangement position of the near-infrared spectral imaging device probe in the brain region disclosed in a specific embodiment of the present invention;

[0050] Figure 8 The figure is a flow chart of a control method of a hand exoskeleton system disclosed in a specific embodiment of the present invention.

[0051] Among them, 1. exoskeleton; 2. embedded control component; 3. base; 4. linear drive component; 5. connecting rod mechanism; 6. first mounting seat; 7. through hole; 8. first bracket; 9. second mounting seat; 10. second bracket; 11. development board; 12. USB interface; 13. power supply; 14. relay; 15. protective layer. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] like Figures 1 to 4 As shown, the present invention provides a hand exoskeleton device, including an exoskeleton 1 and an embedded control component 2, the exoskeleton 1 includes a connecting rod mechanism 5, a linear drive component 4 for driving the connecting rod mechanism 5 to move, and a base 3; the linear drive component 4 is installed on the base 3 and is connected to the embedded control component 2 by signal, and the embedded control component 2 is also connected to the signal acquisition device for collecting user brain signals by signal, one end of the connecting rod mechanism 5 is hinged with the telescopic end of the linear drive component 4, the middle part of the connecting rod mechanism 5 is hinged with the base 3 through the matching structure of the arc-shaped long strip hole and the pin shaft, the end of the connecting rod mechanism 5 away from the linear drive component 4 is used to be fixed on the user's finger, and the base 3 is used to cover the back of the user's hand. When in use, the signal acquisition device collects the user's brain signal, and the embedded control component 2 controls the telescopic end of the linear drive component 4 to extend and retract according to the user's intention to drive the connecting rod mechanism 5 to move forward and backward (i.e., the length direction of the finger), so that the patient's finger can curl and relax.

[0055] In one embodiment of the present invention, the signal acquisition device includes a functional near-infrared spectroscopy imaging device and an acquisition head cap; the linear drive component 4 has various structural forms, such as an electric telescopic rod, a gear rack structure, a screw nut structure, etc. In one embodiment of the present invention, the linear drive component 4 is a linear motor, which realizes the curling and relaxation movement of the finger by controlling the forward and reverse rotation of the motor. The embedded control component 2 includes a development board 11, a relay 14, a USB interface 12 and a power supply 13, wherein the development board 11 is an esp32 development board, and of course other structures that can play the same role are also possible. The development board 11 is respectively connected to the relay 14, the USB interface 12 and the power supply 13, wherein the USB interface 12 is used to connect to the corresponding serial port of the computer.

[0056] The connection mode between the middle part of the connecting rod mechanism 5 and the base 3 can be that an arc-shaped long hole is opened on the base 3, and the middle part of the connecting rod mechanism 5 is hinged to the arc-shaped long hole through a pin shaft, or that an arc-shaped long hole is opened in the middle part of the connecting rod mechanism 5, and the base 3 is hinged to the arc-shaped long hole through a pin shaft. Regardless of which form is adopted, the arc-shaped long hole extends along the length direction of the connecting rod mechanism 5, and the distance between the centers of the circles where the two ends of the arc-shaped long hole are located is adapted to the length required for the curling and relaxation of the finger, so that the hinge point between the middle part of the connecting rod mechanism 5 and the base 3 moves along the length direction of the connecting rod mechanism 5 (that is, the length direction of the finger). In one embodiment of the present invention, an arc-shaped long hole is opened in the middle part of the connecting rod mechanism 5, and a second bracket 10 is provided on the base 3, and the pin shaft passes through the second bracket 10 and the arc-shaped long hole so that the middle part of the connecting rod mechanism 5 is hinged to the base 3. A first bracket 8 is also provided on the base 3 , and a housing of the linear drive assembly 4 is hinged to the first bracket 8 so as to install the linear drive assembly 4 on the base 3 .

[0057] The number of the connecting rod mechanisms 5 is any one from one to five, and the number of the linear drive components 4 and the number of the first brackets 8 and the second brackets 10 on the base 3 are all adapted to the number of the connecting rod mechanisms 5. In actual use, the number of the connecting rod mechanisms 5 can be set according to the needs of the user. Figures 1 to 3 In the illustrated embodiment, three groups of connecting rod mechanisms 5 are provided, and correspondingly three linear drive assemblies 4 and three groups of first brackets 8 and second brackets 10 are provided.

[0058] The end of the connecting rod mechanism 5 away from the linear drive component 4 is fixed to the user's finger through a first connecting component (not shown in the figure), and the first connecting component is an easily torn adhesive device (such as a structure similar to medical tape) or a bandage that is easy to disassemble, wherein the bandage that is easy to disassemble can be a bandage with Velcro at both ends, or a bandage with a buckle structure that is interlocked at both ends. The base 3 is covered on the back of the user's hand through a second connecting component (not shown in the figure), and the second connecting component is also an easily torn adhesive device or a bandage that is easy to disassemble. In one embodiment of the present invention, the first connecting component and the second connecting component are both bandages with Velcro at both ends, and the end of the connecting rod mechanism 5 close to the finger is provided with an arc-shaped first mounting seat 6, and the end of the base 3 close to the wrist is provided with a plate-shaped second mounting seat 9, and the first mounting seat 6 and the second mounting seat 9 are both provided with through holes 7 for passing the strap.

[0059] To improve wearing comfort, a soft and skin-friendly protective layer 15 is provided on the side of the base 3 close to the back of the user's hand. The protective layer 15 can be a sheet structure whose size matches that of the base 3, or a glove-like structure. In one embodiment of the present invention, the protective layer 15 is a glove woven from a flexible fabric. In addition, to reduce the weight of the hand exoskeleton device and further improve the wearing comfort of the user, the exoskeleton 1 is made of a lightweight material, which can be a lightweight metal or plastic. In one embodiment of the present invention, the exoskeleton 1 is printed from a resin material.

[0060] In an exemplary embodiment, Figure 5 As shown, the present invention provides a hand exoskeleton system, including a signal acquisition module, a signal processing module, a motion execution module and a human-computer interaction module. The signal acquisition module and the signal processing module communicate via TCP, the signal processing module and the motion execution module communicate via a serial port, and the human-computer interaction module and the signal acquisition module communicate via a serial port.

[0061] The human-computer interaction module is used to send acquisition instructions to the signal acquisition module when the user performs a motor imagery task.

[0062] The signal acquisition module is used to: after receiving an acquisition instruction, acquire the target fNIRS signal of the cerebral cortex when the user performs a motor imagery task.

[0063] A signal processing module is used to determine movement control instructions based on the target fNIRS signal.

[0064] The motion execution module is used to control the hand exoskeleton device to execute the action corresponding to the motion control instruction according to the motion control instruction.

[0065] The hand exoskeleton system extracts features from the functional near-infrared spectral signals of the user's cerebral cortex, identifies the current user's motion control intention and converts it into control instructions for the hand exoskeleton device. Based on the hand exoskeleton system, the user can actively control the exoskeleton device worn on the affected hand through motor imagination, which helps to induce the plasticity of the motor neural circuit and promote the effect of hand rehabilitation movement.

[0066] In the present invention, the hand rehabilitation training is divided into two parts: offline training and online control training. Correspondingly, the hand exoskeleton system also includes an offline training submodule and an online control submodule.

[0067] Among them, the offline training submodule is used to:

[0068] Acquire a sample data set; the sample data set includes a number of sample fNIRS signals and an action label corresponding to each sample fNIRS signal;

[0069] Preprocessing the sample fNIRS signal to obtain a sample time series signal; the sample time series signal includes a time series signal of oxygenated hemoglobin and a time series signal of deoxygenated hemoglobin;

[0070] Extract features from sample time series signals to obtain sample neural signal features;

[0071] The motion intention recognition model is trained using the sample neural signal features and the action labels corresponding to the sample neural signal features to obtain a trained motion intention recognition model.

[0072] Online control submodule, used for:

[0073] Preprocessing the target fNIRS signal to obtain a target time series signal; the target time series signal includes a time series signal of oxygenated hemoglobin and a time series signal of deoxygenated hemoglobin;

[0074] Extract features of the target time series signal to obtain target neural signal features;

[0075] Input the target neural signal features into the trained motion intention recognition model to obtain the motion intention recognition result;

[0076] The motion intention recognition results are converted into motion control instructions, and the motion control instructions are sent to the motion execution module.

[0077] The human-computer interaction module sends a flag bit (e.g., number 01) to the signal acquisition module at the start of motor imagination to mark the start of the task in each trial in the fNIRS signal timing sequence. The signal acquisition module transmits the collected fNIRS signal and task label information to the signal processing module. The signal processing module analyzes and processes the received fNIRS signal, obtains the recognition result of whether the user has the intention of motor control in the current trial, converts the recognition result into a motion execution instruction, and transmits the motion execution instruction to the motion execution module. The motion execution module receives the control instruction from the signal processing module and controls the hand exoskeleton device to perform the corresponding action.

[0078] The signal acquisition module includes a functional near-infrared spectroscopy imaging device and a collection head cap, which collects the fNIRS signals of the user's cerebral cortex in real time during offline training and online exoskeleton control. When collecting fNIRS signals from the user's brain, a special collection head cap is worn on the user's head. The head cap is equipped with a near-infrared light source transmitting probe position S and a receiving light source absorption probe position D, as shown in the figure. Figure 6 As shown in the figure, the head cap is equipped with 18 light source emitters S and 19 detectors D, which are symmetrically distributed on the left and right sides of the head cap. The distance between the light source emitters and detectors is 3 cm. A measurement channel is formed between adjacent light source emitters and detectors, with a total of 64 measurement channels. The arrangement of the measurement channels refers to the international 10-20 system. Figure 6 The detector marked R in the figure corresponds to the Cz position in the international 10-20 system. Figure 7 As shown in the figure, according to the Brodmann partition, 64 measurement channels are distributed in the prefrontal lobe (BA9, BA10, BA11) and sensorimotor cortex of both sides of the brain, including the primary motor area (M1, BA4), premotor cortex (PMC) and supplementary motor area (SMA, BA8, BA6) of the precentral gyrus, and the primary somatosensory area (S1, BA3, BA1, BA2) of the postcentral gyrus. The signal acquisition module uses three wavelengths of 740nm, 808nm and 850nm near-infrared light as the light source, collects fNIRS signals on all measurement channels at a sampling frequency of 8Hz, and sends the collected signals to the signal processing module. For the offline training process, the signal acquisition module transmits the collected sample fNIRS signals to the signal processing module after all offline training trials are completed; for the online control training process, the signal acquisition module transmits the collected sample fNIRS signals to the signal processing module in real time.

[0079] The signal processing module is used to preprocess, extract features and identify movement intentions of the received fNIRS signals. Specifically, in offline training, the sample fNIRS signals are preprocessed, feature extracted and screened, and the movement intention recognition model is trained; in online control training, the signals are preprocessed, feature extracted, movement intention recognized, and the movement intention recognition results are converted into movement control instructions and transmitted to the movement execution module. The specific process of offline training and online control training is as follows:

[0080] The user first performs offline training, and the specific process includes the following steps 101 to 103.

[0081] Step 101: The user wears an fNIRS acquisition head cap, sits in front of the display device of the human-computer interaction module, and performs several trials of motor imagery / relaxation tasks according to the prompts; in this embodiment, 80 trials are used, 40 times of motor imagery / relaxation tasks each, and the order is random. While the user is performing the task, the signal acquisition module collects the user's sample fNIRS signal, and after all trials are completed, the collected sample fNIRS signal and the task label of each trial (i.e., whether the motor imagery task or relaxation task is performed) are sent to the signal processing module.

[0082] Step 102: The signal processing module preprocesses the sample fNIRS signal to obtain a sample time series signal, including the time series signals of oxyhemoglobin (HbO) and deoxyhemoglobin (HbR), and extracts a number of sample neural signal features from the time series signals of HbO and HbR.

[0083] The specific method of preprocessing is as follows: for each measurement channel, first, convert the sample fNIRS signal into optical density data; use the sliding window method to detect the motion artifacts contained in the optical density data, and use the spline interpolation method to correct the detected data segments containing motion artifacts to obtain the corrected optical density data; then, perform bandpass filtering on the corrected optical density data (specifically, 0.01-0.2 Hz bandpass filtering can be used) to remove the system noise and human global physiological noise that may be contained in the optical density data; finally, use the modified Beer-Lambert law to calculate the time curve of HbO and HbR concentrations on each measurement channel, and further perform baseline correction by subtracting the average HbO and HbR concentrations 2s before the start of the task from the HbO and HbR concentration values ​​of each trial, and obtain the time series signals of HbO and HbR concentrations within each trial duration after baseline correction. The time series signals of HbO and HbR concentrations within each trial duration constitute the sample time series signals corresponding to each trial. The above preprocessing process is implemented through the GUI provided by the built-in open source toolkit HOMER3 in MATLAB.

[0084] Feature extraction and feature screening are performed on the sample time series signals obtained after preprocessing to obtain a sample neural signal feature set related to brain movement intention, and the sample neural signal feature set includes several sample neural signal features.

[0085] The specific method of feature extraction is as follows: for the HbO and HbR concentration time series signals on each measurement channel obtained after preprocessing, the average value of the HbO and HbR concentration signals within each trial duration (specifically, the average value of the concentration within the time window of 5-20s after the start of the task), peak value, variance, slope and other channel-level time domain features are calculated; further, based on the average value of the HbO and HbR concentration signals at the measurement channel level, the mean values ​​of the HbO and HbR concentration signals in the left prefrontal lobe, right prefrontal lobe, left sensorimotor area and right sensorimotor area are calculated as the features at the brain region level; further, based on the mean values ​​of the HbO and HbR concentrations at the brain region level, the Pearson correlation coefficient method is used to calculate the correlation coefficients between each brain region on the mean values ​​of HbO and HbR concentrations as the features across brain regions. Finally, a statistical analysis method is used to analyze the trials of the motor imagery task and the relaxation task to see whether there are statistically significant differences in all the features extracted above. Features with p values ​​less than or close to 0.05 obtained by statistical analysis are selected as sample neural signal features for constructing a motor intention recognition model in step 103.

[0086] Step 103: Based on the extracted sample neural signal features and task label data (task label data is action label), a machine learning method is used to build a user's movement intention recognition model.

[0087] The specific method is as follows: add the action label data corresponding to each trial to the sample neural signal features screened in step 102 (1 for performing the motor imagery task and 0 for performing the relaxation task) to construct a complete labeled data set; then, randomly divide the data set into a training set (90%) and a validation set (10%) according to a certain ratio, respectively use the support vector machine (SVM) and linear discriminant analysis (LDA) algorithms, adopt a 10-fold cross validation method, train and test the model, and take the average classification accuracy obtained from 10 tests as the final classification accuracy. Compare the final classification results of the SVM and LDA algorithms, and select the model with a higher classification result as the movement intention recognition model for the subsequent online training process.

[0088] After obtaining the user's trained movement intention recognition model, online control training is performed. In the online control training, the user wears the fNIRS acquisition head cap in the signal acquisition module and the hand exoskeleton device in the movement execution module, and performs the movement imagination task according to the task prompt information given by the display device in the human-computer interaction module. The signal processing module receives the sample fNIRS signal sent by the signal acquisition module, pre-processes the signal, extracts features and recognizes movement intention, converts the movement intention recognition result into a movement control instruction, and transmits it to the movement execution module. The specific process includes the following steps 201 to 203.

[0089] Step 201: The user wears an fNIRS acquisition head cap and a hand exoskeleton device, sits in front of the display device of the human-computer interaction module, and performs a motor imagery / relaxation task according to the prompts; while the user is performing the task, the signal acquisition module collects the user's target fNIRS signal in real time and sends it to the signal processing module.

[0090] Step 202: The signal processing module preprocesses the received target fNIRS signal to obtain a target timing signal, which includes timing signals of oxyhemoglobin (HbO) and deoxyhemoglobin (HbR), and extracts several specific target neural signal features from the timing signals of HbO and HbR.

[0091] The specific technical solution adopted is the same as that of steps 101 and 102 of the offline training process. Real-time preprocessing is achieved by calling the corresponding function programming provided by the open source toolkit HOMER3 in MATLAB, which will not be repeated here.

[0092] The preprocessed target fNIRS signal is subjected to feature extraction to extract the target neural signal feature. This process is the same as step 102 and will not be described in detail here.

[0093] Step 203: Based on the extracted target neural signal features and the trained motion intention recognition model of the user obtained through offline training, determine whether the current user has motion control intention (i.e., whether he is imagining hand movements), and obtain a motion intention recognition result. If the motion intention recognition result is that the user has motion control intention, a motion control instruction is generated and sent to the motion execution module, and the motion execution module controls the hand exoskeleton device to perform a grasping action; if the motion intention recognition result is that the user has no motion control intention, no motion control instruction is sent to the motion execution module.

[0094] The extracted target neural signal features are input into the trained motion intention recognition model obtained in step 103, the target neural signal features are decoded to obtain the motion intention recognition result, the motion intention recognition result is converted into a digital representation (1 indicates that the motion intention recognition result is that the user has performed a motor imagination task, and 0 indicates that the motion intention recognition result is that the motor imagination task has not been performed) as a motion control instruction, and the motion control instruction is sent to the motion execution module.

[0095] Step 202 and step 203 are for one trial, and the online training process includes several trials. The system repeats the above steps 202 and step 203 until the training of all trials is completed. In this embodiment, the online training process includes 60 trials, including 40 motor imagery tasks and 20 relaxation tasks, and the presentation order is random.

[0096] The motion execution module is a lightweight hand exoskeleton device, which is worn on the hand of the user who needs to be trained. The control is realized by connecting to the hand exoskeleton device through the corresponding serial port of the computer. The functional near-infrared spectroscopy imaging device in the signal acquisition module transmits the collected user target fNIRS signal to the signal processing module, which reads and processes the target fNIRS signal, identifies the user's motion intention according to the processing result of the target fNIRS signal (target neural signal characteristics), and sends the motion control instruction corresponding to the motion intention recognition result to the corresponding serial port, and realizes communication through the development board 11; the development board 11 receives the signal from the corresponding serial port, controls the forward and reverse rotation of the current according to the signal, and then controls the exoskeleton.

[0097] After receiving the motion control instruction, the motion execution module drives the user's hand to complete the corresponding motion action. Specifically, after receiving the "1" motion control instruction sent by the signal processing module, the motion execution module drives the hand exoskeleton device to perform two consecutive grasping actions. If a "0" motion control instruction is received, no action is performed. Steps 202 to 203 are performed once for each trial, that is, the system recognizes the user's motion intention once for each trial and drives the exoskeleton device to perform the action.

[0098] The human-computer interaction module is used to give users visual prompts about tasks and training progress during offline training and online training. In offline training and online control training, users imagine that their hands perform corresponding actions according to the prompt pictures they see, or do not perform motor imagery (i.e., perform relaxation tasks). The human-computer interaction module includes a computer and a display device, and the display device can be a screen or other display devices such as a virtual reality head-mounted display. A 27-inch computer screen is used in this embodiment. Offline training includes 80 trials, and users perform motor imagery or relaxation tasks, 40 trials each, and the presentation order is random. Online training includes 60 trials, and users perform motor imagery tasks 40 times and relaxation tasks 20 times, and the presentation order is random. Each trial includes 3 stages: a 5s preparation period, a 20s motor imagery period, and a 15s rest period. At the beginning of each trial, the text "Please prepare" appears in the center of the screen for 5 seconds, prompting the user that the picture is about to appear and to prepare to perform the task; then, the prompt picture corresponding to the task is displayed on the screen for 20 seconds. When the animation of repeated fist clenching appears on the screen, the user concentrates on imagining that his hand performs the corresponding repeated fist clenching action, and imagines it for 20 seconds until the animation ends. If a static picture of an open palm appears on the screen, the user does not perform motor imagination (that is, just perform the relaxation task); after that, the fist clenching animation or the picture of the open palm on the screen disappears, and a fixed cross fixation point appears in the center of the screen. The user's eyes are fixed on the cross position and remain still for 15 seconds. After all trials are completed, the text "Experiment End" appears on the screen.

[0099] At the moment when motor imagery begins in each trial (i.e., when the fist-clenching animation appears on the screen), the human-computer interaction module sends the number "01" to the signal acquisition module through the serial port as a flag for the start of motor imagery, marking the start of motor imagery in the fNIRS timing signal to facilitate subsequent signal processing.

[0100] The hand exoskeleton system described above can guide the user to control the hand exoskeleton device to perform corresponding movement movements by imagining the hand performing grasping movements, helping the user to actively participate in hand rehabilitation training, and helping to promote the reconstruction and rehabilitation of the brain's motor neural circuits.

[0101] Based on the same inventive concept, the present application also provides a control method for a hand exoskeleton system. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme described in the above hand exoskeleton system, so the specific limitations of one or more control method embodiments of hand exoskeleton systems provided below can refer to the limitations of hand exoskeleton systems above, and will not be repeated here.

[0102] In an exemplary embodiment, Figure 8As shown, the present invention provides a control method for a hand exoskeleton system, including the following steps 301 to 305.

[0103] Step 301: Obtain a sample data set; the sample data set includes a number of sample fNIRS signals and an action label corresponding to each sample fNIRS signal.

[0104] Step 302: Use the sample data set to train the motion intention recognition model to obtain a trained motion intention recognition model.

[0105] Step 303: Collecting target fNIRS signals from the user's cerebral cortex during motor imagery.

[0106] Step 304: Determine a motion control instruction according to the target fNIRS signal.

[0107] Step 305: Control the hand exoskeleton device to execute the action corresponding to the motion control instruction according to the motion control instruction.

[0108] It should be noted that it is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any figure mark in the claims should not be regarded as limiting the claims involved.

Claims

1. A hand exoskeleton device, characterized in that: The invention comprises an exoskeleton and an embedded control component, wherein the exoskeleton comprises a linkage mechanism, a linear drive component and a base; The linear drive component is installed on the base and is connected to the embedded control component by signal. The embedded control component is also connected to the signal acquisition device for acquiring user brain signals. One end of the connecting rod mechanism is hinged to the telescopic end of the linear drive component. The middle part of the connecting rod mechanism is hinged to the base through the matching structure of the arc-shaped long strip hole and the pin shaft. The end of the connecting rod mechanism away from the linear drive component is used to be fixed on the user's finger, and the base is used to cover the back of the user's hand.

2. The hand exoskeleton device according to claim 1, characterized in that: The linear drive assembly is a linear motor.

3. The hand exoskeleton device according to claim 2, characterized in that: The base is provided with a first bracket and a second bracket, and the linear drive assembly is detachably connected to the first bracket; An arc-shaped long hole is provided in the middle of the connecting rod mechanism, and the arc-shaped long hole extends along the length direction of the connecting rod. The distance between the centers of circles where the two ends of the arc-shaped long hole are located is adapted to the length required for curling and relaxing the fingers, and the second bracket is hinged to the arc-shaped long hole.

4. The hand exoskeleton device according to claim 3, characterized in that: The number of the connecting rod mechanisms is any one from one to five, and the number of the linear drive components and the number of the first brackets and the second brackets on the base are all adapted to the number of the connecting rod mechanisms.

5. The hand exoskeleton device according to any one of claims 1 to 4, characterized in that: The end of the connecting rod mechanism away from the linear drive assembly is mounted on the user's finger through a first connecting assembly, and the first connecting assembly is an easily tearable adhesive device or a strap that is easy to disassemble; The base is mounted on the back of the user's hand via a second connecting component, and the second connecting component is also an easily tearable adhesive device or a strap that is easy to disassemble.

6. The hand exoskeleton device according to claim 1, characterized in that: A soft protective layer is arranged on a side of the base close to the back of the hand.

7. A hand exoskeleton system, characterized in that: It includes signal acquisition module, signal processing module, motion execution module and human-computer interaction module; The human-computer interaction module is used to: send a collection instruction to the signal collection module when the user performs a motor imagery task; The signal acquisition module is used to: after receiving the acquisition instruction, acquire the target fNIRS signal of the cerebral cortex of the user when performing the motor imagery task; The signal processing module is used to determine the motion control instruction according to the target fNIRS signal; The motion execution module is used to control the hand exoskeleton device according to claim 1 to execute the action corresponding to the motion control instruction according to the motion control instruction.

8. The hand exoskeleton system according to claim 7, characterized in that: The hand exoskeleton system includes an online control submodule; the online control submodule is used to: Preprocessing the target fNIRS signal to obtain a target time series signal; the target time series signal includes a time series signal of oxygenated hemoglobin and a time series signal of deoxygenated hemoglobin; Extracting features of the target time series signal to obtain target neural signal features; Inputting the target neural signal feature into a trained movement intention recognition model to obtain a movement intention recognition result; The motion intention recognition result is converted into a motion control instruction, and the motion control instruction is sent to the motion execution module.

9. The hand exoskeleton system according to claim 7, characterized in that: The hand exoskeleton system further includes an offline training submodule; the offline training submodule is used to: Acquire a sample data set; the sample data set includes a plurality of sample fNIRS signals and an action label corresponding to each of the sample fNIRS signals; Preprocessing the sample fNIRS signal to obtain a sample timing signal; the sample timing signal includes a timing signal of oxygenated hemoglobin and a timing signal of deoxygenated hemoglobin; Extracting features from the sample time series signal to obtain sample neural signal features; The sample neural signal features and the action labels corresponding to the sample neural signal features are used to train the movement intention recognition model to obtain a trained movement intention recognition model.

10. A method for controlling a hand exoskeleton system according to any one of claims 7 to 9, characterized in that: The control method comprises: Acquire a sample data set; the sample data set includes a plurality of sample fNIRS signals and an action label corresponding to each of the sample fNIRS signals; Using the sample data set to train the motion intention recognition model to obtain a trained motion intention recognition model; Collecting a target fNIRS signal of the cerebral cortex of the user during motor imagery; determining a movement control instruction according to the target fNIRS signal; According to the motion control instruction, the hand exoskeleton device of claim 1 is controlled to execute an action corresponding to the motion control instruction.

Images