Motion intention detection method, medical robot system, storage medium and product

By simultaneously collecting and processing EEG signals and combining the fusion decision of multi-objective particle swarm algorithm, the accuracy of medical robots in motion intention detection is improved, the problem of low accuracy of single signal detection is solved, and the recovery is improved The effectiveness and efficiency of training.

CN119026014APending Publication Date: 2024-11-26TIANJIN POLYTECHNIC UNIV +2
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410929821.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing medical robots rely on a single electrical signal in motion intention detection, resulting in low detection accuracy, especially poor rehabilitation training for patients with stroke and other diseases.

Method used

The electroencephalogram and electromyography signals are collected simultaneously, and feature extraction and classification are performed through pre-trained processing models, and fusion decisions are made in combination with the multi-objective particle swarm algorithm to improve the detection accuracy of motion intentions.

Benefits of technology

By fusing the advantages of EEG and EEM signals, the shortcomings of single signal detection are overcome, the accuracy of identifying exercise intentions is significantly improved, and the effectiveness and efficiency of rehabilitation training are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026014A_ABST
    Figure CN119026014A_ABST
Patent Text Reader

Abstract

The invention provides a motion intention detection method, a medical robot system, a storage medium and a product, and relates to the technical field of medical robotics.The method comprises the steps that electroencephalogram signals and electromyographic signals of a user are collected at the same time; processing the electroencephalogram signal to obtain a first detection result; processing the electromyographic signal to obtain a second detection result; and fusing the first detection result and the second detection result, and determining the motion intention according to a fusion result. The electroencephalogram signals and the electromyographic signals are subjected to advantage complementation in a fusion mode, the problem that the detection result is inaccurate due to the fact that the electroencephalogram signals are weak in strength and the electromyographic signals are prone to being affected by the muscle fatigue degree of the user is solved, and the exercise intention recognition accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of medical robot technology, and more specifically, to a motion intention detection method, a medical robot system, a storage medium and a product. Background Art

[0002] In recent years, the number of users in my country suffering from movement disorders due to diseases such as stroke has increased year by year, and the demand for rehabilitation training among these users is huge.

[0003] Traditional rehabilitation training is generally passive movement rehabilitation training, in which passive movement rehabilitation training refers to the user passively performing a large number of repetitive limb movements with the assistance of professional therapists. However, the effect of this passive movement rehabilitation training is not good. Therefore, in this case, active medical robots based on user movement intentions came into being.

[0004] Most of the current medical robots use a single electrical signal to detect movement intentions, and then assist users in performing limb movements based on the detected movement intentions. However, this method of detecting movement intentions using a single electrical signal has a low detection accuracy. Summary of the invention

[0005] The present application provides a motion intention detection method, a medical robot system, a storage medium and a product, so that the accuracy of the detected motion intention is higher.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a method for detecting motion intention is provided, which is applied to a medical robot system. The method includes: simultaneously collecting the user's electroencephalogram (EEG) signal and electromyography (EMG) signal; processing the EEG signal to obtain a first detection result; processing the EMG signal to obtain a second detection result; fusing the first detection result and the second detection result, and determining the motion intention based on the fusion result.

[0008] In one embodiment, the movement intention is determined based on the fusion result, including: when the fusion result is greater than a first threshold, the first limb movement intention is determined; when the fusion result is less than the first threshold, the second limb movement intention is determined; when the fusion result is equal to the first threshold, the resting state movement intention is determined.

[0009] In one embodiment, the first detection result includes a first probability value and a second probability value, the first probability value indicates the probability that the movement intention corresponding to the EEG signal is the first movement intention, and the second probability value indicates the probability that the movement intention corresponding to the EEG signal is the second movement intention. The EEG signal is processed to obtain the first detection result, including: calling a pre-trained EEG signal processing model to extract features of the EEG signal to obtain EEG signal features; and classifying the EEG signal features to obtain the first probability value and the second probability value.

[0010] In one of the embodiments, a pre-trained EEG signal processing model is called to perform feature extraction on the EEG signal to obtain EEG signal features, and the EEG signal features are classified to obtain a first probability value and a second probability value, including: filtering the EEG signal to obtain the filtered EEG signal; calling the pre-trained EEG signal processing model to perform feature extraction on the filtered EEG signal based on the common space pattern algorithm and the Riemann manifold method to obtain the EEG signal features; and classifying the extracted EEG signal features based on the SVM classifier to obtain the first probability value and the second probability value.

[0011] In one embodiment, the second detection result includes a third probability value and a fourth probability value, the third probability value indicates the probability that the movement intention corresponding to the electromyographic signal is the first movement intention, and the fourth probability value indicates the probability that the movement intention corresponding to the electromyographic signal is the second movement intention, and the electromyographic signal is processed to obtain the second detection result, including: calling a pre-trained electromyographic signal processing model to extract the time-frequency domain characteristics of the electromyographic signal based on the wavelet packet decomposition method; processing the time-frequency domain characteristics based on the residual network module to obtain the second detection result.

[0012] In one of the embodiments, the first detection result and the second detection result are fused, including: using a multi-objective particle swarm algorithm to determine a fusion coefficient with the goal of minimizing the mean absolute error and the mean square error; and fusing the first detection result and the second detection result according to the fusion coefficient.

[0013] In the second aspect, a medical robot system is provided, which includes: a signal acquisition module for simultaneously acquiring the user's EEG signal and EMG signal; a signal processing module for processing the EEG signal to obtain a first detection result; processing the EMG signal to obtain a second detection result; a fusion module for fusing the first detection result with the second detection result, and determining the motion intention based on the fusion result; and a motion control module for controlling the medical robot to perform corresponding actions based on the motion intention.

[0014] In one embodiment, the motion control module is specifically used to execute a preset upper limb rehabilitation exercise training mode, and the upper limb rehabilitation exercise training mode includes unilateral upper limb rehabilitation exercise and bilateral upper limb rehabilitation exercise, wherein unilateral upper limb rehabilitation exercise includes single-joint rehabilitation exercise and multi-joint rehabilitation exercise, single-joint rehabilitation exercise includes left elbow joint rehabilitation exercise, left shoulder joint rehabilitation exercise, right elbow joint rehabilitation exercise and right shoulder joint rehabilitation exercise, multi-joint rehabilitation exercise includes left shoulder and elbow joint rehabilitation exercise and right shoulder and elbow joint rehabilitation exercise; bilateral upper limb rehabilitation exercise includes bilateral same-joint rehabilitation exercise and bilateral multi-joint rehabilitation exercise, bilateral same-joint rehabilitation exercise includes bilateral shoulder joint rehabilitation exercise and bilateral elbow joint rehabilitation exercise, and bilateral multi-joint rehabilitation exercise includes bilateral shoulder and elbow joint rehabilitation exercise.

[0015] In a third aspect, a medical robot system is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is run by the processor, the motion intention detection method as described in any one of the above-mentioned first aspects is executed.

[0016] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is run on a processor, the motion intention detection method as described in any one of the first aspects is executed.

[0017] In a fifth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is run on a computer, the computer executes the motion intention detection method as described in any one of the above-mentioned first aspects.

[0018] In the motion intention detection method provided in the embodiment of the present application, the user's EEG signals and EMG signals are simultaneously collected, and then the EEG signals and EMG signals are processed, analyzed and fused for decision making. By combining the characteristics of the two different signals, accurate judgment of the user's motion intention is achieved, thereby improving the accuracy of the motion intention detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A block diagram showing a structure of a medical robot system provided in an embodiment of the present application.

[0020] Figure 2 A schematic diagram showing the structure of a computer device in a hardware operating environment involved in an embodiment of the present application is shown.

[0021] Figure 3 A flow chart of a motion intention detection method provided in an embodiment of the present application is shown.

[0022] Figure 4 A schematic diagram showing the location of EEG electrodes.

[0023] Figure 5 A schematic diagram showing the location of an EMG electrode.

[0024] Figure 6 A schematic diagram comparing Riemann distance and Euclidean distance is shown.

[0025] Figure 7 A schematic diagram of a ResNet-18 residual network structure is shown.

[0026] Figure 8 A structural diagram of a ResNet residual block integrating a compression-inspired attention mechanism is shown.

[0027] Fig. 9 A schematic diagram showing the classification of rehabilitation exercise training patterns.

[0028] Fig.10 A structural block diagram of a medical robot system provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0030] Stroke is a disease characterized by symptoms of cerebral ischemia and hemorrhagic injury, with extremely high mortality and disability rates. Stroke has an acute onset and high mortality rate, and is one of the most important fatal diseases in the world. According to the latest survey by the World Health Organization: Nearly 80% of Chinese stroke users suffer from motor dysfunction to varying degrees. According to the "China Stroke Report 2020", in recent years, the number of users in my country who have motor disorders due to diseases such as stroke has increased year by year, so rehabilitation treatment for such users is an urgent problem to be solved.

[0031] Traditional rehabilitation training is generally passive movement rehabilitation training, in which passive movement rehabilitation training refers to the user passively performing a large number of repetitive limb movements with the assistance of a professional therapist. However, the effect of this passive movement rehabilitation training is not good. Therefore, in this case, active medical robots based on user movement intentions have emerged. This medical robot can assist users in limb movement based on their movement intentions, and the rehabilitation effect and efficiency are significantly improved.

[0032] The working principle of the medical robot is to detect and identify the movement intention with the help of a single electrical signal, and control the corresponding limb movement according to the identified movement intention. However, the accuracy of movement intention based on a single electrical signal is low.

[0033] For example, movement intention is detected based on a single EEG signal (EEG signal), but EEG signals have problems such as weak signal strength, low signal-to-noise ratio, and susceptibility to the influence of other signals. Therefore, the accuracy of movement intention detection and recognition based on a single EEG signal is low.

[0034] For another example, the detection of movement intention is based on a single electromyographic signal (sEMG signal). However, the collection of electromyographic signals requires a high level of muscle movement ability of the user. When performing movement intention detection and recognition, the accuracy of the detection results drops significantly when facing users with severe movement disorders and lack of movement ability. In addition, the collection of electromyographic signals is also affected by the degree of muscle fatigue. When users collect electromyographic signals, changes in the degree of muscle fatigue may also lead to errors in the recognition of movement intention.

[0035] In order to solve the above technical problems, the embodiment of the present application provides a method for detecting movement intention, which includes simultaneously collecting the user's EEG signal and electromyographic signal; processing the EEG signal to obtain a first detection result; processing the electromyographic signal to obtain a second detection result; fusing the first detection result and the second detection result, and determining the movement intention according to the fusion result. By fusing the EEG signal and the electromyographic signal, the advantages of the signal are complemented, the problem of inaccurate detection results caused by weak EEG signal strength and the influence of the user's muscle fatigue on the electromyographic signal is overcome, and the recognition accuracy of the movement intention is improved.

[0036] Among them, the motion intention detection method can be applied to a medical robot system. The embodiment of the present application also provides a medical robot system, which includes a signal acquisition module, a signal processing module, a signal fusion module and a motion control module, wherein the signal acquisition module is used to simultaneously collect the user's EEG signal and electromyographic signal, the signal processing module is used to process the EEG signal to obtain a first detection result; the electromyographic signal is processed to obtain a second detection result; the signal fusion module fuses the first detection result and the second detection result, and determines the motion intention according to the fusion result, and the motion control module is used to control the medical robot to perform corresponding actions according to the motion intention, and the corresponding actions are such as limb movement actions. The medical robot system solves the problems of low accuracy of EEG signals and susceptibility of electromyographic signals to muscle fatigue by simultaneously collecting EEG signals and electromyographic signals for processing, analysis and fusion decision-making, realizes accurate judgment of the user's motion intention, and at the same time, coordinates the movement of the medical robot based on the judgment result, and performs rehabilitation treatment on the user according to the preset plan. Compared with traditional rehabilitation training methods, the rehabilitation effect is significantly improved, shortening the user's treatment cycle.

[0037] Reference Figure 1 , Figure 1A block diagram of the structure of a medical robot system provided in an embodiment of the present application is shown, wherein the medical robot system includes an electroencephalogram signal acquisition device, an electromyography signal acquisition device, an interface guidance device, a computer device, and a medical robot actuator.

[0038] The EEG signal acquisition device is used to acquire EEG signals, including a plurality of EEG electrodes, which are arranged according to the international standard 10-10 electrode positions, and transmits the acquired EEG signals to a computer device. Optionally, the EEG signal acquisition device is, for example, an EEG cap.

[0039] The electromyographic signal acquisition device is used to acquire electromyographic signals, and includes a plurality of electromyographic electrodes, which are arranged at the biceps and triceps of the bilateral upper limbs closely related to upper limb movements. The electromyographic signal acquisition device transmits the acquired electromyographic signals to a computer device.

[0040] The interface guidance device includes a display screen, wherein a guidance image is displayed on the display screen to guide the patient to perform motor imagination. During the user's motor imagination, the electroencephalogram signal and the electromyography signal are simultaneously collected based on the electroencephalogram signal collection device and the electromyography signal collection device.

[0041] Computer equipment is used for signal processing and motion control of the medical robot's actuators.

[0042] The medical robot's actuator is controlled by a computer device and fixed to the patient's limbs, driving the patient's limbs to perform rehabilitation exercises.

[0043] In the embodiment of the present application, after the computer device acquires the EEG signal and the EMG signal, it can determine the user's movement intention based on the movement intention detection method provided in the embodiment of the present application, and then control the movement of the medical robot actuator according to the user's movement intention.

[0044] Optionally, in another implementation, the medical robot system provided in the embodiment of the present application may also include a host computer, a slave computer, a motion controller, a joint actuator drive unit, a medical robot actuator, an angle sensor, etc.

[0045] Among them, the medical robot can use a programmable logic controller (PLC) to control the motor motion position and speed. The medical robot can communicate with the motor motion controller through the EtherCat communication protocol. The medical robot can use the Modbus tcp / ip protocol to send the running results of the upper computer Matlab program to the lower computer, and update the motor motion control variables in the lower computer memory in real time. The lower computer controls the movement of components such as the joint actuator drive unit, the medical robot actuator, and the angle sensor based on the motor motion control variables and the motion controller.

[0046] The medical robot system provided in the embodiment of the present application can realize a variety of rehabilitation training, such as unilateral single-joint and multi-joint upper limb exercises, and bilateral upper limb same-joint and multi-joint exercises. According to different users, personalized and targeted rehabilitation exercises can be realized based on personalized exercise guidance.

[0047] Reference Figure 2 , Figure 2 The structural schematic diagram of the computer device of the hardware operating environment involved in the embodiment scheme of the present application is shown. The computer device includes a processor, a memory and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. Among them, the processor may include one or more processing units. The processor may be a CPU (Central Processing Unit) or a DSP (Digital Signal Processing) and the like. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a motion intention detection method is implemented.

[0048] It should be noted that the implementation of each module in the computer device provided in the embodiment of the present application may be in the form of a computer program. The computer program can be run on the computer device. The program module constituted by the computer program can be stored in the memory of the computer device. When the computer program is executed by the processor, the steps of the method described in the embodiment of the present application are implemented.

[0049] Those skilled in the art will understand that Figure 2 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0050] The motion intention detection method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0051] Please refer to Figure 3 , Figure 3 A schematic flow chart of a motion intention detection method provided by an embodiment of the present application is shown. Each step of the method is described in detail below.

[0052] Step 301, simultaneously collect the user's electroencephalogram signal and electromyography signal.

[0053] In an embodiment of the present application, the medical robot system also includes two dedicated computers (PC1 and PC2), one computer (PC1) is an interface guidance device, and the other computer (PC2) is a computer device. When using the medical robot system based on EEG signal-electromyographic signal joint control to collect EEG signals and EMG signals from the user, the user needs to interact with the two dedicated computers at the same time.

[0054] Before each motor imagery, sound and visual prompts are used to help users focus on the upcoming motor imagery task. Specifically, the motor imagery guidance interface is displayed on PC1, and users observe the guidance of PC1 to perform motor imagery of the corresponding action to promote their neural activity. The motor imagery guidance interface program provided to users includes multiple links such as experimental program introduction, text prompts, and sound prompts. This program not only provides subjects with coherent experimental process guidance, but also sends precise event markers to the data acquisition software to ensure the accuracy of data alignment and segmentation.

[0055] At the same time, PC2 controls the EEG signal acquisition device (EEG cap) and the EMG signal acquisition device (EMG electrodes) to work synchronously to collect bioelectric signals, and synchronizes time information through the EEG multi-parameter synchronization box to ensure that the collected data has the same event markers, laying the foundation for subsequent signal processing.

[0056] Among them, the EEG signal collection positions selected the C3, C4, Cz and surrounding adjacent electrodes in the internationally accepted standard 10-10 standard electrode positions, a total of 19 electrode positions. Figure 4 As shown, Figure 4 A schematic diagram showing the position of an EEG electrode is shown. Figure 4 The electrode positions circled in the middle are the electrode positions selected when collecting EEG signals. Optionally, the 19 electrode positions may include CP1, CP2, CP3, CP4, CP5, CP6, CZ, C1, C2, C3, C4, C5, C6, FC1, FC2, FC3, FC4, FC5, and FC6.

[0057] The electromyographic signal acquisition device selects the electromyographic signal acquisition position generally at the biceps and triceps of the bilateral upper limbs that are closely related to upper limb movement, and the reference electrode is placed on the outside of the upper limb arm of the human body, such as Figure 5 As shown, Figure 5 A schematic diagram showing the location of an EMG electrode.

[0058] In the embodiment of the present application, compared to the user's irregular movement imagination, the user is guided to perform movement imagination according to a pre-set movement method based on the movement guidance method. By setting the movement images used for movement imagination to movement images that are conducive to the user's recovery training, it can help the user to perform recovery training.

[0059] Step 302, processing the EEG signal to obtain a first detection result, and processing the EMG signal to obtain a second detection result.

[0060] In one implementation, the first detection result includes a first probability value and a second probability value, the first probability value indicates the probability that the movement intention corresponding to the EEG signal is the first movement intention, and the second probability value indicates the probability that the movement intention corresponding to the EEG signal is the second movement intention, PC2 can call a pre-trained EEG signal processing model to extract features from the EEG signal to obtain EEG signal features; classify the EEG signal features to obtain the first probability value and the second probability value. Optionally, PC2 can filter the EEG signal to obtain a filtered EEG signal; then call a pre-trained EEG signal processing model to extract features from the filtered EEG signal based on the common space pattern algorithm and the Riemann manifold method to obtain EEG signal features, classify the extracted EEG signal features based on the SVM classifier, and obtain the first probability value and the second probability value.

[0061] In one implementation, the second detection result includes a third probability value and a fourth probability value. The third probability value indicates the probability that the movement intention corresponding to the electromyographic signal is the first movement intention, and the fourth probability value indicates the probability that the movement intention corresponding to the electromyographic signal is the second movement intention. PC2 can call a pre-trained electromyographic signal processing model to extract the time-frequency domain features of the electromyographic signal based on the wavelet packet decomposition method; the time-frequency domain features are processed based on the residual network module to obtain the second detection result.

[0062] The specific processing process of EEG signals and EMG signals is described in detail below:

[0063] First, signal filtering.

[0064] (1) EEG signal filtering

[0065] ① Use a 5th-order Butterworth filter to perform bandpass filtering on the collected 1-30 Hz EEG signals to remove electromagnetic interference and power frequency interference in the high-frequency band of the EEG signals and highlight the frequencies related to movement intention. The Butterworth filter transfer function formula is expressed as:

[0066]

[0067] Among them, n and w care the filter order and passband cutoff frequency of the Butterworth filter respectively.

[0068] ②Based on step ①, a 50Hz notch filter is used to suppress the noise of a specific frequency in the EEG signal (i.e., power line noise). The formula for eliminating power line noise is expressed as:

[0069]

[0070] Among them, f is the signal frequency, f0 is the suppressed center frequency, and Q is the quality factor, which indicates the selectivity of the filter. A high Q value means that the filter has a narrower bandwidth near the center frequency, thereby more accurately suppressing noise of a specific frequency.

[0071] ③ On the basis of step ②, the whole brain average re-reference method is used to reduce the information coupling between electrodes and improve the identifiability of the signal, while removing the common noise components and enhancing the features related to movement intention in the EEG signal. The formula is expressed as:

[0072]

[0073] Where m is the number of leads, and are the ith and jth lead potentials, respectively. is the filtered potential of the ith lead.

[0074] ④ Based on step ③, the independent component analysis (ICA) method is used to separate statistically independent source signals from the mixed signals (separating EEG signals from eye movements, ECG noise, etc.), and the formula is expressed as:

[0075] X(t)=AS(t)

[0076] Among them, X(t) is the observed signal, A is the mixing matrix, and t is the source signal.

[0077] Among them, ICA aims to find a separation matrix such that:

[0078] Y(t)=WX(t)

[0079] Among them, Y is the estimated independent component and W is the separation matrix.

[0080] In the embodiments of the present application, the EEG signals are preprocessed by using passband filtering, notch filtering, whole-brain average re-referencing, and independent component analysis (ICA) to remove various artifacts and environmental interferences and improve the data signal-to-noise ratio.

[0081] (2) EMG signal filtering

[0082] ① Use a bandpass filter to perform 30-150Hz bandpass filtering on the collected electromyographic signals to filter out most of the electrocardiographic influence and retain the frequency band that reflects the muscle movement signal. The formula is:

[0083] y(t)=F -1 [F[x(t)]·H(f)

[0084]

[0085] ω0=2πf0

[0086]

[0087] Where x(t) is the original signal, F represents the Fourier transform, H(f) is the transfer function of the Butterworth bandpass filter, s is the complex frequency, ω0 is the center angular frequency, f0 is the center frequency, f1, f h are the lower and upper frequency limits of the bandpass filter, and n is the filter order.

[0088] ② Use a 50Hz notch filter to suppress the noise of a specific frequency in the electromyographic signal (power line noise). The formula is:

[0089]

[0090] Among them, f is the signal frequency, f0 is the suppressed center frequency, and Q is the quality factor, which indicates the selectivity of the filter. A high Q value means that the filter has a narrower bandwidth near the center frequency, thereby more accurately suppressing noise of a specific frequency.

[0091] The embodiment of the present application pre-processes the electromyographic signal by means of bandpass filtering and notch filtering, thereby improving the data signal-to-noise ratio.

[0092] It should be noted that, in one implementation, the sample size of the filtered EEG signals and EMG signals can be expanded by using a data expansion method that combines random noise and sliding window technology, which can improve the generalization ability and robustness of the subsequent model.

[0093] Second, signal processing

[0094] (1) EEG signal processing

[0095] In the embodiment of the present application, the EEG signal processing model (gaCSP-SVM network model) includes a feature extraction step based on the common spatial pattern algorithm and the Riemann manifold method, and a step of classifying the extracted EEG signal features by an SVM classifier. The specific implementation method is as follows:

[0096] The gaCSP-SVM network model is constructed by combining the common space pattern algorithm (CSP), the Riemann manifold method, and the SVM classifier, and then the EEG signals are processed based on the network model. Among them, the Riemann manifold method can improve the robustness of the classification model to abnormal values ​​of EEG signals and improve the classification accuracy. The CSP algorithm can effectively remove components and noise that are irrelevant to the classification task and optimize the classification effect. The two algorithms are described below.

[0097] Co-spatial pattern algorithm

[0098] By using the diagonalization of the matrix, a set of optimal spatial filters is found for projection, so that the difference in variance values ​​of the two types of signals is maximized, thereby obtaining a feature vector with higher discrimination.

[0099] ① Determine the spatial filter ω to achieve the extreme value of J(ω), which can be expressed as:

[0100]

[0101] Among them, ∑1ω and ∑2ω are the average covariance matrices of the two types of EEG signals (left arm and right arm).

[0102] ②Use the Lagrange multiplier method to transform it into an extreme value problem:

[0103] L(ω,λ)=ω T ∑1ω-λ(ω T ∑2ω-1)

[0104] Calculate the partial derivative of ω and make it zero, and we get:

[0105]

[0106] Σ1ω=λΣ2ω

[0107] ③Perform feature extraction:

[0108]

[0109] in, is the space-time signal matrix of a single experiment.

[0110] n and t are the recording electrode of EEG signals and the number of sampling points on the electrode, respectively.

[0111] var(·) is the variance operation.

[0112] sum(·) is a sum operation.

[0113] Riemannian manifold method

[0114] The Riemannian distance is used instead of the Euclidean distance through the Riemann manifold method, such as Figure 6 As shown, Figure 6 A schematic diagram comparing Riemann distance and Euclidean distance is shown, and the calculation method of the spatial filter in the common space mode algorithm is modified.

[0115] ①The objective function is modified as follows:

[0116]

[0117] ω T ω=I m

[0118] in, represents the i-th SPD matrix of class c, is the Riemann mean of class c, is the Riemann mean of the entire set of SPD matrices.

[0119] ②The final filter is

[0120] ③The feature extraction process is modified as follows:

[0121]

[0122] Where d(·) represents extracting diagonal matrix elements to construct a diagonal matrix.

[0123] In the embodiment of the present application, the common spatial pattern algorithm and the Riemann manifold method are used to process the EEG signal to obtain the EEG signal features, and then the extracted EEG signal features are classified by the SVM classifier, and the classification results are output in the form of probability, that is, the first probability value and the second probability value, wherein the first probability value may refer to the probability value P of left motor imagination L The second probability value may refer to the probability value P of right motor imagination. R .

[0124] In the embodiment of the present application, a gaCSP-SVM network model is constructed by combining the CSP algorithm, the Riemann manifold method, and the SVM classifier. Compared with classification in Euclidean space, this model achieves high-precision classification of EEG signals.

[0125] (2) EMG signal processing

[0126] In the embodiment of the present application, the electromyographic signal processing model (SE-ResNet network model) includes the steps of extracting the time-frequency domain characteristics of the electromyographic signal based on the wavelet packet decomposition method and processing the time-frequency domain characteristics based on the residual network module. Among them, wavelet packet decomposition (CWT) is used to extract the time-frequency domain characteristics of the electromyographic signal during the motor imagery process, and Morlet wavelet is selected as the basis function. CWT can obtain the local time-frequency information of the electromyographic signal by applying the Morlet wavelet basis function at different scales. The formula is expressed as follows:

[0127]

[0128] Among them, W(a,b) is the wavelet coefficient, is the wavelet basis function, a is the scale parameter, and b is the translation parameter.

[0129] By processing the EEG signal using the above wavelet function, a wavelet coefficient matrix can be obtained, the data can be visualized, and the time point of the EMG signal amplitude transformation and the corresponding frequency band can be obtained.

[0130] Then, based on the ResNet-18 residual network model, the compressed attention mechanism is introduced to construct the SE-ResNet model. The SE-ResNet network model is composed of a residual network and a compressed excitation attention mechanism. Figure 7 A schematic diagram of a ResNet-18 residual network structure is shown, where Figure 7 The ResNet-18 shown in the figure contains 18 layers, including 17 convolutional layers and 1 fully connected layer. The basic building block of the network is the residual block, each of which consists of two convolutional layers with the same number of output channels. These convolutional layers use 3×3 convolution kernels, with intervals of batch normalization and ReLU activation functions. The output of each residual block is added to its input through a skip connection to form the final output. Optional, such as Figure 8 As shown, Figure 8 The schematic diagram of the structure of a ResNet residual block integrating a compression-excitation attention mechanism is shown. By introducing the compression-excitation attention mechanism (SE) into the residual block of the residual network to construct an attention mechanism residual network (SE-ResNet), the network's sensitivity to important features is enhanced and unimportant features are suppressed.

[0131] In the embodiment of the present application, the SE-ResNet model is used to process the electromyographic signal, the input is the electromyographic signal after filtering, and the output is the second detection result, wherein the second detection result includes a third probability value and a fourth probability value, and the third probability value may refer to the probability value P of left motor imagination. LThe fourth probability value may refer to the probability value P of right motor imagination. R .

[0132] In the embodiments of the present application, a compressed incentive attention mechanism and a residual neural network structure are introduced to construct a SE-ResNet network model, which can effectively improve the classification accuracy while avoiding the problems of model overfitting and gradient disappearance.

[0133] Step 303: Fuse the first detection result and the second detection result, and determine the motion intention according to the fusion result.

[0134] In the embodiment of the present application, a multi-objective particle swarm algorithm (MOPSO) is used to determine the fusion coefficient with the goal of minimizing the mean absolute error and the mean square error. The fusion coefficient is the EEG and sEMG decision value that meets the Pareto optimality.

[0135] ① Mean absolute error

[0136]

[0137] ② Mean square error

[0138]

[0139] By using a multi-objective particle swarm algorithm to find the α and β values ​​that meet the Pareto optimality according to the above objectives, the parameters and algorithm flow are as follows:

[0140] ① The parameters of the multi-objective particle swarm algorithm are shown in Table 1:

[0141] Table 1

[0142] Parameter name Parameter Value Population size 10 Maximum number of iterations 50 Coefficient of inertia ω 0.15 Number of cells per hypercube 10 <![CDATA[Individual learning coefficient c1]]> 1.5 <![CDATA[Overall learning coefficient c2]]> 1.5

[0143] ② Generate population POP based on the given population size.

[0144] ③ Randomly initialize the position and velocity of each particle and assign other initialization parameters.

[0145] ④Build the repository REP.

[0146] ⑤Update the particle velocity v and position x according to the following formula.

[0147] v i (t+1)=ωv i (t)+c1r1(xbest i (t)-x i (t))+c2r2(REP(h)-x i (t))

[0148] x i (t+1)=xi (t)+v i (t+1)

[0149] Among them, r1 and r2 represent random parameters in the range [0,1]. i (t) represents the velocity of vector i at the tth iteration. i (t) represents the position of vector i at the tth iteration. i (t) represents the vector that obtains the best position at the tth iteration. REP(h) represents the position of the global optimal solution in the entire particle swarm at the tth iteration. h is selected as follows:

[0150] Ⅰ. Count the number of particles in all hypercubes containing particles and divide them by 10 (determined by the number of grids in each hypercube in the parameter table).

[0151] II. Calculate the distribution probability P of hypercube k k :

[0152]

[0153] Where m represents the number of hypercubes, F k represents the fitness parameter of hypercube k

[0154] III. Calculation of the cumulative probability Q of the hypercube k k :

[0155]

[0156] IV. Randomly generate an array T of length k, whose values ​​are in [0,1], and sort the parameter values ​​from large to small. If the cumulative probability Q k If it is greater than the element T[k] in the array, individual k is selected. If it is less than T[k], the next individual k+1 is compared until an individual is selected, and that individual is the selected hypercube.

[0157] V. Randomly select a particle from the selected hypercube k, which is h.

[0158] ⑥Update the content of REP and the position information of particles in the hypercube.

[0159] ⑦If the current particle position is better than the previous position, the particle is updated to the latest position.

[0160] ⑧If the termination condition is met, stop the iteration. Otherwise, repeat step ⑤.

[0161] ⑨The position coordinates of the optimal particles selected by the multi-objective particle swarm algorithm are α and β, where α and β are fusion coefficients.

[0162] (3) The fusion result of the EEG signal and the EMG signal is calculated based on the fusion coefficient, which can also be called the decision value. The decision value is used to determine the movement intention. The calculation formula of the fusion result is as follows:

[0163] P m =(P L ) m -(P R ) m ,m∈{EEG,sEMG}

[0164] ψ=αP EEG -βP sEMG

[0165] The final result of the motion intention detection is output based on the fusion result, specifically including: when the fusion result is greater than the first threshold, the first limb motion intention is determined; when the fusion result is less than the first threshold, the second limb motion intention is determined; when the fusion result is equal to the first threshold, the resting state motion intention is determined.

[0166] This application adopts a multi-objective particle swarm algorithm to make fusion decisions on EEG signals and EMG signals, which can dynamically find the optimal proportion of EEG signals and EMG signals in the fusion decision, avoiding the problem that the artificially set fusion decision proportion is contrary to the actual situation of the user.

[0167] The following is an explanation of the process of motion control in the embodiment of the present application.

[0168] In the embodiment of the present application, PC2 can control the medical robot actuator to perform corresponding rehabilitation actions through a programmable logic unit, a motion controller and a joint actuator drive unit.

[0169] Among them, the medical robot controls the movement position and speed of the medical robot motor through a programmable logic controller, communicates with the motion controller through the EtherCat communication protocol, uses the Modbus tcp / ip protocol to send the running results of the upper computer Matlab program to the lower computer, and updates the motor motion control variables in the lower computer memory in real time.

[0170] In the embodiment of the present application, the medical robot system collects human bioelectric signals (EEG signals and EMG signals) through an EEG cap and EMG electrodes, and then uses passband filtering, notch filtering, whole-brain average re-reference, and independent component analysis to filter the collected signals. After that, the EEG signal processing model is used to process the EEG signal, and the EMG signal processing module is used to process the EMG signal. The processing results of the EEG signal and the EMG signal are fused and decided by the multi-objective particle swarm algorithm (MOPSO), and the user's movement intention is determined according to the fusion result. In this scheme, the brain-computer interface mode of fusion decision of EEG signal and EMG signal is adopted, which reduces the influence of single EEG signal, sEMG signal and muscle fatigue degree on the accuracy of movement intention detection and recognition.

[0171] Compared with the prior art, the solution provided by the embodiment of the present application has the advantages of high accuracy in motion intention recognition, diverse and adaptable treatment modes, and short user rehabilitation period.

[0172] Based on the embodiments of this application, please refer to Fig. 9 , Fig. 9 A schematic diagram of the classification of rehabilitation exercise training modes is shown. The embodiment of the present application is classified according to the upper limb rehabilitation exercise training modes, which may include unilateral upper limb rehabilitation exercise and bilateral upper limb rehabilitation exercise.

[0173] Among them, unilateral upper limb rehabilitation exercises can be divided into single-joint rehabilitation exercises and multi-joint rehabilitation exercises. Four rehabilitation training methods are designed for single-joint rehabilitation exercises, namely: left elbow rehabilitation exercises, left shoulder rehabilitation exercises, right elbow rehabilitation exercises and right shoulder rehabilitation exercises. Two rehabilitation training methods are designed for multi-joint rehabilitation exercises, namely left shoulder and elbow rehabilitation exercises and right shoulder and elbow rehabilitation exercises.

[0174] Bilateral upper limb rehabilitation exercises can be divided into bilateral same-joint rehabilitation exercises and bilateral multi-joint rehabilitation exercises. Two rehabilitation training methods are designed for bilateral same-joint rehabilitation exercises, namely bilateral shoulder joint rehabilitation exercises and bilateral elbow joint rehabilitation exercises. A rehabilitation training method is designed for bilateral multi-joint rehabilitation exercises, namely bilateral shoulder and elbow joint rehabilitation exercises.

[0175] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0176] Based on the same inventive concept, an embodiment of the present application also provides a medical robot system. The implementation solution for solving the problem provided by the medical robot system is similar to the implementation solution recorded in the above method. Therefore, the specific limitations in the medical robot system embodiment provided below can be referred to the limitations on the motion intention detection method in the above text, and will not be repeated here. Fig.10 A structural block diagram of a medical robot system provided in an embodiment of the present application is shown. Fig.10 As shown, the medical robot system includes: a signal acquisition module 1001, a signal processing module 1002, a fusion module 1003 and a motion control module 1004, wherein the signal acquisition module 1001 is used to simultaneously acquire the user's EEG signal and EMG signal; the signal processing module 1002 is used to process the EEG signal to obtain a first detection result; the EMG signal is processed to obtain a second detection result; the fusion module 1003 is used to fuse the first detection result and the second detection result, and determine the motion intention according to the fusion result; the motion control module 1004 is used to control the medical robot to perform corresponding actions according to the motion intention.

[0177] In one embodiment, the fusion module 1003 is specifically used to determine the first limb movement intention when the fusion result is greater than the first threshold; to determine the second limb movement intention when the fusion result is less than the first threshold; and to determine the resting state movement intention when the fusion result is equal to the first threshold.

[0178] In one embodiment, the first detection result includes a first probability value and a second probability value, the first probability value indicates the probability that the movement intention corresponding to the EEG signal is the first movement intention, and the second probability value indicates the probability that the movement intention corresponding to the EEG signal is the second movement intention, and the signal processing module 1002 is specifically used to call a pre-trained EEG signal processing model to extract features of the EEG signal to obtain EEG signal features; classify the EEG signal features to obtain the first probability value and the second probability value.

[0179] In one of the embodiments, the signal processing module 1002 is specifically used to filter the EEG signal to obtain the filtered EEG signal; call a pre-trained EEG signal processing model to extract features of the filtered EEG signal based on the common space pattern algorithm and the Riemann manifold method to obtain EEG signal features; classify the extracted EEG signal features based on the SVM classifier to obtain a first probability value and a second probability value.

[0180] In one embodiment, the second detection result includes a third probability value and a fourth probability value, the third probability value indicates the probability that the movement intention corresponding to the electromyographic signal is the first movement intention, and the fourth probability value indicates the probability that the movement intention corresponding to the electromyographic signal is the second movement intention, and the signal processing module 1002 is specifically used to call a pre-trained electromyographic signal processing model to extract the time-frequency domain characteristics of the electromyographic signal based on the wavelet packet decomposition method; the time-frequency domain characteristics are processed based on the residual network module to obtain the second detection result.

[0181] In one embodiment, the fusion module 1003 is specifically configured to adopt a multi-objective particle swarm algorithm to determine a fusion coefficient with the goal of minimizing the mean absolute error and the mean square error; and fuse the first detection result and the second detection result according to the fusion coefficient.

[0182] Each module in the above-mentioned medical robot system can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0183] Based on the above embodiments, the present application also provides a medical robot system, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is run by the processor, the steps of the motion intention detection method disclosed in the above embodiments are executed.

[0184] Based on the above embodiments, the present application also provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed, the steps of the motion intention detection method disclosed in the above embodiments are implemented.

[0185] Based on the above embodiments, the present application also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, it enables the computer to execute the steps of the motion intention detection method disclosed in the above embodiments.

[0186] It is understandable that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0187] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a base station or a terminal. Of course, the processor and the storage medium can also be present in a base station or a terminal as discrete components.

[0188] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it may also be an optical medium, such as a DVD; it may also be a semiconductor medium, such as a solid state disk (SSD).

[0189] In the various embodiments of the present application, unless otherwise specified or provided for in any logical conflict, the terms and / or descriptions between the different embodiments are consistent and may be referenced to each other, and the technical features in the different embodiments may be combined to form new embodiments according to their inherent logical relationships.

[0190] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of this application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula of this application, the character " / " indicates that the previous and next associated objects are in a "division" relationship.

[0191] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.

Claims

1. A method for detecting motion intention, characterized in that: Applied to a medical robot system, the method comprises: Collect the user's EEG and EMG signals at the same time; Processing the EEG signal to obtain a first detection result; Processing the electromyographic signal to obtain a second detection result; The first detection result and the second detection result are fused, and the motion intention is determined according to the fusion result.

2. The method according to claim 1, characterized in that: The determining of the motion intention according to the fusion result includes: When the fusion result is greater than a first threshold, determining the first limb movement intention; When the fusion result is less than the first threshold, determining the second limb movement intention; When the fusion result is equal to the first threshold, the resting state movement intention is determined.

3. The method according to claim 1, characterized in that The first detection result includes a first probability value and a second probability value, the first probability value indicates the probability that the movement intention corresponding to the EEG signal is the first movement intention, and the second probability value indicates the probability that the movement intention corresponding to the EEG signal is the second movement intention, and the processing of the EEG signal to obtain the first detection result includes: A pre-trained EEG signal processing model is called to extract features of the EEG signal to obtain EEG signal features; and the EEG signal features are classified to obtain the first probability value and the second probability value.

4. The method according to claim 3, characterized in that The calling of a pre-trained EEG signal processing model to extract features of the EEG signal to obtain EEG signal features, and classifying the EEG signal features to obtain the first probability value and the second probability value includes: Performing filtering on the electroencephalogram signal to obtain the electroencephalogram signal after filtering; Calling a pre-trained EEG signal processing model to extract features of the EEG signal after filtering based on a common spatial pattern algorithm and a Riemann manifold method to obtain the EEG signal features; The extracted EEG signal features are classified based on a SVM classifier to obtain the first probability value and the second probability value.

5. The method according to claim 3, characterized in that: The second detection result includes a third probability value and a fourth probability value, the third probability value indicates the probability that the movement intention corresponding to the electromyographic signal is the first movement intention, and the fourth probability value indicates the probability that the movement intention corresponding to the electromyographic signal is the second movement intention, and the electromyographic signal is processed to obtain the second detection result, including: Calling a pre-trained electromyographic signal processing model to extract the time-frequency domain features of the electromyographic signal based on a wavelet packet decomposition method; The time-frequency domain features are processed based on the residual network module to obtain the second detection result.

6. The method according to claim 5, characterized in that The fusing the first detection result and the second detection result includes: A multi-objective particle swarm algorithm is used to determine the fusion coefficient with the goal of minimizing the mean absolute error and mean square error; The first detection result and the second detection result are fused according to the fusion coefficient.

7. A medical robot system, characterized in that: The system comprises: A signal acquisition module, used to simultaneously acquire the user's EEG signals and EMG signals; A signal processing module, used to process the EEG signal to obtain a first detection result; and to process the EMG signal to obtain a second detection result; a fusion module, configured to fuse the first detection result and the second detection result, and determine the motion intention according to the fusion result; A motion control module is used to control the medical robot to perform corresponding actions according to the motion intention.

8. A medical robot system, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the motion intention detection method according to any one of claims 1 to 6 is executed.

9. A computer-readable storage medium, characterized in that: The computer program is stored therein, and when the computer program is run on a processor, the motion intention detection method according to any one of claims 1 to 6 is executed.

10. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is executed on a computer, the computer is enabled to execute the motion intention detection method according to any one of claims 1 to 6.

Citation Information

Cited By

  • Method for decoding motion intention based on multi-modal information

    CN121971075A

  • Method for decoding motor intention based on multi-modal information

    CN121971075B