Robot control method and system based on multi-source information perception and electronic device

Abstract

The application discloses a robot control method and system based on multi-source information perception and electronic equipment, and the method comprises the following steps: acquiring the brain electrical signal, the inertial signal, the electromyographic signal and / or the force tactile signal of a patient with low motor function; using a random forest machine model to recognize the movement intention for each first signal respectively, and obtaining the corresponding movement intention recognition result; acquiring the weight parameter corresponding to each first signal; preliminarily determining the movement intention of the patient according to the movement intention recognition result and the corresponding weight parameter of each first signal; judging whether the preliminarily determined movement intention conforms to the logic according to the logic decision table; if yes, the final movement intention is determined; determining the movement control signal of the upper limb rehabilitation exoskeleton according to the finally determined movement intention; and controlling the upper limb rehabilitation exoskeleton to move according to the movement control signal. The application can improve the movement intention recognition accuracy of the patient with low motor function in the upper limb rehabilitation training process.

Description

Technical Field

[0001] This invention relates to the field of medical rehabilitation technology, specifically to a robot control method, system, and electronic device based on multi-source information perception. Background Technology

[0002] Upper limb motor dysfunction is a common clinical manifestation in stroke patients. While patients with low motor function exhibit limb motor impairment, they still retain some motor ability. In this state, the upper limbs can generate relatively stable electromyographic signals when the patient expresses a motor intention. Current stroke rehabilitation approaches tend towards repetitive, task-oriented functional training. Upper limb rehabilitation exoskeleton robots can perform this task. Human-machine interaction is key to the research of upper limb exoskeleton robots. High accuracy in recognizing human motor intentions enables higher human-machine compatibility in rehabilitation training, which is significant for improving training effectiveness and patient safety. Currently, upper limb exoskeleton... Methods for motion intention recognition are typically based on single-mode signal recognition. Existing input signals for intention recognition algorithms include physical signals and bioelectrical signals. Physical signals, such as inertial, angular, and pressure signals, offer advantages in stability. Bioelectrical signals, such as electroencephalogram (EEG) and electromyography (EMG), provide advantages in globality and predictability. However, due to the weakened limb control and frequent tremors in patients with low motor function, especially those with stroke-related low motor function, the acquired physical signals may not accurately represent the patient's true motor intention. Furthermore, bioelectrical signals suffer from poor robustness and are easily affected by external information. Therefore, regardless of the signal type used for intention recognition, the accuracy is not very high. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a robot control method, system and electronic device based on multi-source information perception to solve the problems of low accuracy in recognizing the movement intention of patients based on single-mode signals during upper limb rehabilitation training of patients with low motor function.

[0004] According to a first aspect, embodiments of the present invention provide a robot control method based on multi-source information perception, comprising:

[0005] Acquire a first movement-related signal from a patient with low motor function, the first signal including at least two of the following: electroencephalogram (EEG) signals, inertial signals of the upper limb, electromyographic signals of the upper limb, and force-tactile signals of the upper limb;

[0006] For each type of the first signal, a random forest machine model is used to perform motion intent recognition on the first signal to obtain the motion intent recognition result corresponding to the first signal;

[0007] Obtain the weight parameters corresponding to each of the first signals;

[0008] Based on the motion intention recognition result and corresponding weight parameters corresponding to each of the first signals, a first motion intention is determined, which is the motion intention of the patient with low motor function initially determined.

[0009] Determine whether the first motion intention conforms to logic based on the pre-established logical decision table;

[0010] If the first motion intention is logically consistent, the first motion intention is determined as the final determined motion intention;

[0011] The motion control signals of the upper limb rehabilitation exoskeleton are determined based on the final determined motion intention;

[0012] The upper limb rehabilitation exoskeleton is controlled to move according to the motion control signal.

[0013] In some optional implementations, before obtaining the weight parameters corresponding to each of the first signals, the method further includes:

[0014] Acquire sample signals, wherein the signal type of the sample signals is consistent with the signal type of the first signal, and each type of sample signal includes multiple samples;

[0015] For each of the sample signals, the true motion intent corresponding to each sample signal and the motion intent recognition result obtained by recognizing the sample signal using the random forest machine model are obtained respectively;

[0016] For each of the sample signals, the recognition accuracy is determined based on the actual motion intent and motion intent recognition results corresponding to the multiple sample signals respectively.

[0017] For each of the sample signals, a corresponding weight parameter is determined based on the recognition accuracy.

[0018] In some optional implementations, determining the first motion intent based on the motion intent recognition result and corresponding weight parameters for each of the first signals includes:

[0019] For the first signal that has the same motion intent recognition result, the weight parameters corresponding to the first signal are summed to obtain the confidence level of the corresponding motion intent recognition result;

[0020] For a first signal whose motion intent recognition result is inconsistent with all other first signals, the weight parameter corresponding to the first signal is directly used as the confidence level of the corresponding motion intent recognition result.

[0021] The motion intent recognition result corresponding to the highest confidence level is determined as the first motion intent.

[0022] In some optional implementations, determining the motion intent recognition result corresponding to the maximum confidence level as the first motion intent includes:

[0023] Determine whether the minimum confidence level is greater than a preset threshold;

[0024] If the minimum confidence level is less than or equal to the preset threshold, then the motion intent recognition result corresponding to the maximum confidence level is determined as the first motion intent.

[0025] In some optional embodiments, the method further includes:

[0026] If the minimum confidence level is greater than the preset threshold, then for each of the first signals, the random forest machine model is used again to perform motion intent recognition on the first signal to obtain the motion intent recognition result corresponding to the first signal, and the first motion intent is determined based on the newly obtained motion intent recognition result.

[0027] In some optional embodiments, the upper limb rehabilitation exoskeleton includes multiple joints, each joint corresponding to one or more drive motors;

[0028] The process of determining the motion control signals for the upper limb rehabilitation exoskeleton based on the finally determined motion intention includes:

[0029] Based on the motion intention, drive signals for the drive motors corresponding to at least some joints are determined, and the drive signals include at least one of drive time, rotational speed of the drive motor, and running direction of the drive motor.

[0030] In some optional embodiments, after controlling the upper limb rehabilitation exoskeleton to move according to the motion control signal, the method further includes:

[0031] Obtain the positional information of each joint of the upper limb rehabilitation exoskeleton;

[0032] Adjust the posture of the virtual model corresponding to the upper limb rehabilitation exoskeleton in the virtual scene according to the posture information;

[0033] Obtain the deviation information between the current pose and the target pose of the virtual model;

[0034] The posture adjustment signal of the upper limb rehabilitation exoskeleton is generated based on the deviation information;

[0035] The upper limb rehabilitation exoskeleton is controlled to adjust its posture according to the posture adjustment signal.

[0036] According to a second aspect, embodiments of the present invention provide a robot control system based on multi-source information perception, comprising:

[0037] The first acquisition module is used to acquire a first movement-related signal from a patient with low motor function, the first signal including at least two of the following: electroencephalogram (EEG) signal, inertial signal of the upper limb, electromyographic signal of the upper limb, and force-tactile signal of the upper limb.

[0038] The recognition module is used to perform motion intent recognition on each of the first signals using a random forest machine model, and obtain the motion intent recognition result corresponding to the first signal.

[0039] The second acquisition module is used to acquire the weight parameters corresponding to each of the first signals;

[0040] The first intention determination module is used to determine the first motion intention based on the motion intention recognition result and the corresponding weight parameter corresponding to each of the first signals. The first motion intention is the motion intention of the patient with low motor function that has been preliminarily determined.

[0041] The judgment module is used to determine whether the first motion intention conforms to logic based on a pre-established logical decision table;

[0042] The second intent determination module is used to determine the first motion intent as the final determined motion intent if the first motion intent is logically consistent.

[0043] The control signal determination module is used to determine the motion control signal of the upper limb rehabilitation exoskeleton based on the finally determined motion intention;

[0044] The control module is used to control the upper limb rehabilitation exoskeleton to move according to the motion control signal.

[0045] According to a third aspect, embodiments of the present invention provide an electronic device, comprising:

[0046] The system includes a memory and a processor, which are interconnected. The memory stores a computer program, which, when executed by the processor, implements any of the robot control methods based on multi-source information perception described in the first aspect above.

[0047] According to a fourth aspect, embodiments of the present invention provide a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements any of the robot control methods based on multi-source information perception described in the first aspect.

[0048] This invention proposes a multi-modal signal fusion-based intention recognition technology. By fusing biological and physical signals, it ensures accurate and timely recognition of the motor intentions of patients with low motor function. Specifically, it uses EEG, EMG, inertial, and force / tactile signals as recognition signal sources, fusing them for identification. Furthermore, to improve the efficiency and accuracy of intention recognition, this invention constructs a random forest machine model to determine the motor intentions of each signal source and implements an adaptive attention mechanism (weighting parameters) to achieve decision-level fusion of the recognition results. This allows patients with low motor function to perform upper limb rehabilitation training according to their own wishes, making it more engaging and increasing their motivation for rehabilitation training compared to repetitive, task-oriented functional training. Attached Figure Description

[0049] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings:

[0050] Figure 1 A flowchart illustrating a robot control method based on multi-source information perception provided in an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of a robot control process based on multi-source information perception provided in an embodiment of the present invention;

[0052] Figure 3 A logical decision table for motion intent provided in embodiments of the present invention;

[0053] Figure 4 This is a schematic diagram illustrating the process of multimodal signal fusion intention recognition for patients with low motor function, provided in an embodiment of the present invention.

[0054] Figure 5 This is a schematic diagram of the control process of an upper limb rehabilitation exoskeleton for rehabilitation training of patients with low motor function, provided in an embodiment of the present invention.

[0055] Figure 6 This is a schematic diagram of a robot control system based on multi-source information perception, provided as an embodiment of the present invention.

[0056] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. In the following descriptions of embodiments, "a plurality of" means two or more, unless otherwise expressly specified.

[0059] Please see Figure 1 This invention provides a robot control method based on multi-source information perception, wherein the robot includes an upper limb rehabilitation exoskeleton, and the method includes:

[0060] S101: Acquire a first movement-related signal from a patient with low motor function, the first signal including at least two of the following: electroencephalogram (EEG) signal, inertial signal of the upper limb, electromyographic signal of the upper limb, and force-tactile signal of the upper limb; the inertial signal and the force-tactile signal have the advantage of good stability, while the electromyographic signal and the EEG signal have the advantages of globality and anticipation;

[0061] For information on acquiring the first signal, please refer to [link / reference]. Figure 2 This can involve playing pre-designed motor task prompt videos to patients with low motor function (i.e., providing visual guidance for motor tasks), guiding them to imagine motor movements, and collecting their first signal when they intend to perform the corresponding movement after imagining it.

[0062] S102: For each type of the first signal, use a random forest machine model to perform motion intention recognition on the first signal to obtain the motion intention recognition result corresponding to the first signal;

[0063] S103: Obtain the weight parameters corresponding to each of the first signals;

[0064] S104: Based on the motion intention recognition result and corresponding weight parameters corresponding to each of the first signals, determine the first motion intention, which is the motion intention of the patient with low motor function that has been preliminarily determined.

[0065] S105: Based on a pre-established logical decision table (e.g.) Figure 3 (As shown) Determine whether the first intention of movement conforms to logic;

[0066] S106: If the first motion intention is logically correct, determine the first motion intention as the final determined motion intention;

[0067] If the first motion intention is illogical, it is considered an identification error, and a second intention identification is performed; if the second intention identification result is still incorrect, the previously determined motion intention is used as the current identification result.

[0068] S107: Determine the motion control signals for the upper limb rehabilitation exoskeleton based on the finally determined motion intention;

[0069] S108: Control the upper limb rehabilitation exoskeleton to move according to the motion control signal.

[0070] Specifically, the execution devices for steps S101-S106 and steps S107-S108 can be different. For example, steps S101-S106 can be executed by an intention recognition device, while steps S107-S108 can be executed by an upper limb rehabilitation robot (including a processor and an upper limb rehabilitation exoskeleton). However, both the intention recognition device and the upper limb rehabilitation robot belong to the upper limb rehabilitation system. In this case, after determining the movement intention of the patient with low motor function, the intention recognition device needs to send the patient's movement intention to the upper limb rehabilitation robot so that the upper limb rehabilitation robot can control the movement of its upper limb rehabilitation exoskeleton according to the patient's movement intention, thereby driving the patient's upper limb to perform rehabilitation movements.

[0071] Steps S101-S106 and S107-S108 can also be performed by the upper limb rehabilitation robot. In other words, no additional intention recognition device is needed, and intention recognition is also achieved by the upper limb rehabilitation robot.

[0072] This invention proposes a multi-modal signal fusion-based intention recognition method. By fusing biological and physical signals, it ensures the accuracy and timeliness of motor intention recognition for patients with low motor function. Specifically, it uses EEG, EMG, inertial, and force / tactile signals as recognition signal sources, fusing them for identification. Furthermore, to improve the efficiency and accuracy of intention recognition, this invention constructs a random forest machine model to determine the motor intention of each signal source and builds an adaptive attention mechanism (weighting parameters) to achieve decision-level fusion of the recognition results. This allows patients with low motor function to perform upper limb rehabilitation training according to their own wishes, making it more engaging and increasing their motivation for rehabilitation training compared to repetitive, task-oriented functional training.

[0073] Furthermore, in this embodiment of the invention, a two-layer fusion decision is used for the intent recognition results of multi-mode signals. The first layer uses weights for fusion decision, and the second layer uses logical judgment for fusion decision. The second-layer fusion decision is formulated based on the temporal and logical relationships between the recognition results.

[0074] For some specific implementation methods, please refer to Figure 4 Before obtaining the weight parameters corresponding to each of the first signals, the method further includes:

[0075] Acquire sample signals, wherein the signal type of the sample signals is consistent with the signal type of the first signal, and each type of sample signal includes multiple samples; the sample signals may be from other patients with low motor function, that is, samples obtained from other patients with low motor function, or samples obtained from healthy individuals, or samples obtained from the same patient with low motor function. Of course, the sample signals may not be collected from the same object; the object of collection may include healthy individuals, other patients with low motor function, and / or the same patient with low motor function.

[0076] For each of the sample signals, the true motion intent corresponding to each sample signal and the motion intent recognition result obtained by recognizing the sample signal using the random forest machine model are obtained respectively; each sample signal has multiple sample signals, and a motion intent recognition result can be obtained by recognizing each sample signal.

[0077] For each of the sample signals, the recognition accuracy is determined based on the actual motion intent and motion intent recognition results corresponding to the multiple sample signals respectively.

[0078] For each of the sample signals, a corresponding weight parameter is determined based on the recognition accuracy.

[0079] In some specific implementations, determining the first motion intent based on the motion intent recognition result and corresponding weight parameters for each of the first signals includes:

[0080] For the first signal that has the same motion intent recognition result, the weight parameters corresponding to the first signal are summed to obtain the confidence level of the corresponding motion intent recognition result; here, each first signal has only one signal, and the signal can be a signal with a certain duration;

[0081] For a first signal whose motion intent recognition result is inconsistent with all other first signals, the weight parameter corresponding to the first signal is directly used as the confidence level of the corresponding motion intent recognition result.

[0082] The motion intent recognition result corresponding to the highest confidence level is determined as the first motion intent.

[0083] For example, the first signal includes four types: electroencephalogram (EEG) signals, upper limb inertial signals, upper limb electromyographic (EMG) signals, and upper limb force-tactile signals. Among these, the motor intention identified based on EEG signals is the first type of motor intention, the motor intention identified based on inertial signals is the second type, the motor intention identified based on EMG signals is the third type, and the motor intention identified based on force-tactile signals is the fourth type. The weighted parameters for EMG signals are q1, EEG signals are q2, force-tactile signals are q3, and inertial signals are q4. Adding the weighted parameter q2 (for EEG signals) to the weighted parameter q4 (for inertial signals) gives (q2 + q4), which is used as the confidence level for the first type of motor intention. The confidence level for the second type of motor intention is q1, and the confidence level for the third type is q3. Comparing (q2 + q4), q1, and q3, if (q2 + q4) is the largest, then the first motor intention of the patient with low motor function is considered to be the first type of motor intention.

[0084] In other specific embodiments, determining the motion intent recognition result corresponding to the maximum confidence level as the first motion intent includes:

[0085] Determine whether the minimum confidence level is greater than a preset threshold;

[0086] If the minimum confidence level is less than or equal to the preset threshold, then the motion intent recognition result corresponding to the maximum confidence level is determined as the first motion intent.

[0087] Taking the example above, if q1 is smaller than q3, then if q1 is less than or equal to the preset threshold, then the first motor intention of the patient with low motor function is considered to be the first type of motor intention (its corresponding confidence level (q2+q4) is the largest). Otherwise, the first motor intention of the patient with low motor function cannot be considered to be the first type of motor intention.

[0088] In this embodiment of the invention, considering the possibility of recognition errors in machine learning, a judgment and feedback mechanism is designed. By establishing a strong correlation with the intention recognition results of each signal source, the mechanism judges whether the fusion intention recognition result of the signal decision level is effective and compensates for this, thereby improving the accuracy of motion intention recognition for low-functioning patients based on multimodal (electromyography, electroencephalography, inertia, force and touch) signals.

[0089] For other specific implementations, please refer to Figure 4 The method further includes:

[0090] If the minimum confidence level is greater than the preset threshold, then for each of the first signals, the random forest machine model is used again to identify the movement intention of the first signal to obtain the movement intention identification result corresponding to the first signal, and the first movement intention of the patient with low motor function is determined based on the newly obtained movement intention identification result.

[0091] In other words, if the minimum confidence level is greater than the preset threshold, the motor intention recognition result corresponding to the maximum confidence level cannot be directly determined as the motor intention of the patient with low motor function, and motor intention recognition needs to be performed again.

[0092] The following details how, for each type of the first signal, a random forest machine model is used to perform motion intent recognition on the first signal to obtain the corresponding motion intent recognition result.

[0093] For electroencephalogram (EEG) signals, wavelet transform is used to filter the EEG signals to remove noise such as electromyography (EMG), electrocardiogram (ECG), and power line noise, extracting the EEG signals and performing baseline correction and bad channel processing. The processed EEG signals are then segmented according to a time window segmentation scheme, and the feature values ​​of each segmented EEG signal unit are calculated. These feature groups are then input into a random forest machine model trained using pre-trained EEG signal samples to obtain the motion intention recognition result. This EEG signal can be acquired in real time, and the recognition of the EEG signal is also performed in real time.

[0094] In addition, given that EEG signals have multiple channels, in this embodiment of the invention, the weights of each channel are determined based on the correlation between the signals of each channel of the EEG signal and the motor intention. The determined channel weights are used to identify the motor intention, and the channel weights are adjusted according to the intention recognition results. This process is repeated until the optimal channel weights are obtained.

[0095] For inertial signals, inertial sensors can be used to collect upper limb motion signals from patients with low motor function, i.e., inertial signals, specifically including acceleration and velocity. Then, a random forest machine model (pre-trained using inertial signal samples) is used in conjunction with a human skeletal model of the patient with low motor function to obtain motion intention recognition results. These motion intention recognition results include the direction of movement, velocity, and acceleration of each joint of the patient with low motor function. Specifically, the human skeletal model of the patient with low motor function can be constructed based on the determined upper limb bone length based on information such as the patient's height and weight.

[0096] For electromyography (EMG) signals, flexible sensors can be used to collect EMG signals from the surface of the upper limbs. Wavelet denoising algorithms are then used to filter the EMG signals, extracting cleaner signals. The extracted, cleaner EMG signals are segmented according to a time window segmentation scheme, and feature values ​​are calculated. These feature groups are then input into a pre-trained random forest machine model to obtain the motion intention recognition result. The acquisition and processing procedures for the EMG signal samples used in training the random forest machine model are consistent with those used in actual detection.

[0097] In addition, for multi-channel electromyography (EMG) signals, the weights of each channel can be determined based on the correlation between the signals of each EMG channel and the motor intention. The determined channel weights are then used to identify the motor intention, and the channel weights are adjusted based on the intention recognition results. This process is repeated until the optimal channel weights are obtained.

[0098] For force-tactile signals, force-tactile gloves can be used to acquire the patient's grasping action signal information, which is referred to here as force-tactile signal. Then, a pre-trained random forest machine model is used to identify the force-tactile signal to obtain the motion intention recognition result.

[0099] In some specific embodiments, the upper limb rehabilitation exoskeleton includes multiple joints, each joint corresponding to one or more drive motors;

[0100] The process of determining the motion control signals for the upper limb rehabilitation exoskeleton based on the finally determined motion intention includes:

[0101] Based on the motion intention, drive signals for the drive motors corresponding to at least some joints are determined, and the drive signals include at least one of drive time, rotational speed of the drive motor, and running direction of the drive motor.

[0102] Specifically, upper limb rehabilitation robots can also be called upper limb rehabilitation exoskeleton robots. Upper limb rehabilitation exoskeleton robots include an upper limb rehabilitation exoskeleton and a processor (or controller), or the upper limb rehabilitation exoskeleton robot is controlled by an external controller (such as a computer). Please refer to [link to relevant documentation]. Figure 5 After acquiring the movement intention of a patient with low motor function, the controller obtains the movement requirements of each joint of the upper limb rehabilitation exoskeleton based on the movement intention, converts the movement requirements into drive signals (also known as control commands) for the drive motors corresponding to each joint, and sends these drive signals to the multi-axis motion control card. The multi-axis motion control card then sends the drive signals to the drivers of the corresponding drive motors. The drivers control the movement of the drive motors according to the drive signals, so that the upper limb rehabilitation exoskeleton in the upper limb rehabilitation exoskeleton robot can perform upper limb rehabilitation movements according to the patient's movement intention.

[0103] In some specific embodiments, after controlling the upper limb rehabilitation exoskeleton to move according to the motion control signal, the method further includes:

[0104] Obtain the positional information of each joint of the upper limb rehabilitation exoskeleton;

[0105] Adjust the posture of the virtual model corresponding to the upper limb rehabilitation exoskeleton in the virtual scene according to the posture information;

[0106] Obtain the deviation information between the current pose and the target pose of the virtual model;

[0107] The posture adjustment signal of the upper limb rehabilitation exoskeleton is generated based on the deviation information;

[0108] The upper limb rehabilitation exoskeleton is controlled to adjust its posture according to the posture adjustment signal.

[0109] Specifically, the motion task prompt video used when acquiring the first signal can be a virtual reality (VR) video. In this embodiment of the invention, a VR interaction module is designed based on VR technology to establish a virtual multi-degree-of-freedom upper limb rehabilitation robot model and a virtual scene model. The pose data of each joint of the upper limb rehabilitation exoskeleton robot is acquired in real time based on the encoder installed on the real upper limb rehabilitation exoskeleton robot. The pose of the upper limb rehabilitation robot model in the virtual scene is adjusted in real time based on this data. The virtual robot model will interact with the virtual environment. If the upper limb movement posture displayed by the upper limb rehabilitation robot model deviates from the target posture in the virtual environment, this movement deviation can be fed back to the robot controller. The controller, based on the deviation analysis, sends corresponding joint adjustment information to the motion control card. The control card then sends the motor control information corresponding to the joint to the driver of the corresponding drive motor. The driver controls the drive motor to perform motion compensation to compensate for the deviation between the target posture and the actual posture, assisting the patient in performing rehabilitation training with standardized movements. Further details can be found in the following section. Figure 5 After receiving the motor control signal from the motion control card, the driver (servo driver) of the drive motor releases the magnetic brake of the corresponding joint and drives the motor to move. The Hall sensor installed in the drive motor monitors the motor's current and rotation angle, feeding this information back to the driver. Torque sensors installed at the robot joints detect the human-machine interaction forces during rehabilitation training; this signal is fed back to the robot controller to guide robot control. Furthermore, to ensure safety during exoskeleton robot rehabilitation training, photoelectric limit switches are installed at each joint of the exoskeleton according to the actual joint movement angle of the human body. If the exoskeleton joint movement exceeds the set limit range, the corresponding signal is fed back to the robot controller, which will then stop the robot's movement, ensuring the safety of the rehabilitation training.

[0110] This invention addresses the interaction issues in rehabilitation training by proposing a virtual interactive scenario based on VR technology. By constructing a near-realistic virtual training scenario, the invention aims to increase patient participation during training, thereby enhancing training effectiveness.

[0111] In addition, electromyographic signals of patients can be collected in real time during training, and the rehabilitation status of upper limbs of patients with low motor function can be judged by monitoring changes in electromyographic signals of the upper limbs.

[0112] Accordingly, please refer to Figure 6 This invention provides a robot control system based on multi-source information perception, the system comprising:

[0113] The first acquisition module 601 is used to acquire a first movement-related signal from a patient with low motor function. The first signal includes at least two of the following: electroencephalogram (EEG) signal, inertial signal of the upper limb, electromyographic signal of the upper limb, and force-tactile signal of the upper limb.

[0114] The recognition module 602 is used to perform motion intention recognition on each of the first signals using a random forest machine model to obtain a motion intention recognition result corresponding to the first signal.

[0115] The second acquisition module 603 is used to acquire the weight parameters corresponding to each of the first signals;

[0116] The first intention determination module 604 is used to determine a first motion intention based on the motion intention recognition result and the corresponding weight parameter corresponding to each of the first signals. The first motion intention is the motion intention of the patient with low motor function that has been preliminarily determined.

[0117] The judgment module 605 is used to determine whether the first motion intention conforms to logic based on a pre-established logical decision table;

[0118] The second intention determination module 606 is used to determine the first motion intention as the final determined motion intention if the first motion intention is logically consistent.

[0119] The control signal determination module 607 is used to determine the motion control signal of the upper limb rehabilitation exoskeleton based on the finally determined motion intention;

[0120] The control module 608 is used to control the upper limb rehabilitation exoskeleton to move according to the motion control signal.

[0121] This invention proposes a multi-modal signal fusion-based intention recognition technology. By fusing biological and physical signals, it ensures the accuracy and timeliness of motor intention recognition for patients with low motor function. Specifically, it uses electroencephalography (EEG), electromyography (EMG), inertial signals, and force / tactile signals as recognition signal sources for fusion recognition. Furthermore, to improve the efficiency and accuracy of intention recognition, this invention constructs a random forest machine model to determine the motor intention of each signal source and implements an adaptive attention mechanism (weighting parameters) to achieve decision-level fusion of the recognition results.

[0122] In some specific embodiments, the system further includes:

[0123] A sample signal acquisition module is used to acquire sample signals, wherein the signal type of the sample signals is consistent with the signal type of the first signal, and each type of sample signal includes multiple samples;

[0124] The third acquisition module is used to acquire, for each of the sample signals, the true motion intention corresponding to each sample signal and the motion intention recognition result obtained by recognizing the sample signal using the random forest machine model;

[0125] The recognition accuracy determination module is used to determine the recognition accuracy for each of the sample signals based on the real motion intention and motion intention recognition results corresponding to the multiple sample signals respectively.

[0126] The weight parameter determination module is used to determine the corresponding weight parameter for each of the sample signals based on the recognition accuracy.

[0127] In some specific implementations, the first intent determination module 604 includes:

[0128] The confidence determination unit is used to sum the weight parameters corresponding to the first signal for the first signal whose motion intention recognition result is consistent with the first signal to obtain the confidence of the corresponding motion intention recognition result; for the first signal whose motion intention recognition result is inconsistent with the other first signals, the weight parameter corresponding to the first signal is directly used as the confidence of the corresponding motion intention recognition result.

[0129] The intent determination unit is used to determine the motion intent recognition result corresponding to the maximum confidence level as the first motion intent.

[0130] In some specific implementations, the intent determination unit is used to determine whether the minimum confidence level is greater than a preset threshold; if the minimum confidence level is less than or equal to the preset threshold, the motion intent recognition result corresponding to the maximum confidence level is determined as the first motion intent.

[0131] In some specific embodiments, the system further includes:

[0132] The re-identification module is used to, when the minimum confidence level is greater than the preset threshold, re-apply the random forest machine model to identify the motion intent of the first signal for each type of the first signal, obtain the motion intent identification result corresponding to the first signal, and determine the first motion intent based on the newly obtained motion intent identification result.

[0133] In some specific embodiments, the upper limb rehabilitation exoskeleton includes multiple joints, each joint corresponding to one or more drive motors;

[0134] The control signal determination module 607 is specifically used to determine the drive signal of the drive motor corresponding to at least some joints according to the motion intention. The drive signal includes at least one of drive time, rotational speed of the drive motor, and running direction of the drive motor.

[0135] In some specific embodiments, the system further includes:

[0136] The pose information acquisition module is used to acquire the pose information of each joint of the upper limb rehabilitation exoskeleton;

[0137] An adjustment module is used to adjust the posture of the virtual model corresponding to the upper limb rehabilitation exoskeleton in the virtual scene according to the posture information;

[0138] The deviation acquisition module is used to acquire the deviation information between the current pose and the target pose of the virtual model;

[0139] The posture adjustment signal generation module is used to generate posture adjustment signals for the upper limb rehabilitation exoskeleton based on the deviation information.

[0140] The posture adjustment module is used to control the upper limb rehabilitation exoskeleton to adjust its posture according to the posture adjustment signal.

[0141] The embodiments of the present invention are system embodiments based on the same inventive concept as the method embodiments described above. Therefore, for specific technical details and corresponding technical effects, please refer to the method embodiments described above, which will not be repeated here.

[0142] This invention also provides an electronic device that may include an intent recognition device and an exoskeleton robot, or it may be an exoskeleton robot (intent recognition is implemented by the exoskeleton robot), such as... Figure 7 As shown, the electronic device may include a processor 71 and a memory 72, wherein the processor 71 and the memory 72 can communicate with each other via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0143] Processor 71 can be a central processing unit (CPU). Processor 71 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0144] Memory 72, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the robot control method based on multi-source information perception in this embodiment of the invention (e.g., Figure 6 The first acquisition module 601, the recognition module 602, the second acquisition module 603, the first intent determination module 604, the judgment module 605, the second intent determination module 606, the control signal determination module 607, and the control module 608 are shown. The processor 71 executes various functional applications and data processing by running non-transitory software programs, instructions, and modules stored in the memory 72, thereby realizing the robot control method based on multi-source information perception in the above method embodiments.

[0145] The memory 72 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 71, etc. Furthermore, the memory 72 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 72 may optionally include memory remotely located relative to the processor 71, and these remote memories may be connected to the processor 71 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0146] The one or more modules are stored in the memory 72, and when executed by the processor 71, they perform the following: Figure 1-5 The robot control method based on multi-source information perception in the illustrated embodiment.

[0147] For specific details regarding the aforementioned electronic devices, please refer to the relevant documentation. Figures 1 to 5 The relevant descriptions and effects in the illustrated embodiments are for understanding purposes only and will not be repeated here.

[0148] Accordingly, this embodiment of the invention also provides a computer-readable storage medium for storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described robot control method embodiment based on multi-source information perception and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0149] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0150] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0151] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A robot control system based on multi-source information perception, characterized in that, include: The first acquisition module is used to acquire a first movement-related signal from a patient with low motor function, the first signal including at least two of the following: electroencephalogram (EEG) signal, inertial signal of the upper limb, electromyographic signal of the upper limb, and force-tactile signal of the upper limb. The recognition module is used to perform motion intent recognition on each of the first signals using a random forest machine model, and obtain the motion intent recognition result corresponding to the first signal. The second acquisition module is used to acquire the weight parameters corresponding to each of the first signals; The first intention determination module is used to determine the first motion intention based on the motion intention recognition result and the corresponding weight parameter corresponding to each of the first signals. The first motion intention is the motion intention of the patient with low motor function that has been preliminarily determined. The judgment module is used to determine whether the first motion intention conforms to logic based on a pre-established logical decision table; The second intent determination module is used to determine the first motion intent as the final determined motion intent if the first motion intent is logically consistent. The control signal determination module is used to determine the motion control signal of the upper limb rehabilitation exoskeleton based on the finally determined motion intention; The control module is used to control the upper limb rehabilitation exoskeleton to move according to the motion control signal; Robot control systems based on multi-source information perception also include: A sample signal acquisition module is used to acquire sample signals, wherein the signal type of the sample signals is consistent with the signal type of the first signal, and each type of sample signal includes multiple samples; The third acquisition module is used to acquire, for each of the sample signals, the true motion intention corresponding to each sample signal and the motion intention recognition result obtained by recognizing the sample signal using the random forest machine model; The recognition accuracy determination module is used to determine the recognition accuracy for each of the sample signals based on the real motion intention and motion intention recognition results corresponding to the multiple sample signals respectively. The weight parameter determination module is used to determine the corresponding weight parameter for each of the sample signals based on the recognition accuracy.

2. The robot control system based on multi-source information perception according to claim 1, characterized in that, The first intent determination module includes: The confidence determination unit is used to sum the weight parameters corresponding to the first signal for the first signal whose motion intention recognition result is consistent with the first signal to obtain the confidence of the corresponding motion intention recognition result; for the first signal whose motion intention recognition result is inconsistent with the other first signals, the weight parameter corresponding to the first signal is directly used as the confidence of the corresponding motion intention recognition result. The intent determination unit is used to determine the motion intent recognition result corresponding to the maximum confidence level as the first motion intent.

3. The robot control system based on multi-source information perception according to claim 2, characterized in that, The intent determination unit is specifically used for: Determine whether the minimum confidence level is greater than a preset threshold; If the minimum confidence level is less than or equal to the preset threshold, the motion intent recognition result corresponding to the maximum confidence level is determined as the first motion intent.

4. The robot control system based on multi-source information perception according to claim 3, characterized in that, Also includes: The re-identification module is used to, when the minimum confidence level is greater than the preset threshold, re-apply the random forest machine model to identify the motion intent of the first signal for each type of the first signal, obtain the motion intent identification result corresponding to the first signal, and determine the first motion intent based on the newly obtained motion intent identification result.

5. The robot control system based on multi-source information perception according to claim 1, characterized in that, The upper limb rehabilitation exoskeleton includes multiple joints, and each joint corresponds to one or more drive motors; The control signal determination module is specifically used to determine the drive signal of the drive motor corresponding to at least some joints according to the motion intention. The drive signal includes at least one of drive time, rotational speed of the drive motor, and running direction of the drive motor.

6. The robot control system based on multi-source information perception according to claim 1 or 5, characterized in that, Also includes: The pose information acquisition module is used to acquire the pose information of each joint of the upper limb rehabilitation exoskeleton; An adjustment module is used to adjust the posture of the virtual model corresponding to the upper limb rehabilitation exoskeleton in the virtual scene according to the posture information; The deviation acquisition module is used to acquire the deviation information between the current pose and the target pose of the virtual model; The posture adjustment signal generation module is used to generate posture adjustment signals for the upper limb rehabilitation exoskeleton based on the deviation information. The posture adjustment module is used to control the upper limb rehabilitation exoskeleton to adjust its posture according to the posture adjustment signal.

Images