Brain-controlled upper limb rehabilitation robot brain-machine fusion grasping system and method

Abstract

The application discloses a brain-controlled upper limb rehabilitation robot brain-machine fusion grasping system and method, relates to the field of brain-computer interfaces, and aims to solve the problems of single control mode of traditional robots and poor autonomy in a dynamic non-structural environment. The system comprises a brain control module based on fine hand movement, an intention reasoning module based on fine hand movement, and a rehabilitation robot body module. The brain control module based on fine hand movement is connected with the intention reasoning module based on fine hand movement through a first wireless communication module, the intention reasoning module based on fine hand movement is connected with the rehabilitation robot body module through a second wireless communication module, the brain control module based on fine hand movement sends brain control instructions to the intention reasoning module based on fine hand movement through the first wireless communication module, and the intention reasoning module based on fine hand movement sends control instructions to the rehabilitation robot body module through the second wireless communication module.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interfaces, specifically to a brain-computer fusion grasping system and method for upper limb rehabilitation robots based on brain-controlled intention and knowledge reasoning for patients with disabilities. Background Technology

[0002] Stroke is a disease caused by impaired blood circulation in the brain, resulting in localized loss of neurological function. It is accompanied by symptoms such as speech and limb dysfunction, and is characterized by high incidence and disability rates. Many stroke patients who recover also suffer from hemiplegia. Stroke rehabilitation often utilizes traditional rehabilitation robots (such as those controlled by handgrips, voice, and touchscreens) to rehabilitate limb function. However, this approach suffers from problems such as poor adaptability, lack of real-time feedback, low patient participation, and complex operation.

[0003] Brain-controlled technology, as a cutting-edge human-computer interaction method, has the advantage of establishing a communication channel directly between the brain and external devices without relying on the peripheral nervous system. It can help patients regain basic limb functions by controlling peripheral devices. Helping rehabilitation robots understand the brain-controlled intentions of patients with limb dysfunction can help restore the basic self-care abilities of these patients. Therefore, it is essential to develop brain-computer interface grasping methods for rehabilitation robots based on brain-controlled intentions.

[0004] However, current brain-controlled rehabilitation robot grasping methods suffer from limited control modes and poor autonomy, making it difficult to adapt to the interactive needs of dynamic environments. Summary of the Invention

[0005] The purpose of this invention is to provide a brain-computer interface grasping system and method for brain-controlled upper limb rehabilitation robots, in order to solve the problems of single grasping control mode and poor autonomy in dynamic unstructured environments of traditional robots.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a brain-computer interface grasping system for a brain-controlled upper limb rehabilitation robot, comprising a brain control module based on fine hand movements, an intention reasoning module based on fine hand movements, and a rehabilitation robot body module; wherein, the brain control module based on fine hand movements is connected to the intention reasoning module based on fine hand movements via a first wireless communication module, the intention reasoning module based on fine hand movements is connected to the rehabilitation robot body module via a second wireless communication module, the brain control module based on fine hand movements sends brain control commands to the intention reasoning module based on fine hand movements via the first wireless communication module, and the intention reasoning module based on fine hand movements sends control commands to the rehabilitation robot body module via the second wireless communication module.

[0007] Furthermore, the brain control module based on fine hand movements includes an EEG signal acquisition submodule and an EEG signal processing submodule, which are connected via a third wireless communication module. The EEG signal acquisition submodule is mainly used to acquire EEG signals when the operator performs fine hand movements and transmit the EEG signals to the EEG signal processing submodule. The EEG signal processing submodule performs preprocessing, multi-domain feature extraction, and decoding of the operator's brain control intention based on a convolutional neural network on the received EEG signals.

[0008] Furthermore, the intention reasoning module based on fine hand movements includes a knowledge reasoning submodule and an intention decision submodule, which are connected by a signal. The knowledge reasoning submodule is implemented by an item availability knowledge graph reasoning unit and an expert system reasoning unit based on the analysis of the patient's intention to grasp items. The intention decision submodule determines the final item to be grasped based on the graspable target inferred by the knowledge reasoning submodule and the multi-attribute factors of the graspable target, and transmits the result to the body module of the rehabilitation robot via a wireless communication module.

[0009] Furthermore, the rehabilitation robot body module includes a visual perception submodule and a motion control submodule. The visual perception submodule is responsible for photographing and labeling the items in the item placement area in the ward, and then transmitting the images to the item availability knowledge graph reasoning unit via the second wireless communication module for online knowledge graph construction. The motion control submodule receives control commands inferred by the intention reasoning module based on fine hand movements via the second wireless communication module to complete the autonomous grasping of items.

[0010] Secondly, the present invention provides a brain-computer interface grasping method for a brain-controlled upper limb rehabilitation robot, which is based on the aforementioned brain-computer interface grasping system for a brain-controlled upper limb rehabilitation robot, and includes the following steps: S1. Identification of the operator's brain control intention based on the decoding of fine hand movement EEG signals; S2. The rehabilitation robot uses an availability knowledge graph of graspable objects for intent reasoning. S3. Expert system knowledge reasoning strategy based on environmental factors; S4. Target grasping decision of rehabilitation robot based on multi-attribute weight analysis; S5. The rehabilitation robot's autonomous grasping motion control of the final object to be grasped.

[0011] Furthermore, the specific steps for identifying the operator's mind-controlled intention based on the decoding of fine motor brain signals of the hand include: S11. Design a brain control paradigm based on fine motor skills of the hands; Six fine hand movements were selected as the target movements of the brain control paradigm: right hand five-finger closing movement, right hand index finger single-finger extension movement, right hand thumb single-finger extension movement, left hand five-finger closing movement, left hand index finger single-finger extension movement, and left hand thumb single-finger extension movement. S12. EEG signal acquisition for fine hand movements; The operator performs one of the six fine hand movements included in the brain control paradigm according to their own needs, and the EEG signal acquisition submodule collects the EEG signals of the operator currently performing the fine hand movement. Among them, the EEG signal acquisition submodule selects the 64-lead system under the international standard 10-20 for acquisition; S13. Multi-domain feature extraction of fine motor brain signals of the hand; The EEG signals acquired in S12 were processed by Butterworth bandpass filtering in the range of 4 to 40 Hz and trend term removal. Then, time domain, frequency domain, spatial domain and time-frequency domain features were extracted. The extracted multi-domain features were optimized using transfer learning and reinforcement learning for subsequent decoding of hand fine motor EEG signals. S14. Construct a convolutional neural network model based on multi-domain feature fusion; The extracted EEG signal multi-domain features are fused in a convolutional neural network using a feature fusion method, and the network model is used to decode the multi-domain features of the fine hand movement EEG signal obtained in S13 to obtain the corresponding brain control command information of the operator.

[0012] Furthermore, the specific steps of the rehabilitation robot's intent reasoning based on the availability knowledge graph of graspable objects include: S21. Establish an offline knowledge graph of item availability based on grabbable items in the ward environment; Based on the definition of item availability, the grasping relationship between commonly used items in the ward and fine motor skills is mapped to item availability to obtain the corresponding relationship between fine motor skills, item availability, and the item to be grasped, and an offline knowledge graph of item availability is constructed. S22. Construct an online knowledge graph of the availability of graspable items in the dynamic unstructured environment in which the rehabilitation robot is located; The visual perception submodule captures images of the dynamic, unstructured environment in which the rehabilitation robot is located, and uses the YOLOv5 method to detect, mark, and output objects that the rehabilitation robot can grasp. S23, Matching mechanism for online knowledge graph units; First, based on the offline knowledge graph of item availability constructed in S21, extract the triple information of fine hand movements, item availability, and items to be grasped, and convert it into a dictionary key-value pair representation of the offline graph list; second, compare the item tag information extracted in S22 with the offline knowledge graph list to obtain the online knowledge graph list; finally, based on the brain control intention command of the operator obtained by decoding in step S14, the list of items to be grasped by the operator is finally determined 1.

[0013] Furthermore, the expert system knowledge reasoning strategy based on environmental factors is to establish an expert system based on the analysis of the patient room object grasping intention, according to the environmental factors in the dynamic unstructured environment in which the rehabilitation robot is located, to determine the relationship between environmental factors and grasped objects. Specific steps include: S31. Establish an expert system knowledge base based on the environmental factors of the operator; Based on the relationship between four factors in the operator's environment—time, temperature, special events, and special times—and the objects being grasped, as well as historical cases of grasping objects, a knowledge base for the expert system is constructed. S32. Establish an expert system inference engine based on a traversal case search strategy; Based on the four factors of time, temperature, special events, and special times input into the expert system, the cases contained in the knowledge base are traversed and searched to determine the list of items to be captured based on environmental factors 2. Furthermore, the specific steps of the rehabilitation robot's target grasping decision based on multi-attribute weight analysis include: S41. Determine the multi-attribute factors based on brain control intention, environment, temperature, time, and event importance, and give their weight values ​​based on experience and common sense. Determine the attribute state values ​​of the decision model based on five attributes: brain control intention and environmental factors. S42. Calculate the state value of each item in the list of items to be grabbed in S23, which is inferred from the mind control intent in the online knowledge graph, and the list of items to be grabbed in S32, which is inferred from the environmental attribute features of the expert system. S43. According to the weights of each feature set in step S41, the scores of the items to be grasped are weighted and summed. The final score is then converted into a probability value. The item corresponding to the highest probability value is the item to be grasped by the rehabilitation robot.

[0014] Furthermore, the autonomous grasping motion control of the rehabilitation robot for the final object to be grasped is achieved by the rehabilitation robot body module autonomously planning the motion trajectory based on the final object to be grasped result obtained in S43, and converting the motion trajectory into motion control commands and sending them to its own motion control submodule.

[0015] Compared with the prior art, the beneficial technical effects of the present invention are as follows: (1) This invention extracts features from EEG signals from four dimensions: time domain, frequency domain, spatial domain and time-frequency domain. It also utilizes a feature optimization algorithm that combines transfer learning and reinforcement learning, and a convolutional neural network model that fuses features to fully apply the features of EEG signals. This can effectively improve the decoding accuracy of EEG signals for fine hand movements and solve the problems of insufficient feature extraction and utilization of EEG signals for fine hand movements in the prior art. (2) The present invention designs a reasoning strategy based on the combination of an item availability knowledge graph and an expert system, which enables more target grasping results to be inferred when there are few brain control instructions. This effectively improves the reasoning ability of the rehabilitation robot to understand the operator's intentions and solves the problem that the brain control strategy in the prior art has few brain control instructions and cannot meet the needs of use in unstructured environments. (3) To address the problems of single control and low execution efficiency in current brain-computer interface control strategies, this invention combines a biological intelligence-based brain control strategy with a knowledge-driven reasoning strategy, enabling the brain-controlled grasping system to fully understand the subject's brain control intentions when selecting objects to grasp. This enhances the intelligence of the brain-controlled rehabilitation robot grasping system and further improves the overall execution efficiency of the brain-computer interface system. Attached Figure Description

[0016] Figure 1 This is a block diagram illustrating the control principle of a brain-computer interface grasping system for a brain-controlled upper limb rehabilitation robot according to the present invention. Figure 2 This is an algorithm flowchart of a brain-computer interface grasping method for a brain-controlled upper limb rehabilitation robot according to the present invention; Figure 3 These are schematic diagrams illustrating the six fine hand movements selected for this invention. Figure 4 This is a schematic diagram of the electrode arrangement of the EEG signal processing submodule in this invention; Figure reference numerals: 110-Brain control module based on fine hand movements, 120-Intention reasoning module based on fine hand movements, 130-First wireless communication module, 131-Second wireless communication module, 133-Third wireless communication module, 140-Rehabilitation robot body module, 210-EEG signal acquisition submodule, 220-Portable EEG signal processing submodule, 310-Knowledge reasoning submodule, 320-Intention decision-making submodule, 410-Visual perception submodule, 420-Motion control submodule, 510-Expert system reasoning unit based on ward item grasping intention analysis, 520-Item availability knowledge graph reasoning unit. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0018] Example 1 Please refer to Figure 1 This embodiment of a brain-computer interface grasping system for an upper limb rehabilitation robot includes a brain control module 110 based on fine hand movements, an intention reasoning module 120 based on fine hand movements, and a rehabilitation robot body module 140. The brain control module 110 is connected to the intention reasoning module 120 via a first wireless communication module 130, and the intention reasoning module 120 is connected to the rehabilitation robot body module 140 via a second wireless communication module 131. The brain control module 110 sends brain control commands to the intention reasoning module 120 via the first wireless communication module 130, and the intention reasoning module 120 sends control commands to the rehabilitation robot body module 140 via the second wireless communication module 131.

[0019] Specifically, the brain control module 110 based on fine hand movements includes an EEG signal acquisition submodule 210 and an EEG signal processing submodule. The EEG signal acquisition submodule 210 and the EEG signal processing submodule are connected through a third wireless communication module 133. The EEG signal acquisition submodule 210 is mainly used to acquire the EEG signals when the operator performs fine hand movements and transmit the EEG signals to the EEG signal processing submodule. The EEG signal processing submodule performs preprocessing, multi-domain feature extraction, and operator brain control intention decoding based on convolutional neural networks on the received EEG signals to obtain the operator's intention commands.

[0020] Please refer to Figure 4 In this embodiment, the EEG signal acquisition submodule 210 worn by the operator adopts a portable 64-channel wireless EEG acquisition device, such as the portable wired NeuSen H64-lead EEG cap. EEG signals from the electrode positions Fz, FC3, FC1, FCz, FC2, FC4, C5, C3, C1, Cz, C2, C4, C6, CP3, CPz, and CP4 are selected under the international standard 10-20 channels.

[0021] Specifically, the intention reasoning module 120 based on fine hand movements includes a knowledge reasoning submodule 310 and an intention decision submodule 320, which are connected by a signal. The knowledge reasoning submodule 310 is implemented by an item availability knowledge graph reasoning unit 520 and an expert system reasoning unit 510 based on the analysis of the intention to grasp items in the ward. The intention decision submodule 320 determines the final item to be grasped based on the graspable target inferred by the knowledge reasoning submodule 310 and the multi-attribute factors of the graspable target, and transmits the result to the body module of the rehabilitation robot via a wireless communication module.

[0022] The working principle of the intention reasoning module 120 based on fine hand movements is as follows: First, an offline knowledge graph of item availability based on graspable objects in the ward environment is established; second, the visual perception submodule 410 in the rehabilitation robot body module 140 identifies the graspable objects contained in the dynamic unstructured environment and transmits the identification results to the intention reasoning module 120 based on fine hand movements through the second wireless communication module 131; the item availability knowledge graph reasoning unit 520 in the knowledge reasoning submodule 310 obtains an online item availability knowledge graph through the trained offline knowledge graph unit and the graspable objects identified by the visual perception submodule 410 in the rehabilitation robot body module 140. The brain control intention of the subject is matched with the online knowledge graph to obtain item 1 to be grasped by the operator; the expert system reasoning unit 510 in the knowledge reasoning submodule 310, based on the analysis of the ward item grasping intention, establishes a knowledge base and inference engine according to the environmental factors of the ward to obtain item 2 to be grasped by the operator; the intention decision submodule 320 performs multi-attribute decision on the obtained item 1 and item 2 to be grasped, and finally obtains the item to be grasped by the rehabilitation robot, and transmits it to the rehabilitation robot body module 140 through the second wireless communication module 131.

[0023] Specifically, the rehabilitation robot body module 140 includes a visual perception submodule 410 and a motion control submodule 420. The visual perception submodule 410 is responsible for photographing and labeling the items in the item placement area in the ward, and then transmitting the images to the item availability knowledge graph reasoning unit 520 via the second wireless communication module 131 for online knowledge graph construction. The motion control submodule 420 receives control commands inferred by the intention reasoning module 120 based on fine hand movements via the second wireless communication module 131 to complete the autonomous grasping of items.

[0024] In this embodiment, the rehabilitation robot body module 140 has a seven-degree-of-freedom robotic arm. The motion control submodule 420 in the rehabilitation robot body module 140 autonomously plans the motion trajectory according to the object to be grasped, so as to realize the grasping motion control of the rehabilitation robot based on brain control intention and knowledge reasoning.

[0025] Example 2 Please refer to Figure 2-4 The specific steps of the brain-computer interface grasping method for a brain-controlled upper limb rehabilitation robot in this embodiment are as follows: Step 610: The operator makes fine hand movements, the EEG signal acquisition submodule 210 acquires EEG signals, and transmits the acquired EEG signals to the EEG signal processing submodule through the third wireless communication module 133.

[0026] In this embodiment, the EEG signal acquisition submodule 210 uses a portable wired NeuSen H64 EEG cap to acquire EEG signals from the positions of electrodes Fz, FC3, FC1, FCz, FC2, FC4, C5, C3, C1, Cz, C2, C4, C6, CP3, CPz, and CP4 under the international 10-20 standard.

[0027] In this embodiment, the selected fine motor movements are as follows: gesture 1: closing movement of the five fingers of the right hand; gesture 2: extending movement of the right index finger; gesture 3: extending movement of the right thumb; gesture 4: closing movement of the five fingers of the left hand; gesture 5: extending movement of the left index finger; gesture 6: extending movement of the left thumb.

[0028] Step 620: The EEG signal processing submodule preprocesses the acquired EEG signals.

[0029] In this embodiment, the acquired raw EEG signals are subjected to fourth-order Butterworth bandpass filtering at 4-40Hz and trend term removal.

[0030] Step 630: Perform feature extraction and decoding on the processed EEG signals.

[0031] In this embodiment, the EEG signal processing submodule extracts time-domain, frequency-domain, spatial-domain, and time-frequency-domain features of the fine motor EEG signals of the hand. The extracted features are optimized by the reinforcement learning network NASNet and the transfer learning network CWTNet, and the extracted features are effectively fused by the feature fusion convolutional neural network (FIF-CNN) to improve the EEG decoding accuracy and decode the operator's intention commands.

[0032] Step 640: Construct an offline knowledge graph of item availability.

[0033] In this embodiment, the item availability knowledge graph reasoning unit 520 selects 60 commonly used items in the ward and classifies them into three categories of item availability related to fine motor skills according to the definition of item availability. At the same time, it classifies the items according to their inherent attributes, such as category. Finally, based on the corresponding relationship between fine motor skills, item availability, and the item to be grasped, an offline item availability knowledge graph containing 67 nodes is constructed.

[0034] Step 650: Construct an online knowledge graph of item availability.

[0035] In this embodiment, the visual perception submodule 410 transmits the captured images of the target environment where the rehabilitation robot is located into the YOLOv5 network model to detect objects in the indoor environment, obtains the label of each object in the image, and saves and outputs it; secondly, the item availability knowledge graph reasoning unit 520 combines the hand fine movement nodes and affordance nodes in the constructed offline item availability knowledge graph with the identified item label information; finally, the construction of the online item availability knowledge graph is completed.

[0036] Step 660: Matching mechanism for knowledge graph units.

[0037] In this embodiment, the item availability knowledge graph reasoning unit 520 extracts triple information of fine hand movements, item availability, and items to be grasped based on the constructed item availability offline knowledge graph and converts it into dictionary key-value pairs to obtain an offline graph list; secondly, it compares the extracted item tag information with the offline knowledge graph list to obtain an online knowledge graph list; finally, based on the operator's brain control intention command decoded in step 530, it finally determines the list 1 of items to be grasped by the operator.

[0038] Step 670: Establish an expert system based on the analysis of the patient ward item grabbing intent.

[0039] First, the expert system reasoning unit 510, based on the analysis of the intent to grab items in the ward, establishes a knowledge base, which is a case library constructed according to the relationship between relevant environmental factors in the operator's environment and the items to be grabbed. Then, the inference engine part is established. The case search strategy set by the expert system used in this invention is a traversal search strategy. Based on the environmental factors input into the expert system, the cases contained in the knowledge base are traversed and searched until the traversal is completed, and a list of items to be grabbed 2 is obtained.

[0040] In this embodiment, the knowledge base is represented as Complete cases in the expert system case library for analyzing the intent of grabbing items in the ward Composed of four-dimensional environmental feature information, represented as ,in, Information for a time period within a day, specifically: This refers to the time period from 8:00 AM to 9:00 AM. This refers to the time period from 12:00 to 13:00. This refers to the time period from 5 PM to 6 PM. The ambient temperature at that time was as follows: This indicates that the temperature is moderate. This indicates that the temperature is too high or too low. This is special time information, specifically... Indicates the time for washing up (7:30 am, 9:30 pm). Indicate the time to take the medicine (9:30, 13:30, and 18:30). Indicates no specific time; For special event information, specifically This indicates that you haven't drunk water for a long time. This indicates that you should eat fruit before grabbing the item. This means to take a tissue before grabbing the item. He said he had taken the remote control before. This means to take toiletries before grabbing the item. This indicates that there were no special events. The environmental information collected at that time in the ward was analyzed, and the environmental characteristics were input into the expert system. The meaning of the specific environmental characteristics is shown in Table 1. Table 1. Representation and meaning of environmental characteristics

[0041] Step 580: Identify the specific decision factors in multi-attribute decision-making.

[0042] In this embodiment, the decision factors are derived from four environmental attribute features (T, W, SE, ST) in the expert system case library and the brain control intention attribute H in the online knowledge graph.

[0043] In this embodiment, based on the knowledge and experience of reasoning about items in the case database of the expert system and the relationship between mind control intentions and items in the online knowledge graph, different weights were assigned to the five factor attribute features according to experience and common sense. The weights were set according to the importance of each factor to the item to be grasped. Since the mind control intention H is the key to determining whether the patient grasps the item, H was given the highest weight. Special events (SE) recommend the next item based on the previous item retrieved, with a weight value of [value missing]. Special time ST is more important than ordinary time T, therefore its weight is [value missing]. However, the importance of ordinary time T and temperature W is equal, with a weight of . The final weights are set as follows: The specific attributes and characteristics variables, including their states and meanings, are shown in Table 2.

[0044] Table 2. States and their meanings included in multi-attribute features.

[0045] The scores of each item are weighted and summed according to the predefined feature weights, and then the final score is converted into a probability value. The item corresponding to the highest probability value is the final item to be grabbed.

[0046] Step 690: Based on the final result of the object to be grasped obtained in step 680, the rehabilitation robot body module 140 autonomously plans the motion trajectory and converts the motion trajectory into motion control commands, which are then sent to its own motion control submodule 420 to realize the grasping motion control of the rehabilitation robot based on brain-controlled intention and knowledge reasoning.

Claims

1. A brain-computer interface grasping system for a brain-controlled upper limb rehabilitation robot, comprising a brain-controlled module based on fine hand movements, an intention reasoning module based on fine hand movements, and a rehabilitation robot body module; wherein, The brain-controlled module based on fine hand movements is connected to the intention reasoning module based on fine hand movements via a first wireless communication module. The intention reasoning module based on fine hand movements is connected to the rehabilitation robot body module via a second wireless communication module. The brain-controlled module based on fine hand movements sends brain-controlled commands to the intention reasoning module based on fine hand movements via the first wireless communication module, and the intention reasoning module based on fine hand movements sends control commands to the rehabilitation robot body module via the second wireless communication module. Its features include: The intention reasoning module based on fine hand movements includes a knowledge reasoning submodule and an intention decision submodule, which are connected by a signal. The knowledge reasoning submodule is implemented by an item availability knowledge graph reasoning unit and an expert system reasoning unit based on the analysis of the patient room item grasping intention. The intention decision submodule, based on the graspable targets inferred by the knowledge reasoning submodule and considering the multi-attribute factors of the graspable targets, decides on the final item to be grasped and transmits it to the body module of the rehabilitation robot through a wireless communication module.

2. The brain-computer interface grasping system for brain-controlled upper limb rehabilitation robots according to claim 1, characterized in that: The brain-controlled module based on fine hand movements includes an EEG signal acquisition submodule and an EEG signal processing submodule, which are connected via a third wireless communication module. The EEG signal acquisition submodule is mainly used to acquire EEG signals when the operator performs fine hand movements and transmit the EEG signals to the EEG signal processing submodule. The EEG signal processing submodule performs preprocessing, multi-domain feature extraction, and decoding of the operator's brain control intention based on a convolutional neural network on the received EEG signals.

3. The brain-computer interface grasping system for brain-controlled upper limb rehabilitation robots according to claim 2, characterized in that: The rehabilitation robot's main body module includes a visual perception submodule and a motion control submodule. The visual perception submodule is responsible for photographing and labeling the items in the ward's item placement area, and then transmitting the images to the item availability knowledge graph reasoning unit via a second wireless communication module for online knowledge graph construction. The motion control submodule receives control commands inferred by the intention reasoning module based on fine hand movements via the second wireless communication module, and completes the autonomous grasping of items.

4. A brain-computer interface grasping method for a brain-controlled upper limb rehabilitation robot, implemented based on the brain-computer interface grasping system for a brain-controlled upper limb rehabilitation robot as described in claim 3, characterized in that... It includes the following steps: S1. Identification of the operator's brain control intention based on the decoding of fine hand movement EEG signals; S2. The rehabilitation robot uses an availability knowledge graph of graspable objects for intent reasoning. S3. Expert system knowledge reasoning strategy based on environmental factors; S4. Target grasping decision of rehabilitation robot based on multi-attribute weight analysis; S5. The rehabilitation robot's autonomous grasping motion control of the final object to be grasped.

5. The brain-computer interface grasping method for a brain-controlled upper limb rehabilitation robot according to claim 4, characterized in that, The specific steps for identifying the operator's mind control intention based on decoding fine motor brainwave signals include: S11. Design a brain control paradigm based on fine motor skills of the hands; Six fine hand movements were selected as the target movements of the brain control paradigm: right hand five-finger closing movement, right hand index finger single-finger extension movement, right hand thumb single-finger extension movement, left hand five-finger closing movement, left hand index finger single-finger extension movement, and left hand thumb single-finger extension movement. S12. EEG signal acquisition for fine hand movements; The operator performs one of the six fine hand movements included in the brain control paradigm according to their own needs, and the EEG signal acquisition submodule collects the EEG signals of the operator currently performing the fine hand movement. Among them, the EEG signal acquisition submodule selects the 64-lead system under the international standard 10-20 for acquisition; S13. Multi-domain feature extraction of fine motor brain signals of the hand; The EEG signals acquired in S12 were processed by Butterworth bandpass filtering in the range of 4 to 40 Hz and trend term removal. Then, time domain, frequency domain, spatial domain and time-frequency domain features were extracted. The extracted multi-domain features were optimized using transfer learning and reinforcement learning for subsequent decoding of hand fine motor EEG signals. S14. Construct a convolutional neural network model based on multi-domain feature fusion; The extracted EEG signal multi-domain features are fused in a convolutional neural network using a feature fusion method, and the network model is used to decode the multi-domain features of the fine hand movement EEG signal obtained in S13 to obtain the corresponding brain control command information of the operator.

6. The brain-computer interface grasping method for a brain-controlled upper limb rehabilitation robot according to claim 5, characterized in that, The specific steps of the rehabilitation robot's intent reasoning based on the availability knowledge graph of graspable objects include: S21. Establish an offline knowledge graph of item availability based on grabbable items in the ward environment; Based on the definition of item availability, the grasping relationship between commonly used items in the ward and fine motor skills is mapped to item availability to obtain the corresponding relationship between fine motor skills, item availability, and the item to be grasped, and an offline knowledge graph of item availability is constructed. S22. Construct an online knowledge graph of the availability of graspable items in the dynamic unstructured environment in which the rehabilitation robot is located; The visual perception submodule captures images of the dynamic, unstructured environment in which the rehabilitation robot is located, and uses the YOLOv5 method to detect, mark, and output objects that the rehabilitation robot can grasp. S23, Matching mechanism for online knowledge graph units; First, based on the offline knowledge graph of item availability constructed in S21, extract the triple information of fine hand movements, item availability, and items to be grasped, and convert it into a dictionary key-value pair representation of the offline graph list; second, compare the item tag information extracted in S22 with the offline knowledge graph list to obtain the online knowledge graph list; finally, based on the brain control intention command of the operator obtained by decoding in step S14, the list of items to be grasped by the operator is finally determined 1.

7. The brain-computer interface grasping method for a brain-controlled upper limb rehabilitation robot according to claim 6, characterized in that, The environmental factor-based expert system knowledge reasoning strategy establishes an expert system based on the analysis of the patient room object grasping intention, according to the environmental factors in the dynamic unstructured environment in which the rehabilitation robot is located, to determine the relationship between environmental factors and grasped objects. Specific steps include: S31. Establish an expert system knowledge base based on the environmental factors of the operator; Based on the relationship between four factors in the operator's environment—time, temperature, special events, and special times—and the objects being grasped, as well as historical cases of grasping objects, a knowledge base for the expert system is constructed. S32. Establish an expert system inference engine based on a traversal case search strategy; Based on four factors input into the expert system—time, temperature, special events, and special times—the cases contained in the knowledge base are traversed and searched to determine a list of items to be captured based on environmental factors.

8. The brain-computer interface grasping method for a brain-controlled upper limb rehabilitation robot according to claim 7, characterized in that, The specific steps of the rehabilitation robot's target grasping decision based on multi-attribute weight analysis include: S41. Determine the multi-attribute factors based on brain control intention, temperature, time, special events, and special times, and give the weight value of each attribute factor based on experience and common sense. Determine the attribute state value of the five attribute decision model based on brain control intention and environmental factors. S42. Calculate the state value of each item in the list of items to be grabbed in S23, which is inferred from the mind control intent in the online knowledge graph, and the list of items to be grabbed in S32, which is inferred from the environmental attribute features of the expert system. S43. According to the weights of each feature set in step S41, the scores of the items to be grasped are weighted and summed. The final score is then converted into a probability value. The item corresponding to the highest probability value is the item to be grasped by the rehabilitation robot.

9. The brain-computer interface grasping method for a brain-controlled upper limb rehabilitation robot according to claim 8, characterized in that, The autonomous grasping motion control of the rehabilitation robot for the final object to be grasped is achieved by the rehabilitation robot body module autonomously planning the motion trajectory based on the final object to be grasped result obtained in S43, and converting the motion trajectory into motion control commands and sending them to its own motion control submodule.

Images