Brain signal processing method, system and equipment for assisting cerebral palsy patient in movement and medium

Abstract

The method is mainly applied to the technical field of brain-computer interface and time sequence signal analysis. The invention discloses a brain signal processing method, system and device for assisting a cerebral palsy patient in movement and a medium. The method comprises the steps that a first electroencephalogram signal and a second electroencephalogram signal are obtained; generating a first instruction set according to the first electroencephalogram signal, and generating a second instruction set according to the second electroencephalogram signal; adjusting the second instruction set by taking the first instruction set as a target to obtain the adjusted second instruction set and taking the adjusted second instruction set as a target instruction set; and according to the target instruction set, controlling preset auxiliary equipment, so that the user completes the motion action through the preset auxiliary equipment. The brain signal of the cerebral palsy patient can be adjusted, the cerebral palsy patient is assisted in completing movement actions, and therefore the movement function of the cerebral palsy patient can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of brain-computer interface and timing signal analysis, and in particular to a brain signal processing method, system, device and medium for assisting cerebral palsy patients in movement. Background Art

[0002] Cerebral palsy, the full name of which is cerebral palsy, refers to early brain development abnormalities that occur before or after birth. It is a syndrome of continuous and progressive brain damage caused by a variety of reasons. The overall incidence of the disease is about 2% to 3%, and the incidence in my country is 2.48%. Cerebral palsy not only causes movement disorders in patients, but also disorders in cognition, perception, sensation, language communication, etc. It may also lead to secondary skeletal muscle system lesions and epilepsy, which seriously affect the growth and development of children. There is a lack of specific drug treatment. Early rehabilitation exercise training for patients with cerebral palsy can effectively improve the prognosis. However, providing a solution for rehabilitation exercise training for patients with cerebral palsy has always been a problem that the industry has not yet solved. Summary of the invention

[0003] The present invention provides a brain signal processing method, system, device and medium for assisting cerebral palsy patients in movement, which can regulate the brain signals of cerebral palsy patients and assist cerebral palsy patients in completing movement movements, thereby helping to improve the motor function of cerebral palsy patients.

[0004] The present invention provides a brain signal processing method for assisting cerebral palsy patients in movement, the method comprising: Acquire a first electroencephalogram signal and a second electroencephalogram signal; generating a first instruction set according to the first EEG signal, and generating a second instruction set according to the second EEG signal; Taking the first instruction set as a target, adjusting the second instruction set to obtain an adjusted second instruction set as a target instruction set; According to the target instruction set, the preset auxiliary device is controlled so that the user can complete the exercise action through the preset auxiliary device.

[0005] Further, taking the first instruction set as a target and adjusting the second instruction set to obtain an adjusted second instruction set as a target instruction set includes: dividing the second instruction set into a plurality of instruction subsets; Filtering out a target instruction subset from the plurality of instruction subsets; The neighborhood of the target instruction subset is used as the target instruction set.

[0006] Furthermore, dividing the second instruction set into a plurality of instruction subsets includes: Through a preset neural network of adaptive learning, all instructions in the second instruction set and the corresponding action videos when the second instruction set is collected are subjected to multimodal unsupervised clustering, so that the second instruction set is clustered into multiple instruction subsets.

[0007] Further, the step of filtering out a target instruction subset from the plurality of instruction subsets comprises: Filtering the multiple instruction subsets according to the number of instructions in each instruction subset, and using the filtered instruction subsets as small instruction subsets; The target instruction subset is selected from the multiple small instruction subsets, wherein the target instruction subset is the small instruction subset with the highest feature similarity to the first instruction set among the multiple small instruction subsets.

[0008] Furthermore, the multiple instruction subsets are screened according to the number of instructions in the instruction subsets, and the screened instruction subsets are used as small instruction subsets, including: Arrange each of the instruction subsets in order from small to large according to the number of instructions in each instruction subset to obtain an instruction subset sequence; The instruction subset whose sorting number is less than the preset sorting value in the instruction subset sequence is regarded as a small instruction subset.

[0009] Furthermore, the target instruction subset is selected from the plurality of small instruction subsets, wherein the target instruction subset is a small instruction subset with the highest feature similarity to the first instruction set among the plurality of small instruction subsets, including: Performing multimodal feature extraction on the mean of the first instruction set and the action video corresponding to the acquisition of the first instruction set through a trained preset neural network autoencoder; The distances between the extracted features and each feature in the small instruction subset are calculated respectively, and the small instruction subset containing the calculation result with the shortest distance is determined as the target instruction subset.

[0010] Furthermore, the preset neural network through adaptive learning performs multimodal unsupervised clustering on all instructions in the second instruction set and the action video corresponding to the acquisition of the second instruction set, so that the second instruction set is clustered into a plurality of instruction subsets, including: Performing feature extraction processing on the instruction format data samples and the video format data samples in the second instruction set through a preset neural network of adaptive learning to obtain format sample features and video sample features; splicing the format sample features and the video sample features into a total feature and performing fusion clustering; When the instruction format data sample and the video format data sample are subjected to feature extraction processing, the preset neural network is trained based on the reconstruction loss and the total clustering loss of the instruction format data sample and the video format data sample until a trained neural network is obtained; The instruction format data samples are classified into corresponding target instruction subsets through the trained neural network.

[0011] Furthermore, the step of performing feature extraction processing on the instruction format data sample includes: Preprocessing the instruction format data sample to obtain a low signal-to-noise ratio ultra-short-time signal; When the low signal-to-noise ratio ultra-short-time signal is a broadband signal, decomposing the low signal-to-noise ratio ultra-short-time signal to obtain a plurality of narrowband signals; When the low signal-to-noise ratio ultra-short-time signal is a narrowband signal, a narrowband signal is obtained; Feature extraction processing is performed according to the characteristic amplitude contained in each narrowband signal, and a feature vector is obtained through the preset neural network.

[0012] Further, the method comprises: The movement action is detected, and when the detection result does not meet the preset conditions, the step of obtaining the first EEG signal and the second EEG signal is returned to execute.

[0013] The present invention provides a system for assisting cerebral palsy patients in exercising, the system comprising a controller module and an auxiliary device module, the controller module comprising: An input unit, used to obtain a first EEG signal and a second EEG signal; a conversion unit, configured to generate a first instruction set according to the first EEG signal, and to generate a second instruction set according to the second EEG signal; an adjusting unit, configured to adjust the second instruction set with the first instruction set as a target, so as to obtain an adjusted second instruction set as a target instruction set; An execution unit, used to control the auxiliary device module according to the target instruction set, so that the user can complete the movement action through the auxiliary device module; The auxiliary device module is used to interact with the user through the wearable hardware device according to the instructions of the execution unit, and support and drive the user's limbs to perform movement movements.

[0014] The present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the brain signal processing method for assisting cerebral palsy patients in movement as described in any one of the above items is implemented.

[0015] The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the brain signal processing method for assisting cerebral palsy patients in movement as described in any one of the above items is implemented.

[0016] The present invention has at least the following beneficial effects: The brain disease of cerebral palsy patients will cause them to want to do the same action as normal people, and the second EEG signal generated is very different from the first EEG signal of normal people, so that the limbs receive the wrong signal and make wrong movements. In this technical solution, a brain signal processing method for assisting cerebral palsy patients in movement can continuously adjust and train the process of cerebral palsy patients to generate EEG signals from thoughts, so that the patient's EEG generation gradually recovers to the direction of normal people's EEG signals. First, obtain the first EEG signal and the second EEG signal, for example, by collecting the EEG signal of a person with normal motor function as the first EEG signal, and by collecting the EEG signal of a cerebral palsy patient as the second EEG signal, then, by converting the EEG signal into an instruction set in the form of data, generate a first instruction set corresponding to the first EEG signal and a second instruction set corresponding to the second EEG signal, and then, with the first instruction set as the target, adjust the second instruction set so that the adjusted second instruction set is closer to the first instruction set than before. The adjusted second instruction set is used as the target instruction set and is used to control the preset auxiliary device, so that the preset auxiliary device can complete the adjusted movement under the control of the action instructions in the target instruction set, so that when the cerebral palsy patient uses the preset auxiliary device as a user, it helps the cerebral palsy patient to correct the originally abnormal movement into a near-normal movement through continuous intensive training. After training, the patient can spontaneously issue a near-correct instruction without the auxiliary device, and then independently make a near-correct movement. This technical solution is conducive to improving various motor functions of cerebral palsy patients by learning from one example. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0018] Figure 1 is a flowchart of the steps of a brain signal processing method for assisting movement in patients with cerebral palsy; Figure 2 The present invention is a schematic diagram of a step of adjusting a second instruction set of a brain signal processing method for assisting movement in patients with cerebral palsy; Figure 3 The schematic diagram is a step of classifying data samples in a brain signal processing method for assisting cerebral palsy patients in locomotion; Figure 4This is a schematic diagram of the training process of an unsupervised autoencoder clustering neural network model used in a brain signal processing method for assisting cerebral palsy patients in locomotion; Figure 5 is a schematic diagram of the structure of a system for assisting movement in patients with cerebral palsy; Figure 6 It is a structural diagram of an electronic device. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0020] In recent years, with the continuous development of medicine, brain-computer interface technology has been applied to the field of rehabilitation treatment for patients with cerebral palsy. It has shown excellent performance and has begun to be deployed in actual scenarios. Brain-computer interface devices can control external devices according to human intentions, activate the cerebral cortex during training, and improve the patient's limb motor function. Exoskeletons can provide highly repeatable task-oriented functional training for stroke patients, induce continuous motor learning, enhance neural remodeling, and promote motor function rehabilitation. Inspired by the combination of brain-computer interface devices and exoskeleton technologies, this invention has developed exoskeleton technology based on brain-computer interface regulation, and has gradually been used in the motor rehabilitation of patients with cerebral palsy. Various embodiments of the solution provided by the present invention are as follows: Please refer to Figure 1 , Figure 1 A flowchart of the steps of a brain signal processing method for assisting movement in patients with cerebral palsy.

[0021] In a first aspect, this embodiment provides a brain signal processing method for assisting cerebral palsy patients in movement, including: S100: Acquire a first electroencephalogram signal and a second electroencephalogram signal.

[0022] S200, generating a first instruction set according to the first EEG signal, and generating a second instruction set according to the second EEG signal.

[0023] S300: Taking the first instruction set as a target, adjusting the second instruction set to obtain an adjusted second instruction set as a target instruction set.

[0024] S400: Control a preset auxiliary device according to a target instruction set, so that a user can complete a movement through the preset auxiliary device.

[0025] In some embodiments, the movement is detected, and when the detection result does not meet the preset condition, the process returns to step S100. Through this cycle, the user can improve or restore the movement function through repeated exercises with the help of auxiliary equipment.

[0026] In step S100 of some embodiments, the first EEG signal is an EEG signal of a person with normal motor function, and the second EEG signal is an EEG signal of a person with cerebral palsy.

[0027] In step S200 of some embodiments, the electrical signals of the brains of normal people and cerebral palsy patients are sampled and analyzed respectively through a non-invasive brain-computer interface, and the collected EEG signals are classified and converted into data sets with the help of a signal conversion module.

[0028] In step S300 of some embodiments, during the adjustment process, the EEG signals emitted by the cerebral palsy patient and the EEG signals emitted by a normal person are gradually converged. When the cerebral palsy patient wants to complete a certain action, an erroneous EEG signal is emitted, and then the EEG signal is converted into an instruction and a second instruction set is generated; the second instruction set is compared with the first instruction set, and the second instruction set is adjusted according to the comparison result; and a new target instruction set is generated.

[0029] In step S400 of some embodiments, the user wears an exoskeleton auxiliary device and starts training to control the brain to correctly send out EEG signals within the adjusted target instruction set. After collecting the user's EEG signals, the exoskeleton auxiliary device drives the cerebral palsy patient to perform the action once if and only if the EEG signal sent by the cerebral palsy patient falls within the adjusted target instruction set; if it falls outside the adjusted target instruction set, the exoskeleton does not perform any action. After completing a round, the user is prompted to send out EEG signals again. After each successful action, the cerebral palsy patient should recall the thoughts and ideas when the exoskeleton was triggered to lead him to complete the action, and then reproduce the intention to perform the action again according to the thoughts and ideas when sending out the EEG signal next time. Through continuous intensive training, the user is familiar with the "feeling" of the adjusted action and control, and the degree of migration of the user's EEG to the target instruction set and the response rate of the exoskeleton are improved. This type of training lasts for weeks or months. When the user's current EEG signal sets in the latest period - week / month (actually selected according to the progress of EEG signal migration during user use) are all distributed within the range of the target instruction set, the intensive training is consolidated and step S400 can be regarded as completed.

[0030] In some embodiments, the user's designated motion after final training is detected. When the detection result does not meet the preset conditions, that is, the user believes that the motion is aesthetically pleasing and still needs further improvement or cannot achieve the purpose of the motion, it is determined that the target instruction set still needs to be adjusted toward the first instruction set, and the process returns to execute step S100.

[0031] In some preferred embodiments, by continuously adjusting the threshold during the adjustment of the second instruction set, the EEG signal emitted by the patient is guided to change from the second instruction set to the first instruction set, and the target instruction set is continuously improved and enhanced. By using the auxiliary effect of preset auxiliary equipment, this feedback mechanism is strengthened, so that the patient's motor function is continuously improved, and ultimately independent movement is achieved without the preset auxiliary equipment.

[0032] In some embodiments, step S300 further includes: (310) Divide the second instruction set into a plurality of instruction subsets.

[0033] (320) Filter out a target instruction subset from multiple instruction subsets.

[0034] (330) The neighborhood of the target instruction subset is taken as the target instruction set.

[0035] See also Figure 2 In a specific embodiment, the first instruction set is named instruction set A, and the second instruction set is named instruction set C. The ideal goal is to completely migrate the C instruction set to the A instruction set. In actual operation, the compromise result of the migration process from C to A is named instruction set B. The preset auxiliary device is a human exoskeleton. When a normal person sends out an EEG signal, it is collected and processed and classified into the first instruction set A. When the exoskeleton actuator receives A, it will make normal human movements; when a cerebral palsy patient sends out an EEG signal, it is collected and processed and classified into the second instruction set C. Because the EEG signals of cerebral palsy patients are erroneous and unstable, and have a certain degree of randomness, it is necessary to collect the EEG signals sent by the patient when he wants to walk multiple times, and cut each EEG signal into a data sample of a specific length. Each data sample is not exactly the same, and these samples are randomly distributed in the second instruction set C (i.e., in the dotted circle C). This embodiment divides the instruction set C into several categories based on an original multimodal unsupervised clustering algorithm, and selects C whose feature mean is closest to the feature of the first instruction set A. A1 Class (ie real coil C A1 ), named as the local optimal class, and constitute the target instruction subset C in the second instruction set A1 Then the target instruction subset C A1 The neighborhood (i.e., the dotted circle C A1 The target instruction set is then divided into several categories, from which the C whose feature mean is closest to the feature of the first instruction set A is selected. A2 Class (ie real coil C A2 and constitute the target instruction subset C in the current target instruction set A2 Then the target instruction subset C A2 The neighborhood (i.e., the dotted circle C A2 Repeat the above process of classifying the optimal instruction subset to generate the target instruction subset CA3 , C A4 , ...C An , and complete the iteration of the target instruction set until the final metastable instruction set with a compromise effect is reached, named instruction set B.

[0036] In some embodiments, step (320) includes: The multiple instruction subsets are screened according to the number of instructions in each instruction subset, and the screened instruction subsets are used as small instruction subsets; a target instruction subset is selected from the multiple small instruction subsets, wherein the target instruction subset is the small instruction subset with the highest feature similarity to the first instruction set among the multiple small instruction subsets.

[0037] In a specific embodiment, in the process of filtering multiple approximate samples from the current instruction set and generating a target instruction subset, the cerebral palsy patient wears a preset auxiliary device (BCI exoskeleton) for training, and the EEG signal emitted by the cerebral palsy patient is in the C A1 In the neighborhood (dashed circle), that is, the data samples converted from the EEG signals emitted by cerebral palsy patients meet the preset approximation requirements and become the optimal instruction subset C A1 When the patient follows the instructions in the BCI exoskeleton, the BCI controls the exoskeleton to drive the patient to walk normally. During this process, the cerebral palsy patient needs to wear the preset auxiliary equipment (BCI exoskeleton) for repeated training until the EEG signals spontaneously emitted by the cerebral palsy patient when he wants to walk are completely transferred from the C neighborhood to the C A1 Neighborhood, target instruction subset C A1 The neighborhood becomes the current instruction set. Similarly, based on multimodal clustering and approximate metric algorithm, C A1 , C A2 , ..., C An Class, thereby generating the target instruction subset C A3 , C A4 , ...C An The iteration of the current instruction set is completed until the user thinks that the action is beautiful and can achieve the action purpose, with a certain compromise effect. The current target instruction set is correspondingly migrated to the final metastable instruction set B.

[0038] See also Figure 3In a specific embodiment, at the beginning of the movement training, each time a cerebral palsy patient "steps forward" (or an erroneous movement made when intending to move forward) walks, a gait video is captured and an EEG signal is collected simultaneously, and the two together constitute a multimodal sample. The sample sets collected repeatedly are subjected to unsupervised clustering and divided into three categories. The instruction subsets generated by the EEG signals in each category of samples correspond to C1, C2, and C3, respectively. There are c1, c2, and c3 instructions in the three subsets, and c3≤c2≤c1. The preset sorting value is 2. C2 and C3 are used as small instruction subsets, and multimodal features are extracted from the mean of the normal person's instruction set A and the corresponding action video when the instruction set A is collected; the distance between the extracted features and each feature in the subsets C1 and C2 is calculated, and the small instruction subset C3 with the closest calculation result is determined as the target instruction subset, named , The data in the class are the data closest to the normal person's "taking a step forward" among the three classes.

[0039] In some embodiments, a specific implementation method of filtering multiple instruction subsets according to the number of instructions in the instruction subset and using the filtered instruction subsets as small instruction subsets includes: arranging each instruction subset in sequence from small to large according to the number of instructions in each instruction subset to obtain an instruction subset sequence; and using the instruction subset in the instruction subset sequence whose sorting number is less than a preset sorting value as a small instruction subset.

[0040] In some embodiments, a specific implementation method of selecting a target instruction subset from multiple small instruction subsets includes: performing multimodal feature extraction on the mean of the first instruction set and the action video corresponding to the collection of the first instruction set through a trained preset neural network autoencoder; calculating the distance between the extracted features and each feature in the small instruction subset, and determining the small instruction subset with the closest calculation result as the target instruction subset.

[0041] In step (310) of some embodiments, all instructions in the second instruction set and the corresponding action video when the second instruction set is collected are subjected to multimodal unsupervised clustering through a preset neural network for adaptive learning, so that the second instruction set is clustered into multiple instruction subsets.

[0042] Specifically, see Figure 4, through the preset neural network of adaptive learning, feature extraction processing is performed on the instruction format data samples and video format data samples in the second instruction set, the instruction format data samples are used as input samples A, input to the adaptive learning autoencoder A, obtain feature A, and restore output A through decoder A and obtain reconstruction loss A; the video format data samples are used as input samples B, input to the adaptive learning autoencoder B, obtain feature B, and restore output B through decoder B and obtain reconstruction loss B; feature A and feature B are spliced ​​into total features and then fused and clustered, and the total clustering loss is obtained. Based on the reconstruction loss A of the instruction format data samples, the reconstruction loss B of the video format data samples and the total clustering loss, encoder A and encoder B are jointly optimized. In this process, the reconstruction loss A, reconstruction loss B, and total clustering loss are cyclically calculated, and the preset neural network is continuously trained accordingly until the trained neural network is obtained. Then, the instruction format data samples are clustered into several instruction subsets through the trained neural network.

[0043] In some embodiments, the step of performing feature extraction processing on instruction format data samples includes: preprocessing the instruction format data samples to obtain a low signal-to-noise ratio ultra-short time signal; when the low signal-to-noise ratio ultra-short time signal is a broadband signal, decomposing the low signal-to-noise ratio ultra-short time signal to obtain multiple narrowband signals; when the low signal-to-noise ratio ultra-short time signal is a narrowband signal, obtaining a narrowband signal; performing feature extraction processing according to the characteristic amplitude contained in each narrowband signal, and obtaining a feature vector through a preset neural network.

[0044] It can be understood that when the low signal-to-noise ratio ultra-short time signal is a broadband signal, it is necessary to decompose the low signal-to-noise ratio ultra-short time signal into multiple narrowband signals. During decomposition, a group of bandpass filters are used to decompose the low signal-to-noise ratio ultra-short time signal into multiple narrowband components, each narrowband component is a narrowband signal; the bandpass filter can be implemented based on the filter circuit before the signal analog-to-digital conversion, or it can be implemented based on the digital filter after the signal analog-to-digital conversion. Affected by noise interference, the amplitude of the narrowband signal changes with time, so different characteristic amplitudes will be solved based on different double-point pairs. These required characteristic amplitudes jointly describe the characteristics of the narrowband signal and are used together to identify the narrowband signal. When obtaining the characteristic vector, the characteristic amplitude of each narrowband signal is extracted based on several double-point pairs, and the amplitudes of all narrowband signals together constitute the characteristics of the narrowband signal.

[0045] Second, reference Figure 5 , Figure 5 A schematic diagram of the structure of a system for assisting movement in patients with cerebral palsy.

[0046] This embodiment provides a system for assisting cerebral palsy patients in exercising. The system includes a controller module 510 and an auxiliary device module 520 .

[0047] The controller module 510 includes: The input unit 511 is used to obtain the first EEG signal and the second EEG signal.

[0048] The conversion unit 512 is used to generate a first instruction set according to the first EEG signal, and to generate a second instruction set according to the second EEG signal.

[0049] The adjustment unit 513 is used to adjust the second instruction set with the first instruction set as the target, so as to obtain the adjusted second instruction set as the target instruction set.

[0050] The execution unit 514 is used to control the auxiliary device module according to the target instruction set, so that the user can complete the movement action through the auxiliary device module.

[0051] The auxiliary device module 520 is used to interact with the user through the wearable hardware device according to the execution unit instructions, and support and drive the user's limbs to perform movements.

[0052] In some embodiments of the system for assisting cerebral palsy patients in locomotion, the assistive device module is an exoskeleton module.

[0053] In addition, the present invention also provides an embodiment implemented in an application scenario: First, the rehabilitation goal for cerebral palsy patients is determined to be high leg raising. The electroencephalogram (EEG) of the cerebral cortex reflects the movement control information of the human body, and the electromyogram (EMG) of the body's muscle tissue reflects the muscle's response information to the brain's control. All muscles work together to complete high leg raising. A set of correct electromyographic signals is required to generate correct electromyographic signals. A set of correct neural instructions is required to generate the correct electromyographic signals. Let this instruction set be A, which is the instruction set generated by the EEG signals of normal people.

[0054] The first step is to let a normal person wear a brain-computer interface and perform high-leg raising movements on the spot for many times. Through 8 electrode channels and adjusting the sampling rate to 250Hz, the EEG signals emitted by the normal person when raising their legs are collected, that is, the first EEG signals, and generate the first instruction set A.

[0055] The second step is to inform the cerebral palsy patient that his task is to raise his legs high, and order the patient to repeat the action many times and use the intention of raising his legs high to control the body to perform the action; through 8 electrode channels and adjusting the sampling rate to 250Hz, the EEG signals emitted by the cerebral palsy patient when raising his legs high are collected, that is, the second EEG signals, and the second instruction set C is generated.

[0056] Patients with cerebral palsy want to do the correct high-leg lift movement, but this intention cannot generate the corresponding correct EEG signal. At the same time, there are great individual differences between different patients, which will produce different types of neural command errors, leading to different types of movement disorders, such as kicking, tiptoeing, squatting and other various incorrect movements.

[0057] The EEG signals generated by multiple attempts of cerebral palsy patients are collected and cut into samples of specific length; each EEG sample is not exactly the same and is randomly distributed in the neighborhood of instruction set C; based on the original multimodal unsupervised clustering algorithm, the instruction set C is classified into C1, C2, and C3, and there are c1, c2, and c3 instructions in the three subsets respectively, and c1≤c2≤c3, the preset sorting value is 2, C1 and C2 are used as small instruction subsets, and the mean of instruction set A and the corresponding action video when instruction set A is collected are used for multimodal feature extraction; the distance between the extracted features and each feature in subsets C1 and C2 is calculated respectively, and the small instruction subset C1 where the calculation result with the closest distance is located is determined as the target instruction subset, which is named the target instruction subset ,Will The neighborhood of is defined as the target instruction set.

[0058] Next, the patient wore the leg exoskeleton and began the transfer training phase. The patient with cerebral palsy continued to try the high leg lift intention movement. When the patient's command fell on When the patient gives a command, the exoskeleton will lead the cerebral palsy patient to perform a high leg lift to strengthen the thought and intention. When the patient gives other commands, the exoskeleton will not move. Repeat this process until the patient gradually becomes familiar with issuing commands under the reinforced feedback of the exoskeleton. The "feel" of the command, each subsequent command is stable In the process, a transition from C to The process of neighborhood migration ends this round of exoskeleton training.

[0059] Next, the cerebral palsy patient is asked to continue to try high leg lifting without the exoskeleton while thinking about the feeling of training. The movement made is tested. The movement made by the cerebral palsy patient is still not high leg lifting, but he can stand on tiptoe stably with one leg. It is determined that the target instruction set still needs to be adjusted to the first instruction set. Return to repeat the first step and proceed to the next stage of adjustment. According to the precondition, it can be known that the second instruction set is now the target instruction set. Repeat the classification of the current instruction set to find the target instruction subset C that is closer to A. A2 ,Will The neighborhood of is defined as the target instruction set. Next, the exoskeleton training process is carried out until a Neighborhood to C A2 The process of neighborhood migration. A2The action performed in the stage is standing on tiptoe and bending the knee. This cycle is repeated until the cerebral palsy patient gradually moves from standing on tiptoe, bending the knee on tiptoe, and raising one leg, and approximately completes the high-leg raising action close to that of a normal person. At this time, the target instruction set is the final metastable instruction set B adapted to the specific cerebral palsy patient and the specific high-leg raising action, and the EEG signal processing for high-leg raising is completed.

[0060] It will be appreciated by those skilled in the art that all or some of the steps and devices in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. As is well known to those skilled in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0061] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are the same as those achieved by the above method embodiments.

[0062] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing any of the above brain signal processing methods for assisting cerebral palsy patients in movement when executing the computer program.

[0063] refer to Figure 6 , Figure 6 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes: The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store operating devices and other applications. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the brain signal processing method for assisting cerebral palsy patients in movement in the embodiments of this application; Input / output interface 903, used to implement information input and output; Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.); A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904); The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0064] It can be understood that the contents of the above method embodiments are all applicable to the electronic device embodiment, the functions specifically implemented by the electronic device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement a brain signal processing method for assisting cerebral palsy patients in movement as described in any one of the above-mentioned specific embodiments.

[0066] An embodiment of the present application also discloses a computer program product, including a computer program or computer instructions, wherein the computer program or computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device executes the brain signal processing method for assisting cerebral palsy patients in movement as described in any of the previous embodiments.

[0067] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0068] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0069] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0070] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0071] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0072] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0073] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0074] Although the description of the present application has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims by reference to the attached claims, taking into account the prior art, so as to effectively cover the intended scope of the present application. In addition, the above description of the present application is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present application that have not yet been foreseen may still represent equivalent changes to the present application.

Claims

1. A brain signal processing method for assisting cerebral palsy patients in movement, characterized in that: The method comprises: Acquire a first electroencephalogram signal and a second electroencephalogram signal; generating a first instruction set according to the first EEG signal, and generating a second instruction set according to the second EEG signal; Taking the first instruction set as a target, adjusting the second instruction set to obtain an adjusted second instruction set as a target instruction set; According to the target instruction set, the preset auxiliary device is controlled so that the user can complete the exercise action through the preset auxiliary device.

2. A brain signal processing method for assisting cerebral palsy patients in movement according to claim 1, characterized in that: The step of taking the first instruction set as a target and adjusting the second instruction set to obtain an adjusted second instruction set as a target instruction set includes: dividing the second instruction set into a plurality of instruction subsets; Filtering out a target instruction subset from the plurality of instruction subsets; The neighborhood of the target instruction subset is used as the target instruction set.

3. A brain signal processing method for assisting cerebral palsy patients in movement according to claim 2, characterized in that: The dividing the second instruction set into a plurality of instruction subsets comprises: Through a preset neural network of adaptive learning, all instructions in the second instruction set and the corresponding action videos when the second instruction set is collected are subjected to multimodal unsupervised clustering, so that the second instruction set is clustered into multiple instruction subsets.

4. A brain signal processing method for assisting cerebral palsy patients in movement according to claim 2, characterized in that: The step of selecting a target instruction subset from the plurality of instruction subsets comprises: Filtering the multiple instruction subsets according to the number of instructions in each instruction subset, and using the filtered instruction subsets as small instruction subsets; The target instruction subset is selected from the multiple small instruction subsets, wherein the target instruction subset is the small instruction subset with the highest feature similarity to the first instruction set among the multiple small instruction subsets.

5. A brain signal processing method for assisting cerebral palsy patients in movement according to claim 4, characterized in that: The multiple instruction subsets are screened according to the number of instructions in the instruction subsets, and the screened instruction subsets are used as small instruction subsets, including: Arrange each of the instruction subsets in order from small to large according to the number of instructions in each instruction subset to obtain an instruction subset sequence; The instruction subset whose sorting number is less than the preset sorting value in the instruction subset sequence is regarded as a small instruction subset.

6. A brain signal processing method for assisting cerebral palsy patients in locomotion according to claim 4, characterized in that: The target instruction subset is selected from the plurality of small instruction subsets, wherein the target instruction subset is a small instruction subset with the highest feature similarity to the first instruction set among the plurality of small instruction subsets, including: Performing multimodal feature extraction on the mean of the first instruction set and the action video corresponding to the acquisition of the first instruction set through a trained preset neural network autoencoder; The distances between the extracted features and each feature in the small instruction subset are calculated respectively, and the small instruction subset containing the calculation result with the shortest distance is determined as the target instruction subset.

7. The method for processing brain signals for assisting cerebral palsy patients in locomotion according to claim 3, characterized in that: The preset neural network through adaptive learning performs multimodal unsupervised clustering on all instructions in the second instruction set and the action video corresponding to the acquisition of the second instruction set, so that the second instruction set is clustered into multiple instruction subsets, including: Performing feature extraction processing on the instruction format data samples and the video format data samples in the second instruction set through a preset neural network of adaptive learning to obtain format sample features and video sample features; splicing the format sample features and the video sample features into a total feature and performing fusion clustering; When the instruction format data sample and the video format data sample are subjected to feature extraction processing, the preset neural network is trained based on the reconstruction loss and the total clustering loss of the instruction format data sample and the video format data sample until a trained neural network is obtained; The instruction format data samples are classified into corresponding target instruction subsets through the trained neural network.

8. The method for processing brain signals for assisting cerebral palsy patients in locomotion according to claim 7, characterized in that: The step of performing feature extraction processing on the instruction format data sample comprises: Preprocessing the instruction format data sample to obtain a low signal-to-noise ratio ultra-short-time signal; When the low signal-to-noise ratio ultra-short-time signal is a broadband signal, decomposing the low signal-to-noise ratio ultra-short-time signal to obtain a plurality of narrowband signals; When the low signal-to-noise ratio ultra-short-time signal is a narrowband signal, a narrowband signal is obtained; Feature extraction processing is performed according to the characteristic amplitude contained in each narrowband signal, and a feature vector is obtained through the preset neural network.

9. The method for processing brain signals for assisting cerebral palsy patients in locomotion according to claim 1, characterized in that: The method comprises: The movement action is detected, and when the detection result does not meet the preset conditions, the step of obtaining the first EEG signal and the second EEG signal is returned to execute.

10. A system for assisting cerebral palsy patients in exercising, characterized in that: The system comprises a controller module and an auxiliary equipment module, wherein the controller module comprises: An input unit, used to obtain a first EEG signal and a second EEG signal; a conversion unit, configured to generate a first instruction set according to the first EEG signal, and to generate a second instruction set according to the second EEG signal; an adjusting unit, configured to adjust the second instruction set with the first instruction set as a target, so as to obtain an adjusted second instruction set as a target instruction set; an execution unit, configured to control the auxiliary device module according to the target instruction set, so that the user can complete the motion action through the auxiliary device module; The auxiliary device module is used to interact with the user through the wearable hardware device according to the instructions of the execution unit, and support and drive the user's limbs to perform movement movements.

11. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the brain signal processing method for assisting cerebral palsy patients in movement as described in any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the brain signal processing method for assisting cerebral palsy patients in movement according to any one of claims 1 to 9 is implemented.

Images