Method and device for configuring a motor intention decoding model based on electroencephalogram signal data

Abstract

The application provides a configuration method and device of a motor intention decoding model based on electroencephalogram signal data, and provides a novel training mechanism of the motor intention decoding model. In the training process, the electroencephalogram signal data and the motor posture data obtained by analyzing the video are jointly analyzed, so that the model system can quickly capture the correlation between the neural signals and the movement trajectory, comprehensively capture the motor intention, significantly improve the recognition accuracy of the model for various complex movement modes, reduce the required computing resources of the model processing, make the entire decoding process more efficient, and balance the individual adaptability and the generalization, so that the real-time application requirements of the invasive brain-computer interface research can be well met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical treatment, in particular to a method and device for configuring a movement intention decoding model based on electroencephalogram data. BACKGROUND

[0002] The instructions generated by neural signals command actual movement actions, but the complexity of neural signals and the complexity of actual movement processes limit the decoding process of neural signals. An invasive brain-computer interface (BCI) captures electro-physiological signals, i.e., electroencephalogram (EEG), generated by neuron activity by implanting electrodes into the cerebral cortex to analyze the state of brain function.

[0003] At present, using non-human primates, especially macaques, as research objects to develop brain-computer interfaces is a key step to realize the transformation of invasive brain-computer interfaces to clinical applications. However, there are still some technical challenges to overcome in non-human primate brain-computer interface control experiments, such as movement process analysis and movement intention signal analysis.

[0004] Most current research focuses on classifying electroencephalogram signals of brain-computer interfaces. First, time-frequency domain transformation methods are used to extract electroencephalogram signal features, and then machine learning methods are used for classification. Although with the progress of technology in recent years, deep learning technology has also been used in electroencephalogram signal classification and has achieved good results.

[0005] However, the present inventors have found that the current invasive brain-computer interface research, through the technology of obtaining macaque electroencephalogram signals by implantable electrodes, has gradually matured, but the movement intention decoding model configured by deep learning technology still has limitations in processing performance in real-time analysis and accurate decoding. SUMMARY

[0006] The present application provides a method and device for configuring a movement intention decoding model based on electroencephalogram data, which provides a novel training mechanism for a movement intention decoding model. In the training process, the model system can quickly capture the correlation between neural signals and movement trajectories by jointly analyzing electroencephalogram data and movement posture data obtained from video analysis, fully capturing movement intention, significantly improving the recognition accuracy of the model for various complex movement patterns, and reducing the computational resources required for model processing, making the entire decoding process more efficient, taking into account individual adaptability and generalizability, and can well meet the real-time application requirements of invasive brain-computer interface research.

[0007] In a first aspect, the application provides a method for configuring a motor intention decoding model based on electroencephalogram signal data, the method comprising:

[0008] In the process of the sample motor object performing the specified motor task, monitoring images of the sample motor object are collected.

[0009] The monitoring images are subjected to image recognition processing to recognize the motor posture of the sample motor object in the process of performing the specified motor task, and corresponding sample motor posture data are obtained.

[0010] In the process of the sample motor object performing the specified motor task, electroencephalogram signals of the sample motor object are collected by an invasive brain-computer interface system, and corresponding sample electroencephalogram signal data are obtained.

[0011] The sample motor posture data and the sample electroencephalogram signal data are used as training samples to train a motor intention decoding model, wherein the motor posture data are used as labels, the motor intention decoding model is a deep learning model, and the motor intention decoding model is used to perform motor intention decoding processing on the motor intention of a corresponding motor object based on input electroencephalogram signal data of the model.

[0012] In a first aspect, the application provides a configuration device for a motor intention decoding model based on electroencephalogram signal data, the device comprising:

[0013] A first collection unit is configured to collect monitoring images of a sample motor object in the process of the sample motor object performing a specified motor task.

[0014] An identification unit is configured to perform image recognition processing on the monitoring images to recognize the motor posture of the sample motor object in the process of performing the specified motor task, and obtain corresponding sample motor posture data.

[0015] A second collection unit is configured to collect electroencephalogram signals of the sample motor object by an invasive brain-computer interface system in the process of the sample motor object performing the specified motor task, and obtain corresponding sample electroencephalogram signal data.

[0016] A training unit is configured to use the sample motor posture data and the sample electroencephalogram signal data as training samples to train a motor intention decoding model, wherein the motor posture data are used as labels, the motor intention decoding model is a deep learning model, and the motor intention decoding model is used to perform motor intention decoding processing on the motor intention of a corresponding motor object based on input electroencephalogram signal data of the model.

[0017] In a third aspect, the application provides a processing device comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the method provided in the first aspect of the application when invoking the computer program in the memory.

[0018] In a fourth aspect, the present application provides a computer readable storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method provided in the first aspect of the present application.

[0019] From the above, the present application has the following beneficial effects:

[0020] Focusing on the motion intention decoding target of the invasive brain-computer interface research, the present application provides a novel training mechanism of the motion intention decoding model. In the training process, the electroencephalogram signal data and the motion posture data obtained by video analysis are jointly analyzed, so that the model system can quickly capture the correlation between the neural signals and the motion trajectory, comprehensively capture the motion intention, significantly improve the recognition accuracy of the model for various complex motion patterns, and reduce the computing resources required by the model processing, so that the entire decoding process is more efficient, and the individual adaptability and generalization are considered. The real-time application requirements of the invasive brain-computer interface research can be well met. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 A flowchart of a configuration method of a motion intention decoding model based on electroencephalogram signal data of the present application;

[0023] Figure 2 A structural diagram of a configuration device of a motion intention decoding model based on electroencephalogram signal data of the present application;

[0024] Figure 3 A structural diagram of a processing device of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0026] The terms "first", "second", and the like in the description and in the claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or modules does not necessarily have to be limited to those steps or modules clearly listed, but can include other steps or modules not clearly listed or inherent to the process, method, product, or device. The naming or numbering of the steps appearing in the present application does not mean that the steps in the method flow must be performed in the time / logical order indicated by the naming or numbering, and the named or numbered flow steps can be changed in execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0027] The division of modules appearing in the present application is a logical division, and in actual application, there can be another division manner, for example, multiple modules can be combined or integrated in another system, or some features can be ignored or not executed, in addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, the indirect coupling or communication connection between the modules can be electrical or other similar forms, which are not limited in the present application. Moreover, the modules or sub-modules described as separate components can or can not be physically separated, can or can not be physical modules, or can be distributed into multiple circuit modules, and part or all of the modules can be selected according to actual needs to achieve the purpose of the present application scheme.

[0028] Before introducing the configuration method of the motor intention decoding model based on electroencephalogram signal data provided by the present application, the background content involved in the present application is first introduced.

[0029] The configuration method, device and computer readable storage medium of the motor intention decoding model based on electroencephalogram signal data provided by the present application can be applied to a processing device, and provide a novel training mechanism of the motor intention decoding model. In the training process, the electroencephalogram signal data and the motion posture data obtained by analyzing the video are jointly analyzed, so that the model system can quickly capture the correlation between the neural signals and the motion trajectory, comprehensively capture the motor intention, significantly improve the recognition accuracy of the model for various complex motion modes, and reduce the calculation resources required by the model processing, so that the entire decoding process is more efficient, and the individual adaptability and generalization are considered, which can well meet the real-time application requirements of the invasive brain-computer interface research.

[0030] The execution subject of the motion intention decoding model based on the electroencephalogram signal data mentioned in the present application can be a configuration device of the motion intention decoding model based on the electroencephalogram signal data, or a server, a physical host or a user equipment (UE) and other different types of processing devices integrated with the configuration device of the motion intention decoding model based on the electroencephalogram signal data. Among them, the configuration device of the motion intention decoding model based on the electroencephalogram signal data can be realized in the form of hardware or software, and the UE can be a terminal device such as a smart phone, a tablet computer, a notebook computer, a desktop computer or a personal digital assistant (PDA), and the processing device can be set in the form of a device cluster.

[0031] It should be noted that the present application scheme involves the collection of two types of data, namely monitoring images and electroencephalogram signals. The corresponding camera (which can be further extended to related devices / apparatuses with cameras) and the invasive brain-computer interface system can be used in the form of external devices / external devices, or can be integrated into the processing device in the form of a device cluster, or can involve the integration of the original device functions into the processing device through software and hardware modification. This can be flexibly adjusted according to actual conditions.

[0032] And corresponding to the actual application that the trained model can involve later, the processing device can also divide the model processing mainly involved in the present application scheme and the model application that can be involved later to form a device cluster.

[0033] As above, if the display of the model processing result is also involved, the processing device also needs to be configured with a corresponding display screen (including a touch screen), or other display screen devices or other devices with display screens can be externally connected to meet the result display requirements.

[0034] And when a device cluster or a device part involving multiple parts is involved, the connection between the device parts can be wired or wireless. Wireless connection is suitable for remote communication, such as web service and other communication methods, which correspond to the diversified application requirements in actual conditions.

[0035] As can be understood, the processing device executing the motion intention decoding model based on the electroencephalogram signal data of the present application or carrying the corresponding application service of the motion intention decoding model based on the electroencephalogram signal data of the present application has flexible device types and device deployment forms, which can be adaptively configured according to actual needs.

[0036] Next, the configuration method of the motion intention decoding model based on the electroencephalogram signal data provided by the present application will be introduced.

[0037] Firstly, refer to Figure 1 , Figure 1 A flowchart of the configuration method of the motion intention decoding model based on the EEG signal data of the application is shown. The configuration method of the motion intention decoding model based on the EEG signal data provided by the application can specifically include the following steps S101 to S104:

[0038] Step S101, in the process of the sample motion object performing the specified motion task, the monitoring image of the sample motion object is collected;

[0039] It can be understood that the application focuses on recognizing / detecting the motion intention of the motion object or test object based on the EEG signal. For this purpose, image recognition is introduced to assist model training. As for the data source object of the training sample in the model training process, i.e. the sample motion object, the application specifically needs it to perform the corresponding specified motion task. Based on the specified related motion, the monitoring image and EEG signal with a correlation are captured, and the data processing is further processed to obtain the model training.

[0040] Among them, the monitoring image is usually a monitoring video, and the subsequent data processing is specifically carried out on the video frames parsed from the monitoring video, or the monitoring image may also be a monitoring picture in some cases, which can be adjusted.

[0041] In addition, for the scene of the sample motion object performing the specified motion task, the application can also involve the process of guiding the sample motion object to perform the specified motion task, such as image display, voice prompt, vibration reminder, light reminder, and even artificial guidance, which may exist in actual situations.

[0042] If the sample motion object is a non-human object such as a macaque, it is easy to understand that the execution of the specified motion task usually involves a period of training, for example, 2 months, to fully enable the sample motion object to better complete the specified motion task and provide high-quality monitoring image and EEG signal data.

[0043] Step S102, image recognition processing is carried out on the monitoring image to identify the motion posture of the sample motion object in the process of performing the specified motion task, and the corresponding sample motion posture data is obtained;

[0044] It can be understood that for the previously obtained monitoring image, the image recognition algorithm / model can be pre-configured to recognize the video frames in the image to identify the specific motion posture of the sample motion object involved in the execution of the specified motion task. The corresponding motion action of the specific motion posture is also reflected in the electroencephalogram signal. Thus, the sample motion posture data obtained by mature and high-precision image recognition can be used as a label or prediction truth value in the model training process corresponding to the sample electroencephalogram signal data, to constitute a high-quality training sample to promote the training of the model based on the electroencephalogram signal data for motion intention decoding processing.

[0045] Step S103, in the process of the sample motion object executing the specified motion task, the electroencephalogram signal of the sample motion object is collected by the invasive brain-computer interface system, and the corresponding sample electroencephalogram signal data is obtained.

[0046] It can be understood that in the process of the sample motion object executing the specified motion task, on the one hand, the monitoring image acquisition work is involved, and on the other hand, the electroencephalogram signal acquisition work is also involved, and the specific electroencephalogram signal acquisition process needs to be responsible by the invasive brain-computer interface system.

[0047] Among them, considering how the invasive brain-computer interface system collects the electroencephalogram signal to obtain the corresponding sample electroencephalogram signal data, it is considered to belong to the category of prior art, therefore, it is not expanded here.

[0048] Step S104, the sample motion posture data and the sample electroencephalogram signal data are used as training samples to train the motion intention decoding model, wherein the motion posture data is used as a label, the motion intention decoding model is a deep learning model, and the motion intention decoding model is used for motion intention decoding processing of the motion intention of the corresponding motion object based on the model input electroencephalogram signal data.

[0049] It can be understood that after obtaining the sample electroencephalogram signal data corresponding to the model input involved in the actual application of the model and the sample motion posture data having a direct correlation with the sample electroencephalogram signal and being highly accurate, the two can be configured as a data format of the training sample that can be used by the specific model training link. Thus, the configured training sample can be used to expand the specific model training, and the motion intention decoding model can be trained.

[0050] The motion intention decoding model is a deep learning model, which is built by artificial intelligence (AI) technology with strong learning ability and has better model performance in machine learning models. Therefore, based on the configuration of the training samples, the brain electrical signal data and the motion posture data can be deeply analyzed to capture the corresponding relationship between the brain electrical signal and the motion posture, comprehensively capture the motion intention of the sample motion object, and carry out deep model training. On the one hand, the recognition accuracy of the model for various complex motion patterns can be significantly improved, and on the other hand, the calculation resources required for model processing can be effectively reduced, so that the entire decoding process is more efficient, and the adaptability and generalization of different individual motion behaviors are considered.

[0051] In this case, the model type (or model structure) used by the motion intention decoding model can obviously be adaptively configured according to actual needs. Existing schemes such as long short-term memory (LSTM), recurrent neural network (RNN), etc. can be used, or further optimization and improvement can be made on the basis of existing schemes, or even novel self-developed schemes can be used to further enhance the model performance under a high-quality model training mechanism.

[0052] The specific model training process usually includes:

[0053] In each round of model training, a sample brain electrical signal data is input into the model to prompt the model to carry out corresponding motion intention decoding processing, realize forward propagation, then calculate the loss function based on the motion intention decoding processing result output by the model and the corresponding sample motion posture data, and optimize the model parameters through the loss function calculation result to realize back propagation. After a large number of rounds of model training, if the preset model training requirements such as training time, training times or prediction accuracy are met, the model training is completed, and a motion intention decoding model that can be put into actual use is obtained.

[0054] The model training architecture and the specific loss function used in the training process are similar to the model type introduced above, which can be deployed flexibly according to actual conditions to meet the diversified application requirements in actual applications.

[0055] Continue to the above Figure 1The various steps of the illustrated embodiment and their possible implementation in practical applications are described in detail.

[0056] As an exemplary implementation, the preceding step S102 performs image recognition processing on the monitoring image to identify the motion posture of the sample motion object in the process of performing the specified motion task and obtain corresponding sample motion posture data, which can specifically include:

[0057] 1) Perform target detection processing on the monitoring image using a yolo-v8 algorithm to obtain corresponding hand information, wherein the hand information includes hand position and hand size;

[0058] It can be understood that the computer vision method or target detection processing performed here is a preliminary image recognition processing for hand motion posture to locate the hand from the video frame pictures involved in the overall monitoring image, which can specifically involve hand position and hand size.

[0059] In addition, it can also be seen that the yolo-v8 deep learning algorithm is specifically used in the processing link here, and the yolo-v8 algorithm has excellent target detection performance, so it can well meet the demand for obtaining hand information here.

[0060] 2) Continue hand key point detection processing on the hand information through a key point detection algorithm to obtain corresponding hand key point information;

[0061] After obtaining the hand information including hand position and hand size, the hand information can be subjected to more detailed hand key point positioning detection through the configured related key point detection algorithm, so as to obtain more detailed and specific hand information related to the hand motion posture. For convenience of description, the deep-level hand information obtained here is denoted as hand key point information.

[0062] 3) Perform hand posture estimation processing on the data-associated hand information and hand key point information through a hand posture estimation network based on deep learning to obtain sample motion posture data.

[0063] After obtaining the hand information at the preliminary stage and the deep-level hand key point information, the two can be data-associated to ensure that each detected target is correctly matched with its corresponding key point. This processing can be specifically carried out in terms of relative position relationship, that is, the relative position relationship between the position information of the target and the key points is used for matching.

[0064] Then, the posture of the hand movement can be accurately estimated by using the processing results of the target detection and key point detection of the completed data association, and the hand posture key points can be regressed based on a posture estimation network of deep learning to obtain accurate posture information of the hand movement, and then the hand movement trajectory is constructed to complete the configuration of the sample movement posture data.

[0065] Further, for the sample movement object of non-human primates, especially macaques, the application also designs specific task contents of the designated movement task.

[0066] Correspondingly, as an exemplary embodiment, the designated movement task involved in the application can specifically include the following movement process:

[0067] 1) After the first signal light is lit, touch the first sensor for 500 ms;

[0068] Among them, the first signal light is usually arranged adjacent to the first sensor.

[0069] As an example, the first signal light can be a "Center light" beside the central sensor, and the lighting of the first signal light serves as a signal for the start of a task. The lighting of the first signal light is used to prompt the sample movement object to touch the central sensor and place the right hand on the central sensor and maintain it for 500 ms.

[0070] 2) After touching the first sensor for 500 ms, extinguish the first signal light, and randomly light up one target light among the three second signal lights;

[0071] After the first sensor is continuously touched for 500 ms, the system will extinguish the first signal light, and in the case that the system is configured with one first signal light, three second signal lights, and a sensor corresponding to each second signal light, a second signal light is randomly selected to be lit up to prompt the sample movement object to touch the sensor corresponding to the lit second signal light (i.e. the target sensor).

[0072] 3) After the target light is lit, touch the target sensor corresponding to the target light among the three second sensors for 200 ms.

[0073] It can be understood that in the following touch action of the target sensor, it needs to be maintained for 200 ms.

[0074] Continuing the previous example, if the right hand is used for touch action in the previous training, then the right hand of the sample movement object is moved from the central sensor to the target sensor (target object) for 200 ms.

[0075] And in the training process, when the sample moving object completes the above movement and completes a task, the touching action on the target sensor can be stopped, the grip can be released, and rewards such as drinking water can be given.

[0076] It can be understood that in the embodiment, the application focuses on the hand movement posture of the sample moving object, and a standard movement direction is specified. The effective training of the grip training process can help to realize stable grip action, standardize the hand movement, optimize the accuracy of the training and capture process, and better realize parameterization of the movement process, and lay a good foundation for subsequent model training.

[0077] At the same time, continue to focus on the movement intention decoding target involved in the hand movement posture, and in the specific parameterization configuration work, the application also gives a specific adaptation scheme to realize higher quality hand posture parameterization effect.

[0078] Specifically, as an exemplary embodiment, the sample movement posture data can be data describing the hand movement posture, and different hand movement primitives involved include grip movement, upward movement, downward movement, leftward movement, rightward movement, forward movement, backward movement and stop movement, a total of 8 hand movement primitives.

[0079] The sample movement posture data is specifically configured by decomposing a time window of 200ms window size, parameterization encoding representing different hand movement primitives, and is represented as follows:

[0080] M={m1, m2,..., mx},

[0081] Wherein, M is the sample movement posture data, my is the time series data of the yth time window, y=1, 2,...x.

[0082] As an example, the movement primitive encoding of the above 8 hand movement primitives can be 0-7 corresponding to binary in turn, of course, the specific encoding mechanism can be flexibly configured according to actual conditions.

[0083] In this way, the hand movement process after time window decomposition is converted into a series of movement primitives through one-to-one mapping, and the movement process is parameterized encoded according to the movement primitive encoding.

[0084] In the embodiment, it can be seen that the application constructs a specific motion trajectory coding and decoding technology, captures the hand motion trajectory of the sample motion object, extracts the key process, and through the disassembly of the motion trajectory into multiple motion primitives and the parameterized change of the motion primitives, can effectively improve the recognition accuracy of the system to the hand gripping action, and enhance the real-time and stability of the decoding, which is helpful to enhance the adaptability of the system to individual differences and different environmental conditions, and ensure the continuous and stable work of the system in the dynamic change.

[0085] In addition, it can be understood that, in addition to the motion intention decoding target involved in the hand motion of the motion object, the application can also involve the motion intention decoding target of other body parts of the motion object, or can also involve the motion intention decoding target of multiple body parts or even the whole body, which can be adaptively adjusted according to actual needs, therefore, for the sample motion object, an appropriate specified motion task needs to be configured to obtain corresponding training samples for model training, among which, the operation idea of the above hand motion task can be specifically followed for processing, or other types of improved operation ideas can also be used for processing.

[0086] At the same time, for the collection and processing of the sample electroencephalogram signal data, the application can also involve corresponding preprocessing operations to enhance the data quality.

[0087] Correspondingly, focusing on the preprocessing link, as an exemplary embodiment, step S103 collects the electroencephalogram signal of the sample motion object through the invasive brain-computer interface system during the execution of the specified motion task of the sample motion object, and obtains corresponding sample electroencephalogram signal data, which can specifically include:

[0088] 1) During the execution of the specified motion task of the sample motion object, the electroencephalogram signal of the sample motion object is collected through the invasive brain-computer interface system, and corresponding first sample electroencephalogram signal data with a sampling frequency of 100 Hz per signal channel is obtained;

[0089] It can be seen that the electroencephalogram signal collection frequency of the invasive brain-computer interface system is 1000 Hz, which ensures that the subtle changes of brain activity can be captured with high resolution, and after the initial signal collection is completed, a series of data preprocessing steps can be continued to remove noise and unnecessary interference, so that the obtained neural signal can truly reflect the brain activity during the motion of the macaque.

[0090] 2) The first sample electroencephalogram signal data is processed into second sample electroencephalogram signal data with a sampling frequency of 200 Hz per signal channel through downsampling processing;

[0091] It can be understood that in the first brain electrical signal data collected initially, the sampling frequency of each signal channel is 1000 Hz, which means that 1000 data points are collected per second. Although a high sampling rate can provide higher signal accuracy, in many neural signal processing applications, a higher sampling frequency can lead to a burden of data processing and storage. Therefore, a downsampling process can be continued to reduce the signal frequency to 200 Hz to significantly reduce the amount of data while ensuring that the key information in the data is retained. In particular, in the decoding analysis of the brain motor control area, a sampling frequency of 200 Hz is sufficient to capture important neural activity of the brain during the performance of motor tasks.

[0092] 3) The second sample brain electrical signal data is band-pass filtered using a fourth-order Butterworth filter with a passband of 0.5 Hz to 50 Hz, and the corresponding sample brain electrical signal data is obtained.

[0093] It can be understood that in order to remove low-frequency noise (such as electromyographic signals, power frequency interference, etc.) and high-frequency noise (such as electrode noise and environmental interference), the second brain electrical signal data after downsampling can be further band-pass filtered, and in the specific processing process, a fourth-order Butterworth filter can be used, with a passband of 0.5 Hz to 50 Hz. This band filter can effectively remove low-frequency interference (such as artifacts caused by eye movement or electrocardiographic signals) below 0.5 Hz and high-frequency noise above 50 Hz. The band-pass filtering function is to retain low-frequency and medium-frequency neural signals related to motor control that can reflect the electrical activity of the cerebral cortex, especially the motor cortex.

[0094] In this process, the selection and parameter setting of the filter are crucial, such as the passband of 0.5 Hz to 50 Hz, which can ensure that the filtered signal not only retains useful neural activity components, but also avoids signal distortion or loss of key information.

[0095] In this way, through the pre-processing operation designed in this embodiment, high-quality and highly simplified sample brain electrical signal data is obtained, which lays a good foundation for the subsequent high-quality model training.

[0096] In addition, the training samples involve data related to brain electrical signals and motor gestures. Each corresponding sample brain electrical signal data and sample motor gesture data of the same time motor action serves as a training sample.

[0097] In this regard, the present application can further involve more delicate matching processing between sample brain electrical signal data and sample motor gesture data to further ensure the matching between the two types of data and ensure that the time / time stamp remains synchronized and strictly corresponds to the actual motor process, ensuring the high quality level of the training samples.

[0098] Correspondingly, as an exemplary embodiment, the method of the present application can also include:

[0099] 1) The sample electroencephalogram signal data is divided according to a time window of 200 ms window size, and there is an overlapping part between adjacent time windows in the division process. The divided sample electroencephalogram signal data is represented as follows:

[0100] E = {e1, e2,..., eo},

[0101] Wherein, E is the sample electroencephalogram signal data, ep is the time series data of the pth time window, p = 1, 2,..., o;

[0102] 2) The sample motion posture data and the sample electroencephalogram signal data are combined in units of time windows of 200 ms window size, and the obtained data set as a training sample is represented as follows:

[0103] Q = {(e1, m1), (e2, m2),..., (en, mn)},

[0104] Wherein, the time window of 0 ms window size corresponds to the setting of the sample motion posture data decomposed in the time window of 200 ms window size, Q is the data set as a training sample, n is the number of time series data pairs of the sample motion posture data and the sample electroencephalogram signal data, (eu, mu) is a pair of neural signal segment and motion primitive label, u = 1, 2,.. n.

[0105] In the specific processing process, it can be understood that the sample electroencephalogram signal data and the sample motion posture data (or the monitoring image) will record their own time stamps, so that the time points marked by the time stamps can be matched by the specific calibration method.

[0106] In this case, the time points of the motion event / motion primitive can be used as the calibration basis to match the sample electroencephalogram signal data and the sample motion posture data at each time point. This synchronous processing can ensure the accuracy and reliability of the data, which is helpful for subsequent neural signal decoding and motion behavior analysis.

[0107] As for the collected sample electroencephalogram signal data, the present application divides it according to time windows, so as to associate it with the corresponding motion trajectory. Each 200 ms time window contains a 200 Hz time series, i.e. each time window contains 40 data points. This window size is selected by fully considering the time delay of the brain in performing the motion task and the propagation characteristics of the motion signal. The window size of 200 ms can better capture the neural dynamic changes in the motion execution process, while avoiding signal distortion or information loss caused by too small window size.

[0108] And, in order to ensure consistency in terms of timing, the division of the time window also adopts the same compensation method as the window size. Specifically, the compensation of the time window refers to the overlapping part between each time window, that is, there is a part of overlapping data between two adjacent time windows, and the specific overlapping degree can be flexibly configured according to the actual situation through specific indicators such as data volume, timestamp quantity or time span. This compensation method can maximize the retention of neural information in each time window, reduce signal loss, and improve the accuracy of neural signal classification and decoding.

[0109] In this way, after the time window division, the sample electroencephalogram data E represented above is obtained, where each ep represents the brain activity performance of the sample moving object in a specific time period when performing a certain movement behavior.

[0110] Correspondingly, the movement trajectory of the sample moving object composed of different movement primitives when performing the specified movement task can be obtained, that is, the sample movement posture data, and the movement primitive m can be one-to-one corresponding to the sample electroencephalogram data e in the time window, thereby establishing the relationship between the neural signal and the movement behavior.

[0111] Through the above time window division and labeling process, a complete movement-neural signal data set can be obtained, each item of which includes a neural signal segment e and its corresponding movement primitive label m, that is, the mapping relationship between a neural signal time sequence and a movement behavior, forming a data set Q, and each data point in it contains a 200ms electroencephalogram sequence and its corresponding movement action label. This data set will serve as the basis for subsequent electroencephalogram decoding classification, and through a deep learning method, the movement behavior of the moving object can be accurately predicted.

[0112] In this way, after the above processing, the configuration of high-quality training samples is completed, and obviously, the specific model training process can be started. The movement intention decoding model taking the electroencephalogram data as the model input is trained to perform movement intention decoding processing (or to predict the corresponding movement intention of the moving object) on the movement intention of the corresponding moving object.

[0113] And for the movement intention decoding model, the present application also specially designs a specific model structure scheme to better adapt to the application scenario of the present application, starting from the model itself, to further strengthen the model's decoding classification / prediction performance for movement intention.

[0114] Specifically, as an exemplary embodiment, the motion intention decoding model of the present application is based on Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), and a two-dimensional CNN_GRU fusion neural network framework is created, which can specifically include the following configurations:

[0115] 1) The one-dimensional time series signal data input to the model is converted into a two-dimensional image signal after frequency domain transformation, which is input to the first half of the model. The transformation process includes: using Discrete Fourier Transform (DFT) to convert the signal from time domain to frequency domain, estimating the power spectral density through the amplitude square of the Fourier transform of the signal, and smoothing the estimated power spectral density through the Welch method.

[0116] The discrete Fourier transform can help identify periodic or repetitive patterns in the signal. The Welch method can reduce the impact of noise and help extract the frequency characteristics of the signal more clearly.

[0117] 2) The first half of the model includes three two-dimensional convolution layers and three max-pooling layers. After each two-dimensional convolution layer, a max-pooling layer is used for max-pooling, and then the next two-dimensional convolution layer is input, until the third max-pooling layer at the last position outputs to the second half of the model.

[0118] The max-pooling operation can help reduce the spatial dimension of the features and reduce the computational complexity while preserving key information. The main function of the first half of the model is to extract the spatial features of the input signal, especially the frequency characteristics.

[0119] 3) The second half of the model includes two GRU layers responsible for concatenation. The GRU layer is used to extract the time sequence characteristics, and the output of the second GRU layer is processed by the flatten layer, the fully connected layer, the dropout layer and the output layer in turn to obtain the model output.

[0120] The GRU layer is used to process the feature sequence extracted by the first half of the model and further extract the time sequence characteristics. The GRU controls the flow of information through the gating mechanism, which can avoid the gradient vanishing or gradient explosion problem in traditional RNN.

[0121] The flatten layer, the fully connected layer, the dropout layer and the output layer are responsible for mapping the features extracted from the GRU layer to the final output category space, and mapping the complex features to the final motion intention classification result.

[0122] Thus, under the design of the above model structure, the GRU is embedded as a layer into the overall model to better extract the features of the time series signal, and finally the precise classification of the motion intention signal is realized.

[0123] 4) The specifications of the three two-dimensional convolution layers are 998x26x32, 497x11x64 and 246x3x64, and the specifications of the three max pooling layers are 499x13x32, 248x5x64 and 123x1x64, respectively.

[0124] 5) The specifications of the flatten layer, the fully connected layer, the dropout layer and the output layer are 1x128, 1x64, 1x64 and 1x2, respectively.

[0125] It can be seen that the present application further improves the configuration of the above-mentioned model structure from the parameter aspect to better complete the configuration of the motion intention decoding model, and provides a good model basis for high-performance motion intention decoding classification / prediction tasks, which has better application value and practical significance.

[0126] After the model training is completed, subsequent model application steps can obviously be involved according to application requirements.

[0127] For this purpose, the method of the present application can further include:

[0128] Obtaining target electroencephalogram signal data to be processed;

[0129] Inputting the target electroencephalogram signal data into the motion intention decoding model;

[0130] Obtaining the processing result corresponding to the target electroencephalogram signal data output by the motion intention decoding model.

[0131] It can be understood that the target electroencephalogram signal data can involve the collection and processing of the sample electroencephalogram signal data mentioned above. Of course, in actual application, it can also be directly used as ready-made electroencephalogram signal data, which can be flexibly adjusted according to actual conditions.

[0132] After obtaining the processing result output by the model, local storage, remote storage, result forwarding, result display, output completion prompt or further data analysis and other data application processing can be further involved. Obviously, this step can be flexibly adjusted according to the pre-configuration and real-time configuration of the data application strategy / rule, so that the network model with high-performance decoding classification performance configured in the present application scheme can be used to promote high-quality invasive brain-computer interface research.

[0133] Finally, for the above scheme content, in general, there are:

[0134] Focusing on the motion intention decoding target of the invasive brain-computer interface research, the application provides a novel training mechanism of a motion intention decoding model. In the training process, the EEG signal data and the motion posture data obtained by video analysis are jointly analyzed, so that the model system can quickly capture the correlation between the neural signals and the motion trajectory, comprehensively capture the motion intention, significantly improve the recognition accuracy of the model for various complex motion patterns, reduce the computing resources required by the model processing, make the entire decoding process more efficient, and balance individual adaptability and generalization. The application can well meet the real-time application requirements of invasive brain-computer interface research.

[0135] The above is an introduction to the configuration method of the motion intention decoding model based on EEG signal data provided by the application. In order to better implement the configuration method of the motion intention decoding model based on EEG signal data provided by the application, the application also provides a configuration device of a motion intention decoding model based on EEG signal data from the functional module perspective.

[0136] Reference Figure 2 , Figure 2 is a structural schematic diagram of the configuration device of the motion intention decoding model based on EEG signal data provided by the application. In the application, the configuration device 200 of the motion intention decoding model based on EEG signal data can specifically include the following structure:

[0137] The first acquisition unit 201 is configured to acquire monitoring images of a sample motion object during execution of a specified motion task by the sample motion object.

[0138] The recognition unit 202 is configured to perform image recognition processing on the monitoring images to recognize the motion posture of the sample motion object during execution of the specified motion task, and obtain corresponding sample motion posture data.

[0139] The second acquisition unit 203 is configured to acquire EEG signals of the sample motion object through an invasive brain-computer interface system during execution of the specified motion task by the sample motion object, and obtain corresponding sample EEG signal data.

[0140] The training unit 204 is configured to train the motion intention decoding model by taking the sample motion posture data and the sample EEG signal data as training samples, wherein the motion posture data is used as a label, the motion intention decoding model is a deep learning model, and the motion intention decoding model is configured to perform motion intention decoding processing on the motion intention of a corresponding motion object based on input EEG signal data.

[0141] In an exemplary embodiment, the recognition unit 202 is specifically configured to:

[0142] The yolo-v8 algorithm is used for target detection processing of the monitoring image to obtain corresponding hand information, wherein the hand information includes hand position and hand size;

[0143] The hand information is subjected to hand key point detection processing through a key point detection algorithm to obtain corresponding hand key point information;

[0144] The hand pose estimation network based on deep learning is used to perform hand pose estimation processing on the data-associated hand information and hand key point information to obtain sample motion pose data.

[0145] In another exemplary embodiment, the specified motion task specifically includes the following motion process:

[0146] After the first signal light is turned on, the first sensor is touched for 500 ms;

[0147] After the first sensor is touched for 500 ms, the first signal light is turned off, and one target light of the three second signal lights is randomly turned on;

[0148] After the target light is turned on, the target sensor corresponding to the target light among the three second sensors is touched for 200 ms.

[0149] In another exemplary embodiment, the sample motion pose data is specifically data describing hand motion poses, and different hand motion primitives involved include grasping motion, upward motion, downward motion, leftward motion, rightward motion, forward motion, backward motion, and stop motion;

[0150] The sample motion pose data is configured in a parameterized encoding representing different hand motion primitives, and is represented as follows:

[0151] M = {m1, m2,..., mx},

[0152] Wherein, M is the sample motion pose data, my is the time series data of the yth time window, y = 1, 2,... x.

[0153] In another exemplary embodiment, the second acquisition unit 203 is specifically configured to:

[0154] During the execution of the specified motion task by the sample motion object, the intracranial brain-computer interface system is used to acquire the electroencephalogram signal of the sample motion object, and corresponding first sample electroencephalogram signal data with a sampling frequency of 100 Hz per signal channel is obtained;

[0155] The first sample electroencephalogram signal data is processed into second sample electroencephalogram signal data with a sampling frequency of 200 Hz per signal channel through downsampling processing;

[0156] The second sample brain electrical signal data is band-pass filtered using a fourth-order Butterworth filter with a passband of 0.5 Hz to 50 Hz, and corresponding sample brain electrical signal data is obtained.

[0157] In yet another exemplary embodiment, the apparatus further comprises a processing unit 205 configured to:

[0158] The sample brain electrical signal data is divided into time windows with a window size of 200 ms, and there is an overlap between adjacent time windows during the division, and the divided sample brain electrical signal data is represented as follows:

[0159] E = {e1, e2,..., eo},

[0160] wherein E is the sample brain electrical signal data, ep is the time series data of the pth time window, and p = 1, 2,..., o;

[0161] The sample motion posture data and the sample brain electrical signal data are combined in time windows with a window size of 200 ms, and the obtained data set as a training sample is represented as follows:

[0162] Q = {(e1, m1), (e2, m2),..., (en, mn)},

[0163] wherein Q is the data set as a training sample, n is the number of paired time series data of the sample motion posture data and the sample brain electrical signal data, (eu, mu) is a pair of neural signal segment and motion primitive label, and u = 1, 2,..., n.

[0164] In yet another exemplary embodiment, the motion intention decoding model comprises the following configurations:

[0165] The one-dimensional time series signal data input to the model is transformed into a two-dimensional image signal in the frequency domain, and the two-dimensional image signal is input to the first half of the model, wherein the transformation process comprises: using a discrete Fourier transform to convert the signal from the time domain to the frequency domain, estimating the power spectral density by the amplitude square of the Fourier transform of the signal, and smoothing the estimated power spectral density by the Welch method;

[0166] The first half of the model comprises three two-dimensional convolution layers and three max-pooling layers, after each two-dimensional convolution layer is processed by two-dimensional convolution, the signal is max-pooled by a max-pooling layer, and then input to the next two-dimensional convolution layer, until the third max-pooling layer at the last position outputs to the second half of the model.

[0167] The latter part of the model includes two concatenated GRU layers. The GRU layer is used to extract temporal features. The output of the second GRU layer is processed sequentially through a flatten layer, a fully connected layer, a dropout layer, and an output layer to obtain the model output.

[0168] The specifications of the three two-dimensional convolutional layers are 998×26×32, 497×11×64 and 246×3×64 respectively, and the specifications of the three max pooling layers are 499×13×32, 248×5×64 and 123×1×64 respectively.

[0169] The specifications for the flatten layer, fully connected layer, dropout layer, and output layer are 1×128, 1×64, 1×64, and 1×2, respectively.

[0170] This application also provides a processing device from a hardware architecture perspective, see [link / reference]. Figure 3 , Figure 3 This diagram illustrates a structural schematic of the processing device of this application. Specifically, the processing device may include a processor 301, a memory 302, and an input / output device 303. The processor 301 executes the computer program stored in the memory 302 to implement, for example... Figure 1 The steps of the configuration method for the motion intention decoding model based on EEG signal data in the corresponding embodiment; or, when the processor 301 executes the computer program stored in the memory 302, it implements as follows: Figure 2 Corresponding to the functions of each unit in the embodiment, the memory 302 is used to store the functions executed by the processor 301 as described above. Figure 1 The computer program required for configuring the motion intention decoding model based on EEG signal data in the corresponding embodiment.

[0171] For example, a computer program may be divided into one or more modules / units, one or more of which are stored in memory 302 and executed by processor 301 to complete this application. One or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a computer device.

[0172] The processing device may include, but is not limited to, processor 301, memory 302, and input / output device 303. Those skilled in the art will understand that the illustrations are merely examples of the processing device and do not constitute a limitation on the processing device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the processing device may also include network access devices, buses, etc., and processor 301, memory 302, input / output device 303, etc., are connected via a bus.

[0173] The processor 301 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The processor is the control center of the processing device, and connects various parts of the entire device through various interfaces and lines.

[0174] The memory 302 can be used to store computer programs and / or modules, and the processor 301 realizes various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 302, and calling the data stored in the memory 302. The memory 302 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function, etc.; and the data storage area can store data created according to the use of the processing device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0175] When the processor 301 is used to execute the computer programs stored in the memory 302, the following functions can be realized:

[0176] In the process in which the sample motion object executes the specified motion task, a monitoring image of the sample motion object is collected;

[0177] Image recognition processing is performed on the monitoring image to identify the motion posture of the sample motion object in the process of executing the specified motion task, and corresponding sample motion posture data is obtained;

[0178] In the process in which the sample motion object executes the specified motion task, an electroencephalogram signal of the sample motion object is collected through the invasive brain-computer interface system, and corresponding sample electroencephalogram signal data is obtained;

[0179] The sample motion posture data and the sample electroencephalogram signal data are taken as training samples to train a motion intention decoding model, wherein the motion posture data is taken as a label, the motion intention decoding model is a deep learning model, and the motion intention decoding model is used to perform motion intention decoding processing on a motion intention of a corresponding motion object based on model input electroencephalogram signal data.

[0180] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the configuration device of the motion intention decoding model based on the electroencephalogram signal data, the processing device and the corresponding units thereof described above can be referred to as Figure 1 The configuration method of the motion intention decoding model based on the electroencephalogram signal data in the corresponding embodiments will not be described in detail here.

[0181] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by related hardware controlled by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0182] To this end, the present application provides a computer readable storage medium, which stores a plurality of instructions capable of being loaded by a processor to execute the present application as Figure 1 The steps of the configuration method of the motion intention decoding model based on the electroencephalogram signal data in the corresponding embodiments can be referred to as Figure 1 The configuration method of the motion intention decoding model based on the electroencephalogram signal data in the corresponding embodiments will not be described in detail here.

[0183] The computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0184] Due to the instructions stored in the computer readable storage medium, the present application as Figure 1 The steps of the configuration method of the motion intention decoding model based on the electroencephalogram signal data in the corresponding embodiments, therefore, the present application as Figure 1 The beneficial effects that can be achieved by the configuration method of the motion intention decoding model based on the electroencephalogram signal data in the corresponding embodiments are described in detail above and will not be described here.

[0185] The configuration method and device of the motion intention decoding model based on the electroencephalogram signal data provided by the application, the processing device, and the computer readable storage medium are described in detail above, and the principles and implementation manners of the application are described in this paper. The above example is only used to help understand the core idea of the application; at the same time, for those skilled in the art, according to the idea of the application, the specific implementation manner and application range will be changed, and the above description should not be understood as a limitation of the application.

Claims

1. A method for configuring a motor intention decoding model based on electroencephalogram data, characterized in that, The method comprises: acquiring monitoring images of the sample motion object in the process of the sample motion object performing a specified motion task; performing image recognition processing on the monitoring images to identify the motion posture of the sample motion object in the process of performing the specified motion task, and obtain corresponding sample motion posture data; acquiring electroencephalogram signals of the sample motion object through an invasive brain-computer interface system in the process of the sample motion object performing the specified motion task, and obtain corresponding sample electroencephalogram signal data; training a motion intention decoding model by taking the sample motion posture data and the sample electroencephalogram signal data as training samples, wherein the sample motion posture data is used as annotation, the motion intention decoding model is a deep learning model, and the motion intention decoding model is used for motion intention decoding processing of the motion intention of a corresponding motion object based on model input electroencephalogram signal data; the specified motion task specifically comprises the following motion process: after the first signal lamp is lighted, touch the first sensor for 500 ms; after the first sensor is touched for 500 ms, turn off the first signal lamp and randomly light up one target lamp in the three second signal lamps; after the target lamp is lighted, touch the target sensor corresponding to the target lamp in the three second sensors for 200 ms; the motion intention decoding model comprises the following configuration content: one-dimensional time series signal data input to the model is converted into two-dimensional image signals through frequency domain transformation, and the two-dimensional image signals are input to the first half of the model, wherein the transformation process comprises: converting the signal from time domain to frequency domain by using discrete Fourier transform processing, estimating the power spectral density by the amplitude square of the Fourier transform of the signal, and smoothing the estimated power spectral density by the Welch method; the first half of the model comprises three two-dimensional convolution layers and three maximum pooling layers, after two-dimensional convolution processing of each two-dimensional convolution layer, maximum pooling is performed by one maximum pooling layer, and then the next two-dimensional convolution layer is input, until the third maximum pooling layer at the last position outputs to the second half of the model; the second half of the model comprises two GRU layers responsible for series connection, the GRU layers are used for extracting time sequence features, and the output of the second GRU layer is sequentially processed by a flatten layer, a full connection layer, a dropout layer and an output layer to obtain model output; the specifications of the three two-dimensional convolution layers are 998×26×32, 497×11×64 and 246×3×64 respectively, and the specifications of the three maximum pooling layers are 499×13×32, 248×5×64 and 123×1×64 respectively; the specifications of the flatten layer, the full connection layer, the dropout layer and the output layer are 1×128, 1×64, 1×64 and 1×2 respectively.

2. The method of claim 1, wherein, the image recognition processing on the monitoring images to identify the motion posture of the sample motion object in the process of performing the specified motion task, and obtain corresponding sample motion posture data, comprises: The YOLO-V8 algorithm is used for target detection processing on the monitoring image, and corresponding hand information is obtained, wherein the hand information includes hand position and hand size. The hand information is subjected to hand key point detection processing by a key point detection algorithm, and corresponding hand key point information is obtained. The hand information and the hand key point information are subjected to hand gesture estimation processing by a hand gesture estimation network based on deep learning, and sample motion gesture data is obtained.

3. The method of claim 1, wherein, The sample motion gesture data is data describing hand motion gestures, and different hand motion primitives involved include grasping motion, upward motion, downward motion, leftward motion, rightward motion, forward motion, backward motion and stop motion. The sample motion gesture data is configured in a parameterized code representing the different hand motion primitives, and is expressed as follows: M={m1, m2,..., mx}, wherein M is the sample motion gesture data, my is time series data of the yth time window, and y=1, 2,...x.

4. The method of claim 3, wherein, In the process of the sample motion object performing the specified motion task, the electroencephalogram of the sample motion object is collected by the invasive brain-computer interface system, and corresponding sample electroencephalogram data is obtained, including: In the process of the sample motion object performing the specified motion task, the electroencephalogram of the sample motion object is collected by the invasive brain-computer interface system, and corresponding sample electroencephalogram data is obtained, including: The first sample electroencephalogram data is processed into second sample electroencephalogram data with a sampling frequency of 200Hz per signal channel by downsampling processing. The second sample electroencephalogram data is subjected to band-pass filtering processing by using a fourth-order Butterworth filter with a passband of 0.5Hz to 50Hz, and corresponding sample electroencephalogram data is obtained.

5. The method of claim 4, wherein, The method further includes: The sample electroencephalogram data is divided into time windows with a window size of 200ms, and there is an overlap between adjacent time windows during the division, and the divided sample electroencephalogram data is expressed as follows: E={e1,e2,...,eo}, wherein E is the sample electroencephalogram data, ep is time series data of the pth time window, and p=1, 2,...o; The sample motion gesture data and the sample electroencephalogram data are combined in units of time windows with a window size of 200ms, and the obtained data set as the training sample is expressed as follows: Q={(e1,m1), (e2,m2),..., (en,mn)}, wherein Q is the data set as the training sample, n is the number of time series data pairs of the sample motion gesture data and the sample electroencephalogram data, (eu, mu) is a paired neural signal segment and motion primitive label, and u=1, 2,...n. 6.A configuration device of a movement intention decoding model based on electroencephalogram data, characterized by The device includes: The first acquisition unit is configured to acquire monitoring images of the sample motion object during execution of a specified motion task by the sample motion object; The recognition unit is configured to perform image recognition processing on the monitoring images to identify a motion posture of the sample motion object during execution of the specified motion task and obtain corresponding sample motion posture data; The second acquisition unit is configured to acquire electroencephalogram signals of the sample motion object during execution of the specified motion task by the sample motion object through an invasive brain-computer interface system and obtain corresponding sample electroencephalogram signal data; The training unit is configured to train a motion intention decoding model by taking the sample motion posture data and the sample electroencephalogram signal data as training samples, wherein the sample motion posture data is taken as a label, the motion intention decoding model is a deep learning model, and the motion intention decoding model is configured to perform motion intention decoding processing on a motion intention of a corresponding motion object based on model input electroencephalogram signal data; The specified motion task specifically includes the following motion process: After the first signal lamp is lighted, touch the first sensor for 500 ms; After the first sensor is touched for 500 ms, the first signal lamp is turned off, and one target lamp of the three second signal lamps is randomly lighted; After the target lamp is lighted, touch the target sensor corresponding to the target lamp of the three second sensors for 200 ms; The motion intention decoding model includes the following configuration content: One-dimensional time series signal data input to the model is converted into a two-dimensional image signal through frequency domain conversion, and the two-dimensional image signal is input to the first half of the model, wherein the conversion process includes: converting the signal from time domain to frequency domain by using discrete Fourier transform processing, estimating the power spectral density through the amplitude square of the Fourier transform of the signal, and smoothing the estimated power spectral density through the Welch method; The first half of the model includes three two-dimensional convolution layers and three maximum pooling layers, after two-dimensional convolution processing of each two-dimensional convolution layer, maximum pooling is performed by a maximum pooling layer, and then the next two-dimensional convolution layer is input, until the third maximum pooling layer at the last position outputs to the second half of the model; The second half of the model includes two GRU layers responsible for concatenation, the GRU layers are configured to extract time series features, and the output of the second GRU layer is sequentially processed by a flatten layer, a full connection layer, a dropout layer and an output layer to obtain model output; The specifications of the three two-dimensional convolution layers are 998×26×32, 497×11×64 and 246×3×64 in sequence, and the specifications of the three maximum pooling layers are 499×13×32, 248×5×64 and 123×1×64 in sequence; The specifications of the flatten layer, the full connection layer, the dropout layer and the output layer are 1×128, 1×64, 1×64 and 1×2 in sequence.

7. A processing device, characterized by A computer program product comprising a computer readable storage medium having computer readable program instructions embodied therewith, the computer readable program instructions adapted to be loaded into a processor to implement the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions adapted to be loaded into a processor to implement the method of any one of claims 1 to 5.

Images