Neural signal processing method and system, electronic equipment and storage medium

Abstract

The invention discloses a neural signal processing method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring a tested neural signal; the neural signal is input into a first decoding model for decoding, a first decoding result is obtained, and the first decoding model is obtained through training according to historical brain control data of a subject; in response to the decoding demand change signal, training a second decoding model according to the neural signal and the first decoding result to obtain a target decoding model, the second decoding model being constructed based on the changed decoding demand; and in response to the obtained target decoding model, inputting the neural signal into the target decoding model for decoding to obtain a second decoding result. The technical problem that in the prior art, a decoding model trained based on historical data is low in accuracy is solved.

Description

Technical Field

[0001] This invention relates to the field of neural signal processing, and more specifically, to a neural signal processing method, system, electronic device, and storage medium. Background Technology

[0002] In the field of brain-computer interfaces (BCI), existing brain control methods typically follow a three-step process: offline data collection, online model calibration, and real-time brain control execution. This process begins by collecting EEG signal data offline and using this data to train an initial decoding model. Then, a brief calibration process is performed before the online phase begins to adjust the model parameters. Finally, the model is applied to real-time brain control tasks to achieve control of the target device.

[0003] During the calibration phase, although necessary adjustments are made to the model, this process is usually brief and cannot continuously capture and reflect the dynamic changes in brain states. Especially for mind control applications that require long-term or frequent interaction, the frequency and effectiveness of calibration directly affect the accuracy of control and the user experience.

[0004] Therefore, the accuracy of decoding models trained on historical data in the existing technology is low, and even after short-term online calibration, the accuracy problem still exists.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This invention provides a neural signal processing method, system, electronic device, and storage medium to at least solve the technical problem of low accuracy of decoding models trained based on historical data in the prior art.

[0007] According to one aspect of the present invention, a signal processing method is provided, comprising: The process involves acquiring the subject's neural signals; inputting the neural signals into a first decoding model for decoding to obtain a first decoding result, wherein the first decoding model is trained based on the subject's historical brain control data; responding to a signal indicating a change in decoding requirements, training a second decoding model based on the neural signals and the first decoding result to obtain a target decoding model, wherein the second decoding model is constructed based on the changed decoding requirements; and responding to obtaining the target decoding model, inputting the neural signals into the target decoding model for decoding to obtain a second decoding result.

[0008] Optionally, in response to the decoding requirement change signal, training the second decoding model based on the neural signal and the first decoding result to obtain the target decoding model includes: in response to the decoding requirement change signal, determining the second decoding model corresponding to the decoding requirement change signal from a preset decoding model library; and training the second decoding model based on the neural signal and the first decoding result to obtain the target decoding model.

[0009] Optionally, in response to a decoding requirement change signal, training a second decoding model based on neural signals and a first decoding result to obtain a target decoding model includes: in response to a decoding requirement change signal, training the second decoding model based on neural signals and a first decoding result to obtain an initial decoding model; verifying the mind control success rate of the initial decoding model to obtain a first verification result; and determining the initial decoding model as the target decoding model in response to the first verification result indicating that the mind control success rate of the initial decoding model is greater than a preset success rate threshold.

[0010] Optionally, in response to a signal indicating a change in decoding requirements, training a second decoding model based on neural signals and a first decoding result to obtain a target decoding model includes: in response to a signal indicating a change in decoding requirements, extracting features from neural signals and a first decoding result to obtain first feature data; passing the feature data through a fully connected layer or a convolutional layer to obtain second feature data; and training the second decoding model based on the second feature data to obtain the target decoding model.

[0011] Optionally, the neural signal processing method further includes: performing accuracy verification on the second decoding result to obtain a second verification result; and generating a decoding requirement change signal in response to the second verification result indicating that the accuracy is less than a preset accuracy threshold.

[0012] Optionally, inputting the neural signal into the first decoding model for decoding to obtain the first decoding result includes: performing online calibration on the first decoding model to obtain a calibrated first decoding model; inputting the neural signal into the calibrated first decoding model for decoding to obtain the first decoding result.

[0013] According to another aspect of the present invention, a neural signal processing apparatus is also provided, comprising: The system comprises: an acquisition module for acquiring neural signals from the subject; a first decoding module for inputting neural signals into a first decoding model for decoding to obtain a first decoding result, wherein the first decoding model is trained based on the subject's historical brain control data; a training module for training a second decoding model based on the neural signals and the first decoding result in response to a signal indicating a change in decoding requirements, thereby obtaining a target decoding model, wherein the second decoding model is constructed based on the changed decoding requirements; and a second decoding module for inputting neural signals into the target decoding model for decoding in response to obtaining the target decoding model, thereby obtaining a second decoding result.

[0014] According to another aspect of the present invention, a brain-computer interface system is also provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present invention during runtime.

[0015] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0016] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0017] According to another aspect of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0018] According to another aspect of the present invention, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of the present invention.

[0019] In this embodiment of the invention, neural signals of the subject are acquired; the neural signals are input into a first decoding model for decoding to obtain a first decoding result, wherein the first decoding model is trained based on the subject's historical brain control data; in response to a signal indicating a change in decoding requirements, a second decoding model is trained based on the neural signals and the first decoding result to obtain a target decoding model, wherein the second decoding model is constructed based on the changed decoding requirements; in response to obtaining the target decoding model, neural signals are input into the target decoding model for decoding to obtain a second decoding result. This embodiment of the invention first utilizes a first decoding model trained on historical brain control data, which can perform preliminary decoding based on the subject's past behavioral patterns and quickly respond to initial control commands. When decoding requirements change (such as switching to a more complex control task), the system can quickly adjust and train the second decoding model, i.e., the target decoding model, based on real-time neural signals and the first decoding result. This mechanism ensures that the BCI system can quickly switch between different tasks without having to perform a lengthy offline data collection and model training process again, greatly saving time and resources. Meanwhile, the real-time training mechanism of the target decoding model enables the system to capture and adapt to changes in the subject's EEG characteristics in new tasks in a timely manner, thereby improving the accuracy of decoding and solving the technical problem of low accuracy of decoding models trained based on historical data in existing technologies. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a schematic flowchart of a signal processing method provided according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a calibration-free signal processing method according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating a calibrated signal processing method according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a signal processing device according to an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] According to an embodiment of the present invention, a signal processing method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0024] Figure 1This is a method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps: Step S101: Obtain the neural signals of the subject.

[0025] In this step, acquiring the subject's neural signals involves using a specialized electrophysiological signal acquisition system, such as SpikeGadgets, to capture the electrophysiological signals generated during the subject's brain activity in real time. These signals contain information about the patterns of neuronal firing in the brain and are the foundational data for the brain-computer interface (BCI) system to interpret brain activity. This process requires the signal acquisition system to have high sensitivity and low noise characteristics to ensure the quality of the acquired neural signals and provide a reliable data source for subsequent decoding and analysis.

[0026] Step S102: The neural signal is input into the first decoding model for decoding to obtain the first decoding result. The first decoding model is trained based on the subject's historical brain control data.

[0027] In this step, the first decoding model is pre-trained based on the subject's historical brain control data. This invention is not based on a general dataset or standard model, but rather on customized training tailored to the specific brain activity patterns of a particular subject, thus enabling more accurate interpretation of the subject's neural signals. When neural signals are input into the first decoding model, the model utilizes the learned feature and pattern information to convert the signals into understandable behavioral intentions or control commands—the first decoding result—thereby achieving preliminary interpretation and control of brain activity.

[0028] Step S103: In response to the decoding requirement change signal, the second decoding model is trained based on the neural signal and the first decoding result to obtain the target decoding model, wherein the second decoding model is constructed based on the changed decoding requirements.

[0029] In this step, the decoding requirement change signal refers to the need to adjust the decoding model in real time to adapt to new decoding requirements due to changes in task type, difficulty, or subject state during the brain-controlled task. This feature allows the BCI system in this embodiment to quickly adjust and construct a second decoding model based on current neural signals and the first decoding result to meet new decoding requirements without needing to recollect large amounts of offline data. This method enables the system to respond more flexibly to dynamic changes, improving decoding efficiency and control accuracy.

[0030] In step S104, in response to obtaining the target decoding model, the neural signal is input into the target decoding model for decoding to obtain the second decoding result.

[0031] Once the second decoding model is successfully trained according to the new decoding requirements, it becomes the target decoding model. In this step, the system re-inputs the real-time neural signals into the target decoding model for decoding to obtain a more accurate second decoding result. The key feature of this is that it ensures the BCI system's decoding model can always be optimized according to the latest changes in requirements, thereby significantly improving control accuracy and response speed.

[0032] In this embodiment of the invention, neural signals of the subject are acquired; the neural signals are input into a first decoding model for decoding to obtain a first decoding result, wherein the first decoding model is trained based on the subject's historical brain control data; in response to a signal indicating a change in decoding requirements, a second decoding model is trained based on the neural signals and the first decoding result to obtain a target decoding model, wherein the second decoding model is constructed based on the changed decoding requirements; in response to obtaining the target decoding model, neural signals are input into the target decoding model for decoding to obtain a second decoding result. This embodiment of the invention first utilizes a first decoding model trained on historical brain control data, which can perform preliminary decoding based on the subject's past behavioral patterns and quickly respond to initial control commands. When decoding requirements change (such as switching to a more complex control task), the system can quickly adjust and train the second decoding model, i.e., the target decoding model, based on real-time neural signals and the first decoding result. This mechanism ensures that the BCI system can quickly switch between different tasks without having to perform a lengthy offline data collection and model training process again, greatly saving time and resources. Meanwhile, the real-time training mechanism of the target decoding model enables the system to capture and adapt to changes in the subject's EEG characteristics in new tasks in a timely manner, thereby improving the accuracy of decoding and solving the technical problem of low accuracy of decoding models trained based on historical data in existing technologies.

[0033] This invention, through the implementation of the above steps, achieves dynamic adjustment and optimization of the BCI system decoding model. Specifically, it first captures the subject's neural signals using specialized equipment, then performs preliminary decoding using a customized, trained first decoding model. Subsequently, based on real-time changes in decoding requirements, a second decoding model is rapidly constructed and trained. The final target decoding model can interpret neural signals more accurately based on the latest requirements. This series of steps not only improves the flexibility and adaptability of the BCI system but also significantly enhances control precision and response speed, providing subjects with a better brain-computer interface experience. In application scenarios such as medical rehabilitation and virtual reality game control, the technical solution of this invention can achieve more efficient and precise brain-computer interface control, enhancing the system's practicality.

[0034] Optionally, in step S103, in response to the decoding requirement change signal, the second decoding model is trained according to the neural signal and the first decoding result to obtain the target decoding model, which may include the following steps: in response to the decoding requirement change signal, the second decoding model corresponding to the decoding requirement change signal is determined from the preset decoding model library; the second decoding model is trained according to the neural signal and the first decoding result to obtain the target decoding model.

[0035] The decoding requirement change signal is a signal used in this embodiment of the invention to instruct the system to adjust the currently used decoding model. This signal is typically triggered when changes are detected in the subject's intent, task requirements, or environmental conditions, aiming to ensure that the decoding model can adapt to new decoding requirements and improve decoding accuracy and flexibility.

[0036] The pre-set decoding model library is a collection of decoding models designed according to different decoding requirements stored in this embodiment of the invention. These models are pre-trained and cover different task types from simple to complex. The purpose is to quickly locate the second decoding model most suitable for the current requirement when a signal of change in decoding requirements is received, thereby reducing the time cost of real-time model training.

[0037] Upon detecting a change in decoding requirements, the system immediately accesses the preset decoding model library and, based on the task requirements or environmental changes indicated by the signal, selects the most suitable second decoding model. After selecting the second decoding model, the system uses the latest neural signals and the decoding results of the previous first decoding model to fine-tune or retrain the second decoding model online, thereby obtaining a more accurate target decoding model.

[0038] In this embodiment of the invention, by responding in real-time to signals indicating changes in decoding requirements, quickly selecting and training a second decoding model from a pre-set decoding model library, and finally obtaining the target decoding model, this series of dynamic adjustments significantly improves the flexibility, real-time performance, and accuracy of the system's decoding. This dynamic optimization mechanism for the decoding model reduces redundant operations in traditional BCI systems when faced with changing decoding requirements, such as unnecessary offline data collection and lengthy model building processes. In application scenarios, whether it's timely adjustments to patient recovery plans in the medical rehabilitation field or instant changes to player control strategies in virtual reality games, this embodiment of the invention can respond rapidly, providing the decoding model best suited to the current needs, thereby achieving real-time, efficient, and precise brain-computer interface control, greatly enhancing the user experience and the system's practicality when using the BCI system.

[0039] Optionally, in step S103, in response to the decoding requirement change signal, the second decoding model is trained based on the neural signals and the first decoding result to obtain the target decoding model, which may include the following steps: in response to the decoding requirement change signal, the second decoding model is trained based on the neural signals and the first decoding result to obtain an initial decoding model; the brain control success rate of the initial decoding model is verified to obtain a first verification result; in response to the first verification result showing that the brain control success rate of the initial decoding model is greater than a preset success rate threshold, the initial decoding model is determined as the target decoding model.

[0040] In this embodiment of the invention, when a signal indicating a change in decoding requirements is detected, the system initiates the training process of the second decoding model. This process uses the latest neural signal data and the first decoding result generated by the first decoding model as input, and generates an initial decoding model through algorithm learning and parameter optimization, ready for use in new decoding tasks.

[0041] After the initial decoding model is generated, the system uses a pre-set test dataset to evaluate its brain-control success rate. This validation step aims to ensure the model's performance in real-world applications. The brain-control success rate refers to the percentage of neural signals successfully converted into preset behaviors or actions by the decoding model in a brain-computer interface (BCI) system. It is a crucial metric for evaluating the decoding model's performance, reflecting the accuracy of the model's neural signal decoding and the stability of the system's control.

[0042] If the initial validation results show that the brain control success rate of the initial decoding model exceeds the preset success rate threshold, it means that the model has reached the expected performance standard. At this point, the system officially designates the initial decoding model as the target decoding model for subsequent decoding tasks, ensuring the accuracy of decoding and the stability of control.

[0043] In this embodiment of the invention, through the implementation of the above steps, the system can achieve dynamic optimization and verification of the decoding model, ensuring that the model maintains high efficiency and high accuracy decoding capabilities even after changes in decoding requirements. These features not only focus on model generation but also emphasize the verification of model effectiveness and applicability. By setting a preset success rate threshold, the system can automatically select initial decoding models that meet the performance requirements and identify them as target decoding models, avoiding the use of invalid or inefficient models, thereby significantly improving the accuracy of brain-computer interface control and the user experience. In application scenarios such as medical rehabilitation, game interaction, and assisted education, this rapid adaptation and verification mechanism for decoding models ensures that the system can always provide optimized decoding services when facing different decoding requirements, enhancing the practicality and applicability of this embodiment of the invention.

[0044] Optionally, in step S103, in response to the decoding requirement change signal, the second decoding model is trained based on the neural signal and the first decoding result to obtain the target decoding model, which may include the following steps: in response to the decoding requirement change signal, feature extraction is performed on the neural signal and the first decoding result to obtain first feature data; the feature data is passed through a fully connected layer or a convolutional layer to obtain second feature data; the second decoding model is trained based on the second feature data to obtain the target decoding model.

[0045] In this embodiment of the invention, upon receiving a decoding requirement change signal, the system will begin in-depth analysis of the receiving neural signal and the previously obtained first decoding result. Through pre-defined algorithms, such as time window sliding, filter application, or waveform morphology recognition, key features are extracted to form first feature data. This process helps to eliminate irrelevant information and focus on dynamic features in the neural signal closely related to the change requirement.

[0046] The first feature data is input into either a fully connected layer or a convolutional layer in the deep learning network. The role of the fully connected layer is to integrate feature information and establish global correlations between features; while the convolutional layer focuses on extracting local correlations of features, such as the time series patterns or spatial distribution characteristics of neural signals. After processing by these two layers, the original feature data is transformed into higher-level and more meaningful second feature data, providing optimized data representation for subsequent decoding model training.

[0047] It should be noted that while the first feature data is input into the fully connected layer or convolutional layer in the deep learning network, the number of feature input channels is reduced to ensure that the online prediction time meets the user's needs.

[0048] The obtained second feature data was used to train the second decoding model. The model is designed to solve new tasks after the decoding requirements have changed. Through iterative learning, the model gradually masters the relationship between the second feature data and the expected output until it converges. The resulting target decoding model can then accurately decode the subject's intention or state based on the input neural signals and adapt to the changed decoding requirements.

[0049] Optionally, the neural signal processing method further includes: performing accuracy verification on the second decoding result to obtain a second verification result; and generating a decoding requirement change signal in response to the second verification result indicating that the accuracy is less than a preset accuracy threshold.

[0050] After the decoding task is executed, the system automatically performs an accuracy verification procedure, which compares the second decoding result with a predefined set of correct answers. By comparing these results, the system can calculate the decoding accuracy of the target decoding model in the specific task, thus deriving a second verification result. This step is crucial for monitoring and evaluating the real-time performance of the decoding model.

[0051] When the second verification result shows that the decoding accuracy of the target decoding model has not reached the preset accuracy threshold, the system determines that the current model may not be suitable for the current decoding requirements or task environment. To correct this situation, the system automatically generates a decoding requirement change signal. The generation of this signal instructs the system to begin preparing a new decoding strategy, which may include reselecting the decoding model, adjusting parameters, or updating training data, in order to improve decoding accuracy and meet the desired control requirements.

[0052] By integrating accuracy verification and decoding requirement change signal generation mechanisms into neural signal processing, this invention enables real-time monitoring and automatic optimization of decoding model performance. This feature introduces a feedback loop, allowing the system to automatically adjust its decoding strategy based on the current decoding model's performance, ensuring consistently high decoding accuracy. When the accuracy of the second decoding result falls below a preset threshold, the system generates a decoding requirement change signal, triggering an update to the decoding model. This process requires no manual intervention, significantly improving the system's adaptability and control stability. In specific application scenarios, whether it's complex medical rehabilitation, sophisticated virtual reality experiences, or everyday human-computer interaction tasks, this mechanism ensures that the BCI system of this invention maintains optimal decoding performance in the face of constantly changing needs and environmental conditions, providing subjects with continuous and reliable brain-computer interface services. This enhances user experience while expanding the application boundaries and potential of BCI technology.

[0053] Optionally, inputting the neural signal into the first decoding model for decoding to obtain the first decoding result includes: performing online calibration on the first decoding model to obtain a calibrated first decoding model; inputting the neural signal into the calibrated first decoding model for decoding to obtain the first decoding result.

[0054] During BCI system operation, the system monitors neural signals and collects calibration data reflecting the subject's neural activity patterns during the current task. This information is used to adjust the first decoding model in real time, updating its parameters to better match the subject's real-time signal characteristics. This process typically involves small amounts of data and rapid computation to ensure that model calibration does not impact the system's real-time performance.

[0055] Once the first decoding model completes online calibration, the system uses the calibrated model to interpret real-time neural signals. Through the optimized model, the system can more accurately decode neural signals and convert them into behavioral commands, such as controlling cursor movement in a virtual reality environment. This step is a core function of the BCI system, aiming to achieve efficient and accurate neural signal decoding to control devices or interfaces.

[0056] By integrating an online calibration mechanism into neural signal processing, this invention significantly improves the adaptability and decoding accuracy of the first decoding model. The implementation of online calibration allows the model to dynamically adjust parameters based on the subject's current neural signal characteristics during BCI system operation. This not only avoids the lengthy process of offline training but also ensures that the decoding model can be optimized in real time to adapt to changes in the subject's tasks and environments. In application scenarios such as medical rehabilitation, virtual reality, or augmented reality control, this technical effect enables the BCI system to decode neural signals more accurately and flexibly, providing subjects with a smoother and more natural control experience. This improves control accuracy and enhances the practicality of the BCI system in real-world applications. Overall, this invention achieves dynamic optimization of the decoding model through online calibration, directly improving the performance of the BCI system in neural signal decoding and providing users with a better brain-controlled experience.

[0057] In some embodiments of the present invention, the neural signal processing method provided by the present invention is implemented as follows: 1. The signal intensity threshold for electrophysiological signals is set to 70, and the number of effective channels is set to 400; 2. Utilize a passive observation center-out paradigm to help subjects (e.g., rhesus monkeys) understand the task they are performing; 3. Extract features from the spike signal and cache the data; the data size is m. n and m represent time, n represents the number of channels, and the size is 400; 4. Randomly select 50 channels from the number of channels n to form a new dataset and train the Kalman-filter model; 5. Use the trained Kalman-filter model to predict motion trajectories and train the subject to control a center-out paradigm cursor. 6. Use the Kalman filter method for mind control for more than 20 minutes, continuously caching the extracted feature data, with a size of m. n and m represent time, n represents the number of channels, and the size is 400; 7. Pass the cached data through a fully connected layer or a convolutional layer to extract features again and reduce the number of channels to ensure the time requirements for online prediction are met.

[0058] 8. Use the processed feature data to train or fine-tune the model of method 2, keeping the time under 20 minutes.

[0059] 9. Using method 2, predict the movement trajectory of the center-out paradigm cursor and train subjects to use method 2 to control the cursor using their minds.

[0060] It should be noted that Method 1 refers to decoding neural signals using the first decoding model, while Method 2 refers to decoding neural signals using the second neural signal decoding model.

[0061] Reference Figure 2 , Figure 2 This invention presents a calibration-free neural signal processing method, comprising method 1 and method 2. In the transition from method 1 to method 2, the online calibration step is omitted, as described in detail below.

[0062] Method 1 involves an online phase, which includes the following process.

[0063] Collect raw data: This involves acquiring raw signals of brain neural activity, typically during a specific task or activity, such as when a subject performs a specific motor task, by capturing neural signals using electrophysiological recording devices.

[0064] Feature extraction involves preprocessing and analyzing the collected raw neural signals to extract features relevant to a specific task or activity. Feature extraction may include setting a threshold to eliminate noise, selecting effective signal channels (channel selection), and performing Gaussian smoothing to reduce random fluctuations, thereby obtaining clean and easily analyzable feature data.

[0065] Training the model: The first decoding model is trained using the extracted feature data. Through machine learning methods, the model learns how to map neural signal features to specific outputs (such as motion trajectories or activity types). This process involves optimizing the algorithm and tuning the model parameters.

[0066] Then, the online prediction process begins, using a pre-trained model to make real-time predictions to control external devices or software. This process includes the following steps.

[0067] Collect raw data: First, collect the electrophysiological signals of the subject's brain in real time. This can be electroencephalogram (EEG), magnetoencephalography (MEG), electrocorticogram (ECoG), or firing data of individual neurons.

[0068] Feature extraction: The collected raw data needs to be preprocessed and feature extracted so that the model can interpret it.

[0069] Buffer data: During the prediction process, the raw data or extracted features are temporarily stored in a buffer. The purpose of this step is to ensure that the model has a sufficient data foundation when making predictions, while also handling real-time data issues. For example, when the model needs a period of signal input before making a prediction, the buffer can store this data until the prediction is complete.

[0070] Predicting velocity: Finally, the model makes predictions based on the data in the buffer, specifically predicting the movement speed of the device or software the subject intends to control. This prediction is achieved by feeding the features in the buffer into a pre-trained model (such as a Kalman filter or KalmanNet) and calculating movement commands corresponding to brain activity based on the model.

[0071] After Method 1 successfully ran and completed the speed prediction, Figure 2 Instead of an online recalibration step, the process directly transitions to using the new method 2 for mind control, which includes the following steps.

[0072] Buffer data: Save the EEG signal data collected in real time during the execution of Method 1, ready for use in subsequent model training.

[0073] Feature processing (FC / CNN): FC / CNN refers to Fully Connected Layer (FC) or Convolutional Neural Network (CNN), which is used to further extract and transform features from buffered data. The purpose is to reduce dimensionality and optimize features to facilitate model learning.

[0074] Model training: Using processed feature data to train or further optimize the model of method 2, for example, using deep learning methods.

[0075] Switch method: Once the model training for method 2 is complete, the mind control will be switched from method 1 to method 2, ensuring that the subject does not need to interrupt the mind control task.

[0076] Predict velocity: Finally, the model from Method 2 is used to predict the velocity indicated by brain activity, which is then used to control external devices or software.

[0077] thus, Figure 2 The method shown omits the recalibration step in the online phase, which makes the transition from Method 1 to Method 2 smoother, reduces the complexity and time of experimental operations, and is particularly advantageous for brain-controlled applications that require real-time control and efficient switching, such as robotic arm control, game interaction, or other scenarios that require rapid response, and can significantly improve user experience and control efficiency.

[0078] Reference Figure 3 , Figure 3 This invention presents a calibrated neural signal processing method, including method 1 and method 2, which are described in detail below.

[0079] Method 1 illustrates a process directly from the raw signal to model prediction without the need for online calibration. Method 1 involves an offline stage and an online stage, where the offline stage includes the following steps.

[0080] Collect raw data: This involves acquiring raw signals of brain neural activity, typically during a specific task or activity, such as when a subject performs a specific motor task, by capturing neural signals using electrophysiological recording devices.

[0081] Feature extraction involves preprocessing and analyzing the collected raw neural signals to extract features relevant to a specific task or activity. Feature extraction may include setting a threshold to eliminate noise, selecting effective signal channels (channel selection), and performing Gaussian smoothing to reduce random fluctuations, thereby obtaining clean and easily analyzable feature data.

[0082] Training the model: The first decoding model is trained using the extracted feature data. Through machine learning methods, the model learns how to map neural signal features to specific outputs (such as motion trajectories or activity types). This process involves optimizing the algorithm and tuning the model parameters.

[0083] Then comes the offline recalibration process, a crucial step before the online phase begins, designed to ensure that the trained model accurately reflects the subject's current brain state. This process includes the following steps.

[0084] Collect raw data: Re-collect raw electrophysiological signals of the subject's brain activity.

[0085] Feature extraction: After collecting the raw data, feature extraction is required to identify and process signals related to brain activity. This feature extraction process may also include setting thresholds, channel selection, and Gaussian smoothing, which will not be elaborated on here.

[0086] Recalibration model: After feature extraction is completed, a dedicated recalibration model is used to adjust the parameters or state of the initial model. This recalibration model is trained based on the new raw data and features collected so that the model can better adapt to the subject's current EEG activity pattern.

[0087] The prediction process, which transitions from the offline phase to the online phase, determines the accuracy and real-time performance of BCI in controlling external devices or performing specific tasks. This process includes the following steps.

[0088] Collect raw data: At the start of online prediction, the electrophysiological signals of the subject's brain are collected in real time. This can be electroencephalography (EEG), magnetoencephalography (MEG), electrocorticography (ECoG), or firing data of individual neurons.

[0089] Feature extraction: The collected raw data needs to be preprocessed and feature extracted so that the model can interpret it.

[0090] Buffer data: During the prediction process, the raw data or extracted features are temporarily stored in a buffer. The purpose of this step is to ensure that the model has a sufficient data foundation when making predictions, while also handling real-time data issues. For example, when the model needs a period of signal input before making a prediction, the buffer can store this data until the prediction is complete.

[0091] Predicting velocity: Finally, the model makes predictions based on the data in the buffer, specifically predicting the movement speed of the device or software the subject intends to control. This prediction is achieved by feeding the features in the buffer into a pre-trained model (such as a Kalman filter or KalmanNet) and calculating movement commands corresponding to brain activity based on the model.

[0092] Method 2 is a further optimized and improved mind control strategy based on Method 1, including an online phase, which includes the following process: Buffer data: After the online prediction speed of Method 1, the EEG signal data collected in real time during the operation of Method 1 is saved for subsequent model training.

[0093] Feature processing (FC / CNN): FC / CNN refers to fully connected layers (FC) or convolutional neural networks (CNN), which are used to further extract and transform features from buffered data. The purpose is to reduce dimensionality and optimize features to facilitate model learning.

[0094] Model training: Using processed feature data to train or further optimize the model of method 2, for example, using deep learning methods.

[0095] Switch method: Once the model training for method 2 is complete, the mind control will be switched from method 1 to method 2, ensuring that the subject does not need to interrupt the mind control task.

[0096] Predict velocity: Finally, the model from Method 2 is used to predict the velocity indicated by brain activity, which is then used to control external devices or software.

[0097] Optionally, embodiments of the present invention integrate multimodal signal sources to increase the robustness and information content of decoding, enabling the system to more accurately understand and respond to the subject's intentions.

[0098] First, the offline data collection process is not limited to a single type of neural signal (such as EEG), but also collects multiple types of signals, including but not limited to eye-tracking data, facial expressions, and muscle potentials (EMG). These multimodal signals together constitute a more comprehensive description of the subject's state, which helps the model to decode complex intentions and emotions more accurately.

[0099] During the feature extraction phase, the system employs a hybrid signal processing algorithm to extract unique and complementary information from each type of acquired signal. The first decoding model trained is a joint model based on multimodal features, capable of collaboratively decoding the subject's intent from multiple signals. Even when the neural signals themselves are weak or of poor quality, it can improve decoding accuracy through data compensation from other modalities.

[0100] During the online phase, the system can intelligently and dynamically adjust the weights of signal sources or switch the primary control model based on the real-time quality of various signals. For example, when the EMG signal quality is detected to be good, the system may rely more on EMG data to drive control, and vice versa.

[0101] The Method 2 model may be specifically designed for signal fusion, that is, seamless switching and intelligent fusion between different modal signals, in order to cope with different types of tasks or operating environments and provide a more stable and accurate brain control experience.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0103] Reference Figure 4 This invention also provides a neural signal processing device system, comprising: The acquisition module 401 is used to acquire the neural signals of the subject; the first decoding module 402 is used to input the neural signals into the first decoding model for decoding to obtain the first decoding result, wherein the first decoding model is trained based on the subject's historical brain control data; the training module 403 is used to train the second decoding model based on the neural signals and the first decoding result in response to the decoding requirement change signal to obtain the target decoding model, wherein the second decoding model is constructed based on the changed decoding requirement; the second decoding module 404 is used to input the neural signals into the target decoding model for decoding in response to obtaining the target decoding model to obtain the second decoding result.

[0104] Embodiments of the present invention also provide a brain-computer interface system, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present invention during runtime.

[0105] Optionally, in this embodiment, the processor in the brain-computer interface system described above can be configured to run an executable program to perform the following steps: Step S101: Obtain the neural signals of the subject; Step S102: The neural signal is input into the first decoding model for decoding to obtain the first decoding result. The first decoding model is trained based on the subject's historical brain control data. Step S103: In response to the decoding requirement change signal, the second decoding model is trained based on the neural signal and the first decoding result to obtain the target decoding model, wherein the second decoding model is constructed based on the changed decoding requirement; In step S104, in response to obtaining the target decoding model, the neural signal is input into the target decoding model for decoding to obtain the second decoding result.

[0106] Optionally, in this embodiment, the executable program can be configured to store an executable program for performing the following steps: Step S101: Obtain the neural signals of the subject; Step S102: The neural signal is input into the first decoding model for decoding to obtain the first decoding result. The first decoding model is trained based on the subject's historical brain control data. Step S103: In response to the decoding requirement change signal, the second decoding model is trained based on the neural signal and the first decoding result to obtain the target decoding model, wherein the second decoding model is constructed based on the changed decoding requirement; In step S104, in response to obtaining the target decoding model, the neural signal is input into the target decoding model for decoding to obtain the second decoding result.

[0107] Embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0108] Optionally, in this embodiment, the computer program, when executed by the processor, performs the following steps: Step S101: Obtain the neural signals of the subject; Step S102: The neural signal is input into the first decoding model for decoding to obtain the first decoding result. The first decoding model is trained based on the subject's historical brain control data. Step S103: In response to the decoding requirement change signal, the second decoding model is trained based on the neural signal and the first decoding result to obtain the target decoding model, wherein the second decoding model is constructed based on the changed decoding requirement; In step S104, in response to obtaining the target decoding model, the neural signal is input into the target decoding model for decoding to obtain the second decoding result.

[0109] Embodiments of the present invention also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the methods of various embodiments of the present invention.

[0110] Embodiments of the present invention also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of the present invention.

[0111] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0112] In the several embodiments provided by this invention, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection can be through some interfaces; the indirect coupling or communication connection of units or modules can be electrical or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0116] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method of processing neural signals, the method comprising: include: Acquire the neural signals of the subjects; The neural signal is input into a first decoding model for decoding to obtain a first decoding result, wherein the first decoding model is trained based on the subject's historical brain control data; In response to a signal indicating a change in decoding requirements, a second decoding model is trained based on the neural signal and the first decoding result to obtain a target decoding model, wherein the second decoding model is constructed based on the changed decoding requirements; In response to obtaining the target decoding model, the neural signal is input into the target decoding model for decoding to obtain a second decoding result.

2. The neural signal processing method according to claim 1, characterized in that, The step of training the second decoding model in response to the decoding requirement change signal, based on the neural signal and the first decoding result, to obtain the target decoding model includes: In response to a decoding requirement change signal, the second decoding model corresponding to the decoding requirement change signal is determined from a preset decoding model library; Based on the neural signals and the first decoding result, the second decoding model is trained to obtain the target decoding model.

3. The neural signal processing method according to claim 1, characterized in that, The step of training the second decoding model in response to the decoding requirement change signal, based on the neural signal and the first decoding result, to obtain the target decoding model includes: In response to the signal indicating a change in decoding requirements, the second decoding model is trained based on the neural signal and the first decoding result to obtain an initial decoding model; The success rate of mind control based on the initial decoding model was verified, and the first verification result was obtained. In response to the first verification result indicating that the mind control success rate of the initial decoding model is greater than a preset success rate threshold, the initial decoding model is determined as the target decoding model.

4. The neural signal processing method according to claim 1, characterized in that, The step of training the second decoding model in response to the decoding requirement change signal, based on the neural signal and the first decoding result, to obtain the target decoding model includes: In response to a signal indicating a change in decoding requirements, feature extraction is performed on the neural signal and the first decoding result to obtain first feature data; The feature data is passed through a fully connected layer or a convolutional layer to obtain the second feature data. The second decoding model is trained based on the second feature data to obtain the target decoding model.

5. The neural signal processing method according to claim 1, characterized in that, Also includes: The accuracy of the second decoding result is verified to obtain the second verification result; In response to the second verification result indicating that the accuracy is less than a preset accuracy threshold, the decoding requirement change signal is generated.

6. The neural signal processing method according to claim 1, characterized in that, The step of inputting the neural signal into the first decoding model for decoding to obtain the first decoding result includes: The first decoding model is calibrated online to obtain the calibrated first decoding model; The neural signal is input into the calibrated first decoding model for decoding to obtain the first decoding result.

7. A neural signal processing device, characterized in that, include: The acquisition module is used to acquire the neural signals of the subjects; The first decoding module is used to input the neural signal into the first decoding model for decoding to obtain the first decoding result, wherein the first decoding model is trained based on the subject's historical brain control data; A training module is used to train a second decoding model in response to a decoding requirement change signal, based on the neural signal and the first decoding result, to obtain a target decoding model, wherein the second decoding model is constructed based on the changed decoding requirements; The second decoding module is used to input the neural signal into the target decoding model for decoding in response to obtaining the target decoding model, and to obtain the second decoding result.

8. A brain-computer interface system, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images