Behavior prediction method and device based on neural signal, equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence (AI), in particular to a behavior prediction method and device based on neural signals, equipment and a storage medium. The method comprises the steps of obtaining a target neural signal of a target object, inputting the target neural signal into a trained behavior prediction model, and obtaining predicted behavior data, output by the model, of the target object; through the trained behavior prediction model, cross-time, cross-task and cross-individual neural signal distribution changes can be effectively adapted, the behavior of the target object is accurately and efficiently predicted, and the accuracy and efficiency of behavior prediction are improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a behavior prediction method, device, equipment and storage medium based on neural signals. Background Art

[0002] In recent years, brain-computer interface (BCI) technology has made significant progress in helping paralyzed patients control computers and external devices. However, the instability of neural recordings has always been a key challenge in the application of BCI technology. Because neural activity changes over time, BCI systems need to be frequently recalibrated to maintain stable performance.

[0003] However, the relevant technology can only be applied to BCI calibration in single-object and single-task application scenarios. Once cross-object and cross-task changes occur, the stability of neural recordings cannot be guaranteed. For example, some BCI systems require objects to switch in multi-task scenarios, such as controlling cursor position and switching function options, and traditional technologies cannot obtain the impact of task changes on neural signals, which may cause the cursor to move to the wrong position or the user to be unable to switch to the target function. For another example, when the same BCI system is applied to different objects, due to differences in neurophysiological characteristics of different objects (such as brain structure or neural activity patterns), the signal characteristics are significantly different, resulting in the inability to accurately predict the behavior of each object.

[0004] Therefore, when traditional technologies predict behavior based on the subject's neural signals, they cannot guarantee the accuracy and efficiency of predictions based on individual differences and task diversity. Summary of the invention

[0005] The present invention provides a behavior prediction method, device, equipment and storage medium based on neural signals, which are used to improve the accuracy and efficiency of behavior prediction.

[0006] In a first aspect, the present application provides a behavior prediction method based on neural signals, the method comprising:

[0007] Acquiring a target neural signal of a target object, and inputting the target neural signal into a trained behavior prediction model;

[0008] Obtaining predicted behavior data of the target object output by the behavior prediction model;

[0009] Among them, the behavior prediction model is trained based on a sample data set of multiple sample objects, and the sample data set includes sample neural signals and sample behavior data generated by the corresponding sample objects under multiple neural activity tasks respectively; the behavior prediction model includes a decoder and a meta-aligner, the aligner is used to align the target neural signals to obtain the aligned neural signal features, and the decoder is used to predict the neural signal features and output corresponding predicted behavior data.

[0010] In a second aspect, the present application provides a behavior prediction device based on neural signals, the device comprising:

[0011] an acquisition unit, configured to acquire a target neural signal of a target object and input the target neural signal into a trained behavior prediction model;

[0012] A prediction unit, used to obtain the predicted behavior data of the target object output by the behavior prediction model;

[0013] Among them, the behavior prediction model is trained based on a sample data set of multiple sample objects, and the sample data set includes sample neural signals and sample behavior data generated by the corresponding sample objects under multiple neural activity tasks respectively; the behavior prediction model includes a decoder and a meta-aligner, the aligner is used to align the target neural signals to obtain the aligned neural signal features, and the decoder is used to predict the neural signal features and output corresponding predicted behavior data.

[0014] Optionally, the device further comprises a training unit, configured to:

[0015] Acquire original data sets corresponding to each of the plurality of sample objects, and preprocess the original data sets; the original data sets include original neural signals and original behavioral data respectively generated by the corresponding sample objects under a plurality of neural activity tasks;

[0016] Based on the preset scene conditions, the preprocessed original data set is classified and processed to obtain the corresponding sample data set.

[0017] Optionally, the training unit is further used for:

[0018] Iteratively training the decoder based on the first sample subset in the sample data set until the loss function of the decoder meets a preset convergence condition, thereby obtaining a trained decoder;

[0019] Based on the second sample subset in the sample data set, inner loop training and meta-learning training are performed on the meta-aligner to obtain a trained meta-aligner.

[0020] Optionally, the training unit is specifically used to:

[0021] Based on a preset task construction strategy, sampling processing is performed on the second sample subset to obtain a plurality of training task sets corresponding to the second sample subset;

[0022] Based on the multiple training task sets, obtaining a meta-objective function of the meta-aligner, wherein the meta-objective function represents a total loss value of the meta-aligner on the multiple training task sets;

[0023] Based on the meta-objective function, the model parameters of the meta-aligner are adjusted to obtain a trained meta-aligner.

[0024] Optionally, the training unit is specifically used to:

[0025] Based on the training data in each training task set and the loss function of the meta-aligner, updating the model parameters of the meta-aligner to obtain a plurality of updated parameters;

[0026] Based on the test data in each training task set and the corresponding update parameters, obtain the test error corresponding to the training task set;

[0027] The meta-objective function is obtained by performing summation processing based on various test errors.

[0028] Optionally, the preset scenario condition includes at least one of a cross-time scenario condition, a cross-task scenario and a cross-object scenario; wherein,

[0029] The sample data set corresponding to the cross-time scenario represents: data generated by the same sample object in two different time periods of the same neural activity task;

[0030] The sample data set corresponding to the cross-task scenario represents: data generated by the same sample object in two similar time periods of different neural activity tasks;

[0031] The sample data set corresponding to the cross-object scenario represents: data generated by different sample objects in two similar time periods of the same neural activity task; the similar time period represents that the time interval between the corresponding two time periods meets a preset interval condition.

[0032] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the neural signal-based behavior prediction methods described in the first aspect is implemented.

[0033] In a fourth aspect, the present application provides a computer storage medium, wherein the computer-readable storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement any one of the neural signal-based behavior prediction methods described in the first aspect.

[0034] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer program instructions, which, when executed by a processor, implement any one of the neural signal-based behavior prediction methods in the first aspect described above.

[0035] The beneficial effects of the present invention are as follows:

[0036] The embodiment of the present application provides a behavior prediction method based on neural signals, which obtains the target neural signals of the target object and inputs the target neural signals into a trained behavior prediction model to obtain the predicted behavior data of the target object output by the model. Among them, the present application uses sample data of multiple neural activity tasks to train the model, enhances the adaptability of the behavior prediction model to changes in the distribution of neural signals across tasks, time and individuals, and introduces meta-learning training, so that the model can quickly adapt to new neural signal distributions in unseen scenarios, ensure the accuracy and stability of behavior prediction, and improve the generalization and robustness of the behavior prediction model as a whole, providing reliable technical support for applications in multiple fields such as brain-computer interfaces and neural rehabilitation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 A flowchart of a method for training a behavior prediction model provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of a meta-aligner training process provided in an embodiment of the present application;

[0040] Figure 3 A flowchart of a behavior prediction method based on neural signals provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of a process of model training and model application provided in an embodiment of the present application;

[0042] Figure 5A schematic diagram of the structure of a behavior prediction device based on neural signals provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0045] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, can be two, three or more, and the embodiments of the present application are not limited.

[0046] The term "and / or" in the embodiments of the present application is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0047] It is understood that in the following specific implementations of the present application, related data such as polyolefin production data are involved. When the various embodiments of the present application are applied to specific products or technologies, relevant licenses or consents need to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, relevant volunteers can be recruited and relevant agreements on volunteer authorization data can be signed, and then the data of these volunteers can be used for implementation; or, by implementing within the scope of the authorized organization, the following implementation methods are implemented by using the data of internal members of the organization to manage data; or, the relevant data used in the specific implementation are all simulated data, such as simulated data generated in a virtual scene.

[0048] The embodiments of the present application relate to artificial intelligence and machine learning (ML) technology, and are mainly designed based on machine learning in artificial intelligence.

[0049] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0050] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0051] Machine learning is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0052] Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other technologies. Artificial neural networks (ANN) abstract the human brain neural network from the perspective of information processing, establish a simple model, and form different networks according to different connection methods. A neural network is a computational model composed of a large number of nodes (or neurons) connected to each other. Each node represents a specific output function, called an activation function. The connection between each two nodes represents a weighted value for the signal passing through the connection, called a weight, which is equivalent to the memory of the artificial neural network. The output of the network varies depending on the network connection method, weight value and activation function. The network itself is usually an approximation of a certain algorithm or function in nature, or it may be an expression of a logical strategy.

[0053] The following is a brief introduction to the design concept of the embodiments of the present application.

[0054] In recent years, BCI technology has made significant progress in helping paralyzed patients control computers and external devices. However, the instability of neural recordings has always been a key challenge in the application of BCI technology. Because neural activity changes over time, BCI systems need to be frequently recalibrated to maintain stable performance.

[0055] Traditional techniques for dealing with the instability of neural activity usually use exponentially weighted sliding averages to update interface parameters, track and record non-stationary adjustment parameters, or use low-dimensional projections to offset neural fluctuations. Alternatively, the interface is trained with a large amount of data, and semi-supervised learning is used to align the probability density function of the new predicted motion to the probability density function of the previous typical motion.

[0056] However, since traditional technology can only be applied to application scenarios of a single object and a single task, once changes occur across objects and tasks, traditional technology cannot guarantee the stability of neural recordings. For example, some BCI systems require objects to switch in multi-task scenarios, such as controlling cursor position and switching function options, and traditional technology cannot obtain the impact of task changes on neural signals, which may cause the cursor to move to the wrong position or the user to be unable to switch to the target function. For another example, when the same BCI system is applied to different objects, due to differences in neurophysiological characteristics of different objects (such as brain structure or neural activity patterns), the signal characteristics are significantly different, resulting in the inability to accurately predict the behavior of each object. Obviously, when traditional technology predicts behavior based on the object's neural signals, it cannot guarantee the accuracy and efficiency of predictions based on individual differences and task diversity.

[0057] In view of the above problems, an embodiment of the present application provides a behavior prediction method based on neural signals, which obtains the target neural signals of the target object and inputs the target neural signals into a trained behavior prediction model to obtain the predicted behavior data of the target object output by the model. Among them, the present application uses sample data of multiple neural activity tasks to train the model, enhances the adaptability of the behavior prediction model to changes in the distribution of neural signals across tasks, time and individuals, and introduces meta-learning training, so that the model can quickly adapt to new neural signal distributions in unseen scenarios, ensure the accuracy and stability of behavior prediction, and improve the generalization and robustness of the behavior prediction model as a whole, providing reliable technical support for applications in multiple fields such as brain-computer interfaces and neurorehabilitation.

[0058] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0059] The solution provided in the embodiment of the present application can be applied to most xxx application scenarios, and is used to significantly improve the prediction accuracy of the polyolefin melt index. For example, in an industrial production scenario, a distributed control system is a common automated control system in an industrial production process, which can collect the polymerization parameters (such as temperature, pressure, flow rate, catalyst concentration, etc.) of the polyolefin to be predicted in real time, and predict and feedback the polyolefin melt index according to the index prediction model provided in the embodiment of the present application, and obtain a more accurate melt index prediction result, thereby helping operators to adjust production conditions in real time, optimize process parameters, and improve product quality.

[0060] Of course, the method provided in the embodiment of the present application is not limited to the above application scenarios, but can also be used in other possible application scenarios, which are not limited by the embodiment of the present application. The functions that can be realized by each device in the above application scenarios will be described in the subsequent method embodiments, and no further details will be given here.

[0061] Below, in combination with the application scenarios described above, the method provided by the exemplary embodiment of the present application is described with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation methods of the present application are not limited in this regard.

[0062] The behavior prediction model in the embodiment of the present application is obtained by training with a sample data set of multiple sample objects, and the behavior prediction model can be used to realize the target neural signal of the target object and obtain its predicted behavior data. The model architecture of the behavior prediction model consists of a decoder and a meta-aligner. The trained aligner is used to align the target neural signal to obtain the aligned neural signal features, and the trained decoder is used to predict the neural signal features and output the corresponding predicted behavior data. During the model training process, the decoder can achieve good performance by training on sample data sets that are not well recorded, and correctly predict the behavior evaluated using the test set in each sample data set, which means that the decoder has learned to extract the relationship between the potential state and the motor intention from the neural activity. Other sample data sets are regarded as training tasks for meta-learning, and the meta-aligner is meta-trained, and the decoder remains fixed. The trained meta-aligner can project the changed neural activity recorded by different sample data sets into the historical neural activity space, and then convert it into a potential state, so that the fixed decoder predicts the behavior of the target object from it.

[0063] Therefore, before the behavior prediction model is put into use, it is necessary to collect sample data sets in advance and train the model with the sample data sets to make the behavior prediction model converge. Next, in order to facilitate the description of the model application process, the process of obtaining the sample data set is first introduced:

[0064] In a possible implementation, the embodiment of the present application can obtain original data sets corresponding to multiple sample objects respectively, and preprocess the original data sets. The original data sets include original neural signals and original behavior data generated by the corresponding sample objects under multiple neural activity tasks.

[0065] Specifically, taking two rhesus monkeys "RS" and "NW" as sample objects, three neural activity tasks, namely 8-way joystick, random target joystick, and touch-screen whack-a-mole, were performed on the two sample objects respectively, and the original data sets generated over a period of time were collected, such as 150 sessions spanning 18 months, which included the original neural signals of RS and NW in the M1 and PMD brain regions and the corresponding original task data. For example, the original task data corresponding to the 8-way joystick and the random target joystick are the two-dimensional coordinate position of the cursor, and the original task data corresponding to the touch-screen whack-a-mole is the number of the gopher hole where the gopher appears and is hit.

[0066] In a possible implementation, the preprocessing of the embodiment of the present application may include threshold crossing detection, Gaussian smoothing, affine transformation, data normalization processing and sliding window segmentation. The present application may first extract the pulse (spike) emission in the original neural signal through threshold crossing detection, and use Gaussian smoothing to convert the discrete spike emission into a continuous value, which is recorded as the neural signal X = {x1, x2, ..., xn}. For the discrete behavior data in the original data set, such as the number of the hamster hole where the hamster appears and hits, the embodiment of the present application will also transform the discrete value into a continuous two-dimensional coordinate value through affine transformation. Next, for the X neural signal and its corresponding behavior data Y = {y1, y2, ..., y n} to eliminate the dimensional differences of different data and improve the stability and accuracy of subsequent model training. n} and corresponding behavior data Y = {y1,y2,...,y n}, take all successfully completed trials and extract several subsequences.

[0067] Specifically, threshold crossing detection is used to extract pulse activity from neural signals, that is, a fixed voltage threshold is set for the original neural signal. When the signal value exceeds the threshold, the pulse occurrence at that time point is recorded, thereby converting the continuous original neural signal into sparse pulse events, improving the effectiveness of the signal and reducing noise interference.

[0068] Gaussian smoothing is a signal smoothing technique used to convert discrete pulse activity into a continuous signal representation, and to make the signal smoother by applying a Gaussian kernel function around each pulse event. In the embodiment of the present application, a Gaussian distribution function is applied near the time point of the pulse in the detected original neural signal to smooth the discrete pulse activity into a continuous value, thereby eliminating the noise of the discrete pulse signal and generating a more continuous and model-suitable neural activity signal.

[0069] Affine transformation is used to map data in one coordinate system to another. For discrete behavioral data (such as hamster hole numbers), affine transformation can be used to convert them into two-dimensional coordinate values, which is convenient for unified processing with other task data. By preprocessing discrete data with affine transformation, subsequent regression models can be used to uniformly decode classification problems, which is a prerequisite for solving the cross-task stability of brain-computer interfaces.

[0070] Normalization is done by shifting the data to zero mean and scaling it to unit variance so that it has the same distribution characteristics. The mean and standard deviation of the neural activity and corresponding behavioral data can be calculated separately, and then the normalization formula is applied to each data point. The formula for data normalization is as follows:

[0071]

[0072] in, is the mean of the neural signal X, σ(X) is the standard deviation of X, is the mean of the behavioral data Y, and σ(Y) is the standard deviation of Y.

[0073] Sliding window segmentation is a method of segmenting continuous time series data into several fixed-length subsequences. Subsequences can be extracted from the complete time series according to the set window size and sliding step size, and each subsequence is used as a sample. In the embodiment of the present application, the sliding window method can be used to extract subsequences from the entire time series data X = {x1, x2, ..., x n} and the corresponding behavior data Y = {y1,y2,...,y n}, take all successfully completed trials and intercept several subsequences to generate enough samples for model training while maintaining the local features of the time series. Specifically, assuming that the window size is w and the sliding distance is d, and there are L data points in a trial, the number of samples s after data segmentation is as follows:

[0074]

[0075] Among them, ceil is the floor function, s represents the number of samples generated, and L represents the total length of the time series data.

[0076] In one possible implementation, after preprocessing the original data set, the embodiment of the present application will also classify the preprocessed original data set according to preset scene conditions to obtain a corresponding sample data set to adapt to different scene requirements such as across time, across tasks, and across individuals. It can also enable the model to learn the changes in the distribution of neural activity data across time, across tasks, and across individuals, thereby aligning the neural data to the low-dimensional manifold representation of the latent space, and the relationship between this feature representation and motion intention is relatively stable, which can improve the accuracy of behavior prediction.

[0077] In a possible implementation, the preset scene conditions in the embodiments of the present application may include at least one of cross-time scene conditions, cross-task scenes, and cross-object scenes. The sample data set corresponding to the cross-time scene represents the data generated by the same sample object in two different time periods of the same neural activity task. The sample data set corresponding to the cross-task scene represents the data generated by the same sample object in two similar time periods of different neural activity tasks. The sample data set corresponding to the cross-object scene represents the data generated by different sample objects in two similar time periods of the same neural activity task, and the similar time period refers to the time interval between the corresponding two time periods that meets the preset interval condition. The embodiment of the present application does not specifically limit the size of the time interval. Under any scene condition, the data generated in the first time period (i.e., the first segment of data) will be used for the training of the decoder to ensure that the decoder can interpret the neural activity in a specific scene and learn the mapping relationship between the current neural signal and the behavior. The data generated in the second time period (i.e., the second segment of data) is used for the training of the meta-aligner, so that the meta-aligner can learn the neural signal characteristics of the new time period, new object, and / or new task, and enhance the alignment ability of the model in cross-time, cross-task, and cross-individual situations.

[0078] Specifically, for cross-time scenarios, the embodiment of the present application may select the original data sets generated by a specific rhesus monkey under a specific neural activity task in two periods of time. For example, the first segment of data contains 20 sessions in the first period of time, and the second segment of data contains 40 sessions in the second period of time.

[0079] For cross-time scenarios, the embodiments of the present application may select original data sets generated by a specific rhesus monkey in two similar periods of time under multiple neural activity tasks, for example, the first segment of data contains 40 sessions of the first task, and the second segment of data contains 40 sessions of the second task.

[0080] For cross-individual scenarios, the embodiments of the present application may select original data sets generated by two rhesus monkeys in two similar periods of time under a specific neural activity task, for example, the first segment of data contains 40 sessions of the first rhesus monkey, and the second segment of data contains 40 sessions of the second rhesus monkey.

[0081] Next, in order to facilitate the description of the model application process, the training process of the behavior prediction model is introduced:

[0082] Please refer to Figure 1 , is a schematic diagram of a training process of a behavior prediction model provided in an embodiment of the present application, and the specific implementation process of the training process is as follows:

[0083] Step 101: Based on a first sample subset in a sample data set, iteratively train a decoder until a loss function of the decoder satisfies a preset convergence condition, thereby obtaining a trained decoder.

[0084] In an embodiment of the present application, the sample data set used for model training may include sample neural signals and sample behavior data generated by the corresponding sample objects under multiple neural activity tasks. And through the acquisition process of the aforementioned sample data set, the present application divides the sample data set into a first sample set and a second sample set for training the decoder and the meta-aligner respectively according to a certain division strategy.

[0085] In a possible implementation, the decoder in the present application may be composed of a Gated Recurrent Unit (GRU) encoder and a nonlinear regressor. After obtaining a first sample subset for training the decoder, a sliding window of length T may be used to segment the neural signals of successful trials and their corresponding behavioral data from each session included in the first sample subset. The parameters of the GRU encoder are adjusted through training to learn time-dependent patterns in neural activity.

[0086] Specifically, the generative forward process of the decoder is as follows:

[0087] z i,1...T =GRU enc (x i,1...T )

[0088]

[0089] Among them, GRU enc and W dec are the GRU parameters for encoding the neural activity input, and the nonlinear layer parameters for decoding the behavior from the latent variables.

[0090] x i Represents the neural signal sequence, that is, the neural signal time series segmented by the sliding window, and each sample length is T. i Represents the behavioral data label, that is, the target behavior corresponding to each neural signal sequence, such as the two-dimensional cursor position or the target task number.

[0091] Represents the predicted behavior data of the decoder, the latent variable Z extracted by the GRU encoder i,T Calculated by the nonlinear regression layer. This application makes a prediction on the last step of the output sequence, denoted as For simplicity, the predicted value is

[0092] In a possible implementation, in each iterative training of the decoder, the embodiment of the present application can minimize the loss function through the optimizer until the loss function is minimized, or when the loss function no longer decreases significantly after multiple iterations, or when the preset maximum number of iterations is reached, the training process stops and a trained decoder is obtained. In this way, the encoder is optimized by minimizing the loss function, and the data representation capability of the encoder is improved.

[0093] Specifically, the embodiment of the present application may use the Adam optimizer to minimize the loss function, and the expression of the loss function is as follows:

[0094]

[0095] Among them, y i represents the actual behavior data of the i-th sample, Represents the predicted behavior data obtained by the decoder for the i-th sample, and N is the batch size.

[0096] Lbehavior is the loss function value, which is used to measure the predicted behavior data. and actual behavior data y i The difference between.

[0097] Step 102: Based on the second sample subset in the sample data set, perform inner loop training and meta-learning training on the meta-aligner to obtain a trained meta-aligner.

[0098] In the embodiment of the present application, inner loop training refers to updating the aligner parameters for the training data of a single task, and meta-learning training refers to improving the generalization ability of the aligner through joint optimization of training data of multiple tasks. The meta-aligner trained in this way can quickly adapt to the neural signal distribution of new tasks, new time periods or new individuals, thereby improving the accuracy and efficiency of behavior prediction.

[0099] Specifically, after the decoder completes training, it extracts the appropriate latent space for neural BCI decoding. The present application further takes into account that neural signal recordings will change over time, individuals, and tasks, and the decoder may not be used for future decoding. Therefore, the embodiment of the present application will train a meta-aligner to convert neural activity into a latent space through an alignment function, so that the model can learn a general model initialization in any scenario across time, tasks, and / or individuals, regardless of any changes or drifts in the neural activity recorded by the implanted cortical electrodes over time, tasks, and individuals, and quickly adapt to new tasks to maintain the high performance of the BCI itself.

[0100] In a possible implementation, the embodiment of the present application can use the Transformer encoder as the architecture of the meta-aligner to construct a new transformer-based network architecture Meta-AlignNN, so that the meta-aligner has strong feature extraction and representation capabilities, can capture changes in the distribution of neural activity data in different scenarios, and the evaluation indicators on relevant data sets exceed other traditional methods. And the meta-aligner is trained using a training strategy based on model-agnostic meta-learning (MAML). For example, the meta-aligner will be trained in a multi-task setting in a meta-learning phase, so that the meta-aligner can quickly adapt to new recorded sessions that have not been seen and have neural drift in the case of a small amount of neural data and a small number of training iterations.

[0101] In a possible implementation, the embodiment of the present application will sample and process the second sample subset through a preset task construction strategy to obtain multiple training task sets corresponding to the second sample subset. Through the multiple training task sets, a meta-objective function representing the total loss value of the meta-aligner on the multiple training task sets is obtained. Thus, through the meta-objective function, the model parameters of the meta-aligner are adjusted to obtain a trained meta-aligner.

[0102] Specifically, the present application samples from the second sample subset according to the task distribution p(T) to construct multiple training task sets T i In this way, by covering scenarios across time, tasks, and individuals, and the diversity of task sets reflecting different changes in neural signal distribution, the foundation for the training of the meta-aligner is laid. i Finally, the parameters of each task-specific aligner are optimized through in-task training And obtain the meta-objective function representing the total loss value of the meta-aligner on multiple training task sets, which will be used to evaluate the adaptability and generalization ability of the global parameters of the aligner. Based on the meta-objective function, the global parameter ψ of the meta-aligner can be optimized by the gradient descent method to improve its cross-task adaptability.

[0103] Specifically, the model parameter adjustment process of the meta-aligner in the embodiment of the present application can be as follows: randomly initialize the model parameter ψ of the meta-aligner, and the optimization process randomly samples a batch of task sets T from the task distribution p(T) i ~p(T). For each task T i , respectively from T i Sample K samples from the i ={(x (j) ,y (j) )}. Update model parameters using gradient descent Then from the same task set T i In the process, a new batch of samples S is sampled again i′ ={(x (j) ,y (j) )}, and use the updated parameters Calculate the corresponding loss value Combine the loss values ​​of all tasks to meta-update the model parameters ψ In this way, through multiple rounds of iterations, the meta-aligner is gradually optimized in the training task set across time, tasks, and individuals, and a meta-aligner model that can quickly adapt to the new task distribution is obtained. This model significantly improves the stability and robustness of neural signal alignment and behavior prediction with less calibration data and training iterations.

[0104] In one possible implementation, the process of obtaining the meta-objective function in the embodiment of the present application can be as follows: the model parameters of the meta-aligner are updated through the training data in each training task set and the loss function of the meta-aligner to obtain multiple update parameters, and the test error corresponding to the training task set is obtained through the test data in each training task set and the corresponding update parameters, and the meta-objective function is obtained by adding up each test error.

[0105] Specifically, the multiple training task sets T constructed in the embodiment of the present application i In each task T i Contains training data set S(x,y) and test data set Q(x,y). S(x,y) is used to train task-specific aligner parameters, and Q(x,y) is used to evaluate the adaptability of parameters. Through the training data S(x,y) and the loss function, the initial global parameter ψ of the aligner is optimized, and the update parameter specific to each task is calculated. That is, in each task T i Minimize the loss function on the training data S(x,y) Get updated parameters Next, for each training task set T i After completing the parameter update on the training dataset, the task-specific update parameters Applied to the corresponding test data set Q(x,y) in order to calculate the test error within the task on the test data By integrating the test errors of all training task sets, the test errors are summed up to obtain the meta-objective function, so that the global parameters of the aligner can be optimized through the meta-objective function using methods such as gradient descent to achieve the training of the aligner. In this way, by integrating the test errors of multiple tasks, local adaptation within the task is combined with global generalization between tasks.

[0106] In one possible implementation, reference Figure 2 Shown is a schematic diagram of a meta-aligner training process provided in an embodiment of the present application. Figure 2 The left side of the figure shows the process of meta-learning training, where Day 1, Day m and Day M represent the time span of data collection, covering neural activity data of multiple time periods, which are used to generate training task sets across time, tasks and individuals. These neural activity data are input into the corresponding aligners 1, aligners m and aligners M. The initialization of the meta-training phase represents the initial state of the global parameters of the aligner, which is optimized to the final aligner parameters after meta-training update, and can gradually adapt to the task scenarios of the future M+k days that change over time. The right side contains the test phase and update phase of the inner loop phase. The neural activity data are used for testing and updating the aligner respectively. The channels in the neural activity data represent the multidimensional neural signals collected from different electrode channels, and the time represents the time series signal segments divided by sliding windows. In the test phase of the inner loop phase, the neural activity data is first smoothed and input into the aligner to generate the aligned neural signal, which is then decoded into the behavior prediction value by the decoder with fixed parameters. The behavior loss is calculated by the difference between the actual behavior data and the predicted behavior data, which is used to update the aligner parameters or evaluate its performance. The update process in the inner loop stage is to optimize the aligner parameters based on the behavioral loss of the aligner on the training data of the task, and complete the local adaptation of the aligner within the task. In this way, through the cycle of training and testing within the task, the meta-aligner can be gradually optimized to make it more stable on all tasks and have stronger cross-task adaptability.

[0107] Specifically, in meta-learning, the meta-aligner is trained to adapt to a large number of tasks. A task refers to inputting sample data corresponding to a session into the meta-aligner, and using the aligned neural data to input the decoder to predict behavior. The set of all training tasks corresponding to the second sample subset is denoted as p(T), ψ represents the parameter of the meta-aligner, which is used to align neural activities, and the parameterizable function h with parameter ψ is ψ Stands for Meta Aligner.

[0108] Formally, each task can be written as:

[0109] T={L(x,y),s(x,y),q(x,y)}

[0110] Where L(·) is the loss function, s(x,y) represents the training data, and q(x,y) represents the test data. Specifically, 80% of the sample data is selected from the recorded sessions spanning a certain period of time, and the early sessions used to train the decoder are excluded as the test data of the meta-aligner, and the remaining 20% ​​of the data is used as the test data. Each task contains both training data and test data.

[0111] Specifically, take a training task set T iAs an example, let’s introduce the inner loop training of the meta-aligner: i Randomly sample K samples from the training data set s(x,y) and use the corresponding loss function Perform gradient descent update on it to get the updated parameters

[0112]

[0113] Among them, w dec are the trained decoder parameters, which are kept fixed during the training of the meta-aligner. α is the learning rate during the inner loop training phase. h ψ The latent representation representing the output of the meta-aligner is that the meta-aligner can extract the alignment features according to the input neural signal and the current parameter ψ, and output the aligned neural signal as the input of the decoder. Represents the loss value of the meta-aligner on the training task set, which is used to optimize the meta-aligner parameter ψ to minimize the gap between the predicted behavior and the actual behavior.

[0114] Next, the embodiment of the present application requires that i Test the new sample on the test data q(x,y). Update ψ according to the test error to minimize the prediction error of the new test data.

[0115]

[0116] in, Indicates the use of updated parameters The model ψ represents the model that needs to be updated to minimize the test error.

[0117] In a possible implementation, in order to minimize the meta-objective function of the meta-aligner, the embodiment of the present application may use stochastic gradient descent (SGD) to update the meta-aligner parameter ψ. The specific update formula is as follows:

[0118]

[0119] Here, β represents the meta-learning rate.

[0120] It should be noted that the meta-optimization is performed on the parameter ψ, but the calculation of the objective function depends on the model parameters updated for each task Therefore, when performing meta-optimization, a "gradient passing through a gradient" situation may occur, and the computational overhead is relatively large. The embodiment of the present application may also use a first-order approximation to ignore the second-order derivative to reduce the computational overhead.

[0121] After obtaining the trained behavior prediction model, the embodiment of the present application can perform behavior prediction processing on the target neural signal of the target object through the behavior prediction model to obtain the predicted behavior data of the target object output by the behavior prediction model.

[0122] refer to Figure 3 As shown, it is a flowchart of a behavior prediction method based on neural signals provided in an embodiment of the present application. The specific implementation process of the method is as follows:

[0123] Step 301: Obtain a target neural signal of a target object, and input the target neural signal into a trained behavior prediction model.

[0124] In the embodiment of the present application, after the behavior prediction model is trained, the behavior prediction can be performed on the real-time target neural signal of the target object through the behavior prediction model.

[0125] In a possible implementation, the embodiment of the present application may further fine-tune the behavior prediction model before the actual prediction to further improve the prediction accuracy of the model for the target object.

[0126] Specifically, the original neural signals and corresponding original behavioral data generated by the target object within a certain period of time can be collected in real time, and preprocessed by threshold crossing detection, data standardization and sliding window processing to obtain the corresponding sample data set S = {(x (j) ,y (j) )}, fine-tune the trained meta-aligner h through the sample dataset ψ , and get the fine-tuned aligner The specific fine-tuning process can refer to the in-task training in the aforementioned training process, and this embodiment of the present application will not be described in detail.

[0127] Step 302: Obtain predicted behavior data of the target object output by the behavior prediction model.

[0128] In the embodiment of the present application, the real-time target neural signal of the target object is predicted by using a trained behavior prediction model to obtain the predicted behavior data of the target object.

[0129] Specifically, the behavior prediction model of the embodiment of the present application can be applied to online brain control, and the target neural signal x is obtained by preprocessing the latest neural data collected in real time, such as threshold crossing detection, data standardization, and sliding window processing. After alignment, the decoder w dec Output predicted behavior data The predicted behavior data (e.g., two-dimensional cursor coordinates and hamster hole numbers, etc.) can be displayed in real time through the display interface. In addition, by receiving the behavioral feedback of the online brain control at the previous moment, the latest neural data at the next moment is adjusted and generated, and then the predicted behavior data at the next moment is generated, and the cycle repeats until the session ends.

[0130] In the embodiment of the present application, the prediction process of the predicted behavior data of the target object during the training process is the same as the prediction process during the actual application process. Therefore, the process can refer to the detailed introduction of the aforementioned training process and will not be elaborated here.

[0131] In one possible implementation, reference Figure 4 The figure shows a flow chart of a model training and model application provided by an embodiment of the present application. Figure 4 It includes offline process (model training process) and online process (model application process). The offline process includes four sub-processes: data collection, data preprocessing, data partitioning and model training. Among them, the data collection stage collects the original data set through cortical flexible electrode implantation, which contains the neural signals and behavioral data generated by two rhesus monkeys under three neural activity tasks. Then, enter the data preprocessing stage, perform data standardization on the collected data, apply the threshold crossing detection method to extract the pulses in the neural signal, and then use the sliding window segmentation method to perform window segmentation on the neural signal to generate sample data. Subsequently, the data is partitioned, and the preprocessed data is divided according to cross-time, cross-task and cross-individual scenes, and the first and second sample subsets are divided for the decoder and meta-aligner respectively for subsequent model training. In the model training stage, the decoder and meta-aligner are trained separately based on the divided data set. The meta-aligner will perform inner loop training on the specific task data to optimize the task-specific parameters, and then integrate the test errors of multiple tasks through meta-training to optimize the global parameters of the meta-aligner to improve its cross-task generalization ability. In the online process, the real-time collected neural signal data will be preprocessed, including data standardization, threshold crossing detection and sliding window segmentation, and then the preprocessed data will be input into the offline trained model for model fine-tuning and prediction. By fine-tuning the model parameters to adapt to the changes in real-time neural signals and complete the prediction of movement intentions, the predicted behavioral results will eventually be output as behavioral feedback for interactive control of the brain-computer interface.

[0132] Please refer to Figure 5 Based on the same inventive concept, the embodiment of the present application further provides a behavior prediction device 50 based on neural signals, the device comprising:

[0133] An acquisition unit 501 is used to acquire a target neural signal of a target object and input the target neural signal into a trained behavior prediction model;

[0134] Prediction unit 502, used to obtain predicted behavior data of the target object output by the behavior prediction model;

[0135] Among them, the behavior prediction model is trained based on a sample data set of multiple sample objects, and the sample data set includes sample neural signals and sample behavior data generated by the corresponding sample objects under multiple neural activity tasks; the behavior prediction model includes a decoder and a meta-aligner, the aligner is used to align the target neural signals to obtain the aligned neural signal features, and the decoder is used to predict the neural signal features and output the corresponding predicted behavior data.

[0136] Optionally, the device further includes a training unit 503, configured to:

[0137] Acquire original data sets corresponding to multiple sample objects respectively, and preprocess the original data sets; the original data sets include original neural signals and original behavioral data respectively generated by the corresponding sample objects under multiple neural activity tasks;

[0138] Based on the preset scene conditions, the preprocessed original data set is classified and processed to obtain the corresponding sample data set.

[0139] Optionally, the training unit 503 is further configured to:

[0140] Iteratively training the decoder based on the first sample subset in the sample data set until the loss function of the decoder meets a preset convergence condition, thereby obtaining a trained decoder;

[0141] Based on the second sample subset in the sample data set, inner loop training and meta-learning training are performed on the meta-aligner to obtain a trained meta-aligner.

[0142] Optionally, the training unit 503 is specifically configured to:

[0143] Based on a preset task construction strategy, sampling processing is performed on the second sample subset to obtain a plurality of training task sets corresponding to the second sample subset;

[0144] Based on the multiple training task sets, a meta-objective function of the meta-aligner is obtained, where the meta-objective function represents a total loss value of the meta-aligner on the multiple training task sets;

[0145] Based on the meta-objective function, the model parameters of the meta-aligner are adjusted to obtain a trained meta-aligner.

[0146] Optionally, the training unit 503 is specifically configured to:

[0147] Based on the training data in each training task set and the loss function of the meta-aligner, the model parameters of the meta-aligner are updated to obtain multiple updated parameters;

[0148] Based on the test data in each training task set and the corresponding update parameters, the test error corresponding to the training task set is obtained;

[0149] The meta-objective function is obtained by summing up the errors of each test.

[0150] Optionally, the preset scenario condition includes at least one of a cross-time scenario condition, a cross-task scenario, and a cross-object scenario; wherein,

[0151] Representation of sample datasets corresponding to cross-temporal scenarios: data generated by the same sample subject in two different time periods of the same neural activity task;

[0152] Representation of sample data sets corresponding to cross-task scenarios: data generated by the same sample subject in two similar time periods of different neural activity tasks;

[0153] The sample data set representation corresponding to the cross-object scene is: the data generated by different sample objects in two similar time periods of the same neural activity task; the similar time period represents that the time interval between the corresponding two time periods meets the preset interval condition.

[0154] For the convenience of description, the above parts are divided into each unit module (or module) according to the function and described separately. Of course, when implementing the present application, the functions of each unit (or module) can be implemented in the same one or more software or hardware. It can be understood by those skilled in the art that various aspects of the present application can be implemented as a system, method or program product. Therefore, various aspects of the present application can be specifically implemented in the following forms, namely: a complete hardware implementation method, a complete software implementation method (including firmware, microcode, etc.), or an implementation method combining hardware and software, which can be collectively referred to as "circuit", "module" or "system" here.

[0155] The device can be used to execute the methods shown in the various embodiments of the present application. Therefore, for the functions that can be implemented by the various functional modules of the device, reference can be made to the description of the aforementioned embodiments and no further details will be given.

[0156] See also Figure 6 As shown, based on the same technical concept, the embodiment of the present application further provides a computer device 60. In one embodiment, the computer device may be a BCI system. Figure 6 As shown, it includes a memory 601 , a communication module 603 and one or more processors 602 .

[0157] The memory 601 is used to store computer programs executed by the processor 602. The memory 601 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0158] The memory 601 may be a volatile memory, such as a random-access memory (RAM); the memory 601 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 601 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 601 may be a combination of the above memories.

[0159] The processor 602 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 602 is used to implement the above-mentioned behavior prediction method based on neural signals when calling the computer program stored in the memory 601 .

[0160] The communication module 603 is used to communicate with terminal devices such as signal acquisition devices or other servers.

[0161] The specific connection medium between the memory 601, the communication module 603 and the processor 602 is not limited in the embodiment of the present application. Figure 6 In the embodiment, the memory 601 and the processor 602 are connected via a bus 604. The bus 604 is connected to the processor 602 via a bus 604. Figure 6 The connections between the other components are only for illustration and are not intended to be limiting. The bus 604 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Figure 6 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or one type of bus.

[0162] The memory 601 stores a computer storage medium, and the computer storage medium stores computer executable instructions. The computer executable instructions are used to implement the behavior prediction method based on neural signals in the embodiments of the present application. The processor 602 is used to execute the behavior prediction method based on neural signals in the above-mentioned embodiments.

[0163] Based on the same inventive concept, an embodiment of the present application also provides a storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the steps of the behavior prediction method based on neural signals according to various exemplary embodiments of the present application described above in this specification.

[0164] In some possible implementations, various aspects of the neural signal-based behavior prediction method provided in the present application may also be implemented in the form of a computer program product, which includes a computer program. When the program product is run on a computer device, the computer program is used to enable the computer device to execute the steps of the neural signal-based behavior prediction method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the steps of each embodiment.

[0165] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0166] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on a computer device. However, the program product of the present application is not limited thereto. In the present application, the readable storage medium may be any tangible medium containing or storing a program, and the computer program included therein may be used by or in combination with a command execution system, apparatus or device.

[0167] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0168] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0169] Computer programs for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages.

[0170] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0171] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0172] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0173] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0174] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A behavior prediction method based on neural signals, characterized in that: The method comprises: Acquiring a target neural signal of a target object, and inputting the target neural signal into a trained behavior prediction model; Obtaining predicted behavior data of the target object output by the behavior prediction model; Among them, the behavior prediction model is trained based on a sample data set of multiple sample objects, and the sample data set includes sample neural signals and sample behavior data generated by the corresponding sample objects under multiple neural activity tasks respectively; the behavior prediction model includes a decoder and a meta-aligner, the aligner is used to align the target neural signals to obtain the aligned neural signal features, and the decoder is used to predict the neural signal features and output corresponding predicted behavior data.

2. The method according to claim 1, characterized in that The sample data set is obtained based on the following method: Acquire original data sets corresponding to each of the plurality of sample objects, and preprocess the original data sets; the original data sets include original neural signals and original behavioral data respectively generated by the corresponding sample objects under a plurality of neural activity tasks; Based on the preset scene conditions, the preprocessed original data set is classified and processed to obtain the corresponding sample data set.

3. The method according to claim 1, characterized in that The behavior prediction model is trained based on the following method: Iteratively training the decoder based on the first sample subset in the sample data set until the loss function of the decoder meets a preset convergence condition, thereby obtaining a trained decoder; Based on the second sample subset in the sample data set, inner loop training and meta-learning training are performed on the meta-aligner to obtain a trained meta-aligner.

4. The method according to claim 3, characterized in that The step of performing inner loop training and meta-learning training on the meta-aligner based on the second sample subset to obtain a trained meta-aligner includes: Based on a preset task construction strategy, sampling processing is performed on the second sample subset to obtain a plurality of training task sets corresponding to the second sample subset; Based on the multiple training task sets, obtaining a meta-objective function of the meta-aligner, wherein the meta-objective function represents a total loss value of the meta-aligner on the multiple training task sets; Based on the meta-objective function, the model parameters of the meta-aligner are adjusted to obtain a trained meta-aligner.

5. The method according to claim 4, characterized in that The obtaining the meta-objective function of the meta-aligner based on the multiple training task sets includes: Based on the training data in each training task set and the loss function of the meta-aligner, updating the model parameters of the meta-aligner to obtain a plurality of updated parameters; Based on the test data in each training task set and the corresponding update parameters, obtaining the test error corresponding to the training task set; The meta-objective function is obtained by performing summation processing based on various test errors.

6. The method according to claim 2, characterized in that The preset scenario conditions include at least one of cross-time scenario conditions, cross-task scenarios and cross-object scenarios; wherein, The sample data set corresponding to the cross-time scenario represents: data generated by the same sample object in two different time periods of the same neural activity task; The sample data set corresponding to the cross-task scenario represents: data generated by the same sample object in two similar time periods of different neural activity tasks; The sample data set corresponding to the cross-object scenario represents: data generated by different sample objects in two similar time periods of the same neural activity task; the similar time period represents that the time interval between the corresponding two time periods meets a preset interval condition.

7. A behavior prediction device based on neural signals, characterized in that: The device comprises: an acquisition unit, configured to acquire a target neural signal of a target object and input the target neural signal into a trained behavior prediction model; A prediction unit, used to obtain the predicted behavior data of the target object output by the behavior prediction model; Among them, the behavior prediction model is trained based on a sample data set of multiple sample objects, and the sample data set includes sample neural signals and sample behavior data generated by the corresponding sample objects under multiple neural activity tasks respectively; the behavior prediction model includes a decoder and a meta-aligner, the aligner is used to align the target neural signals to obtain the aligned neural signal features, and the decoder is used to predict the neural signal features and output corresponding predicted behavior data.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images